This was inspired by Luc Trudeau's CfL in Action notebook but doesn't incorporate any chroma-from-luma logic. It turned out that simple linear filters on the chroma planes gave about 90% of the gain from CfL.
The edges are filtered from 2x2 inputs and the interior blocks from 3x3 inputs. The weights were trained on the Kodak Lossless True Color Image Suite.
`C = [[0, 0, 0], [0, 1, 0], [0, 0, 0]],` `H = 13/128[[0, 6, 0], [0, 0, 0], [0, -6, 0]],` `V = 13/128[[0, 0, 0], [6, 0, -6], [0, 0, 0]],` `D = 13/128[[1, 0, -1], [0, 0, 0], [-1, 0, 1]]`
`y = (C*x)` `+ (H*x) 1/4[[1, 1], [-1, -1]]` `+ (V*x) 1/4[[1, -1], [1, -1]]` `+ (D*x) 1/4[[1, -1], [-1, 1]]`
The canvas is drawn from the three planar images below. Try moving the sliders to alter to each plane separately. The Y slider controls luma contrast. The Cb and Cr sliders control the strength of chroma upscaling.