How to add color to a grayscale image based on a colormap from a color photo?

I have two cameras, a very fast one that takes black and white photos, and a slow one that takes color photos. I want to be able to add false color to the gray photos using the colormap from the color photos. My code: gray=imread('Gray.png');

color=imread('Color.png');

[indexImage,cmap]=rgb2ind(color,128);

gray=rgb2gray(gray);

rgb=grs2rgb(gray,cmap);

comp=imcomplement(rgb);

figure

imagesc(comp);

I have to take the complement because for some reason the colors are reversed when adding color to the gray image. But the biggest problem is that the picture comes out very grainy. Is there a better way to do this?

Answer 1

[Image Analyst]

I don't like that method. I think the best way is to convert the color image into HSV color space, and apply the Hue and Saturation images to the gray scale camera image. This assumes the cameras are fairly well aligned and have the same field of view. Even so, the image will be slightly off color. See demo code, test3.m, below the image it creates.

Answer 2

[DGM]

Well, it's Sunday night. The forum is sleeping and I'm bored. I might as well pretend that this thread needs another answer. @Image Analyst already said what needs to be said, but I like to entertain myself with elaboration.

Applying a colormap

To begin, I want to press a point that I haven't. Given a Mx3 list of RGB tuples, I think most people would call it a "colormap", whereas I have developed a (not entirely consistent) habit of referring to it generically as a "color table". The difference is a matter of expectations. If you find some random Mx3 list of tuples lying on the sidewalk, there's no reason to expect that it has any particular properties. The output of rgb2ind() is not linear-lightness or linear-luma in its ordering. It does not map luma to color (as if anything could). Divorced from its index array, the list has no meaningful order. It's for this seemingly obvious reason that I'm hesitant to call these things maps. Without any other information, it's just an arbitrary lookup table.

Yeah, yeah. A LUT is a map, but this is a matter of rather petty subtlety. I think the common expectation of a "colormap" is that dim1 is a (often expectantly linear) map of a generally continuous or coarsely piecewise-continuous trajectory through 3D space.

The point here is that your color table does not, and can not, generally map gray values back to their original color, even for the exact same image. You're projecting 3D information into 1D and then assuming you can unambiguously back calculate the original values despite all the information that's been lost -- by using a randomly-ordered list of colors. The result should be unsurprising.

% an RGB image and a grayscale copy of the exact same image
rgbpict = imread('peppers.png');
graypict = im2gray(rgbpict);

% pretend that the colors of an RGB image can be reduced to a reversible 1D mapping
% gray2pcolor() is equivalent to using imagesc() with the specified CT as a colormap
[~,map] = rgb2ind(rgbpict,256);
grayidx = gray2ind(graypict,256);
A = ind2rgb(grayidx,map);
imshow2(A) % it's complete nonsensical garbage

MIMT does have a set of tools for creating an ordered colormap from an image and then applying it to a grayscale image. That said, there's a couple reasons why that still doesn't work.

MIMT im2ct() takes a color image and attempts to create an ordered map of characteristic colors from the image. The map is either linear or at least monotonically-increasing in brightness. The number of breakpoints and interpolating behavior can be selected. In most cases, you can't actually make a map with more than a handful of breakpoint colors, simply because there's no good general way to interpolate a path through a 3D cloud of color points that results in a progression of colors that has an orderly appearance with regards to any particular color axis.

The second problem is still the same as before. Since any number of colors can map to the same gray value, there's still no way to map the gray values to their original colors. Even if we can obtain a well-ordered colormap, that might only give us a broad color theme. The relationship between objects and their original colors is still lost.

% the same image and a gray copy
rgbpict = imread('peppers.png');
graypict = im2gray(rgbpict);

% use im2ct() to "restore" color to the gray copy
CT = im2ct(rgbpict,'ncolors',256,'nbreaks',8,'uniform',true, ...
    'cspace','lab','fullrange',true,'minsat',0.2);
A2 = gray2pcolor(graypict,CT,imclassrange(class(graypict)),'default');

imshow2(A2) % that's better, but still pretty useless

The only cases where this does work are those where the colors in the original image lie in a narrow band that can be simply sorted by luma (or L*). That's not the general case for photographs though, so it's unlikely to be useful.

As the synopsis indicates, im2ct(), was never really meant to be good at this. It was written to demonstrate an improved techinque to accomplish a questionable task. Are there more elaborate approaches that can use content awareness to recolorize the image based on an unsorted color table? Perhaps, but I don't have one.

Colorization by image blending

Okay, so using rgb2ind() or other means of applying a colortable/map is not helpful. Rather than trying to blindly guess the colors of gray pixels based on another image, we could hopefully assume the images are (or could be) registered. If there is some general spatial correspondence, maybe we could treat this as an image colorization/blending task.

It's likely that the images are not exactly registered. If that's not sufficient, Image Processing Toolbox does provide an Image Registration Estimation app, though to use the registration objects programmatically requires other toolboxes. In this example, I'm just assuming a not-quite-registered image pair and processing them as-is.

MIMT imblend() makes the suggested colorization very simple.

% a nominally-registered RGB image of the same scene
crefpict = imread('peppers_shift.png');

% recreate IA's example in imblend()
% rescale FG, altering global contrast; swap V
B = imblend(simnorm(graypict),crefpict,1,'value');
imshow2(B)

It's dark. It's usually dark. V is the maximum of RGB contributions. It discards a lot of information, and generally results in an image that's brighter than any sensible grayscale representation. If you start with a sensible grayscale representation and pretend that it's V, you'll get a dark image. With a few occasional exceptions, HSV is the worst model for color adjustments or colorizations.

While imblend() makes a simple colorization with HSV into a one-liner, it also makes a relatively complicated colorization with chroma-constrained YPbPr into an equally-mindless one-liner.

% try something that's a bit better
% maintain original contrast; swap luma in YPbPr with chroma constraint
C = imblend(graypict,crefpict,1,'lumac');
imshow2(C)

... and the results are about as good as you can get without improving the registration.