Spatial interpolation between two images/frames.

Hello everybody,

[Wanted]: I want to estimate (interpolate) middle image between two frames in 'xylophone.mp4', for example I would like to to estimate new frame between existing frames 4 and 6.

[What has been done so far]: Here is what I have done so far:

close all; clear all; clc;
vidObj = VideoReader('xylophone.mp4');
vidHeight = vidObj.Height;
vidWidth = vidObj.Width;
s = struct('cdata',zeros(vidHeight,vidWidth,3,'uint8'),...
    'colormap',[]);
k = 1;
while hasFrame(vidObj)
    s(k).cdata = readFrame(vidObj);
    k = k+1;
end

figure(1);
subplot(1,2,1);imshow(s(4).cdata); title('4th frame');
subplot(1,2,2);imshow(s(6).cdata); title('6th frame');

im1=s(4).cdata;im2=s(6).cdata; t=0.5;
imnew = (1-t)*im1 + t*im2; % linear combination of them.

figure(2);
subplot(1,2,1); imshow(s(5).cdata); title('5th frame');
subplot(1,2,2); imshow(imnew); title('5th frame');

[Need help with]

1. Estimate/interpolate image between frame 4 and 6, using following methods: 'nearest','cubic','spline','linear'.

I'm not sure what function should I use ... interp2 or interp3 and how to use it. I have tried to use Vq = interp2(im1,im2,'nearest'); but I got an error, so that is no the way how it should be used.

Thanks in advance for any help (@Image Analyst),

Ivan

Answer 1

[Image Analyst]

I don't know about interp3 - I haven't used it. interp2() required 2D images so to use that you'd use it on each color channel one at a time. It will basically give you the average of the images so you'd have two hands in it, like you have but each would be blended with the background. You would not get one hand in between the two hand positions. There might be algorithms for that, but I don't know what they are.

[Ivan Shorokhov]

Hello @Image Analyst, Thank you very much for your quick response.

Could you help me, please, with code for interp2, with 'spline' method, to simplify the code you can convert images to gray scale. Even average of the images would suite vet well, I just what to see which method can provide better result.

I have tied the following code, but it didn't work:

I = rescale(rgb2gray(im1),256);
class_of_I = class(I);
[numCols,numRows] = size(I);
[x,y] = meshgrid(1:numCols,1:numRows);
[xi,yi] = meshgrid(1:0.1:256);
New_Image = cast(interp2(x,y,double(I),xi,yi,'spline'),class_of_I);

[Image Analyst]

You can't use spline method to get a slice in between two other slices. For one, there are not enough slices for spline. Secondly, you'd have to permute the direction of the data so you're going along the z dimension, not within the x-y plane.

[Ivan Shorokhov]

Thank you, then I'm wondering how the author managed to do it in the following publication http://fbt.tums.ac.ir/index.php/fbt/article/viewFile/50/45 I guess he was using interp3.

[Image Analyst]

Perhaps. I have never looked at that function. Maybe you can ask the author.