add audio to a static image

I have an audio signal and a picture. I'm trying to output a video of the static image for the duration of the audio that's playing. I'm woefully unskilled with the computer vision toolbox (but I have it). Could someone help me with this seemingly simply task?

[xt,fs]=audioread('mywav.wav');
im=imread('myimage.png');

videoFWriter = vision.VideoFileWriter('newvideo.avi','AudioInputPort',true);
videoFrame = im;
step(videoFWriter,videoFrame,xt);
release(videoFWriter);

The output is a video file, but when I try to listen to the audio of it, it's been resampled and all sorts of crazy. Help!

Answer 1

[Pranjal Kaura]

Hey,

The problem with your code seems to be that you’re not setting the frames per second parameter. If you don’t set it, the model assumes the FPS to be 30(default). Now when you add audio using the step command, its rushed/played quickly, to finish within the time frame of numFrames/FPS.

Here’s my solution. Hope this helps.

[data, freq] = audioread('pathtoAudioFile');
img = imread('pathtoImage');
audioLength = length(data)/freq;%duration of audio file
writerObj = vision.VideoFileWriter('newvideo.avi', 'AudioInputPort',true, 'FrameRate', 1);%Setting FPS to 1. Now we need to add atleast
%audioLength number of frames
for i = 1:audioLength
    parsedAudio = data((i-1)*length(data)/audioLength + 1:i*length(data)/audioLength);%parsing the audio into equally sized pieces(play for 1 sec)
    % that can be added with each img frame
    step(writerObj, img, parsedAudio);
end

release(writerObj);

[Shae Morgan]

This is really close to what I'm looking for - but it cuts off some of the audio. Do I need to buffer the audio somehow so the sample rate matches the frame rate of the video so it'll play the whole audio file?

[Pranjal Kaura]

The current solution parses audio at intervals of one sec. Thus if the audio is of length 16.6 seconds, 0.6 seconds of it will be left out. To add more granularity to the audio parsing, you can change the Frame rate and with it parsedAudio variable. For e.eg doubling the frame rate would require you to halve the audio being added to parsedAduio at each step.

[Shae Morgan]

Thank you! that helps a ton.

[SKP]

Thank you for this code.

I could successfully create .avi file using this code. However, I wanted a .mp4 file as output and therefore, I tried to modify one line in this code as below,

writerObj = vision.VideoFileWriter('Filename','newvideo.mp4','FileFormat', 'MPEG4','AudioInputPort',true,'FrameRate', 1); %

This was giving me a warning and error (shown below), which I am not able to resolve. Could you please help?

Warning: The AudioInputPort property is not relevant in this configuration of the System object.

Error using vision.VideoFileWriter/step

Too many input arguments. Expected 1 (in addition to System object), got 2.

Error in Audio_combine_evening (line 51)

step(writerObj, img, parsedAudio);