How can I do Video Skeletal Tracking through Kinect V2 in MATLAB?

I using Kinect for XBox One (Kinect V2) with Windows Adapter to do skeletal tracking through video in MATLAB. It is correctly identifying the location of joints but it is not joining them correctly. Therefore the lines are skewed and no proper skeleton is showing (see attached image). Also, fps are very low so the video is shaky. I am using MATLAB 2016a with Image Acquisition Toolbox Version 5.0, Microsoft Kinect for Windows Support from Image Acquisition Toolbox, Kinect for Windows Runtime 2.0 and Windows 10 64 bit. How can I correctly join the joints and increase fps? I am using the following code:

imaqreset;
%create color and depth kinect videoinput objects
colorVid = videoinput('kinect', 1);
depthVid = videoinput('kinect', 2, 'depth_512x424');
triggerconfig (depthVid,'manual');

framesPerTrig = 1

depthVid.FramesPerTrigger=framesPerTrig;
depthVid.TriggerRepeat=inf;

src = getselectedsource(depthVid);
src.EnableBodyTracking = 'on';

start(depthVid);

himg = figure
while ishandle(himg);
trigger (depthVid);
[depthMap, ~, depthMetaData] = getdata (depthVid);
imshow (depthMap, [0 4096]);

if  sum(depthMetaData.IsBodyTracked) >0
skeletonJoints = depthMetaData.DepthJointIndices (:,:,depthMetaData.IsBodyTracked);
hold on;
plot (skeletonJoints(:,1), skeletonJoints(:,2),'*-');
hold off;
end
end
stop(depthVid);

[Izza Ali]

Hi Rao, I am facing this same issue while using depth image. The joints are correctly placed but the interlinking lines are messed up. Also, how can i do skeleton tracking on RGB video?

Answer 1

[Dymiargani Milono]

 try this one instead

    imaqreset;
%create color and depth kinect videoinput objects
colorVid = videoinput('kinect', 1);
depthVid = videoinput('kinect', 2);
triggerconfig (depthVid,'manual');
framesPerTrig = 1;
depthVid.FramesPerTrigger=framesPerTrig;
depthVid.TriggerRepeat=inf;
src = getselectedsource(depthVid);
src.EnableBodyTracking = 'on';  
start(depthVid);
himg = figure;
SkeletonConnectionMap = [ [4 3];  % Neck
                          [3 21]; % Head
                          [21 2]; % Right Leg
                          [2 1];
                          [21 9];
                          [9 10];  % Hip
                          [10 11];
                          [11 12]; % Left Leg
                          [12 24];
                          [12 25];
                          [21 5];  % Spine
                          [5 6];
                          [6 7];   % Left Hand
                          [7 8];
                          [8 22];
                          [8 23];
                          [1 17];
                          [17 18];
                          [18 19];  % Right Hand
                          [19 20];
                          [1 13];
                          [13 14];
                          [14 15];
                          [15 16];
                        ];
while ishandle(himg);
trigger (depthVid);
[depthMap, ts, depthMetaData] = getdata (depthVid);
anyBodiesTracked = any(depthMetaData.IsBodyTracked ~= 0);
trackedBodies = find(depthMetaData.IsBodyTracked);
nBodies = length(trackedBodies);
colors = ['r';'g';'b';'c';'y';'m'];
imshow (depthMap, [0 4096]);
if  sum(depthMetaData.IsBodyTracked) >0
skeletonJoints = depthMetaData.DepthJointIndices (:,:,depthMetaData.IsBodyTracked);
hold on;
for i = 1:24
for body = 1:nBodies
X1 = [skeletonJoints(SkeletonConnectionMap(i,1),1,body); skeletonJoints(SkeletonConnectionMap(i,2),1,body)];
Y1 = [skeletonJoints(SkeletonConnectionMap(i,1),2,body), skeletonJoints(SkeletonConnectionMap(i,2),2,body)];
line(X1,Y1, 'LineWidth', 2, 'LineStyle', '-' , 'Marker', '+', 'Color', colors(body));
end
end
hold off;
end
end
stop(depthVid);

[Patrick Kuba]

is there any chance I can track the skeleton and save the coordinates of the joints in a txt-file? I need them for further investigation to calculate some distances and angles.

[chian lim]

Do you figure out the solution already ?

IF yes, please send me the code to this email: [e-mail]

TQ very much

[Dymiargani Milono]

@Patrick Kuba sorry for the late reply, i was working on this for my bachelor's thesis and never touched matlab ever since lol. The skeleton coordinate will be in the metadata after you save stopped the video and you can convert it from metadata to a 25 x 3 matrix. After that you can save it manually to a txt file.

@chian lim https://library.ui.ac.id/detail?id=20489438&lokasi=lokal you can download my thesis here. I put all of my code here

[William Sánchez]

@Dymiargani Milono I can´t get your tesis, I need an account of your university, could you give or send the information at the foro, I really need the code or only know how did you do it

[Dymiargani Milono]

@William Sánchez i am terribly sorry that i just read your reply. I'll try to attach my thesis in here but if i failed to do that you can email me on [e-mail] and i'll send my thesis to you

[Homero Vladimir]

Dymiargani.

I was not able to enable skeleton tracking.

I tried your code and works very well. Thanks !

Best regards,

Homero

[Daniel Franco]

@Dymiargani Milono

Good evening, congratulations on your work. I'm having problems with slow processing of the image (the frames). I saw that your code works one frame at a time, is there anything I can change to improve this aspect? Thank you very much in advance