Kinect(Microsoft SDK) Skeleton (recorded ) Data from pixel to 3d World Coordinates

I have a dataset of partial joints(right elbow , r shoulder and r wrist)taken from a fellow who acquired this data with OpenNi. The joints are in pixel as regards x and y ,while z in mm. I have to convert them in the world space to match them with data acquired by me (using Microsoft Sdk) for an application of gesture recognition. I' working in Matlab.

Searching on web and papers i found that a floor reference is necessary for the conversion but i don't have any, so how this conversion could be done,possibly in matlab, and which candidate should i pick ?(maybe height of kinect from the floor?) Thanks in anticipation.

Answer 1

[Geoff]

Are you saying you acquired the same gesture data from the same unit but in different formats? Are your time stamps the same?

Note: Maybe you could post a link to the article you are considering implementing.

My instinct is this:

Any conversion to world-space is going to be arbitrary. Without any other reference you just need to assume the unit itself as the axis system. In a way, that actually makes things easier.

Presumably your conversion will involve known field of view and aspect ratio of the cameras. You ought to be able to take whatever algorithms are out there, write them in MatLab and generate your points. If it insists on a floor reference, use zero.

Now you need to match them up. This is simply about finding the transformation that rotates one system to line up with the other. If you're lucky it'll only be a matter of offset (which you can infer by plotting the individual components of the wrist in both sets together and looking at the difference). ie compare the X, Y and Z values of your two sets and obtain the offset.

If you're not lucky, the Microsoft SDK will have been clever by working out the exact orientation angle of the Kinect and correcting for it. Now you need to find a rotation. Provided the two datasets are synchronised with a common time value, this should be straight forward:

Start with a unit 4x3 transformation matrix [1 0 0; 0 1 0; 0 0 1; 0 0 0], and bung it into a solver. To transform a point P (given as a 1x4 vector [x y z 1]) by your matrix T you multiply P * T. You want to minimize the sum of squared distances between your transformed points and your reference data.

The output should be the a transformation matrix that maps your data from one set to the other. You might want to make sure the first three rows are orthogonal. You have now calibrated your Kinect.

Bear in mind I have not used Kinect before! I'm just a programmer who uses MatLab. Hope this helps.