How can I store the feature descriptors for all 3D points found in Structure from Motion?

I am running the example on Structure from Motion from MathWorks. I have managed to find 3D points from my 2D images, but I also need to store feature descriptors for each 3D point, so that I can perform 3D-2D matching later on. How do I go about this? I can save the features for each image separately, but I don't know how to later match them to each 3D point.

Answer 1

[Qu Cao]

You can use imageviewset to store the feature points associated with each view and the connections between the views. You can also use worldpointset to store the correspondences between 2-D image points and 3-D world points.

[Markus Iversflaten]

I am currently storing my data in an imageviewset, including the individual images' features. However, when I use the findTracks, I don't know how to associate each feature with each point in the pointTrack object.

[Qu Cao]

pointTrack doesn't store the index of the image point. A workaround is to find the index by comparing the image [x, y] positions. Note that the positions are stored in the imageviewset. Since pointTrack stores both the view ID and the image point position, you should be able to identify the image point corresponding to a view in the pointTack. Basically, you need to do something like the following:

vSet = imageviewset;
% Add views
tracks = findTracks(vSet);
% Find the image point of the first view in the first point track
find(tracks.Points(1,:) ==  vSet.Views.Points{viewId})

We understand this is a pain. We will consider enahnce this feature in a feature release.

[Markus Iversflaten]

I've managed to extract the features that were used to create the 3D points now. However, it seems that the features are not ordered correctly, even though I am extracting them in the order of the pointTracks:

features = zeros(length(tracks),64);

for i=1:length(tracks)
    track = tracks(i);
    viewID = track.ViewIds(1);
    index = find(track.Points(1,:) == vSet.Views.Points{viewID}.Location,1);
    feature = vSet.Views.Features{viewID}(index,:);
    features(i,:) = feature;
end

I believe that running this line:

xyzPoints = triangulateMultiview(tracks, camPoses, intrinsics);

might "scramble" the 3D points, so that they are no longer ordered in the same way as tracks. Am I right about this? If so, how can I remedy this?

[Qu Cao]

The first view in each point track may not correspond to the same view. You may need to do some debugging by inspecting the data in tracks.

In triangulateMultiview, xyzPoints and tracks have 1-to-1 correspondence. The order is not changed.