Optimize tracking using KLT algorithm

Hello, I wrote a code to track points in a video with KLT algorithm. First I tried the code in a video with 5 generic points spaced by at least 1cm and it worked properly. Then I tried the code in a video with 36 points (6x6) spaced by 2mm and after 96 frames it starts to losing points and at the end it doesn't track any. I tried to change the tracker parameters but it still doesn't work. Actually my parameters are:

'NumPyramidLevels',3,'MaxBidirectionalError',2,'BlockSize',[101 101],'MaxIterations',30

Do you have any suggestion about how to solve the problem and how to set the parameters?

I attach the entire code:

vid0=VideoReader('IMG_7180.mov');

NumFrames=vid0.NumberOfFrames;

pointTracker = vision.PointTracker('NumPyramidLevels',3,'MaxBidirectionalError',2,'BlockSize',[101 101],'MaxIterations',30) ;

I0 = read(vid0, 1);

I0gray=rgb2gray(I0);

level0=0.4;

I0thresh=im2bw(I0gray,level0);

[centersCam0, radiiCam0] = imfindcircles(I0thresh,[3 14],'ObjectPolarity','dark','Sensitivity',0.9);

points0=centersCam0;

sortedpoints0=sortrows(points0);

initialize(pointTracker,sortedpoints0,I0gray);

NumMarkers = size(centersCam0);

NumMarkers = NumMarkers(1,1);

Cam0_coordinates = zeros (NumMarkers,2,NumFrames);

for k=NumFrames

    I0 = read(vid0, k); 
    I0gray=rgb2gray(I0);
[points0framek,point_validity0] = step(pointTracker,I0gray);
I0controllo=I0gray;
Cam0_coordinates(:,:,k)=points0framek(:,:);
end

Thank you everybody!