Blocking pixel label data for semantic segmentation DL training

I'm trying to block images and their pixel labels for training a unet. I can use a blockedImageDatastore for the input images, but I don't know how to get this blocking behavior from the pixelLabelDatastore that holds the expected labels. I can get the behavior myself by splitting all the images beforehand and saving them to disk, but I'd rather not have to deal with the file cleanup or lose the dynamic changing of blocking. Does anyone know a way to achieve this?

Answer 1

[Malay Agarwal]

Hi @Software Developer,

Please refer to the following link for an example on how to train a U-Net on multispectral images: https://www.mathworks.com/help/images/multispectral-semantic-segmentation-using-deep-learning.html

The example suggests using "blockedImage" to preprocess both your training samples and the labels. Specifically, you can refer to the following section of the example for sample code: https://www.mathworks.com/help/images/multispectral-semantic-segmentation-using-deep-learning.html#SemanticSegmentationOfMultispectralImagesExample-7.

In the code:

inputTileSize = [256 256];
bim = blockedImage(train_data(:,:,1:6),BlockSize=inputTileSize);
bLabels = blockedImage(labelsTrain,BlockSize=inputTileSize);
bmask = blockedImage(maskTrain,BlockSize=inputTileSize);

• "bim" represents the first 6 channels of the training image, blocked using a block size of "[256 256]".
• "bLabels" are the corresponding labels, blocked using the same block size.
• "bmask" is the binary mask which represents the valid segmentation region, made using the 7th channel of the image and blocked using the same block size.

The example then finds the blocks of the image that overlap the mask using the following code:

overlapPct = 0.185;
blockOffsets = round(inputTileSize.*overlapPct);
bls = selectBlockLocations(bLabels, ...
    BlockSize=inputTileSize,BlockOffsets=blockOffsets, ...
    Masks=bmask,InclusionThreshold=0.95);

After one-hot encoding the labels, it then creates two "blockedImageDatastore" objects, one for the image and one for the labels. It uses the "BlockLocationSet" name-value argument to filter out only those image blocks and labels that overlap with the mask:

bimds = blockedImageDatastore(bim,BlockLocationSet=bls,PadMethod=0);
bimdsLabels = blockedImageDatastore(bLabels,BlockLocationSet=bls,PadMethod=0);

Finally, it combines the block images and the labels into a single datastore using the "combine" function. This combined datastore can be used to train the U-Net.

Please refer to the following links for more information:

• "blockedImage" documentation - https://www.mathworks.com/help/images/ref/blockedimage.html.
• "selectBlockLocations" documentation - https://www.mathworks.com/help/images/ref/selectblocklocations.html.
• "blockedImageDatastore" documentation - https://www.mathworks.com/help/images/ref/blockedimagedatastore.html.
• "BlockLocationSet" name-value argument documentation - https://www.mathworks.com/help/images/ref/blockedimagedatastore.html#mw_1c8c9014-3714-48cb-b151-4e5852e838df.

Hope this helps!

Answer 2

[Ashish Uthama]

Have not tried this - but instead of a pixelLabelDatastore, could you try to use another blockedImageDatastore to read the label data, and then use a transform() call to convert pixel data into label categories?