Export Ground Truth Object to Custom and COCO JSON Files

Open Live Script

This example shows you how to export a ground truth object to a custom data format JavaScript Object Notation (JSON) file, and to a COCO [1] data format JSON file.

You can parse JSON files to use for training and inference in machine learning frameworks. The format of the custom data format JSON file is similar to the COCO data format. The difference is that the format for representing label annotations for the JSON file is modified to include various label types supported by the ground truth object.

This example uses a ground truth object that contains labels previously generated using the Video Labeler app.

To export a ground truth object from the Labeler app to the JSON file, follow these steps:

  1. Use the Image Labeler or the Video Labeler App to label images or videos respectively.

  2. Export the labels to file or the workspace to generate a ground truth object.

  3. Follow the steps in this example.

Load Ground Truth Object

Load a ground truth object.

load("VehicleImageSequenceGroundTruth.mat")

Display the label definitions.

VehicleImageSequenceGroundTruth.LabelDefinitions
ans=3×6 table
      Name        Type               LabelColor                 Group       Description     Hierarchy  
    _________    _______    _____________________________    ___________    ___________    ____________

    {'Car'  }    Polygon     0.5862     0.8276     0.3103    {'Vehicle'}    {0×0 char}     {1×1 struct}
    {'Lane' }    Line       0.49412    0.18431    0.55686    {'Road'   }    {0×0 char}     {0×0 double}
    {'Sunny'}    Scene            1    0.41176    0.16078    {'None'   }    {0×0 char}     {0×0 double}

Add Info and License Information (Optional)

Add the metadata to the annotations by specifying the Info and the License Name-Value pairs.

info = struct( ...
    'year', 2020, ...
    'version', '1', ...
    'description', 'Vehicles Dataset', ...
    'date_created', datetime ...
    );

licenses = struct( ...
    'url', {'www.mathworks.com', 'www.mathworks.com'}, ...
    'name', {'License 1', 'License 2'} ...
    );

Export the Ground Truth Object to a Custom Data Format JSON File

Use exportGroundTruthToJSON function to export the ground truth object to a JSON file.

The exportGroundTruthToJSON function parses the data in the ground truth object, encodes it in a custom data format that is described below, and then generates the JSON file. 

annotationsFileName = 'vehicles_annotations.json';
exportGroundTruthToJSON(VehicleImageSequenceGroundTruth, annotationsFileName, 'Info', info, 'License', licenses)

Export the Object Detection Ground Truth Object to a COCO Data Format JSON File

You can use the exportGroundTruthToJSON helper function in this example to convert polygon and pixel labels for object detection into the COCO format.

To convert the labels:

  • Set the function’s COCO property to true,

  • Use Polygon labels for individual object instances.

  • Use Pixel labels for crowd regions. The pixel label name must include the suffix “_crowd”.

For example, if you are labeling people and regions in the image:

  • Create a polygon label named "person" for individual people.

  • Create a pixel label named "person_crowd". for crowd regions. The _crowd suffix ensures that the crowd label is linked to the same category as the polygon label.

Note: When exporting to COCO object detection format, you might see a warning about ignored label types. In this example, the ground truth objects include Line labels, which will be ignored.

annotationsFileName = 'vehicles_annotations_coco.json';
exportGroundTruthToJSON(VehicleImageSequenceGroundTruth, annotationsFileName, ...
    'Info', info, 'License', licenses, 'COCO', true)
Warning: Label types, Line, Scene, are not supported and are ignored during export to a COCO data format JSON file

Custom JSON Data Format

The exported JSON file contains these five fields:

  1. info

  2. licenses

  3. categories

  4. images/ video

  5. annotations

Info

The Info field specifies the dataset and annotation information by using the 'Info' name-value pair argument. The value must be a scalar structure.

Default Value:

{
  "year": Current year,
  "version": "1",
  "description": "Created using MATLAB",
  "date_created": Current datetime
}

Licenses

The Licenses field specifies information about the licenses by using the License name-value pair argument. The value must be a structure array.

Default Value: []

Categories

The Categories field contains the contents of the ground truth label definitions table. The variable names in the table are used as fields and are modified to be similar to the COCO data format. An additional field Id identifies each label definition.

Each category object contains the following two additional fields which replace the 'Hierarchy' variable if it exists:

  1. Attributes: Contains attributes definitions of the label.

  2. Sublabels: Contains sublabels definitions of the label.

Images/ Video

The Images/ Video field contains a list of the images (or video) objects and related information.

Data Format:

{
  "id": Unique Image ID,
  "time_stamp": Time stamp of the image frame (Only applicable to data sources with timestamps),
  "width": Width of the image/video,
  "height": Height of the image/video,
  "file_name": Name of the file,
  "file_path": Absolute file path,
  "date_captured": Last modified date of the image/ Current datetime
}

Annotations

The Annotations field contains a list of annotation objects. The data format of the annotation objects depends on the label type and is as follows:

Rectangle ROI

{
  "id": Unique Annotation ID,
  "image_id": Image ID , 
  "category_id": Category ID,
  "position":  [x,y,width,height] bounding box location,
  "attributes": Contains attributes data,
  "sublabels": Contains sublabels data
}

  • x and y specify the upper-left corner of the rectangle and are 0-indexed.

  • w specifies the width of the rectangle, which is its length along the x-axis.

  • h specifies the height of the rectangle, which is its length along the y-axis.

Polygon ROI

{
  "id": Unique Annotation ID,
  "image_id": Image ID , 
  "category_id": Category ID,
  "position":  [[x1 y1 x2 y2 ...  xN yN]] for N points in the polygon,
  "attributes": Contains attributes data,
  "sublabels": Contains sublabels data
}

PolyLine ROI

{
  "id": Unique Annotation ID,
  "image_id": Image ID,
  "category_id": Category ID,
  "position": [x1 y1 x2 y2 ...  xN yN] for N points in the polyline.,
  "attributes": Contains attributes data,
  "sublabels": Contains sublabels data
}

  • x1, y1,... specify the point location and are 0-indexed.

ProjectedCuboid ROI

{
"id": Unique Annotation ID,
"image_id": Image ID,
"category_id": Category ID,
"position": List of the form [xctr, yctr, zctr, xlen, ylen, zlen, xrot, yrot, zrot],
  "attributes": Contains attributes data,
  "sublabels": Contains sublabels data
}

  • xctr, yctr, and zctr specify the center of the projected cuboid and are 0-indexed.

  • xlen, ylen, and zlen specify the length of the projected cuboid along the x-axis, y-axis, and z-axis, respectively, before rotation has been applied.

  • xrot, yrot, and zrot specify the rotation angles for the projected cuboid along the x-axis, y-axis, and z-axis, respectively. These angles are clockwise-positive when looking in the forward direction of their corresponding axes.

PixelLabel ROI

{
  "image_id": Image ID,
  "file_name": Name of the pixel label image file,
  "file_path": Path to the pixel label image file,
  "segments_info": List of segment objects
}

Segment Objects format: Each segment in the pixel label image is represented as a segment object.

{
"id": PixelLabelID of the label or R+G*256+B*256^2 in case if PixeLabelID is an array,
"category_id": Category ID,
"area": Area covered by the pixels
}

Scene

{
  "id": Unique Annotation ID,
  "image_id": Image ID,
  "category_id": Category ID of the scene label applicable to the image
}

Limitations

  1. Custom label types cannot be exported to the JSON file.

  2. You cannot export data from a custom reader ground truth datasource to a JSON file format.

  3. The exportGroundTruthToJSON helper function used in this example only exports Polygon and crowd Pixel label ROIs to the COCO data format for object detection. For details see the Export to COCO format section.

  4. The JSON file generated by the exportGroundTruthToJSON helper function cannot be imported back into MATLAB® as a ground truth object.

References

[1] Lin TY. et al. (2014) Microsoft COCO: Common Objects in Context. In: Fleet D., Pajdla T., Schiele B., Tuytelaars T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8693. Springer, Cham. https://doi.org/10.1007/978-3-319-10602-1_48