# trackOSPAMetric

Optimal subpattern assignment (OSPA) metric

## Description

The `trackOSPAMetric` System object™ computes the optimal subpattern assignment metric between a set of tracks and the known truths. You can enable different types of OSPA metrics by configuring these properties:

• Traditional OSPA — Specify the `Metric` property as `"OSPA"` and specify the `LabelingError` property as `0`. The traditional OSPA metric, which evaluates instantaneous tracking performance, contains two components:

• Localization error component — Accounts for state estimation errors between assigned tracks and truths.

• Cardinality error component— Accounts for the number of unassigned tracks and truths.

• Labeled OSPA — Specify the `Metric` property as `"OSPA"` and specify the `LabelingError` property as a positive scalar. The labeled OSPA (LOSPA) metric, which evaluates instantaneous tracking performance and includes penalties for incorrect assignments, contains three components:

• Localization error component — Accounts for state estimation errors between assigned tracks and truths.

• Cardinality error component— Accounts for the number of unassigned tracks and truths.

• Labeling error component — Accounts for the error of incorrect assignments.

• OSPA(2) — Specify the `Metric` property as `"OSPA(2)"`. The OSPA(2) metric evaluates cumulative tracking performance for a duration of time.

For more details, see Algorithms and References.

To use `trackOSPAMetric`:

1. Create the `trackOSPAMetric` object and set its properties.

2. Call the object with arguments, as if it were a function.

## Creation

### Syntax

``OSPAMetric = trackOSPAMetric``
``OSPAMetric = trackOSPAMetric(Name,Value)``

### Description

````OSPAMetric = trackOSPAMetric` creates a `trackOSPAMetric` System object, `OSPAMetric`, with default property values.```
````OSPAMetric = trackOSPAMetric(Name,Value)` sets properties for the `trackOSPAMetric` object using one or more name-value pairs. For example, ```OSPAMetric = trackOSPAMetric('CutoffDistance',5)``` creates a `trackOSPAMetric` object with the cut off distance equal to 5. Enclose property names in single quotes.```

## Properties

expand all

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the `release` function unlocks them.

If a property is tunable, you can change its value at any time.

Metric option, specified as `"OSPA"` or `"OSPA(2)"`.

• `"OSPA"` — Computes the traditional OSPA metric by default, or computes the labeled OSPA metric by additionally specifying the `LabelingError` property as a positive value.

• `"OSPA(2)"` — Computes the OSPA(2) metric, which evaluates cumulative tracking performance. Selecting this option enables these properties for configuring the metric:

• `WindowLength`

• `WindowSumOrder`

• `WindowWeights`

• `WindowWeightExponent`

• `CustomWindowWeights`

Selecting this option also disables two properties used to evaluate the labeling error component:

• `HasAssignmentInput`

• `LabelingError`

When selecting this option, the object internally saves the accumulated track and truth history up to a number of steps defined by the `WindowLength` property.

Data Types: `char` | `string`

Threshold for cutoff distance between track and truth, specified as a real positive scalar. If the computed distance between a track and the assigned truth is higher than the threshold, the actual distance incorporated in the metric is reduced to the threshold.

Example: `40`

Data Types: `single` | `double`

Order of the metric, specified as a positive integer.

Example: `3`

Data Types: `single` | `double`

Distance type, specified as `'posnees'`, `'velnees'`, `'posabserr'`, or `'velabserr'`. The distance type specifies the physical quantity used for distance calculations:

• `'posnees'` – Normalized estimation error squared (NEES) of track position

• `'velnees'` – NEES error of track velocity

• `'posabserr'` – Absolute error of track position

• `'velabserr'` – Absolute error of track velocity

• `'custom'` – Custom distance error

If you specify the `Distance` property as `'custom'`, you must also specify the distance function in the `DistanceFcn` property.

Custom distance function, specified as a function handle. The function must support the following syntax:

`d = myCustomFcn(Track,Truth)`
where `Track` is a structure or an object of track information, `Truth` is a structure or an object of truth information, and `d` is the distance between the truth and the track. See `objectTrack` for an example on how to organize track information.

Example: `@myCustomFcn`

#### Dependencies

To enable this property, set the `Distance` property to `'custom'`.

Desired platform motion model, specified as `'constvel'`, `'constacc'`, `'constturn'`, or `'singer'`. This property selects the motion model used by the `tracks` input.

The motion models expect the `'State'` field of the `tracks` to have a column vector containing these values:

• `'constvel'` — Position is in elements [1 3 5], and velocity is in elements [2 4 6].

• `'constacc'` — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

• `'constturn'` — Position is in elements [1 3 6], velocity is in elements [2 4 7], and yaw rate is in element 5.

• `'singer'` — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

The `'StateCovariance'` field of the `tracks` input must have position, velocity, and turn-rate covariances in the rows and columns corresponding to the position, velocity, and turn-rate of the `'State'` field of the tracks input.

Track identifier function, specified as a function handle. The function extracts track ID from the track input. The function must support the following syntax:

`Trackids = trackIdentifier(Tracks)`
where `Tracks` is an array of structures or objects containing the information of tracks, and `Trackids` is a numeric array of the same size as `Tracks`. For an example of track object, see `objectTrack`. For the default identifier function, `defaultTrackIdentifier`, the track ID must be contained in `Tracks` as the value of the `TrackID` field or property.

Example: `@myTrackIdetifier`

Truth identifier function, specified as a function handle. The function extracts truth ID from truth input. The function must support the following syntax:

`TruthIDs = truthIdentifier(Truths)`
where `Truths` is an array of structures or objects containing the information of truths, and `TruthIDs` is a numeric array of the same size as `Truths`. For the use of the default identifier function, `defaultTruthIdentifier`, the truth ID must be contained in `Truth` as a value of the `PlatformID` field or property.

Example: `@myTruthIdetifier`

OSPA(2)-Only Properties

Sliding window length for the OSPA(2) metric, specified as a positive integer. The window length defines the number of time steps from a previous time to the current time used to evaluate the metric. For more details, see OSPA(2) Metric.

#### Dependencies

To enable this property, set the `Metric` property to `"OSPA(2)"`.

Data Types: `single` | `double`

Order of the weighted sum for the track and truth history, specified as a positive scalar. For more details, see OSPA(2) Metric.

#### Dependencies

To enable this property, set the `Metric` property to `"OSPA(2)"`.

Data Types: `single` | `double`

Options for window weights, specified as `"auto"` or `"custom"`.

• `"auto"` — Automatically generates the window weights using the algorithm given in OSPA(2) Metric.

• `"custom"` — Customizes the window weights using the `CustomWindowWeights` property.

#### Dependencies

To enable this property, set the `Metric` property to `"OSPA(2)"`.

Data Types: `single` | `double`

Exponent for automatic weight calculation, specified as a nonnegative scalar. An exponent value, r, of `0` represents equal weights in the window. A higher value of r assigns more weight to recent data. For more details, see OSPA(2) Metric.

#### Dependencies

To enable this property, set the `WindowWeights` property to `"auto"`.

Data Types: `single` | `double`

Custom weights in the time window, specified as an N-element of vector of nonnegative values, where N is the window length specified in the `WindowLength` property.

#### Dependencies

To enable this property, set the `WindowWeights` property to `"custom"`.

Data Types: `single` | `double`

LOSPA-Only Properties

Penalty for incorrect assignment of track to truth, specified as a real positive scalar. The function decides if an assignment is correct based on the provided known `assignment` input. If the assignment is not provided as an input, the last known "optimal" assignment is assumed to be correct.

Example: `5`

#### Dependencies

To enable this property, set the `Metric` property to `"OSPA"`.

Data Types: `single` | `double`

Enable assignment input, specified as `true` or `false`.

#### Dependencies

To enable this property, set the `Metric` property to `"OSPA"`.

Data Types: `logical`

## Usage

### Syntax

``metric = OSPAMetric(tracks,truths)``
``metric = OSPAMetric(tracks,truths,assignment)``
``[metric,local] = OSPAMetric(___)``
``[metric,local,card] = OSPAMetric(___)``
``[metric,local,card,label] = OSPAMetric(___)``

### Description

````metric = OSPAMetric(tracks,truths)` returns the tracking performance metric between the set of tracks and truths.```
````metric = OSPAMetric(tracks,truths,assignment)` specifies the known assignment between tracks and truths at the current time step. To use this syntax, specify the `HasAssignmentInput` property as `true`.```
````[metric,local] = OSPAMetric(___)` returns the localization error component of the OSPA metric using any of the input combinations in the previous syntaxes.```

example

````[metric,local,card] = OSPAMetric(___)` also returns the cardinality error component of the OSPA metric.```
````[metric,local,card,label] = OSPAMetric(___)` also returns the labeling error component of the OSPA metric.To use this syntax, specify the `Metric` property as `"OSPA(2)"`.```

### Input Arguments

expand all

Track information, specified as an array of structures or objects for non-custom (built-in) distance functions. Each structure or object must contain `State` as a field or property. Additionally, if an NEES-based distance (`posnees` or `velnees`) is specified in the `Distance` property, each structure or object must also contain `StateCovariance` as a field or property. Moreover, if the default track identifier function is used in the `TrackIdentifierFcn` property, then each structure or object must also contain `TrackID` as a field or property.

Data Types: `struct` | `object`

Truth information at the current time, specified as an array of structures or objects for noncustomized (built-in) distance functions. Each structure or object must contain `Position` and `Velocity` as fields or properties. If the default truth identifier function is used in the `TruthIdentifierFcn` property, then each structure or object must also contain `PlatformID` as a field or property.

Data Types: `struct` | `object`

Known assignment, specified as an N-by-2 matrix of nonnegative integers. The first column elements are track IDs, and the second column elements are truth IDs. The IDs in the same row are assigned to each other. If a track or truth is not assigned, specify 0 as the same row element.

Note that the assignment must be a unique assignment between tracks and truths. Redundant or false tracks should be treated as unassigned tracks by assigning them to the "0" `TruthID`.

Data Types: `single` | `double`

### Output Arguments

expand all

Tracking performance metric, returned as a nonnegative real scalar. Depending on the values of the `Metric` and `LabelingError` properties, the returned metric can be traditional OSPA, labeled OSPA (LOSPA), or OSPA(2).

`Metric` Property Value`LabelingError` Property ValueMetric
`"OSPA"``0`OSPA
`"OSPA"`Positive scalarLOSPA
`"OSPA(2)"`Not applicableOSPA(2)

Example: 10.1

Localization error component, returned as a nonnegative real scalar.

Example: 8.5

Cardinality error component, returned as a nonnegative real scalar.

Example: 6

Labeling error component, returned as a nonnegative real scalar.

Example: 7.5

## Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named `obj`, use this syntax:

`release(obj)`

expand all

 `step` Run System object algorithm `release` Release resources and allow changes to System object property values and input characteristics `reset` Reset internal states of System object `clone` Create duplicate System object

## Examples

collapse all

Load prerecorded track data and truth data.

`load trackmetricex tracklog truthlog`

Construct a `trackOSPAMetric` object.

`tom = trackOSPAMetric;`

Initialize output variables.

```ospa = zeros(numel(tracklog),1); cardOspa = zeros(numel(tracklog),1); locOspa = zeros(numel(tracklog),1);```

Calculate three OSPA components in a loop.

```for i = 1:numel(tracklog) tracks = tracklog{i}; truths = truthlog{i}; [ospa(i), locOspa(i), cardOspa(i)] = tom(tracks, truths); end```

Visualize the results.

```figure() plot(ospa,'g'); hold on; plot(locOspa,'r:'); plot(cardOspa,'b--'); legend('OSPA','Localization OSPA','Cardinality OSPA');```

Load prerecorded tracking data that includes tracks and truth trajectories into the workspace.

```load ("ospa2TestLog.mat","trackLog","truthLog"); steps = numel(trackLog);```

Show the tracks and truth trajectories. The recorded data contains three tracks and three truth trajectories.

```positionSelector = [1 0 0 0 0 0; 0 0 1 0 0 0]; figure hold on for i = 1:steps tracksi = trackLog{i}; if ~isempty(tracksi) xyTrackPositions = positionSelector*[tracksi.State]; plot(xyTrackPositions(1,:),xyTrackPositions(2,:),"b.") end truthsi = truthLog{i}; if ~isempty(truthsi) xyTtruthPositions = cat(1,truthsi.Position)'; plot(xyTtruthPositions(1,:),xyTtruthPositions(2,:),"r.") end end xlabel("x (m)") ylabel("y (m)") title("Tracks vs. Truths")```

Create a `trackOSPAMetric` object and enable the ${\mathrm{OSPA}}^{\left(2\right)}$ metric. Specify the window length as `75`.

```ospa2Obj = trackOSPAMetric(Metric="OSPA(2)", ... WindowLength=75, ... CutoffDistance=50, ... WindowWeightExponent=3,... Order=1,... Distance="posabserr");```

Loop over the data to obtain the ${\mathrm{OSPA}}^{\left(2\right)}$ metric over time.

```ospa2 = NaN(steps,1); for i = 1:numel(trackLog) tracks = trackLog{i}; truths = truthLog{i}; ospa2(i) = ospa2Obj(tracks, truths); end```

Visualize the results.

```figure plot(ospa2) axis padded xlabel("Steps") ylabel("OSPA(2) metric")```

expand all

## References

[1] Schuhmacher, B., B. -T. Vo, and B. -N. Vo. "A Consistent Metric for Performance Evaluation of Multi-Object Filters." IEEE Transactions on Signal Processing, Vol, 56, No, 8, pp. 3447–3457, 2008.

[2] Ristic, B., B. -N. Vo, D. Clark, and B. -T. Vo. "A Metric for Performance Evaluation of Multi-Target Tracking Algorithms." IEEE Transactions on Signal Processing, Vol, 59, No, 7, pp. 3452–3457, 2011.

[3] M. Beard, B. -T. Vo, and B. -N. Vo. “OSPA (2) : Using the OSPA Metric to Evaluate Multi-Target Tracking Performance.” 2017 International Conference on Control, Automation and Information Sciences, IEEE, 2017, pp. 86–91.

## Version History

Introduced in R2019b