Post Processing Functions#

Navigation: ← Tensor Format Conversions

It is a modular framework that works on both x_plus_ml_ort and x_plus_ml_vart applications irrespective of model for implementing post-process functions for neural network inference results. The framework provides a base class architecture that allows developers to easily add new post-processing algorithms while maintaining consistency and reusability across the VART-X library. The base class also handles tensor data conversion for INT8, FLOAT32, BF16, and FP16 tensors, so derived functions can operate on FLOAT32 buffers regardless of the model’s quantized data type.

For INT8 models, model-params.quant-scale-factors in postprocess-config can be omitted when the model was compiled with experiment_features: ["SkipDequantizeRemoval"]; see Quantization Parameters for VART Applications.

The framework currently includes the following implemented post-process functions, organized by use case (classification, object detection, and segmentation). These functions are tested with x_plus_ml_ort and x_plus_ml_vart applications.

Classification Functions#

Following are the classification functions that are implemented in the framework. These functions are tested with ResNet50, ResNet18, VGG16, VGG19, and EfficientNetB0 models.

Functions	Enum type	Description	Output Expected
Argmax	`ARGMAX`	Finds the index of the maximum value in a tensor.	Log file prints Top K logit values with labels. Output image overlays Top K labels of prediction.
Bias Correction	`BIAS_ CORRECTION`	Adjusts logits by subtracting a bias vector and applies Platt scaling.	Log file prints Top K probability values with labels. Output image overlays Top K labels of prediction.
Calibration (Platt)	`CALIBRATION _ PLATT`	Calibrates probabilities using Platt scaling.	Log file prints Top K probability values with labels. Output image overlays Top K labels of prediction.
Calibration (Temperature)	`CALIBRATION_ TEMPERATURE`	Calibrates probabilities using temperature scaling.	Log file prints Top K probability values with labels. Output image overlays Top K labels of prediction.
Label Mapping	`LABEL _ MAPPING`	Maps top-k class indices to human-readable labels.	Log file prints Top K logit values with labels. Output image overlays Top K labels of prediction.
Normalization	`NORMALIZATION`	Min-max normalizes logits to the [0, 1] range.	Log file prints Top K logit values with labels. Output image overlays Top K labels of prediction.
Outlier Detection	`OUTLIER_ DETECTION`	Detects outlier predictions based on Z-score.	Log file prints outlier logit values with labels higher than threshold. Output image overlays all labels.
SoftMax	`SOFTMAX`	Converts logits to a probability distribution.	Log file prints Top K probability values with labels. Output image overlays Top K labels of prediction.
Threshold	`THRESHOLD`	Filters predictions based on a value threshold.	Log file prints logit values with labels higher than threshold. Output image overlays all labels.
Top K	`TOPK`	Extracts the top K elements with their indices and values.	Log file prints Top K logit values with labels. Output image overlays Top K labels of prediction.
Uncertainty Estimation	`UNCERTAINTY_ ESTIMATION`	Computes entropy to estimate prediction uncertainty.	Log file prints entropy value (unusually high is an issue). Output image overlays no label.

Object Detection Functions#

The following object detection functions are implemented in the framework. With the exception of Anchor Adjustments, these functions are tested with the Yolov2, Yolov5lu, Yolov8x, Yolov9e, Yolov12s, and YOLOX models. Anchor Adjustments is tested with the Yolov2 model.

Functions	Enum type	Description	Output Expected
Anchor Adjustments	`ANCHOR_ADJUSTMENT`	Decodes bounding boxes using anchor boxes and grid position.	Log file only makes the boxes (no label or confidences). Image has anchor boxes on each detection. No label expected.
Classwise NMS	`CLASSWISE_ NMS`	NMS applied independently per class.	Log file prints detection box, label, and confidences. Image has detection box with labels.
Distance IOU NMS	`DISTANCE_ IOU_NMS`	NMS using the Distance IoU metric.	Log file prints detection box, label, and confidences. Image has detection box with labels.
NMS	`NMS`	Standard Non-Maximum Suppression.	Log file prints detection box, label, and confidences. Image has detection box with labels.
Object Count	`OBJECT_COUNT`	Detects objects with NMS and counts per class.	Log file prints detection box, label, and confidences. Image has detection box with labels.
Soft NMS	`SOFT_NMS`	Soft Non-Maximum Suppression (Gaussian score decay).	Log file prints detection box, label, and confidences. Image has detection box with labels.

Segmentation Functions#

The following segmentation functions are implemented in the framework. These functions are tested with the DeepLabV3+ ResNet50 model (deeplab3plus-resnet50). All segmentation functions expect a 4-D output tensor of shape [batch, height, width, num_classes] and produce a per-pixel class-index map (uint16_t) of length height * width for each batch element.

Functions	Enum type	Description	Output Expected
SoftmaxSeg	`SOFTMAXSEG`	Applies per-pixel softmax across class channels followed by per-pixel argmax to produce a single-channel segmentation map.	Log file prints the number of segmentation maps generated. Output image overlays a color-coded segmentation mask, with each pixel colored according to its predicted class id.
Sigmoid	`SIGMOIDSEG`	Applies per-class sigmoid (`1 / (1 + exp(-x))`) followed by per-pixel argmax to produce a single-channel segmentation map.	Log file prints the number of segmentation maps generated. Output image overlays a color-coded segmentation mask, with each pixel colored according to its predicted class id.
ArgmaxSeg	`ARGMAXSEG`	Performs per-pixel argmax directly on the raw logits (no probability transform) to produce a single-channel segmentation map.	Log file prints the number of segmentation maps generated. Output image overlays a color-coded segmentation mask, with each pixel colored according to its predicted class id.

How to run tests#

Each post-processing function described in detail in the preceding section can be run and tested in a similar fashion by creating a JSON configuration file for the function and running the application with the JSON file.

Example: SoftMax classification post-process#

The following example shows the JSON configuration for the SOFTMAX post-process function, used here with a classification model (ResNet50).

JSON fields#

Field	Type	Description	Example	Default
type	String	Must be set to “SOFTMAX” if testing for SoftMax post process.	“SOFTMAX”	(required)
topk	Integer	Number of top predictions to report.	1	`1`
label-file-path	String	Path to label file for mapping indices to labels.	“/etc/vai/models/resnet50_int8/data/imagenet-classes-1000.txt”	`""` (no labels loaded)
post-process-print	String/bool	Whether to print post-processing results to console.	“false”	`false`

JSON example#

{
  "postprocess-config": {
    "topk": 1,
    "label-file-path" : "/etc/vai/models/resnet50_int8/data/imagenet-classes-1000.txt",
    "type": "SOFTMAX",
    "post-process-print" : "false"
  }
}

Extra fields for other classification functions#

The fields shown in Example: SoftMax classification post-process (type, topk, label-file-path, and post-process-print) are common to every classification post-process. Only the functions in the following list require additional fields. Functions not listed here (ARGMAX, TOPK, LABEL_MAPPING, NORMALIZATION, UNCERTAINTY_ESTIMATION) work with just the common fields.

Function (`type`)	Extra field	Type	Description	Example	Default
`BIAS_ CORRECTION`	`function-options.platt-param-a`	Float	Platt scaling parameter `A` applied after bias correction.	`0.8`	`1.0`
`BIAS_ CORRECTION`	`function-options.platt-param-b`	Float	Platt scaling parameter `B` applied after bias correction.	`0.2`	`0.0`
`CALIBRATION_ PLATT`	`function-options.platt-param-a`	Float	Platt scaling parameter `A`.	`0.8`	`1.0`
`CALIBRATION_ PLATT`	`function-options.platt-param-b`	Float	Platt scaling parameter `B`.	`0.2`	`0.0`
`CALIBRATION_ TEMPERATURE`	`function-options.temperature-scaling`	Integer	Temperature divisor applied to logits before softmax.	`2`	`1`
`THRESHOLD`	`function-options.threshold`	Float	Minimum value (inclusive) for a class to be reported.	`5.0`	`4.0`
`OUTLIER_ DETECTION`	`outlier-threshold`	Float	Z-score threshold above which a class is flagged as an outlier.	`3.0`	`3.0`

Note

In the preceding table, function-options.<param-name> denotes a field that lives inside a nested function-options JSON object. In the JSON configuration file it is written as:

"function-options": {
  "<param-name>": <value>
}

Example: CALIBRATION_TEMPERATURE config with the extra field:

{
  "postprocess-config": {
    "type": "CALIBRATION_TEMPERATURE",
    "topk": 1,
    "label-file-path": "/etc/vai/models/resnet50_int8/data/imagenet-classes-1000.txt",
    "post-process-print": "false",
    "function-options": {
      "temperature-scaling": 2
    }
  }
}

Example: NMS object detection post-process#

The following example shows the JSON configuration for the NMS post-process function, used here with an object detection model (Yolov2). The same skeleton applies to CLASSWISE_NMS, DISTANCE_IOU_NMS, SOFT_NMS, and OBJECT_COUNT (with the type field updated accordingly).

JSON fields#

Field	Type	Description	Example	Default
type	String	Must be set to `"NMS"` (or the corresponding detection function name).	`"NMS"`	(required)
topk	Integer	Number of top detections to keep.	1	`1`
label-file-path	String	Path to the label file used to map class indices to labels.	`"/etc/vai/models/yolox_m_int8/data/coco_data_labels.txt"`	`""` (no labels loaded)
nms-threshold	Float	IoU threshold used by Non-Maximum Suppression.	0.4	`0.4`
conf-threshold	Float	Confidence threshold for filtering detections.	0.4	`0.5`
post-process-print	String/bool	Whether to print post-processing results to the console.	`"false"`	`false`
model-params.is-obj-score-included	String/bool	Whether an objectness score is included in the model output.	`"true"`	`false`
model-params.grid-shape	Array of arrays	Grid shape `[H, W]` for each output scale.	`[[13, 13]]`	(required)
model-params.anchors	Array of arrays	Anchor box dimensions for each output scale.	`[[0.57, 0.67, 1.87, 2.06, 3.33, 5.47, 7.88, 3.52, 9.77, 9.16]]`	`[]` (no anchors)
model-params.input-layout	Array of strings	Layout of each input tensor (`NCHW` / `NHWC` / `HCWNC4`).	`["NCHW"]`	(required)
model-params.output-layout	Array of strings	Layout of each output tensor (`NCHW` / `NHWC` / `NCH` / `NHC`).	`["NCHW"]`	(required)
model-params.is-scaling-required	String/bool	Whether bbox coordinates need to be scaled to the input image size.	`"true"`	`false`
model-params.apply-sigmoid-to-obj-conf	String/bool	Whether to apply sigmoid to objectness/class scores during decoding.	`"false"`	`false`
model-params.num-anchorboxes	Array of integers	Number of anchor boxes for each output scale.	`[5]`	`[]` (no anchor boxes)

JSON example#

{
  "postprocess-config": {
    "type": "NMS",
    "topk": 100,
    "label-file-path": "/etc/vai/models/yolox_m_int8/data/coco_data_labels.txt",
    "nms-threshold": 0.4,
    "conf-threshold": 0.4,
    "post-process-print": "false",
    "model-params": {
      "is-obj-score-included": "true",
      "grid-shape": [[13, 13]],
      "anchors": [[0.57, 0.67, 1.87, 2.06, 3.33, 5.47, 7.88, 3.52, 9.77, 9.16]],
      "input-layout": ["NCHW"],
      "output-layout": ["NCHW"],
      "is-scaling-required": "true",
      "apply-sigmoid-to-obj-conf": "false",
      "num-anchorboxes": [5]
    }
  }
}

Example: SoftmaxSeg segmentation post-process#

The following example shows the JSON configuration for the SOFTMAXSEG post-process function, used here with a segmentation model (DeepLabV3+ ResNet50). The same skeleton applies to SIGMOIDSEG and ARGMAXSEG (with the type field updated accordingly).

JSON fields#

Field	Type	Description	Example	Default
type	String	Must be set to `"SOFTMAXSEG"` (or `"SIGMOIDSEG"` / `"ARGMAXSEG"`).	`"SOFTMAXSEG"`	(required)
topk	Integer	Parsed for parity with classification configs; not used by the core segmentation-map generation.	5	`1`
label-file-path	String	Path to the label file used to map class indices to labels. Set this to `<path_to_segmentation_labels>/segmentation_labels.txt` for your deployed segmentation model.	`"<path_to_segmentation_labels>/segmentation_labels.txt"`	`""` (no labels loaded)
post-process-print	String/bool	Whether to print post-processing results to the console.	`"false"`	`false`

JSON example#

{
  "postprocess-config": {
    "type": "SOFTMAXSEG",
    "topk": 5,
    "label-file-path": "<path_to_segmentation_labels>/segmentation_labels.txt",
    "post-process-print": "false"
  }
}

Testing with x_plus_ml_ort application#

The following example demonstrates how to test the SoftMax post-process function using resnet50_softmax.json from ../cpp_examples/x_plus_ml_ort/json_configs. Update the top-level JSON file x_plus_ml_ort.json (for example under /etc/vai/x_plus_ml_ort/json_configs on the board) to reference this config. The output image postproc0_overlay.jpg is generated in the application’s output folder.

Command line#

x_plus_ml_ort --input-file <input_clip> --app-config <top_level_json_file> --log-level <info_level> --dump-all

#Example
x_plus_ml_ort --input-file /etc/vai/models/data/classification.jpg --app-config /etc/vai/x_plus_ml_ort/json_configs/x_plus_ml_ort.json --log-level 3

Output Snippet#

Running for 1 iterations
Pipeline 0 - Model supports batch size: 1
Pipeline 0 - Available images in input: ALL (processing until EOF)
[RESULT] vart_context.cpp:408  ==== Post Process : Classification (SOFTMAX) =====
[RESULT] /usr/src/debug/cpp-examples/1.0/common/src/post_process.cpp:257  Results for Pipeline 0 :
[RESULT] /usr/src/debug/cpp-examples/1.0/common/src/post_process.cpp:260  Post processing batch 0 frame number: 1
[RESULT] /usr/src/debug/cpp-examples/1.0/common/src/post_process.cpp:268  Classification Label : brain coral (confidence 0.990228)
[RESULT] /usr/src/debug/cpp-examples/1.0/common/src/post_process.cpp:268  Classification Label : coral reef (confidence 0.006672)
[RESULT] /usr/src/debug/cpp-examples/1.0/common/src/post_process.cpp:268  Classification Label : electric ray, crampfish, numbfish, torpedo (confidence 0.001159)
[RESULT] /usr/src/debug/cpp-examples/1.0/common/src/post_process.cpp:268  Classification Label : puffer, pufferfish, blowfish, globefish (confidence 0.000548)
[RESULT] /usr/src/debug/cpp-examples/1.0/common/src/post_process.cpp:268  Classification Label : eel (confidence 0.000332)
[RESULT] /usr/src/debug/cpp-examples/1.0/common/src/post_process.cpp:286  ===========================

Total number of samples processed on all pipelines: 1
Total number of samples processed on per pipeline: 1
Pipeline 0 outputs dumped with 320x256 resolution and JPEG format

Testing with x_plus_ml_vart application#

The following example demonstrates how to test the SoftMax post-process function using resnet50_float_softmax.json from ../cpp_examples/x_plus_ml_vart/json_configs. Update the top-level JSON file x_plus_ml_vart_1model.json (under /etc/vai/x_plus_ml_vart/json_configs on the board) to reference this config. The output image postproc0_overlay.jpg is generated in the application’s output folder.

Command line#

x_plus_ml_vart --input-file <input_clip> --app-config <top_level_json_file> --log-level <info_level> --dump-all

#Example
x_plus_ml_vart --input-file /etc/vai/models/data/classification.jpg --app-config /etc/vai/x_plus_ml_vart/json_configs/x_plus_ml_vart_1model.json --log-level 3

Output Snippet#

Inference0 runner init time: 149.612 ms
[RESULT] postprocess.cpp:1165  ==== Post Process : Classification (SOFTMAX) =====
[RESULT] postprocess.cpp:591   Frame: 0
[RESULT] postprocess.cpp:602   Classification Label : brain coral (confidence 0.989553)
[RESULT] postprocess.cpp:602   Classification Label : coral reef (confidence 0.006668)
[RESULT] postprocess.cpp:602   Classification Label : electric ray, crampfish, numbfish, torpedo (confidence 0.001488)
[RESULT] postprocess.cpp:602   Classification Label : puffer, pufferfish, blowfish, globefish (confidence 0.000547)
[RESULT] postprocess.cpp:602   Classification Label : eel (confidence 0.000426)

---------------------------------------------------------------------------------------
Total number of frames processed: 1
Model [/etc/vai/models/resnet50_int8/resnet50_int8.rai] with device batch size 1 processed 1 frames
---------------------------------------------------------------------------------------

Navigation: ← Tensor Format Conversions

Post Processing Functions

Contents

Post Processing Functions#

Classification Functions#

Object Detection Functions#

Segmentation Functions#

How to run tests#

Example: SoftMax classification post-process#

JSON fields#

JSON example#

Extra fields for other classification functions#

Example: NMS object detection post-process#

JSON fields#

JSON example#

Example: SoftmaxSeg segmentation post-process#

JSON fields#

JSON example#

Testing with x_plus_ml_ort application#

Command line#

Output Snippet#

Testing with x_plus_ml_vart application#

Command line#

Output Snippet#