pranksegmentationalgorithmsfromfolders

Synopsis

Description

The operator pranksegmentationalgorithmsfromfolders ranks several segmentation algorithms according to their performances. Performances are evaluated by discrepancy measures between segmentation results and ground truths.

Performance is calculated from 10 discrepancy measures between segmentations and the corresponding ground truth. Five discrepancy indicators are evaluated and for each indicator two measures are calculated with value between 0 and 1:

• Indicator 1: Detection accuracy. The two measured errors are:
  • Recall error accounts for proportion of false negative regions.
  • Precision error accounts for proportion of false positive regions.
• Indicator 2: Fragmentation consistency. The two measured errors are:
  • Under-segmentation error accounts for proportion of regions grouped into a single segment.
  • Over-segmentation error accounts for proportion of segments grouped into a single region.
• Indicator 3: boundary precision. The two measured errors are:
  • Pixel deficit error accounts for proportion of missed pixels in correctly detected regions.
  • Pixel excess error accounts for proportion of false pixels added to correctly detected region.
• Indicator 4: Shape fidelity. The two measured errors are:
  • Omission shape error accounts for distance of deleted shape to true shape.
  • Commission error accounts for distance of added shape to true shape.
• Indicator 5: Topology preservation. The two measured errors are:
  • Hole addition error accounts for the proportion of false holes.
  • Hole detection error accounts for the proportion of missed holes.

For each indicator i, the most acceptable error between the two should be specified by the way of the parameter acceptable_error_i. (See section "Parameters")

The results are stored in two collections. The first collection col_out1 contains the value of each performance indicator for each algorithm. The second collection col_out2 contains the rank of each algorithm.

Parameters

• -v : verbose mode
• matching_algorithm_id: the index of the matching algorithm to be used:
  • 0: for 1-n and n-1 matching. A segment in the segmentation result can merge several regions in the reference segmentation (under-segmentation), and a region in the reference segmentation can be split into several segments in the segmentation result (over-segmentation). However, a segment or a region cannot be both part of over-segmentation and under-segmentation.
  • 1: for 1-1 matching. A segment can only detect one region at the most, and a region can only be detected by one segment at the most.
• matching_threshold: minimum proportion of overlapping area between a region and a segment to accept a detection. The value is in the interval [0,1] where value x corresponds to (x*100)% of overlapping.
• acceptable_error1, acceptable_error2, acceptable_error3, acceptable_error4, acceptable_error5: the value indicates the preferences :
  1. Both errors are acceptable (no penalty).
  2. Both errors are undesirable.
  3. Prefer error1 to error2.
  4. Prefer error2 to error1.
  5. Do not penalize error1.
  6. Do not penalize error2.
  7. Prohibit error1.
  8. Prohibit error2.
• priority1, priority2, priority3, priority4, priority5: The relative priority of the related indicator. It is an integer between 1 and 5, where ties are possible. The less the priority value, the higher the importance.
• segmentation_result_path: path to segmentation results folder. This folder can be organized into subfolders.
• reference_segmentation_path: path to reference segmentation folder. The folder is divided in as much as subfolders as they are expertises. Each subfolder is organized in the same way as each subfolder of segmentation_result_path.

Outputs

• col_out1: a collection with indicator values of each algorithm.
• col_out2: a collection with the rank of each algorithm.

Result

Returns SUCCESS or FAILURE.

Examples

Rank all the algorithms from their segmentation results stored in subfolders of folder 'resultimages'. Priority between indicators is: first detection accuracy, second fragmentation consistency, third shape fidelity, fourth boundary precision and finally fith topology preservation:

pranksegmentationalgorithmsfromfolders 0 0.5 4 1 5 2 4 4 2 3 1 5 resultimages groundtruths indicators.pan rank.pan

See also

Prototype C++