Corvus ISR Publishes Synthetic Benchmark Showing Tracker Performance Improvements

AIThis post was created with the assistance of artificial intelligence (AI).

Corvus ISR tracker benchmark matrix (seed 1337) — The published matrix — every row reproducible. Source: corvusisr.com/benchmark

Corvus ISR, a leader in wide-area motion imagery (WAMI) exploitation, has unveiled its latest public tracker benchmark comparing two distinct models on a synthetic scene with perfect ground truth. This decision underscores the importance of transparency in evaluating tracking algorithms, especially when differences are measured against an idealized, synthetic environment where all variables are controlled and known. The benchmark, available here, uses a fixed seed (1337) to ensure reproducibility, with identical sensor models, detection generation, and metric definitions across models, allowing fair and precise comparisons.

The two models under test are designed with different strategies: the v1 “greedy nearest-neighbour” employs a two-pass association with constant-velocity prediction and fixed 2s coasting, representing a deliberately simple baseline. The v2 “confirmed-track auction” introduces more sophisticated features such as track confirmation, three-tier auction association, velocity-consistency gating, and noise-scaled reservation prices. These enhancements aim to improve tracking accuracy, especially under challenging conditions.

Results from the benchmark reveal significant improvements: for a scene with 150 movers, ID switches per minute decreased from 2,042 to 1,183, a reduction of 42.1%. In a denser scene with 400 movers, the number dropped from 14,032 to 8,040, a 42.7% decrease. The models were also tested under degraded conditions—such as lower frame rates and occlusions—and still showed substantial reductions in identity errors, demonstrating robustness. The detection rate remains identical for both models, as it is determined solely by the sensor properties, emphasizing that these metrics isolate tracker performance.

Why publish failure numbers? In synthetic scenes with perfect ground truth, these metrics provide an honest appraisal of algorithm performance. While many vendors highlight successes, Corvus ISR’s transparency in reporting thousands of identity errors per minute under stress conditions offers a more meaningful measure of real-world limitations. Such data—accessible and reproducible—encourages the development of more reliable tracking systems.

From an engineering perspective, the v2 model operates efficiently, averaging around 1.2ms per sensor tick at a scene density of 400, with the worst case around 5ms—well within real-time constraints for browser-based testing. The entire benchmark can be reproduced live with no sign-up or NDA required. This openness is made possible because v2 was built by an AI executor, evaluated against a written acceptance contract, and independently reviewed before deployment.

All results are based on a fully synthetic environment, where every pixel is generated, and no real persons, vehicles, or locations are involved. This methodology allows for precise ground truth and repeatability, crucial for benchmarking complex tracking algorithms. By publishing these detailed failure metrics, Corvus ISR demonstrates that transparency and rigorous testing are vital for advancing the state of the art in motion tracking technology.

Science-minded readers interested in the details of this benchmarking process can explore the public benchmark and reproduce it live. This approach underscores the importance of empirical measurement over marketing claims, especially when evaluating the true capabilities of synthetic tracking models. Feel free to run the benchmark yourself and see how your algorithms stack up against these standards.