Need for Speed Dataset

I feel the need...the need for speed!

We propose the first higher frame rate video dataset (called Need for Speed - NfS) and benchmark for visual object tracking. The dataset consists of 100 videos (380K frames) captured with now commonly available higher frame rate (240 FPS) cameras from real world scenarios. All frames are annotated with axis aligned bounding boxes and all sequences are manually labelled with nine visual attributes - such as occlusion, fast motion, background clutter, etc. Our benchmark provides an extensive evaluation of many recent and state-of-the-art trackers on higher frame rate sequences. We ranked each of these trackers according to their tracking accuracy and real-time performance. One of our surprising conclusions is that at higher frame rates, simple trackers such as correlation filters outperform complex methods based on deep networks. This suggests that for practical applications (such as in robotics or embedded vision), one needs to carefully tradeoff bandwidth constraints associated with higher frame rate acquisition, computational costs of real-time analysis, and the required application accuracy. Our dataset and benchmark allows for the first time (to our knowledge) systematic exploration of such issues.


To download the dataset, *nix users can run the following command:

curl -fSsl | bash -

The MD5 hashes of each file can be verified here.

Video previews are available on the embedded YouTube playlist.

About the Dataset

We provide each video sequence as a seperate .zip file, each containing bounding box annotations and frames (as JPEG files) for the two scenarios described in our paper: 240 FPS capture, and 30 FPS with synthesized motion blur (generated with Adobe After Effects).

The annotation files are plain text, and were generated with Vatic.

We also make available all results from the trackers evaluated in our paper. (MD5)

The full tracking suite used to generate our results, and interface with our dataset is also available here. Each individual tracker is the property of the respective authors, and we make no claim of ownership to these works. By making this tracking suite available, we allow our results to be replicated by interested parties. [COMING SOON]


The following publications are associated with our dataset.

H. Kiani Galoogahi, A. Fagg, C. Huang, D. Ramanan, S.Lucey. Need for Speed: A Benchmark for Higher Frame Rate Object Tracking, 2017, arXiv preprint arXiv:1703.05884 - [PDF]

If you use the NfS dataset, we ask that you cite us.


For more information or help please email

Hamed Kiani
Ashton Fagg
Chen Huang
Deva Ramanan
Simon Lucey

Copyright © 2017, Carnegie Mellon University