Welcome to the FastTrack user manual. This manual will present the tracking software and how to use it. Please contact by email at benjamin.gallois@fasttrack.sh if you need more information or to signal a bug. Subscribe to the mailinglist to get the FastTrack dev team’s last updates.
FastTrack is a cross-platform application designed to track multiple objects in video recording. Stable versions of the software are available for Linux, Mac, and Windows. The source code can be downloaded at https://github.com/FastTrackOrg/FastTrack.
Two main features are implemented in the software:
The FastTrack user interface is implemented with Qt, the image analysis with the OpenCV library. This allows a performant and responsive software amenable to process large video recording.
FastTrack was first a Ph.D. thesis side project started by Benjamin Gallois in his spare time that has then taken dedicated time in his Ph.D. project. The software’s core is still maintained in his spare time; therefore, new features implementation, bug fixes, and help can take some time.
Not sure if you want to use FastTrack? Check these five most common questions:
What video quality is required? FastTrack is designed to work with any video quality and frame rate.
What type of objects and numbers can FastTrack handled?
How it performs? Tracking performances depend on systems (number and type of objects). However, with the built-in ergonomic tool, it is possible to achieve 100% tracking accuracy with minimal effort.
It is free? FastTrack is a free software under the GPL3 license.
Do I need programming skills? No.
| NOTE |
| During the installation on Windows and Mac systems, security alerts are displayed because the FastTrack executable does not possess an EV code signing certificate. These alerts can be ignored. FastTrack executable can be verified easily (and freely) by comparing the MD5 checksum. |
Stable versions of FastTrack are released for Linux, Mac (as dmg), and Windows (installer). The nightly version is available on the Github Actions artifacts.
sudo dnf install fasttrack fasttrack-cliyay -S fasttrack fasttrack-cliqmake src/FastTrack.pro ; make ; sudo make install ; qmake src/FastTrack-Cli.pro ; make ; sudo make installFastTrack will display a message at the start-up when a new release is available.
For Windows:
Search the FastTrackUpdater in the Windows Start Menu or execute the MaintenanceTool.exe in the installation folder directly and follow the provided instructions.
For Linux:
The FastTrack AppImage does not currently support the automatic update. Replace the current AppImage with the latest AppImage released.
For Mac:
The FastTrack App does not currently support the automatic update. Replace the current App with the latest App released.
This section details step by step the process to test FastTrack using data from the Two-Dimensional Tracking Dataset. Illustrations originate from v4.x.y and can differ from the current stable version.
In this example, we will use the challenging movie ZFJ_001. This movie features 14 very active juvenile zebrafish. The principal difficulty of this movie is the frequent and complex occlusions. First, download the zip file and extract it. The image sequence is located inside the images folder.
Note: these parameters were found by trials and errors.
When the tracking is completed, FastTrack automatically opens the Tracking Inspector which allows the user to review and correct the tracking.
With the previous tracking analysis, only seven corrections and six deletions are necessary to achieve a perfect tracking accuracy.
| image | id | delete | swap with |
|---|---|---|---|
| 17-20 | 1-0 | x | |
| 17-20 | 0 | x | |
| 22 | 0 | 1 | |
| 23 | 1 | 4 | |
| 27-28 | 1 | x | |
| 63 | 1 | 8 | |
| 97 | 6 | 12 | |
| 50-57 | 1 | x | |
| 109-121 | 0 | x | |
| 114 | 9 | 13 | |
| 116-118 | 0 | x | |
| 122 | 0 | 13 | |
| 124 | 4 | 12 |
FastTrack is able to open image sequences if they follow the leading 0 naming convention (name000.xyz, name001.xyz, name002.xyz, etc…). .bmp, .dib, .jpeg, .jpg, .jpe, .jp2, .png, .pbm, .pgm, .ppm, .sr, .ras, .tiff, .tif formats are supported. To open an image sequence, select the first image of the sequence.
FastTrack is able to open video files, and a lot of codecs are supported.
To navigate inside a video rapidly, FastTrack provides a tool called the timeline. Hover the mouse cursor above the timeline to move across the video. Right-click to place the cursor at a given position. This will save this position when the cursor exit the timeline. Double left-click to place a marker, right-click on this marker to delete it. Keyboard shortcuts are available to move the cursor frame by frame:
The Interactive panel provides a means to perform a tracking analysis and review it in an interactive environment. Several steps have to be performed in the right order (some are mandatory, some are optional) to perform a successful tracking analysis. 
The first step of a tracking analysis is to open a video file. FastTrack supports video files and image sequences. Click on the file or on an image of a sequence to automatically load the movie. 
The background can be computed or imported. To compute the background, select a method and a number of images. Images are selected in the image sequence at regular intervals, and three methods of computation by z-projection are available:
The images can be registered before the z-projection. Three methods of registration are available. 
To select a region of interest, draw on the display a rectangle with the mouse and click on the Crop button. Cancel the crop by clicking on the reset button. 
To compute the binary image from the background image and the image sequence, select the threshold value, and see the result on the display. The background type is automatically selected after the background computation. However, it can be modified: select Dark Background if the objects are light on a dark background, and Light background if the objects are dark on a light background. 
It is possible to apply a morphological operation on the binary image. Select a morphological operation, kernel size, and geometry. See the result on the display. For more information about the different operations, see https://docs.opencv.org/trunk/d9/d61/tutorial_py_morphological_ops.html. 
Objects are detected by their size. Select the maximum and minimum size of the detected objects. The detected objects will be colored in green in the display, and the rejected object will be displayed in red. 
Several parameters can be modified to ensure a good tracking analysis. See this page for more details:
Hard parameters have to be set manually by the user:
The soft parameters can be leveled automatically by clicking on the Level button. This will automatically compute the soft parameters as each contribution weighs one quarter of the total cost. It has to be manually tuned by the user to find the optimal soft parameters with the system’s knowledge. For example, for a system where the objects’ direction is not relevant, the user will select the Normalization angle equal to 0.
The image registration is the process to correct small displacements and rotation of the camera during the movie recording. FastTrack provides several methods for registering the movie:
Image registration is very computationally intensive and can drastically decrease the speed of the program.
The tracking can be previewed on a sub-sequence of images. It can be useful to tune parameters if the tracking is slow.
Several display options are available and unlocked at each step of the analysis.
Several layouts and themes are available in the layout menu in the top bar. The user can also build his layout by dragging the option docks in the window.
The Batch Tracking panel is an advanced tool to track a large number of movies automatically. Several behaviors can be combined to load image sequences in a batch with specific background images or parameter files.
The user can open several image sequences by clicking on the Open folder (1) button and select one or several folders. FastTrack can automatically load a background and/or a parameters file if a Tracking_Result folder is provided with the image sequence; check the Autoload (10) tick to activate this behavior. After opening, image sequences are added to the Processing stack (4). If a background image and/or a set of parameters are automatically loaded, the path will be displayed in the second and third columns. If not, the user can select them with the (5) and (6) buttons after importation. By default, if no background image and parameter file are selected, FastTrack will use the parameters provided in the Parameters table (9) before the image sequence importation. The user can delete an image sequence by selecting the corresponding line in the Processing stack (4) and click on the Remove (14) button. The user can clear all the Processing stack (14) by clicking the Clear (13) button. To process the stack, click the Start Tracking (12) button.
The user can append a suffix to the imported folders folder_path/ + suffix/ For example, it can be usefull with a folder tree like this one:
The user can easily select in one time the folders:
And then add the suffix images/ to select the desired folders without having to do it manually three times.
The user can select a unique background image. Open an image with the Unique background (2) button, and all the sequences in the stack and sequences that will be imported will be using this background image. The user can use the Clear (12) to reset the default behavior.
To apply the same parameters file to all the imported sequences:
Manual selection:
With a file:
With a file:
The Tracking Inspector is a tool to display the result of a tracking analysis and to correct the tracking manually if necessary. For example, the user can delete an object to remove an artifact or change the object ID to correct a tracking error. To make the user’s life easier, an ergonomic interface with build-in keyboard shortcuts are provided. FastTrack alleviates the tedious work of review and correction, and the user can achieve 100% tracking accuracy rapidly and efficiently.
To load a tracking analysis previously tracked in FastTrack, first click on the Open button (1) and select a movie or an image of an image sequence. If the movie was tracked several times, the last tracking analysis is stored in the Tracking_Result folder and the previous tracking analysis in the Tracking_Result_Date folders and can be loaded using the Open Tracking_result directory button (2) (can only be activated if a movie is loaded). Click on the Reload) button (3) to reload the tracking data if necessary. The software can only load a tracking analysis if the folder architecture is preserved, .ie the folder with the image sequence has to have a sub-folder named Tracking_Result containing at least the tracking.txt file.
Several tracking overlay options are available on the tracking overlay panel (24):
Several useful information on the selected object can be found in the information table (18). The user can go to the image where the object has appeared for the first time by clicking directly on the table’s corresponding cell.
The tracking can be inspected by moving the display cursor (19), see the image number (20), and automatically play the movie (21) at a selected frame rate (22). Automatically detected occlusions (overlapped objects) can be reviewed by clicking on the Previous (12) and Next (13) occlusion buttons (this function is experimental and can miss some occlusions).
The user can annotate any image of the tracking. Write the annotation in the annotate text entry (17). The user can search across annotations with the find bar (14) and the buttons (15)(16). All the annotations are saved in the annotation.txt file in the Tracking_Result folder.
The user can correct an error by swapping two object’s ID from the current image to the end of the sequence as follow:
To delete one object of several frames:
To delete one object on the current frame:
A set of keyboard shortcuts are available to speed-up the tracking correction.
All the changes made in the inspector are automatically saved in the original tracking.txt file.
To export a movie of a tracking analysis, select the desired display overlay and click on the Save button (3). Select a folder and a name to save the file. Only .avi format is supported.
Note: Movie with many objects by frame can be challenging to load and review in the tracking Inspector.
A command-line interface is available for macOS, Linux, and by using WSL for Windows. It can be downloaded on the release page.
The full list of parameters can be found by calling ./fasttrack-cli --help. Parameters can be declared individually by calling ./fasttrack-cli --path path/to/movie.webm --parameter1 value --parameter2 value or in batch with a parameters file ./fasttrack-cli --path path/to/movie.webm --cfg path/cfg.toml. Note that the path option need to be the first option.
fasttrack-cli does not support Windows natively. The workaround is to use WSL. * Install WSL https://docs.microsoft.com/en-us/windows/wsl/install-win10. * Install FastTrack in a Linux terminal:
wget https://github.com/FastTrackOrg/FastTrack/releases/download/continuous_cli/fasttrack-cli-x86_64.AppImage
chmod +x fasttrack-cli-x86_64.AppImage
./fasttrack-cli-x86_64.AppImage --appimage-extract
sudo ln -s ~/squashfs-root/usr/bin/fasttrack-cli /usr/local/bin/FastTrack can be called inside a Python script to automate the tracking.
import os
cmd = "./fasttrack-cli --maxArea 500 --minArea 50 --lightBack 0 --thresh 80 --reg 0 --spot 0 --nBack 50 --regBack 0 --methBack 1 --xTop 0 --yTop 0 --xBottom 0 --yBottom 0 --morph 0 --morphSize 0 --morphType 0 --normArea 0 --normPerim 0 --normDist 50 --normAngle 45 --maxDist 150 --maxTime 10 --path ZFJ_001.avi --backPath dataSet/images/Groundtruth/Tracking_Result/background.pgm "
os.system(cmd)import os
cmd = "./fasttrack-cli --path ZFJ_001.avi --cfg Tracking_Result_ZFJ_001/cfg.toml"
os.system(cmd)This section details how to select the relevant tracking features to be included in the cost function and how to tune them.
FastTrack uses the so-called Hungarian method to solve the assignment problem of each object between two frames. This method is based on minimizing the global cost of the association pairs of objects.
The cost is calculated from a cost function that can be constructed from several parameters, in the following, i is indexing the image n, and j the image n + 1: * The distance \(d{ij} = \sqrt{(x_i - x_j)^2 + (y_i - y_j)^2}\) * The angle \(a_{ij} = min(\theta_i - \theta_j)\) * The area \(ar_{ij} = abs(area_i - area_j)\) * The perimeter \(p_{ij} = abs(perimeter_i - perimeter_j)\)
The relative weight of these contributions to the cost function are set by 4 normalization parameters: \[ c_{ij} = \frac{d{ij}}{D} + \frac{a{ij}}{A}+ \frac{ar{ij}}{AR} + \frac{p{ij}}{P}\] These parameters can be set to 0 to cancel one or several tracking feature from the cost computation. All these features are not always relevant and has to be chosen carrefully for the best tracking accuracy. For example, for tracking circles of radius r, and squares of the same area moving at 10px/image, it is best to set \[(D=10, A=0, AR=0, P=2r(\pi-2\sqrt{\pi}))\]. For tracking fish of the same size, travelling at 35px/image, doing small reorientation of 20°, it is best to set \[(D=35, A=20, AR=0, P=0))\]. For tracking fish of different size, travelling at 35px/image, doing small reorientation of 20°, with size difference of 100px it is best to set \[(D=35, A=20, AR=100, P=0))\].
The best way to set the parameter is to first set the normalization parameters to the mean of the variable, .ie the typical change between two consecutive images: * \(D = mean(d_{ij})\) where i and j are the same object. * \(A = mean(a_{ij})\) where i and j are the same object. * \(AR = mean(ar_{ij})\) where i and j are the same object. * \(P = mean(p_{ij})\) where i and j are the same object. In this case, each tracking feature has the same contribution to the cost. To tune the cost function by weighting more (resp. less) a tracking feature, decrease (resp. increase) the normalization parameter of this feature, or increase (resp. decrease) all the normalization parameters of the others.
A parameter of memory named maximal time can be set to account for disappearing objects. If the maximal time is set to m, one object can only disappear during m image. If it reappears after, it will be considered as a new object.
To speed-up the tracking, the maximal distance (L) parameter sets an infinite cost for all the pairs of objects to such as \(d_{ij} > L\). In practice, L is corresponding to the maximal distance an object can disappear.
The spot to track will determine if the distance and the angular difference will be calculated from the head, the tail, or the body of the object. Area and perimeter are always computed from the body. Head is defined as the bigger half of the object, separated alongside the object’s minor axis.
Setting the tracking parameters can be tedious. It can be best achieved by trials and errors (see the Preview option in the Interactive panel). to summarize: 1. Choose the right tracking features. 2. Set the normalization parameters equal to the tracking feature’s std, ie, the typical value change. 3. Tune the normalization parameters to increase or decrease the relative weight between each contribution.
They are several tricks that can be used to increase tracking accuracy and select the optimal parameters.
The detection parameters reject objects that are smaller or bigger than a specific size. To increase the tracking accuracy, we want to reject noise and artifacts and reject blobs constitute of more of one object. If all the objects are of similar size, these two parameters can be selected easily in four steps:
Tracking parameters are mostly found by trials and errors. However, some rules of thumbs can be applied.
Spot to track has to be set to Body for quasi-symmetric objects and low-resolution objects. For deformable objects with enough resolution, select Head or Tail according to the part that predicts best the object’s traveling direction.
For each tracking analysis, FastTrack will save the parameters used in cfg.toml that can be reloaded in the software or in fasttrack_cli. Before FastTrack version 5.2.1, the software used to saved the parameters in parameter.param, these files can be converted as following (left: old file, right: new file):
title = "FastTrack cfg""
[parameters]
Light background = 0 lightBack = 0
Maximal size = 170 maxArea = 170
Maximal occlusion = 200 maxDist = 200
Maximal time = 100 maxTime = 100
Background method = 1 methBack = 1
Minimal size = 50 minArea = 50
Morphological operation = 8 morph = 8
Kernel type = 2 morphSize = 2
Kernel size = 0 morphType = 0
Number of images background = 20 nBack = 20
Maximal angle = 90 normAngle = 90
Binary threshold = 60 normArea = 0
Normalization area = 0 normDist = 100
Maximal length = 100 normPerim = 0
Normalization perimeter = 0 reg = 0
Registration = 0 regBack = 0
Background registration method = 0 spot = 0
Spot to track = 0 thresh = 60
ROI bottom x = 0 xBottom = 0
ROI top x = 0 xTop = 0
ROI bottom y = 0 yBottom = 0
ROI top y = 0 yTop = 0After a tracking analysis (or an analysis preview), FastTrack saves several files inside the Tracking_Result folder located inside the image sequence folder or inside the Tracking_Result_VideoFileName for a video file:
The tracking result file is simply a text file with 20 columns separated by a ’ character. This file can easily be loaded to subsequent analysis see this Python and this Julia.
| xHead | yHead | tHead | xTail | yTail | tTail | xBody | yBody | tBody | |
|---|---|---|---|---|---|---|---|---|---|
| Float64 | Float64 | Float64 | Float64 | Float64 | Float64 | Float64 | Float64 | Float64 | |
| 1 | 514.327 | 333.12 | 5.81619 | 499.96 | 327.727 | 6.10226 | 508.345 | 330.876 | 5.94395 |
| 2 | 463.603 | 327.051 | 0.301279 | 449.585 | 330.323 | 0.245547 | 458.058 | 328.346 | 0.238877 |
| 3 | 23.9978 | 287.715 | 3.70646 | 34.9722 | 278.836 | 3.99819 | 29.2056 | 283.505 | 3.84844 |
| 4 | 372.536 | 230.143 | 0.194641 | 354.226 | 231.604 | 6.08737 | 364.822 | 230.759 | 0.0515087 |
| 5 | 480.58 | 213.482 | 1.28236 | 478.125 | 228.52 | 1.53303 | 479.428 | 220.543 | 1.42567 |
| 6 | 171.682 | 143.55 | 6.09077 | 155.507 | 140.116 | 6.1146 | 164.913 | 142.113 | 6.08216 |
| 7 | 498.151 | 121.32 | 6.00177 | 483.712 | 119.285 | 0.0223247 | 492.683 | 120.55 | 6.15298 |
| 8 | 329.56 | 123.418 | 6.08726 | 312.526 | 119.042 | 5.9098 | 322.531 | 121.614 | 6.01722 |
| 9 | 465.256 | 115.045 | 4.44359 | 470.057 | 99.911 | 4.40559 | 467.106 | 109.205 | 4.40862 |
| 10 | 423.663 | 66.3789 | 0.0888056 | 409.105 | 67.2971 | 6.12053 | 417.615 | 66.7623 | 0.0292602 |
| 11 | 424.487 | 40.4232 | 5.48198 | 411.594 | 30.3912 | 5.88869 | 418.96 | 36.1192 | 5.64923 |
| 12 | 370.591 | 35.2147 | 5.99688 | 354.672 | 29.5633 | 5.89121 | 364.007 | 32.8767 | 5.94008 |
| 13 | 498.502 | 20.2527 | 5.66339 | 487.254 | 9.19499 | 5.39497 | 493.758 | 15.5781 | 5.5026 |
| 14 | 367.791 | 5.03034 | 6.05933 | 352.076 | 6.75603 | 0.653641 | 361.12 | 5.75904 | 0.152688 |
| 15 | 512.965 | 332.575 | 5.86617 | 499.435 | 327.759 | 6.052 | 507.626 | 330.673 | 5.95102 |
| 16 | 463.385 | 324.659 | 0.707 | 451.431 | 332.193 | 0.246265 | 458.959 | 327.443 | 0.542368 |
| 17 | 19.4579 | 293.022 | 4.28861 | 25.5579 | 281.206 | 4.18379 | 21.8962 | 288.302 | 4.23379 |
| 18 | 379.037 | 230.527 | 6.10571 | 361.728 | 229.616 | 0.199343 | 371.74 | 230.144 | 6.25939 |
| 19 | 478.884 | 206.712 | 1.27832 | 475.454 | 221.757 | 1.40929 | 477.197 | 214.108 | 1.35472 |
| 20 | 173.923 | 143.042 | 0.00732468 | 157.261 | 142.182 | 6.00453 | 167.066 | 142.689 | 6.20403 |
| 21 | 498.561 | 122.687 | 5.83253 | 486.357 | 118.196 | 6.13893 | 493.718 | 120.906 | 5.95151 |
| 22 | 328.812 | 124.134 | 6.05932 | 312.848 | 119.605 | 5.98617 | 322.331 | 122.294 | 6.00901 |
| 23 | 461.738 | 116.731 | 4.47649 | 466.371 | 101.736 | 4.40285 | 463.615 | 110.656 | 4.41641 |
| 24 | 428.631 | 69.2715 | 5.87139 | 415.665 | 64.6444 | 6.13862 | 423.218 | 67.3364 | 5.96558 |
| 25 | 425.821 | 44.9942 | 5.59983 | 414.84 | 33.2028 | 5.37159 | 421.248 | 40.0897 | 5.461 |
| 26 | 368.362 | 35.6219 | 5.97427 | 353.22 | 30.4625 | 5.88261 | 362.109 | 33.4891 | 5.94605 |
| 27 | 503.484 | 22.7293 | 5.76026 | 489.632 | 16.6315 | 5.92136 | 497.924 | 20.2857 | 5.86668 |
| 28 | 369.184 | 5.84074 | 6.15994 | 352.622 | 4.25328 | 6.24787 | 362.144 | 5.16766 | 6.19236 |
| 29 | 510.519 | 331.417 | 5.88883 | 495.784 | 327.366 | 6.12889 | 504.484 | 329.758 | 6.02088 |
| 30 | 464.242 | 323.533 | 0.290639 | 451.756 | 328.194 | 0.532686 | 459.432 | 325.326 | 0.37736 |
Positions are in pixels, in the frame of reference of the original image, zero is in the top left corner. Lengths and areas are in pixels. Angles are in radians in the intervale 0 to 2*pi.
0 -- > x
¦
v
y Note: If several tracking analyses are performed on the same image sequence, the previous folder is not erased. It will be renamed as Tracking_result_DateOfTheNewAnalysis.
The tracking file can be opened for subsequent analysis:
# Python
data = pandas.read_csv("tracking.txt", sep='\t')# Julia
using CSV
CSV.read("tracking.txt", delim='\t')# R
read.csv("tracking.txt", header=T sep="\t")