Researchers from the University of Oxford Developed a Deep Learning-Based Software for Precision Tracking of Fish Movement in Complex Environments

Automated animal tracking software has revolutionized behavioral studies, particularly in monitoring laboratory creatures like aquarium fish, which is pivotal across neuroscience, medicine, and biomechanics. Despite advancements, current open-source tracking tools often need more accuracy in diverse conditions, especially when dealing with obstacles or complex environments.

Commonly used tracking solutions employ techniques like background subtraction or blob detection, facing limitations in natural settings or aquariums due to reflections, ripples, and dynamic backgrounds. While specialized software for specific fish models like zebrafish works well under typical conditions, it struggles in varied practical scenarios due to inherent method limitations.

Addressing these challenges, a UK-based research team introduced a hybrid method, merging deep learning and traditional computer vision techniques to enhance tracking accuracy for fish in complex experiments.

Unlike background subtraction or blob detection techniques, the bew proposed technique employs adaptive object detection using deep learning, allowing accurate tracking of the Picasso triggerfish amidst varying backgrounds, occlusions, or deformations. By integrating optical flow computation with object detection and tracking, this approach ensures robustness to changes in the fish’s appearance or occlusion by obstacles, providing precise trajectory information despite complex conditions that often challenge basic methods like background subtraction or blob detection.

This innovative approach combines deep learning’s adaptability with classical vision’s precision in centroid tracking, providing a more robust solution for monitoring fish behavior in challenging environments.

Their paper outlines a pioneering method for analyzing Picasso triggerfish behavior via video processing in controlled tank settings. It utilizes a GoPro Hero 5 camera and advanced tools like EﬃcientDet and optical flow techniques.

The deep learning part involves the use of object detection and tracking. Specifically, the paper utilizes a deep-learning-based object detector (EﬃcientDet) to identify both Picasso triggerfish and cylindrical obstacles in the video frames. This detector is retrained to detect these specific objects within the video data accurately.

On the other hand, traditional computer vision techniques are used in the tracking process. The authors employ classical optical flow estimation between consecutive frames, a conventional method in computer vision, to compute the fish’s trajectory and movement and accurately identify fish trajectories amidst obstacles. By using optical flow between consecutive frames, they determine the fish’s movement, aiding in understanding how obstacles affect fish behavior.

Initially, manual annotations in the videos trained the deep-learning object detector, supplemented by an object tracker to fill detection gaps using nearby high-confidence identifications.

Crucially, the method identified gates and areas between obstacles where fish move, employing Voronoi cell methods. For tank boundary gates, imaginary obstacles were introduced to aid gate identification.

Despite challenges like partial occlusion and proximity to obstacles impacting accuracy, the method achieved a remarkable 97% alignment between computed and manual fish trajectories. The researchers released their software, dataset, and tutorial under a Creative Commons license, supporting the broader scientific community in utilizing computer vision tools for animal tracking.

However, adapting this method to complex scenarios or multiple animals might require further refinement, considering challenges like partial occlusion or intricate environments.

In summary, this innovative fusion of deep learning and traditional computer vision techniques significantly advances animal tracking accuracy, particularly for fish in complex experimental setups. While achieving impressive results, challenges remain, urging further refinement for broader applications beyond controlled settings. The released assets and tutorial provide crucial resources for potential adaptations and advancements in automated animal tracking.

Check out the Pre-Print Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 35k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, LinkedIn Group, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.