Train a custom model to detect species in camera trap images
Let AI process the imagery for your ecological project
Table of contents
Introduction
Wildlife populations are commonly monitored with motion-triggered camera traps. Because analysing this data manually is labor-intensive, automation using AI computer vision models is becoming increasingly popular. Through this tutorial I hope to give you an introduction to customised machine learning models able to identify species of your choice, allowing you to efficiently manage and interpret large volumes of image data.
EcoAssist
In this tutorial, I’ve tried to make object detection as accessible as possible by creating a GitHub-hosted open-source application called EcoAssist [1]. EcoAssist is free to use for everybody.
Tutorial
The core idea of training a model is that a computer recognises specific objects by learning from a set of annotated example images. I’ll show you how you can annotate, train and deploy your own custom object detection model using EcoAssist. This tutorial will guide you through the process visualised below. In this tutorial, we’ll train a species classifier that will detect black bears and crows, but feel free to follow along with your own images to create a model that will detect the species of your choice.
Before we can start, we’ll need the EcoAssist software installed. The training function of EcoAssist has been removed in recent versions, but you can still access it by downloading version 4.3. More information about why the train feature has been removed and how to download that specific version can be found in the FAQ section here.
Step 1 — Annotate
The first part of object detection is also the most tedious one. To enable the computer to recognise our species, we must identify each instance of it in the images — a process known as an annotation. If you don’t have your own images and just want to follow this tutorial, you can download an already annotated dataset from my Google Drive (≈850MB) and skip to step 2. This dataset is a subset of the ENA24-detection dataset [3] and consists of 2000 images, of which 891 are bears, 944 are crows and 165 are backgrounds (empty images). If you want to annotate your own images, follow the steps below.
1.1 — Collect all your images in one folder. There should not be any subfolders or files other than images meant for training. It should look like this:
─── 📁dataset
|──image_1.jpg
|──image_2.jpg
|──image_3.jpg
:
└──image_N.jpg
1.2 — Manually annotating every instance can be a boring task. That’s why EcoAssist allows you to leverage the knowledge of existing models. In this case, we’ll use the MegaDetector [2] model to initiate the annotation process. To do that, open EcoAssist and navigate to the ‘Deploy’ tab.
1.3 — Select the folder containing your images using the ‘Browse’ option.
1.4 — Tick the ‘Process all images in the folder specified’.
1.5 — Click the ‘Deploy model’ button. This will open a new window which will show the deployment progress. Depending on your computer, this might take some time.
1.6 — When finished, the annotation feature will become available. Press ‘Start’.
1.7 — This will open a new window where you can specify which images you’d like to annotate. if you’re only interested in animals, increase the confidence range for class ‘animal’ to 0.2–1.0 and untick the classes ‘person’ and ‘vehicle’. To update the image counts, press the ‘Update counts’ button. Press ‘Start review process with selected criteria’ to start annotating.
1.8 — As you can see, MegaDetector assisted us by already drawing bounding boxes around what it thought were animals. You’ll only have to review them and adjust where necessary. You can adjust labels by right-clicking or pressing CRTL + E. Now it’s time to remove the boxes around the animals that you aren’t interested in, and change the labels of the boxes around the animals you are interested in. The annotation progress and shortcuts will be displayed in a separate window. It’s important to let EcoAssist know that you have verified the image by pressing the space bar. Continue doing this until all images are verified.
1.9 — Close the window when all images are verified. You will be prompted if you’d like to continue to the final step, click ‘Yes’. Then you’ll see an overview of the number of instances verified per class. Continue to export these images as training data by pressing ‘Yes’, choosing a folder to export to, and choosing whether you’d like to move or copy the images. If all went well, you end up with a folder like the one below.
─── 📁train
|──20230914141603-image_1.jpg
|──20230914141603-image_1.xml
|──20230914141603-image_2.jpg
|──20230914141603-image_2.xml
|──20230914141603-image_3.jpg
|──20230914141603-image_3.xml
:
|──20230914141603-image_N.jpg
└──20230914141603-image_N.xml1.10 — You’re ready to start training! Backup your final image training dataset in case disaster strikes.
Step 2 — Train
In this phase, we’ll train a model to detect your species.
2.1 — Navigate to the ‘Train’ tab.
2.2 — Specify values for the required parameters.
- Training type — Here you can select whether you want to continue an existing training or start a new one. Leave it on the default ‘Start new training’.
- Folder with labelled data — Select the folder with images and labels created at the annotation step.
- Retrain from — Here, you can select which model you’d like to use for transfer learning. You can also choose to train your model from scratch. If you’re training on the dataset of this tutorial, we’ll choose ‘MegaDetector 5a’.
- A number of epochs — An epoch means that all training data will pass through the algorithm once. This number thus defines how many times your computer will go through all the images. We’ll go with 300 epochs.
- Destination folder — Select a folder in which you want the results to be saved. This can be any folder.
- Project name — This option will only appear after the destination folder is selected. Here you can choose a name for your project, e.g., ‘tutorial’. The project name option will be a dropdown menu if you selected a destination folder in which previous projects have already been stored.
2.3 — It’s possible to set values for more parameters, but it’s not mandatory. For this tutorial, we’ll keep it simple.
2.4— Click ‘Start training’ to initiate the training process. You’ll see quite some output in the ‘Console output’ — most of which you can ignore. The training has begun if you see something like the output below.
Plotting labels to tutorial\exp\labels.jpg...
Image sizes 1280 train, 1280 val
Using 4 dataloader workers
Logging results to [1mincl-backgrounds\exp[0m
Starting training for 300 epochs...
Epoch GPU_mem box_loss obj_loss cls_loss Instances Size
0%| | 0/156 [00:00<?, ?it/s]
0/299 10.6G 0.1024 0.06691 0.02732 20 1280: 0%| | 0/156 [00:03<?, ?it/s]
0/299 10.6G 0.1024 0.06691 0.02732 20 1280: 1%| | 1/156 [00:08<21:48, 8.44s/it]
0/299 10.6G 0.1033 0.06553 0.02709 15 1280: 1%| | 1/156 [00:09<21:48, 8.44s/it]
0/299 10.6G 0.1033 0.06553 0.02709 15 1280: 1%|1 | 2/156 [00:09<11:01, 4.29s/it]
0/299 11.5G 0.1035 0.0647 0.02714 14 1280: 1%|1 | 2/156 [00:11<11:01, 4.29s/it]2.5 — It’s possible to calculate how long the total process will approximately take when the training is running. If you see [01:02<01:19, 1.29s/it], it means that it’ll take an epoch (01:02 + 01:19 =) 2:21 minutes to complete. Multiply this with the number of epochs, and you’ll have an estimation of the total time required for the training. Allow some extra time for data loading and testing after each epoch.
2.6 — When the training is done, you’ll find the results in the destination folder you specified in step 2.2.
Step 3— Deploy
Your model is now ready for deployment.
4.1 — Navigate to the ‘Deploy’ tab.
4.2 — Select a folder with images or videos you’d like to analyse at ‘Browse’. If you don’t have any test images, you can download some from my Google Drive (≈5MB).
4.3 — Select “Custom model” from the dropdown menu at the ‘Model’ option and select the model you trained. You can find it at <your-destination-folder>/exp/weights/best.pt.
4.4 — Tick the checkboxes to analyse images and/or videos.
4.5 — Select any other options you’d like to enable. See the ‘Help’ tab for more information about those.
4.6 — Click ‘Deploy model’.
Step 4— Post-process
At this point, you have successfully trained and deployed your own species recognition model. If you’d like to manually review a subset of the results, you can do that with the annotation feature. All the information you need is in the image_recognition_file.json for images or video_recognition_file.json for videos. You can use this file to process the images to your specific needs in Python, R or another programming language of your choice. However, I understand that programming can be a bit daunting, and I have thus prepared some generic post-processing features which will hopefully help you on your way.
- Separate files into subdirectories — This function divides the files into subdirectories based on their detections. For example, they will be divided in folders called
black bear,crow,black-bear_crow, andempty. - Draw bounding boxes and confidences — This functionality draws boxes around the detections and prints their confidence values.
- Crop detections — This feature will crop the detections and save them as separate images.
- Export results in CSV files — This will transform the JSON files to CSV files to enable further processing in spreadsheet applications such as Excel.
Follow the steps below to execute these post-processing features. The ‘Post-processing’ frame will become available after the deployment has finished.
5.1 — Select the folder in which you want the results to be placed at ‘Destination folder’.
5.2 — Tick the checkbox of the post-process feature you’d like to execute. It’s possible to select multiple.
5.3 — Select a confidence threshold to determine the minimum confidence. Detections below this value will not be post-processed to avoid false positives. If you don’t know what the best value is, read the explanation in the ‘Help’ tab, or just stick with the default value.
5.4 — Click ‘Post-process files’.
As you can see, the model correctly located all the bears and crows, whilst ignoring all the other animals and backgrounds. Job well done!
Closing remarks
You should now have a general understanding of object detection models, and how to use them. For this tutorial I kept it simple - to train an accurate model, you’ll have to perform multiple trainings to get an insight into your data, model performance, and possible areas of improvement. It’s a constant circle of evaluating, adjusting parameters, wrangling datasets, and setting optimisation techniques like image augmentation and hyperparameters evolution.
Since EcoAssist is an open-source project, feel free to contribute on GitHub and submit fixes, improvements or add new features.
References
- van Lunteren, P. (2022). EcoAssist: A no-code platform to train and deploy YOLOv5 object detection models [Computer software]. https://doi.org/10.5281/zenodo.7223363
- S. Beery, D. Morris and S. Yang, Efficient pipeline for camera trap image review (2019), arXiv preprint arXiv:1907.06772.
- H. Yousif, R. Kays, H. Zhihai, Dynamic Programming Selection of Object Proposals for Sequence-Level Animal Species Classification in the Wild (2019), Institute of Electrical and Electronics Engineers: Transactions on Circuits and Systems for Video Technology.
