.. _tutorial: ======== Tutorial ======== .. warning:: 🚧 Work in progress This section is partly outdated and currently under active development and may change without notice. Content, examples, and screenshots may be incomplete or updated soon. `Contributors:` `Tim R. Hofmeester (Swedish University of Agricultural Sciences, Department of Wildlife, Fish and Environmental Studies)` `Karolina Kuczkowska and Jakub W. Bubnicki (Open Science Conservation Fund)` `Claude Sonnet 4 (AI)` The guides below describe how to use TRAPPER core functionalities. The first manual describes the process of uploading your data to the Trapper server. To start classifying *Resources*, you need to create a *Classification project* and a *Classificator*, as described in the second manual, and a third manual for classifying *Resources* once you have a running *Classification project*. Opening and logging in to TRAPPER +++++++++++++++++++++++++++++++++ Go to the `demo instance `_ and register an account on the TRAPPER server. After your account is activated by an admin, you can log in on the same page using your email address and password. How to upload data into TRAPPER +++++++++++++++++++++++++++++++ 1. Create a research project ============================ In the top menu, select *Research > Add research project*. .. image:: images/tutorial/top_menu.png :alt: top menu with Add Research project option shown You now get a menu with different options to fill in: .. image:: images/tutorial/add_research_project_form.png :alt: add research project form Fill in all information for your research project and click *Add research project*. Make sure you choose an acronym that is more than 3 characters long, which will make it easier to find it with the auto-fill function in Trapper. .. note:: Right after adding your research project, you will see a notification that your project has been successfully added but needs to be activated by an admin. In practice, this means that your project request has been placed in the Admin dashboard and must be manually approved by an admin. This is one of the security measures and ways to control the amount and content of projects in your institution/team. If you are an admin yourself, you can go to the Admin dashboard and approve the project. As soon as an admin approves your new project, it will appear in your *Research* view in a *Research projects* table presenting the project Name, Acronym, Keywords, and Actions. In *Research projects*, all the projects you have access to are listed. You can filter your projects according to their content and/or the role you play in them. .. _tutorial-add-locations: 2. Add locations to TRAPPER =========================== Once you have created a *Research project*, you can add *Locations* to your project. There are two ways to do this in TRAPPER: You can add locations manually, or you can upload a file with location data. To manually add locations, select *Map* > *Locations* > *Add location*. .. image:: images/tutorial/top_menu_add_location.png :alt: top menu with add locations button shown You now go to the map view where you can manually add information for a camera trap location. .. image:: images/tutorial/add_location_map.png :alt: map view for adding a single location Most of the time, you want to add multiple locations at once, which can be done using a .csv or .gpx file with location information. It is easiest to use a .csv file, which should be comma- separated and should contain the following columns: - ``locationID``: the ID/name of the location - ``longitude``: the longitude of the location in WSG84 decimal degrees (e.g., 19.58536776) - ``latitude``: the latitude of the location in WSG84 decimal degrees (e.g., 63.55417296) Once you have created the file, you can add the locations to TRAPPER using *Map* > *Locations* > *Import locations*. .. image:: images/tutorial/import_locations_form.png :alt: import locations form Select the .csv file, make sure you select the right Timezone (e.g., Europe/Stockholm) and select the *Research project* to which you want to add the locations. Once you have done this, you can select *Upload* to add the *Locations* to your *Research project*. 3. Set up Trapper-Tools ======================= .. warning:: The legacy **Trapper Client** is no longer supported for data uploads. Use **Trapper Tools** instead. We provide a Python CLI application called **Trapper Tools** to handle data conversion, packaging, and upload processes. Trapper Tools creates standardized data packages with metadata and uploads them securely to the Trapper server via FTPS. Trapper Tools is available from the GitLab repository: ``_ **Installation Requirements:** - Python 3.11 or higher - FFmpeg (for media conversion) - ExifTool (for metadata extraction) - Access to a Trapper server instance **Installation:** You can install Trapper Tools using pip: .. code-block:: bash pip install "git+https://gitlab.com/trapper-project/trapper-tools.git" **Docker-based Setup:** For a containerized setup, create a ``docker-compose.yml`` file: .. code-block:: yaml services: trapper-tools: image: registry.gitlab.com/trapper-project/trapper-tools:latest container_name: trapper-tools volumes: - trapper_tools_settings:/home/trapper/.trapper-tools - ${DATA_PATH:-./data}:/data env_file: - .env volumes: trapper_tools_settings: Run commands with: .. code-block:: bash export DATA_PATH=/path/to/camera_trap_data docker compose up -d docker compose exec --user trapper trapper-tools bash .. note:: When using Docker, FFmpeg and ExifTool are already included in the container image, so you don't need to install them separately on your host system. **Configuration:** Create a configuration file to store your Trapper server credentials and project settings: .. code-block:: bash trapper-tools config --init This creates a configuration file at ``~/.trapper-tools/default.toml``. Edit this file to include: - **Server URL**: The URL of your Trapper server (e.g., ``https://your-trapper-instance.org/``) - **Username**: Your Trapper account email - **Password**: Your Trapper account password - **Project ID**: The numeric ID of the research project you want to upload data to - **FTP settings**: Connection details for secure file transfer **Example Configuration:** .. code-block:: toml [GENERAL] host = "https://your-trapper-instance.org" login = "your-email@example.com" password = "your-password" project_id = 123 verify_ssl = true [UPLOAD] ftp_tls = false trigger = true .. note:: Make sure you have an FTP account connected to your Trapper account (ask the admin of your Trapper server for this). Trapper Tools uses FTPS for secure file transfer. **Data Organization:** Organize your camera trap data in the following structure: .. code-block:: project_directory/ β”œβ”€β”€ collection_name_1/ β”‚ β”œβ”€β”€ deployment_id_1/ β”‚ β”‚ β”œβ”€β”€ filename_1.jpg β”‚ β”‚ β”œβ”€β”€ filename_2.jpg β”‚ β”‚ └── ... β”‚ β”œβ”€β”€ deployment_id_2/ β”‚ β”‚ β”œβ”€β”€ filename_1.jpg β”‚ β”‚ └── ... └── collection_name_2/ └── ... Ensure deployment folder names match the deployment codes in your TRAPPER database for validation. **Testing Installation:** Verify your installation and configuration: .. code-block:: bash # Test external tools trapper-tools helpers test_external_tools # Test server connection trapper-tools helpers test_connection **Complete Automated Workflow:** For the most efficient workflow, use the **pipeline command** to automate the entire process: .. code-block:: bash # Complete workflow: convert β†’ package β†’ upload β†’ trigger AI processing trapper-tools pipeline --data-path /path/to/data --convert --trigger This single command handles: - Media conversion to web-friendly formats - Data package creation with metadata extraction - Secure FTPS upload to the server - Automatic triggering of AI processing pipeline For more detailed control, you can run individual steps: .. code-block:: bash # Convert media files trapper-tools convert --data-path /path/to/data # Create data package trapper-tools package --data-path /path/to/data # Upload to server trapper-tools upload --file-yaml package.yaml --file-zip package.zip **Citizen-Science interface - Alternative Upload Method** For smaller deployments or for user-friendly uploads, you can use the `trapper-frontend `_ web interface: - **Drag-and-drop uploads**: Upload individual deployment folders through the web browser easily (up to ~10k files in a single request). - **Real-time processing**: Get immediate feedback on upload progress and AI processing results. - **User-friendly interface**: Ideal for interactive exploration of AI-processed data, expert validation, generation of object-based training data for AI models, citizen science projects, distributed network monitoring, and occasional users. - **Single-deployment focus**: Best suited for uploading one deployment at a time. Access the upload interface through the **trapper-frontend** web application at your TRAPPER instance. See :ref:`citizen-science-interface` for more details. **When to Use Each Method** - **trapper-tools**: Best for large datasets, batch processing, automated workflows, and technical users - **trapper-frontend**: Best for small deployments, occasional uploads, training users, and citizen science projects .. note:: Both methods support the same AI processing pipeline and produce identical results in TRAPPER. .. _tutorial-add-deployments: 4. Add deployments to TRAPPER ============================= Now that you have added your locations to TRAPPER, you can start adding *Deployments*. The main differences between Locations and Deployments are that *Locations* only specify the coordinates where a camera trap was located, while *Deployments* also specify the start and end date and time of a camera trap in the field. A *Location* can have several *Deployments*. Usually, a *Deployment* is determined as one session of putting out the camera and picking it up, or a session between changing cards and batteries. Again, it is possible to manually add *Deployments*, but this is rather error-prone. Therefore, in this manual we will only deal with adding *Deployments* based on the camera trap data itself. This will ensure that deployment times are correct for all *Deployments*, even those with cameras that stopped working before they were picked up. You can create a .csv file with your *Deployment* information using Trapper Tools. For this to work, the data must be organized in the correct way: a main directory with sub-directories for each *Collection*. A *Collection* can, for example, be one round of *Deployments*. Within each *Collection*, there are further sub-directories for each *Deployment*. These sub-directories should be labeled in congruence with other information provided to TRAPPER. The *Deployment* label is a combination of a unique code and the *Location* label. For example, you can use a unique code ``R1`` (for round 1) for the first *Deployment* at a certain *Location*, e.g., ``32_01``. The Deployment label then becomes: ``R1-32_01``. Use Trapper Tools to generate a deployment template: .. code-block:: bash trapper-tools helpers template --data-path /path/to/data This will analyze your media files and generate a .csv file with the *Deployment* table containing deployment metadata. You can then upload the .csv to TRAPPER by selecting *Map* > *Deployments* > *Import deployments*. .. image:: images/tutorial/import_deployments.png :alt: import deployments form .. note:: The automatically generated *Deployment* table contains only a minimum set of metadata fields needed to import deployments into TRAPPER. See https://camtrap-dp.tdwg.org/data/#deployments for a full list of supported metadata fields that can be imported into TRAPPER. .. note:: Please note that if your *Deployment* table contains ``longitude`` and ``latitude`` fields, location objects associated with imported deployments can be created automatically, and you can skip the previous step of importing the *Location* table (:ref:`tutorial-add-locations`). Select the previously generated .csv file and the *Research project* to which the round of *Deployments* belongs, and click *Submit*. 5. Add resources to TRAPPER =========================== Once the *Deployments* are added, the *Resources* (e.g. images) made by the camera traps can be uploaded into TRAPPER. TRAPPER supports multiple upload methods depending on your needs. **Upload Methods** TRAPPER offers two main approaches for uploading data: 1. **trapper-tools** (Recommended): Command-line tool for batch uploads with automated processing 2. **trapper-frontend**: Web interface for single deployments and smaller datasets. See :ref:`citizen-science-interface` for more details. .. note:: The media files must be organized in the directory structure described earlier: main directory with sub-directories for each *Collection*, and within each *Collection*, sub-directories for each *Deployment* labeled as described in the previous section (e.g., ``R1-32_01``). **Recommended: Automated Pipeline with trapper-tools** For the most efficient workflow, use the pipeline command with the ``--trigger`` flag to enable automatic processing: .. code-block:: bash # Complete automated workflow with AI processing trapper-tools pipeline --data-path /path/to/data --convert --trigger The ``--trigger`` flag is **critical** for automatic processing. This single command will: - Convert media files to web-friendly formats (if ``--convert`` is specified) - Create a data package with metadata extraction - Upload the package securely via FTPS - **Automatically trigger the AI processing pipeline** including: - Object detection (animals, humans, vehicles) - Species classification - Sequence building - Privacy blurring Without the ``--trigger`` flag, data will be uploaded but **not automatically processed**, requiring manual intervention through the web interface. **Alternative: Step-by-Step Process** If you prefer more control over the upload process, you can run individual steps: **Step 1: Create a data package** .. code-block:: bash trapper-tools package --data-path /path/to/data This command will: - Analyze your media files and validate that corresponding deployments exist in TRAPPER - Create a .zip file containing all your media data - Generate a .yaml file with the package metadata (package definition file) **Step 2: Upload the data package via FTPS** .. code-block:: bash trapper-tools upload --file-yaml package.yaml --file-zip package.zip --trigger **Important**: Always include the ``--trigger`` flag to enable automatic processing. **Additional Examples** .. code-block:: bash # Convert media first, then package and upload with automatic processing trapper-tools convert --data-path /path/to/data trapper-tools package --data-path /path/to/data trapper-tools upload --file-yaml package.yaml --file-zip package.zip --trigger # Process specific collections only with automatic processing trapper-tools pipeline --data-path /path/to/data --collection summer_2024 --convert --trigger # Upload with custom package name and trigger processing trapper-tools package --data-path /path/to/data --package-name "winter_survey_2024" trapper-tools upload --file-yaml winter_survey_2024.yaml --file-zip winter_survey_2024.zip --trigger **Manual Loading via Web Interface (Alternative)** If you need to manually load data that was uploaded without the ``--trigger`` flag: 1. Go to *Storage* > *Upload collection* on your TRAPPER server 2. Select the .yaml file that you uploaded and click 'Upload' .. image:: images/tutorial/upload_collection_1.png :alt: upload collection form step 1 3. Select the .zip file you uploaded in the second window and click 'Upload' to load your data package into TRAPPER .. image:: images/tutorial/upload_collection_2.png :alt: upload collection form step 2 You have now created a *Collection* containing the *Resources* of all the *Deployments* that you used while creating the *Collection*. This *Collection* is added to the correct *Research project* automatically. You can browse the *Resources* in the *Collection* by going to *Storage* > *Resources (available)*. .. seealso:: :ref:`administration-ai-providers` for AI Provider configuration and :ref:`configuration-ai-pipeline` for AI Pipeline setup required for automatic processing. How to create a Classificator and Classification project in TRAPPER +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ .. _tutorial-create-classificator: 1. Create a Classificator ========================= In TRAPPER, images are classified in *Classification projects* using a *Classificator*. One of the great advantages of TRAPPER is that researchers or students can build their own *Classificator* and *Classification project* depending on the research question. A *Classificator* basically describes a set of attributes that are used while annotating an image or sequence of images. In the top menu, select *Classification* > *Add classificator*. First there are some required classificator properties that you have to fill in, such as name and template. Furthermore there are several attributes that have become mandatory in the latest version of Trapper, which are species, observation type, count and β€˜is_setup’. The β€˜is_setup’ and observation type were added to make it easier to include a first screening of images by Microsoft's MegaDetector, a machine learning algorithm that automatically identifies empty images from images containing a human, an animal, or a vehicle. Also, you can add species you want to classify based on entries in the species list. .. note:: Admins can add species to the overall species list in the Admin panel under `Species` either a single species at a time or using a .csv file, or download taxa for entire classes from Catalogue of Life. .. image:: images/tutorial/classificator_form_1.png :alt: classificator form part 1 Further down on the page, you see a lot of different options, which I will explain step by step. There are two types of attributes: static and dynamic attributes. Static attributes are attributes that are only given once for each resource, e.g., β€˜is setup’. Dynamic attributes are attributes that can be given multiple times for one resource, e.g., the species or sex of an animal recorded by the camera trap. There are several pre-defined attributes available to quickly add to your *Classificator* by checking the boxes. .. image:: images/tutorial/classificator_form_2.png :alt: classificator form part 2 One of the pre-defined options is *Group-level annotations*. When enabled, the classification form exposes an extra toggle letting experts record an aggregate group observation (species, sex, age, and a group-size ``count``) without drawing a bounding box β€” useful for frames where a herd or flock is visible but drawing boxes around every individual is impractical. Per-object rows (with bounding boxes and ``count=1`` per track) and group-level rows can coexist on the same resource; leave the flag off if your project only needs per-object annotations. Then you can also add custom attributes in the custom attribute form. This form gives a lot of freedom to determine your own attributes. Below, you find the attributes of an example *Classificator*. .. image:: images/tutorial/example_classificator.png :alt: finished example classificator You can check all the *Classificators* on your TRAPPER server by going to *Classification* > *Classificators*. 2. Create a Classification project ================================== Once you have created your *Classificator*, you can create a *Classification project* to start classifying your resources. You can create a new *Classification project* by going to *Classification* > *Add classification* project. .. image:: images/tutorial/classification_project_form.png :alt: add classification project form Here you can specify the name of the *Classification project*, the *Research project* to which the *Classification project* belongs, which *Classificator* and default *AIProvider* to use, and which users are involved in the *Classification project*. When you have finished filling in all details, click *Add classification project*. You have now created a *Classification project*. You can check all the Classification projects to which you have access by going to *Classification* > *Classification projects*. 3. Add Resources to a Classification project =============================================== In order to be able to classify *Resources*, you need to add them to your *Classification project*. To do this, first go to your *Research project* by going to *Research* > *Research project* and clicking on the magnifying glass next to your research project. Now you will see a selection of all *Collections* that are linked to the *Research project*. .. image:: images/tutorial/research_project_collections.png :alt: research project collections list Select a *Collection* by clicking on it and under *Actions* > *More* select β€˜Add to Classification project’. You now have *Resources* in your *Classification project* that can be classified. How to classify images in TRAPPER +++++++++++++++++++++++++++++++++ 1. Go to the classification page ================================ In the top menu, select *Classification* > *Classification projects*. You are now at the starting screen of all classification projects that you have access to. You will see information on each *Classification project*, including its name, the *Research project* it belongs to, its status and *Actions*. .. image:: images/tutorial/classification_projects_list_2.png :alt: classification projects list To select the *Classification project* you want to work in, select the magnifying glass underneath *Actions*. .. image:: images/tutorial/classification_project_details.png :alt: classification projects details You are now on the starting screen of the selected *Classification project*. Here you see more information on the *Classification project*, including which *Classificator* is used, which *Collections* are in the project, and to which *Research project* the *Classification project* belongs. Furthermore, you can see how many of the *Resources* have been Classified and Approved. Go to the bottom of the page, where you can find the different *Collections* in the *Classification project* (for example, the different rounds during which cameras were placed out in the field). Type the name of the *Deployment* within a *Collection* that you would like to classify and select it. This ensures that you are classifying a specific *Deployment*, which makes it easier to keep an overview during the classification. Once you have selected the *Deployment*, select the eye symbol to start the classification. .. note:: Make sure you generate sequences for your collection by selecting the collection and clicking β€˜(Re)build sequences’. You can specify a time interval between sequences. The default is 5, regular use is 15 minutes. 2. Classify a sequence using the initial classification forms ============================================================= You will now see a screen such as the one below. You can see the different *Resources* (images) with an orange number. This is the *Sequence* number. .. image:: images/tutorial/classify_sequences.png :alt: list of resources in sequences to classify .. note:: To circumvent problems of two people classifying the same Deployment, only start classifying a Deployment if none of the Sequences have been classified yet. You can see if a Resource has been classified by a green mark (note 3 Resources - one Sequence - classified and others not classified in the screen cap above). At the same time, make sure that you classify all Sequences in a Deployment (keep track of which Deployment you are classifying). If you cannot finish a deployment, contact your admin. You can now enlarge pictures by clicking on the thumbnail. You can scroll through the enlarged pictures with your arrow keys. It’s always good to check the whole sequence before classifying it to check you didn't miss any individuals in the thumbnails. .. image:: images/tutorial/resource_enlarged.png :alt: single resource view When you have decided how many individuals and which species is visible in the *Sequence*, it is time to classify the *Sequence*. To do this, click on the sequence number. This will select all images in the sequence. You can now click β€˜Classify’ in the top of the screen to classify the whole sequence. **Make sure to check for empty images at the end of the sequence as it is** **good to classify those separately as β€˜observation_type = blank’.** In the classification view, fill in the right information for the sequence. You classify β€˜is_setup’ as `True` for setup and test images, you can then delete all dynamic attributes by clicking on the red cross. .. note:: Classify **empty resources** as `Observation type = Blank`. Otherwise, keep the default `False` and add the dynamic attribute information. .. image:: images/tutorial/classify_modal_form.png :alt: modal with classification form Dynamic attributes are attributes that can be given multiple times for one sequence. In this case, it gives information about the type of observation, the species of animal, if the observation contains one or more animals, sex, age and behaviour of the animals, and optional custom str value (as specified in the *Classifier*) and individual id. In the standard classification form there are three required attributes: 1. Observation type: the type of observation (Human/Vehicle/Animal/Blank/Unknown/Unclassified) 2. Species: The animal species shown in the image (to be selected from *Species* from *Classificator*) 3. Count: The number of animals/humans/vehicles, default is 1 There are also several attributes that can be assigned, as described in the *Classificator* adding manual. If there are multiple animals in the field with different attributes, you can add a row by clicking the green button with a plus sign to specify the attributes of all animals in the sequence. Click β€˜Classify’ to finish the classification. The *Sequence* is now classified and you can go to the next *Sequence* by clicking on the orange sequence number. Follow the steps above until you have classified all resources in a *Deployment*. .. note:: **FOR GROUPS OF ANIMALS**: If there is one image in which all animals in the group are present, please classify the sequence per picture. In this way we can estimate retention time of individuals (especially with groups <5). With big groups, or groups where animals are never all in the picture, classify the whole group in one sequence. This ensures that we have the right number of animals passing in the analysis by just selecting the maximum per sequence. Again, make sure that empty pictures that are part of the sequence are classified as empty. .. note:: If a person is in the frame with a dog, add **two observations**: one with observation_type = human and species `Homo sapiens` and the other with observation_type = animal and species `Canis familiaris`. 3. Adding bounding boxes for training dataset for Machine Learning ================================================================== TRAPPER provides the capability to add bounding boxes to each classification to delimit where objects (animals, humans, or vehicles) are located. These bounding boxes aid in training Machine Learning algorithms for automatic species identification. **Using the Citizen Science Frontend** For most users, the recommended approach is to use the new **citizen science frontend** which provides an intuitive interface for drawing bounding boxes directly on images during the classification process. To access the citizen science interface: 1. Navigate to your classification project 2. Select the deployment you want to work on 3. Click on a resource to open it in the citizen science interface 4. Draw bounding boxes by clicking and dragging on the image 5. Associate each bounding box with the appropriate classification attributes The citizen science frontend automatically saves bounding boxes as part of the classification process and provides a streamlined workflow for rapid annotation. **Using the Expert Interface** For advanced users who need access to detailed classification features, you can also add bounding boxes through the expert interface. To do this, go to the 'classic' classification view by clicking on 'Details' underneath a thumbnail. .. image:: images/tutorial/bboxes1.png :alt: classify resource detail view You can add a bounding box by clicking on the green hand symbol on the user classification. This will lead you to the bounding box drawing interface. .. image:: images/tutorial/bboxes2.png :alt: draw bboxes view Draw a box around the object as tightly as possible. You can then specify which dynamic attribute field should be linked to this bounding box. .. image:: images/tutorial/bboxes3.png :alt: draw bboxes view with bboxes drawn Click 'Save' to add the bounding box to the classification. You can see the saved bounding boxes in the classification screen. .. image:: images/tutorial/bboxes4.png :alt: classify resource detail view with bboxes drawn Dual Annotation Interfaces in TRAPPER ++++++++++++++++++++++++++++++++++++++ TRAPPER provides two different interfaces for image annotation and classification, each designed for different use cases and user types: Expert Interface vs. Citizen Science Frontend ============================================== **Expert Interface (Classic)** The expert interface provides comprehensive access to all TRAPPER features and is designed for: - Researchers and experienced users who need full control over classification parameters - Projects requiring detailed metadata capture and complex annotation workflows - Administrative tasks and advanced project management - Users who need access to all classificator attributes and advanced features Access the expert interface through the main TRAPPER web application at your server's URL. **Citizen Science Frontend** The citizen science frontend offers a streamlined, user-friendly interface designed for: - Volunteer contributors and citizen scientists - Large-scale data annotation projects requiring many participants - Projects where ease of use is prioritized over feature completeness - Training new users before transitioning to the expert interface The citizen science frontend simplifies the classification process while maintaining data quality and compatibility with the main TRAPPER system. **Choosing the Right Interface** - Use the **expert interface** for detailed research work, administrative tasks, and when you need access to all features - Use the **citizen science frontend** for volunteer annotation projects, training sessions, and when prioritizing ease of use - Both interfaces work with the same underlying data, so you can use them interchangeably based on your needs **Data Compatibility** Classifications and annotations created in either interface are fully compatible and can be viewed and edited in both systems. This allows for flexible workflows where initial classifications might be done in the citizen science frontend and later refined in the expert interface. Camtrap DP Data Export ++++++++++++++++++++++ TRAPPER provides comprehensive support for exporting data in the **Camera Trap Data Package (Camtrap DP)** format, which is the TDWG standard for camera trap data exchange. This functionality allows you to export your TRAPPER data for analysis, archival, or sharing with other researchers and platforms. What is Camtrap DP? =================== Camtrap DP is a standardized data format developed by the Biodiversity Information Standards (TDWG) organization specifically for camera trap data. It ensures that camera trap datasets can be easily shared, compared, and analyzed across different platforms and research groups. The format includes: - Structured metadata about deployments, locations, and media files - Standardized observation and annotation data - Taxonomic information aligned with standard biodiversity databases - Comprehensive project and dataset documentation For more information, see the `official Camtrap DP documentation `_. Exporting Data from TRAPPER ============================ **Export via Web Interface** Navigate to your classification project details view in TRAPPER and click the **Export results** button: .. code-block:: bash https://{your-server}/media_classification/project/detail/{project_id}/ Or navigate directly to the export page: .. code-block:: bash https://{your-server}/media_classification/classification/export/{project_id}/ Fill in the export options form: .. figure:: /images/tutorial/export_camtrapdp.png :alt: Camtrap DP export options form Configure your Camtrap DP package **Generated Files** The export process creates a complete Camtrap DP package containing: - ``datapackage.json``: Main package descriptor with metadata - ``deployments.csv``: Deployment information (when, where cameras were active) - ``media.csv``: Media file information and metadata - ``observations.csv``: Classification and observation data Camtrap DP API for Data Package Generation ++++++++++++++++++++++++++++++++++++++++++ TRAPPER provides a REST API for programmatically generating Camtrap DP data packages. This API is particularly useful for automated workflows, large-scale data processing, and integration with external analysis tools. API Endpoint ============ **Generate and Download Camtrap DP Package** .. code-block:: bash GET /api/media-classifications/package/{project_id}/?parameter=value Authorization: Token your-api-token The API uses a single GET endpoint that generates the package (or returns a cached version) and provides a download URL in the response. API Parameters ============== All parameters are passed as query parameters in the URL: **Package Generation Options:** - ``clear_cache``: Boolean, force regeneration of cached package (default: false) - ``release``: Boolean, create a released version of the package (default: false) - ``get_released``: Boolean, retrieve the released version if available (default: false) - ``export_format``: Export format, only "camtrapdp" is supported (default: "camtrapdp") - ``export_filetype``: File compression type (default: "csv.gz") **Data Filtering Options:** - ``approved_only``: Boolean, include only approved classifications (default: true) - ``exclude_blank``: Boolean, exclude blank/empty observations (default: false) - ``all_deployments``: Boolean, include all deployments in project (default: true) - ``filter_deployments``: String, comma-separated list of deployment IDs to include - ``include_events``: Boolean, include event-level data aggregation (default: false) - ``events_count_var``: String, variable to use for event counting (default: "count") **Privacy and Access Options:** - ``private_human``: Boolean, apply privacy settings to human observations (default: true) - ``private_vehicle``: Boolean, apply privacy settings to vehicle observations (default: true) - ``private_species``: List of species names to apply privacy settings to - ``trapper_url_token``: Boolean, include TRAPPER-specific URL tokens (default: true) **Package Metadata:** - ``name``: String, custom name for the data package - ``version``: String, version identifier for the package (default: "1.0") - ``title``: String, human-readable title for the package - ``description``: String, description of the package contents - ``keywords``: Comma-separated list of keywords for the package - ``licenses``: Comma-separated list of license identifiers Authentication ============== API access requires authentication using API tokens: 1. Log in to TRAPPER web interface 2. Go to *User Profile* > *API Tokens* 3. Generate a new API token 4. Include the token in the ``Authorization`` header: ``Token your-api-token`` Response Format =============== The API returns a JSON response with the following structure: .. code-block:: json { "data": { "message": "Package successfully generated and saved to cache.", "errors": null, "package": "https://{your-server}/storage/packages/download/{package_id}/" } } Example Usage ============= **Python Example** .. code-block:: python import requests # Configuration BASE_URL = "https://your-trapper-instance.org" API_TOKEN = "your-api-token" PROJECT_ID = 123 headers = {"Authorization": f"Token {API_TOKEN}"} # Request package generation with custom parameters params = { "approved_only": "true", "exclude_blank": "true", "clear_cache": "false", "export_format": "camtrapdp", "name": "wildlife_survey_2024", "version": "2.0", "title": "Wildlife Survey Data 2024", "description": "Camera trap data from forest monitoring project" } response = requests.get( f"{BASE_URL}/api/media-classifications/package/{PROJECT_ID}/", headers=headers, params=params ) if response.status_code == 200: result = response.json() download_url = result["data"]["package"] print(f"Package ready: {download_url}") # Download the package package_response = requests.get(download_url, headers=headers) with open("camtrap_dp_export.zip", "wb") as f: f.write(package_response.content) else: print(f"Error: {response.status_code}") print(response.json()) **R Example** .. code-block:: r library(httr2) library(jsonlite) # Configuration base_url <- "https://your-trapper-instance.org" api_token <- "your-api-token" project_id <- 123 # Request package with parameters response <- request(paste0(base_url, "/api/media-classifications/package/", project_id, "/")) |> req_headers(Authorization = paste("Token", api_token)) |> req_url_query( approved_only = "true", exclude_blank = "true", export_format = "camtrapdp", name = "wildlife_survey_2024", title = "Wildlife Survey Data 2024" ) |> req_perform() if (resp_status(response) == 200) { result <- resp_body_json(response) download_url <- result$data$package # Download the package request(download_url) |> req_headers(Authorization = paste("Token", api_token)) |> req_perform(path = "camtrap_dp_export.zip") cat("Package downloaded successfully\n") } else { cat("Error:", resp_status(response), "\n") cat(resp_body_string(response), "\n") } **cURL Example** .. code-block:: bash # Generate and get download URL curl -H "Authorization: Token your-api-token" \ "https://your-trapper-instance.org/api/media-classifications/package/123/?approved_only=true&name=my_survey" # Download the package (using URL from response) curl -H "Authorization: Token your-api-token" \ -o camtrap_export.zip \ "https://your-trapper-instance.org/storage/packages/download/package_id/" Caching and Performance ======================= **Automatic Caching** The API automatically caches generated packages to improve performance: - Identical requests return cached results unless ``clear_cache=true`` is specified - Cached packages are stored server-side and accessible via download URLs - Cache duration is managed automatically by the server **Performance Considerations** - Large classification projects may take several minutes to process initially - Subsequent requests for the same parameters return cached results immediately - Use ``approved_only=true`` to reduce package size and processing time - Filter deployments using ``filter_deployments`` parameter to limit scope Error Handling ============== The API returns standard HTTP status codes: - ``200 OK``: Package generated successfully or returned from cache - ``400 Bad Request``: Invalid parameters or package generation failed - ``401 Unauthorized``: Invalid or missing API token - ``403 Forbidden``: Insufficient permissions to access the classification project - ``404 Not Found``: Classification project not found Error responses include details in the JSON response: .. code-block:: json { "data": { "message": "Bad request", "errors": "Invalid parameter: export_format must be 'camtrapdp'", "package": null } } Common Issues ============= **Permission Errors** Ensure your API token has access to the specified classification project. **Empty Packages** If the generated package contains no data: - Check that the project has approved classifications (if using ``approved_only=true``) - Verify that deployments contain classified resources - Ensure filter parameters are not excluding all data **Large Package Timeouts** For very large projects: - Use ``filter_deployments`` to process data in smaller batches - Consider using ``exclude_blank=true`` to reduce package size - Contact your TRAPPER administrator about server timeout settings Timezone behaviour in TRAPPER +++++++++++++++++++++++++++++ The default timezone for TRAPPER app is **Europe/Warsaw**. You can set your working timezone in your User Profile settings tab (click your username in top right corner to access this view). .. image:: images/tutorial/user_profile.png :alt: user profile settings view Your working timezone will be used: - to correctly display system notifications (e.g. asynchronous task completion times on your dashboard), - as a default timezone value in forms. Daylight Savings Time ===================== While not recommended, it is possible to bypass default behaviour regarding DST (Daylight Saving Time). If your camera trap at a given location *does not use* DST (i.e. does not move its timer one hour forward in the spring and one hour back in the fall), you should mark its ``ignore_DST`` flag ``True``. With that setting, all timestamps in recorded metadata from that location (``Deployment.start_date``, ``Deployment.end_date`` and ``Resource.date_recorded``) will be processed ignoring DST (it would be considered in processing timestamps otherwise). .. note:: Whenever this documentation mentions two *Locations* having **the same timezone specification**, it means both the ``timezone`` field and the ``ignore_DST`` flag having the same value, as only both values together correctly describe the logic of processing timestamps (i.e. datetime objects) in TRAPPER. Timezone is set for Location ============================ *Location* objects have a *timezone* field and an ``ignore_DST`` flag. You have to specify them in the form when adding location in one of the following ways: - using the map view with the form for adding a single location, - importing locations from CSV or GPX file (bulk import), - creating locations automatically using deployments bulk import. It is very important to correctly set timezone information for every *Location*, because this data is used to process timestamps in all *Deployments* and *Resources* connected to given *Location*. Timezone behaviour during CSV import ==================================== When importing *Deployments* or *Locations* from CSV file, you have to provide their ``timezone`` and ``ignore_DST`` values in the import forms. This means that you can only import locations from the same ``timezone`` and with the same ``ignore_DST`` flag value. Data that do not meet this condition has to be imported using separate CSV files. When importing *Locations*, values from the form will be set for all imported objects. When importing *Deployments*, they can be linked to different locations, but all locations need to have the same timezone specification. If locations are being created, their timezone info will be set to data from the form. If *Deployments* are uploaded to existing *Locations*, the file will be validated to make sure all locations referenced in the data have the same timezone specification as in the form. This also applies to uploading *Collections* - when using `trapper-tools `_, *Collections* can only be created from *Resources* linked to *Locations* with the same timezone specification. The timezone specification is provided in the YAML file describing given *Collection*. When uploading a *Collection*, the validation process is performed to check that provided timezone matches *Locations* in the database as well. AI Provider and Classification Project Configuration +++++++++++++++++++++++++++++++++++++++++++++++++++++ TRAPPER provides sophisticated artificial intelligence integration through **AI Providers** and **Classification Projects** that can leverage machine learning models for automated image analysis. This functionality is particularly useful for processing large datasets and providing initial classifications that can be reviewed and refined by human annotators. **Overview** AI Providers in TRAPPER are configurable services that provide automated image analysis capabilities: - **Empty Image Detection**: Filtering out images that contain no objects of interest - **Object Detection**: Identifying and localizing animals, humans, and vehicles in images - **Species Classification**: Automatically identifying species based on image content **Configuration** TrapperAI Providers are configured by administrators through the Django admin interface and can be associated with Classification Projects to provide automated assistance during annotation. Trapper AI currently supports multiple deep learning architectures (including Darknet, YOLOR, YOLOv5, YOLOv8, YOLOv9, YOLOv10, YOLOv11, RT-DETR, and ViT). Available AI models include: - **MegaDetector v5**: General-purpose camera trap object detection; `link `__ - **MegaDetector v6 (PyTorch Wildlife)**: General-purpose camera trap object detection; `link `__ - **DeepFaune v1.2/v1.3**: European mammal species classification; `link `__ - **TRAPPER AI Species Classifier v02.2024/v08.2024**: European mammal species classification; `LINK `_ .. seealso:: For detailed TrapperAI Provider configuration instructions, see :ref:`administration` β†’ TrapperAI Provider Configuration. For environment variables and system settings, see :ref:`configuration` β†’ AI Pipeline Configuration. .. _tutorial-automatic-ai-pipeline-workflow: Automatic AI Pipeline Workflow ++++++++++++++++++++++++++++++ TRAPPER features a sophisticated **automatic AI pipeline** that can process uploaded camera trap images through multiple stages of machine learning analysis. This pipeline streamlines the annotation process by automatically detecting objects, classifying species, building sequences, and applying privacy filters. **Pipeline Overview** The AI pipeline consists of four sequential stages that are automatically triggered when new collections are uploaded to classification projects with AI processing enabled: 1. **Sequence Building**: Group related images into meaningful temporal sequences 2. **Object Detection**: Identify and locate animals, humans, and vehicles in images 3. **Privacy Blurring**: Automatically blur human faces and vehicle license plates 4. **Species Classification**: Classify detected animals to species level Each stage builds upon the results of the previous stage, creating a comprehensive automated workflow that significantly reduces manual annotation effort. **Using the Pipeline with Trapper-Tools** The most efficient way to utilize the AI pipeline is through the **trapper-tools pipeline** command, which automates the entire workflow from data preparation to AI processing: .. code-block:: bash # Complete automated workflow: convert β†’ package β†’ upload β†’ trigger AI processing trapper-tools pipeline --data-path /path/to/data --convert --trigger This single command will: - Convert media files to web-friendly formats - Create standardized data packages with metadata - Upload packages securely via FTPS - Trigger automatic AI pipeline processing upon upload **Pipeline Benefits** - **Automated Processing**: Reduces manual annotation effort by 60-80% - **Consistent Quality**: Applies standardized AI models across all images - **Privacy Protection**: Automatically protects sensitive content - **Scalable**: Processes thousands of images efficiently - **Quality Control**: Provides confidence scores and review workflows .. seealso:: For detailed pipeline configuration, see :ref:`configuration` β†’ AI Pipeline Configuration. For AI provider setup and monitoring, see :ref:`administration` β†’ AI Provider and Classification Project Configuration. How to generate Deployments template ==================================== First, ensure your data follows the required directory structure with collections and deployment folders. Then use the helper command to generate a deployment template: .. code-block:: bash trapper-tools helpers template --data-path /path/to/data This analyzes your data structure and creates a CSV template with deployment information extracted from your media files. You can then review and edit this CSV file before importing it into TRAPPER via *Map* > *Deployments* > *Import deployments*. .. note:: For detailed instructions on Trapper Tools installation, configuration, and advanced usage, see https://gitlab.com/trapper-project/trapper-tools. .. _tutorial-upload-tools: Uploading data with Trapper Tools (end-to-end) ++++++++++++++++++++++++++++++++++++++++++++++ This tutorial walks through ingesting a real camera-trap dataset from raw SD-card files to a fully populated Classification Project, using ``trapper-tools`` for everything between the SD card and the Trapper server. Run it once on a small test dataset before applying it to your season's worth of data. Pre-flight checklist ==================== - A *Research Project* exists in TRAPPER (see step 1 of :ref:`tutorial`). - The *Locations* you plan to associate with deployments are imported (see :ref:`tutorial-add-locations`). - Your TRAPPER user account has an FTP account (see :ref:`admin-users` β†’ *Create FTP accounts*). - ``trapper-tools`` is installed and configured against your TRAPPER instance β€” ``trapper-tools configure`` to verify. - ExifTool and FFmpeg are on your ``PATH`` (the helpers package ``trapper-tools[full]`` installs them). Step 1 β€” Lay out the data on disk ================================= The expected directory structure for a single ingestion session is: .. code-block:: text /data/2025_summer/ <-- collection root β”œβ”€β”€ camera_alpha_2025_06_01_2025_06_30/ <-- one directory per deployment β”‚ β”œβ”€β”€ IMG_0001.JPG β”‚ β”œβ”€β”€ IMG_0002.JPG β”‚ └── ... β”œβ”€β”€ camera_alpha_2025_07_01_2025_07_30/ β”‚ └── ... └── camera_beta_2025_06_01_2025_07_15/ β”œβ”€β”€ VID_0001.MP4 └── ... The deployment-folder name is the most important convention: it must match the deployment ID in your TRAPPER database. Names can be free- form (Trapper Tools doesn't impose a date format), but if you let the tool generate the deployments template (see Step 2) it'll suggest ``{location}_{start}_{end}`` automatically. Step 2 β€” Generate a deployments CSV =================================== If you haven't yet imported the deployments for this collection, generate a Camtrap-DP-flavoured CSV template from the raw files: .. code-block:: bash trapper-tools helpers template \ --data-path /data/2025_summer \ --output 2025_summer_deployments.csv The tool walks the directory tree, reads timestamps from EXIF / video metadata, and writes one row per deployment folder with the columns required by the Trapper deployments importer: ``locationID``, ``deploymentID``, ``deploymentStart``, ``deploymentEnd``, ``setupBy``, ``cameraID``, ``cameraModel``, plus optional rows for height / tilt / heading / habitat / feature type. Open the CSV in your favourite editor, fill in the location IDs to match the *Locations* you've already imported, and any optional metadata you have on hand. Save. Import the CSV via *Map β†’ Deployments β†’ Import deployments* in the web UI; see :ref:`tutorial-add-deployments` for the full walkthrough of that page. Step 3 β€” Convert media to web-friendly formats ============================================== Camera-trap files are often inconvenient: full-resolution JPEGs at 8MP+ each, MOV / H.265 videos that don't stream well in browsers. Convert in-place: .. code-block:: bash trapper-tools convert \ --data-path /data/2025_summer \ --max-image-size 1920 \ --video-codec h264 This produces ``.converted/`` subdirectories alongside the originals with web-friendly versions. Originals are not touched. The conversion is idempotent β€” re-running it skips files already converted. Step 4 β€” Package the data ========================= A *package* is a self-contained directory that ``trapper-tools upload`` can ship to TRAPPER. Packaging adds: - A ``manifest.json`` listing every file with its blake3 checksum. - Per-file metadata extracted from EXIF / video container. - Camtrap-DP-aligned deployment / media metadata tables. .. code-block:: bash trapper-tools package \ --data-path /data/2025_summer \ --collection-name "Summer 2025 β€” main grid" \ --research-project-acronym BIALOWIEZA The output is a ready-to-ship package directory under ``/data/2025_summer/.package/``. Step 5 β€” Upload =============== Two transport modes: **FTPS (large bulk uploads).** Most efficient for thousands of files. Uses your Trapper FTP credentials (created via the :ref:`admin-users` action). .. code-block:: bash trapper-tools upload \ --data-path /data/2025_summer \ --transport ftps **HTTP chunked (anything else).** Falls back to the FastAPI uploader at port 8088. Resumable, transparent to firewalls that block FTP. .. code-block:: bash trapper-tools upload \ --data-path /data/2025_summer \ --transport http Both transports verify blake3 checksums after upload and refuse to proceed past a corrupted file. Step 6 β€” Trigger the pipeline ============================= After upload, TRAPPER processes the package automatically: creates ``Resource`` rows, links them to the imported deployments, and β€” if a ``ClassificationProject`` is configured to consume this collection β€” fires the AI pipeline. Watch progress in: - The web UI's *Collection* page (resource count rises as packages are ingested). - The Celery Flower dashboard (``http://localhost:5555``) for AI pipeline task progress. - The *AI Classification Job* admin (one row per submitted job; see :ref:`admin-ai-jobs`). The whole sequence (Steps 3–6) can also be run as a single command: .. code-block:: bash trapper-tools pipeline \ --data-path /data/2025_summer \ --convert \ --transport ftps \ --collection-name "Summer 2025 β€” main grid" \ --research-project-acronym BIALOWIEZA \ --trigger Use this for unattended batch ingestions (e.g. cron jobs after every SD-card swap). .. todo:: Add screenshots: trapper-tools pipeline progress, deployments CSV import view in the Trapper Expert UI, AI pipeline tasks in Flower. .. _tutorial-classification-project-end-to-end: Configuring a classification project end to end +++++++++++++++++++++++++++++++++++++++++++++++ A *Classification Project* is what binds Resources, a Classificator schema, an AI provider stack, and human annotators together. This tutorial walks through standing one up from scratch. You need (or will create here): 1. A *Research Project* you own or manage. 2. At least one *Collection* with Resources. 3. A *Classificator* β€” the schema of attributes annotators fill in. 4. AI Providers, if you want automatic processing (see :ref:`admin-ai-providers`). 5. The Classification Project itself. Step 1 β€” Create or pick a Research Project ========================================== In the top menu, *Research β†’ Add research project*. Fill in name, acronym (3+ chars), keywords, and the sampling design fields. Your request goes to an admin for approval. Once approved, the project appears in *Research projects* with you as the owner. Step 2 β€” Add a Collection ========================= A Collection is a grouped set of Resources, typically the contents of one ingestion session. From the Research Project page: #. Click *Collections β†’ Add collection*. #. Set the name, owner, and Research-Project link. #. Save. To populate it: either import via :ref:`tutorial-upload-tools` (the ``--collection-name`` flag wires Resources into a freshly named collection) or attach already-uploaded resources via the *Add resources* action on the Collection page. Step 3 β€” Build a Classificator ============================== A Classificator (``Media Classification β†’ Classificators β†’ Add``) is the schema of fields annotators fill in for each detected object. See :ref:`tutorial-create-classificator` for the full step-by-step. If you have an AI provider with a species mapping ready, the fastest way to build a Classificator is the **Create classificator from species mapping** (``create_classificator_from_species_mapping``) admin action β€” select the AIProvider row, run the action, and the resulting Classificator has the species list pre-populated. Step 4 β€” Configure AI Providers =============================== If your project will run automatic detection / species classification: #. Make sure the AI Manager has the model registered. Run ``sync_models_from_schemas`` on the AI Manager and ``sync_ai_models`` on the Expert (the admin wizard does this on first deploy). #. Verify both providers are present and ``is_active=True`` in the Expert admin (``Media Classification β†’ AI Providers``). #. If you want manual edits to ``categories`` or thresholds to survive future syncs, untick **Keep synced**. Full background: :ref:`admin-ai-providers`. Step 5 β€” Create the Classification Project ========================================== Navigate to *Media Classification β†’ Classification Projects β†’ Add*. Required: - **Name** and **Description**. - **Research Project** β€” the parent you created in step 1. - **Classificator** β€” from step 3. - **Collections** β€” the resource pool to classify. - **Owner / managers** β€” typically yourself plus team leads. AI configuration (optional but recommended): - **Object Detection AI Model** β€” e.g. *MegaDetector v6*. - **Species AI Model** β€” e.g. *DeepFaune v1.3*. - **Required AI** β€” enable to require AI to finish before annotators can work. - **Copy AI classifications** β€” enable to auto-fork AI Classifications into editable USER Classifications when annotators start. - **Species matching IOU threshold** β€” 0.5 default; loosen for small / partly-occluded animals. - **Confidence warning thresholds** β€” visual flags only, don't change workflow. - **Blur humans / vehicles** β€” enable for projects on public sites. Privacy / annotation policy: - **Exclude humans / blank** β€” hide these from the citizen science frontend. - **Hide classification attributes for non-animals** β€” simpler form when ``observation_type β‰  animal``. Save. Step 6 β€” Assign annotators ========================== In the Classification Project page, *Members β†’ Add* β€” pick the users and their role (annotator / reviewer / manager). For project-wide bulk role assignment, use the :ref:`set-roles ` admin action on the User changelist. Step 7 β€” Verify =============== Open the project page in the citizen-science frontend (or in the Expert classify view). You should see: - The collections you attached, with counts. - AI Classifications populated on resources (after the pipeline finishes β€” watch Flower at port 5555). - The classification form rendered with your Classificator's fields. .. todo:: Add screenshots: Classification Project add form, AI configuration fieldset, member assignment. .. _tutorial-ai-providers-bootstrap: Bootstrapping AI providers (manual path) ++++++++++++++++++++++++++++++++++++++++ The :ref:`installation` admin wizard registers the bundled AI providers automatically. You only need this tutorial when: - You added a brand-new model to ``trapper-schemas`` and want to wire it up before the next release. - You're testing an external / experimental model not in the schemas catalog. - You want to point at a second AI Manager instance (e.g. a GPU box on a different network). Step 1 β€” Configure the AI Manager connection ============================================ In the Expert admin, *Media Classification β†’ AI Provider Connections β†’ Add*. Fill in: - **API URL** β€” base URL of your AI Manager (e.g. ``https://ai.example.com``). - **API Auth Login** / **Password** β€” credentials of the AI Worker user on the AI Manager. - **Trapper Instance URL** β€” base URL of *this* Expert (used for callback URLs sent to the AI Manager). - **Is active** β€” true. Step 2 β€” Register the model on the AI Manager ============================================= If the model lives in ``trapper-schemas``, just run on the AI Manager: .. code-block:: bash ./admin.py manage trapper-ai-web sync_models_from_schemas -p Otherwise, create the ``PredictionModel`` row by hand in the AI Manager admin: upload the weights file, set ``predictor_class`` to one of the registered classes (see ``trapper-ai-worker``), set ``model_file_hash`` to the SHA-256 of the weights, and toggle ``is_active=True``. Step 3 β€” Pull the catalog into the Expert ========================================= .. code-block:: bash ./admin.py manage trapper sync_ai_models -p The command pulls the AI Manager's ``PredictionModel`` list, creates or updates AIProvider rows (matched by ``model_file_hash``), copies the bundled ``trapper-schemas`` ``categories`` JSON onto each provider, and resolves ``gbifSpeciesKey`` β†’ ``Species.taxon_id`` for every species AttributeLabel. If the model is *not* in ``trapper-schemas``, you'll need to hand-author the ``categories`` JSON in the Expert admin (see :ref:`admin-ai-providers` for the format). Step 4 β€” Smoke-test =================== #. Upload a small batch of test images to a collection that's wired into a ClassificationProject pointing at the new provider. #. Watch the *AI Classification Job* row appear and progress through stages. #. After completion, open one of the resources in the classify view and verify AI Classifications + bounding boxes. #. Use the :ref:`reparse-results ` action if anything went wrong with parsing β€” it lets you fix the ``categories`` JSON and re-run post-processing without redoing inference. .. _tutorial-distance-calibration: Distance calibration & estimation walkthrough +++++++++++++++++++++++++++++++++++++++++++++ Estimating the distance from each detected object to the camera lets you compute density estimators (REM, capture-recapture distance sampling, …) directly from camera-trap data. The calibration is per-camera-and-lens-combination, so you typically do it once per piece of hardware and then reference it from every deployment using that hardware. Concept refresher: see :ref:`configuration-distance` for what the ``CalibrationModel`` and ``DistanceEstimationConfig`` actually do. Step 1 β€” Capture calibration footage ==================================== In the field, set up the camera the way you'll deploy it (same height, tilt, lens). Place a target with a clear visual marker (a tripod with flagging tape works) at known distances β€” e.g. 2 m, 5 m, 10 m, 15 m, 20 m. Capture 5–10 photos / a short video at each distance. Record each ``(distance, target visual ID)`` pair in your field notebook. A handful of distinct distances (5–8) is plenty. The calibration is a function-fit, not a brute-force lookup. Step 2 β€” Create a calibration deployment ======================================== In the Expert UI: #. *Map β†’ Deployments β†’ Add deployment*. #. Fill in the camera + location as usual. #. Tick **Is calibration**. #. Save. The "is calibration" flag is what tells the depth pipeline to fit a model from this deployment instead of consuming one. Step 3 β€” Upload the calibration footage ======================================= Use any of the upload paths (CS frontend, ``trapper-tools``, FTPS). The resources land in the calibration deployment. Step 4 β€” Mark reference points ============================== In the calibration-deployment view, open each resource and use the *Reference points* annotation tool to: - Click the visual marker on the target in the image. - Enter the known distance in metres. Repeat for every distance you captured. The tool stores ``(image_pixel_xy, world_distance_m)`` pairs on the Deployment row. .. todo:: Add a screenshot of the reference-points annotation UI showing the marker overlay and the distance entry field. Step 5 β€” Fit the calibration ============================ In the Deployment edit form's *Calibration* fieldset, configure: - **Method** β€” start with ``linear`` for typical fixed-focal cameras. Switch to ``curve_knots`` if the linear fit residuals are high. - **Alignment** β€” ``mean`` is a sensible default. Save the form. Then, in the Deployment changelist, select the calibration deployment and run the **Rerun depth calibration** (``rerun_depth_calibration``) admin action. This invokes the depth model on each reference image and fits the ``CalibrationModel`` parameters. Step 6 β€” Apply to a real deployment =================================== In a non-calibration Deployment that uses the same camera + lens: #. Open the Deployment edit form. #. Set the **Calibration deployment** FK to the calibration row from step 2. #. Configure the **Distance estimation** fieldset β€” the ~20-field group described in :ref:`configuration-distance`. Defaults are reasonable; the field most often worth tuning per project is ``smoothing_window`` (try 3 / 5 / 9 frames). #. Save. Step 7 β€” Run distance estimation ================================ For new resources, the AI pipeline runs distance estimation automatically once the depth-model AI Provider is set on the ClassificationProject. For existing resources, select the deployment(s) in the changelist and run the **Rerun distance estimation** (``rerun_distance_estimation``) admin action. The action's form lets you override the estimation parameters one-shot for the run, which is useful when iterating on ``smoothing_window`` or Kalman tuning. Step 8 β€” Verify =============== Open a resource and check the per-object distance values rendered in the classification panel. The exact metric is "median distance across the smoothed track"; you'll see it as a numeric column on each ``ClassificationDynamicAttrs`` row and in the per-frame chart in the expert UI. A useful sanity check: pick the calibration footage *itself*, run the estimation pipeline against it, and verify the predicted distance matches the ground truth at each marker. If results look off, common causes: - **Calibration deployment used the wrong camera.** Re-shoot. - **Reference points only at one or two distances.** Add more. - **Linear fit on a non-linear lens.** Switch to ``curve_knots``. - **High inter-frame jitter.** Increase ``smoothing_window``. - **SAM mask leaks into background.** Lower ``sam_threshold`` or set ``sam_use_best_box=true``. For cross-system context, see also: depth-model schemas in ``trapper-schemas/src/trapper_schemas/depth/`` and predictor implementations in ``trapper-ai-worker/packages/predictors-torch-depth/`` and ``trapper-ai-worker/packages/predictors-hailo-depth/``.