Actinidia Dataset Preprocessing

Preprocessing scripts for the Multi-Modal Actinidia chinensis Phenology Dataset.

This repository contains Python utilities for standardizing agricultural imaging datasets, enabling reproducible dataset preparation workflows for phenological stage detection in kiwifruit orchards.

---

Table of Contents

• Overview
• Project Status
• Technology Stack
• Dependencies
• Installation Instructions
• Architecture
• Usage
• Repository Structure
• Dataset Structure
• Known Issues
• Documentation and Resources
• Citation
• Contributors
• Acknowledgments
• Contact
• License

---

Overview

This repository provides two command-line utilities for dataset preparation:

• Image Resolution Standardization: Normalizes images to 1024×1024 pixels using LANCZOS resampling
• GPX Track Generation: Creates georeferenced GPS tracks for video data through linear interpolation

These scripts support the reproducibility of the dataset creation workflow described in:

Pinheiro, I., et al. (2026). A Multi-Modal Dataset for Automated Phenological Stage Mapping in Actinidia chinensis. Scientific Data (under review).

The dataset addresses the challenge of automated phenological stage detection in kiwifruit orchards, enabling precision agriculture applications for optimal harvest timing and yield prediction.

---

Project Status

Status: Production-ready and stable

This project is completed. The scripts were successfully used to prepare the Multi-Modal Actinidia chinensis Phenology Dataset currently under review at Scientific Data (Nature). The codebase is stable, tested, and ready for reuse in similar agricultural dataset preparation workflows.

---

Technology Stack

Component	Technology	Version
Language	Python	3.8+
Image Processing	Pillow	≥ 10.0.0
Video Processing	OpenCV	≥ 4.8.0
Progress Visualization	tqdm	≥ 4.65.0
Output Formats	JPEG, GPX	-

Type: Standalone command-line utilities for dataset preprocessing

---

Dependencies

Install all required dependencies using:

pip install -r requirements.txt

Core dependencies:

• Python 3.8+
• Pillow >= 10.0.0
• tqdm >= 4.65.0
• opencv-python >= 4.8.0

Optional system dependency:

• ffmpeg: Fallback for video duration extraction if OpenCV fails
  • Ubuntu/Debian: sudo apt install ffmpeg
  • macOS: brew install ffmpeg
  • Windows: Download from ffmpeg.org

---

Installation Instructions

Follow these steps to set up the project in your local environment.

1. Clone the repository

   git clone https://github.com/INESCTEC/actinidia-dataset-preprocessing.git
   cd actinidia-dataset-preprocessing

2. Create and activate a virtual environment (recommended)

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies

   pip install -r requirements.txt

4. Verify installation

   python scripts/resize_images.py --help
   python scripts/generate_gpx.py --help

---

Architecture

Data Processing Workflow

Raw Dataset
    │
    ├─> Images (various resolutions)
    │        │
    │        └─> scripts/resize_images.py ──> Standardized Images (1024×1024 JPEG)
    │
    └─> Videos (with start/end coordinates)
             │
             └─> scripts/generate_gpx.py ──> Georeferenced GPX Tracks

Processing Pipeline

1. Image Standardization: LANCZOS resampling maintains high quality while ensuring consistent 1024×1024 resolution
2. GPX Generation: Linear interpolation between start/end coordinates creates georeferenced tracks synchronized with video timestamps
3. Quality Control: Configurable JPEG quality (default: 95) and track point intervals (default: 1s)

---

Usage

1. Image Resolution Standardization

Standardizes images to 1024×1024 pixels for dataset consistency using LANCZOS resampling.

Basic usage:

python scripts/resize_images.py --input path/to/original/images --output path/to/resized/images

Options:

• --input, -i: Input directory containing original images (required)
• --output, -o: Output directory for resized images (required)
• --size, -s: Target resolution in pixels (default: 1024, creates square images)
• --quality, -q: JPEG quality 1-100 (default: 95)

Example:

python scripts/resize_images.py -i ./raw_images -o ./processed_images -s 1024 -q 95

Supported formats: .jpg, .jpeg, .png, .bmp

---

2. GPX Track Generation

Generates georeferenced GPX tracks for video files using linear interpolation between start and end coordinates.

Basic usage:

python scripts/generate_gpx.py --coords coordinates.txt --videos video1.mp4 video2.mp4 video3.mp4

Options:

• --coords, -c: Path to coordinates file (required)
• --videos, -v: Video files, space-separated list (required)
• --output, -o: Output directory for GPX files (default: ./gpx_tracks)
• --interval, -i: Time interval between track points in seconds (default: 1.0)
• --prefix, -p: Prefix for output filenames (default: track)

Coordinates file format:

Each line contains one coordinate pair (start_lat start_lon end_lat end_lon):

41.291389 -8.661667 41.291500 -8.661500
41.126077 -8.548733 41.126200 -8.548600
41.054937 -8.587750 41.055000 -8.587600

Example:

python scripts/generate_gpx.py -c coordinates.txt -v zone1.mp4 zone2.mp4 zone3.mp4 -o ./gpx_output -p video_track

Output: Creates video_track_1.gpx, video_track_2.gpx, video_track_3.gpx with WGS84 coordinates and timestamps.

---

Complete Workflow Example

# 1. Standardize images
python scripts/resize_images.py -i ./original_images -o ./dataset/images -s 1024

# 2. Generate GPX tracks for videos
python scripts/generate_gpx.py -c field_coordinates.txt -v *.mp4 -o ./dataset/gpx_tracks

---

Repository Structure

Folder/File	Description
/scripts	Preprocessing utilities
/scripts/resize_images.py	Image standardization script
/scripts/generate_gpx.py	GPX track generation script
/docs	Documentation and institutional logos
README.md	Project documentation
LICENSE	MIT License
.gitignore	Git ignore rules
requirements.txt	Python dependencies

---

Dataset Structure

The preprocessing scripts prepare data for the following structure:

Multi-Modal Actinidia chinensis Phenology Dataset/
├── Labelled_Images_Component/
│   ├── Images/              (1,665 JPEG files, 1024×1024 pixels)
│   └── Annotations/         (Pascal VOC XML files)
└── Georeferenced_Videos_Component/
    ├── Videos/              (24 MP4 files, 3840×2160 pixels)
    ├── Coordinates/         (24 GPX files)
    └── Validation/          (4 Excel files with ground truth counts)

Complete dataset available at: Zenodo

---

Known Issues

• Video duration extraction: If OpenCV fails to read video metadata, the script automatically falls back to ffprobe, which must be installed separately (sudo apt install ffmpeg on Ubuntu/Debian)
• Large batch processing: For datasets with thousands of images, consider processing in smaller batches to avoid potential memory issues on systems with limited RAM
• Coordinate precision: GPX generation uses linear interpolation, which is suitable for relatively straight paths but may not accurately represent curved trajectories
• JPEG quality vs. file size: The default quality of 95 provides excellent visual quality but produces larger files. Adjust the --quality parameter based on storage/bandwidth constraints

---

Documentation and Resources

• Dataset Paper: Pinheiro, I., et al. (2026). A Multi-Modal Dataset for Automated Phenological Stage Mapping in Actinidia chinensis. Scientific Data (under review).
• Dataset Repository: Zenodo DOI: 10.5281/zenodo.17371025
• Image Annotation Tool: CVAT
• Image Uniqueness Detection: Fiftyone

---

Citation

If you use this code or dataset, please cite:

@article{pinheiro2026actinidia,
  title={A Multi-Modal Dataset for Automated Phenological Stage Mapping in Actinidia chinensis},
  author={Pinheiro, Isabel and Moura, Pedro and Rodrigues, Leandro and Moreira, Germano and Coutinho, Rui Manuel and Terra, Francisco and Valente, António and Cunha, Mário and Santos, Filipe Neves dos},
  journal={Scientific Data},
  year={2026},
  note={under review}
}

---

Contributors

We thank the following team members for their contributions to this project:

Name	Affiliation	Email
Isabel Pinheiro	INESC TEC, UTAD	[email protected]
Pedro Moura	INESC TEC, UTAD	[email protected]
Leandro Rodrigues	INESC TEC, FCUP	[email protected]
Germano Moreira	INESC TEC, FCUP	[email protected]
Rui Manuel Coutinho	INESC TEC, FEUP	[email protected]
Francisco Terra	INESC TEC	[email protected]
António Valente	UTAD, INESC TEC	[email protected]
Mário Cunha	FCUP, INESC TEC	[email protected]
Filipe Neves dos Santos	INESC TEC, FEUP	[email protected]

---

Acknowledgments

This prototype was funded and developed under the following projects:

• WATSON - A holistic framework with anticounterfeit and intelligence-based technologies that will assist food chain stakeholders in rapidly identifying and preventing the spread of fraudulent practices.

               

---

Contact

For support, questions, or collaboration inquiries:

Isabel Pinheiro
INESC TEC & UTAD
Email: [email protected]

You may also open an issue in this repository for bug reports or feature requests.

---

License

This project is licensed under the MIT License - see the LICENSE file for details.

Copyright (c) 2025 INESC TEC

---