Add AprilTag detection and hand-eye calibration system for A1 Mini camera

[Copilot]

Completes integration of AprilTag detection with 6DOF pose estimation and hand-eye calibration for robotic arm control. The core implementation (detection, calibration, and coordinate transformation scripts) was already present from previous commits - this PR adds missing dependencies, fixes path references, and provides comprehensive documentation.

Changes

• Dependencies: Added pupil-apriltags and scipy to requirements.txt for AprilTag detection and rotation transformations
• Path fixes: Updated collect_hand_eye_data.py default paths from images/ to apriltag_images/ to match actual directory structure
• Documentation: Added comprehensive README.md with usage examples for detection, camera calibration, and hand-eye calibration workflows
• Changelog: Documented AprilTag detection (6DOF pose, multiple tag families), camera calibration (0.27px RMS error), and hand-eye calibration (OpenCV CALIB_HAND_EYE_TSAI) features

Usage

# Detect AprilTag with 6DOF pose
python detect_apriltag.py apriltag_images/img1_flipped_vertical.png \
  --tag-size 0.03 --save-json pose.json

# Collect hand-eye calibration data
python collect_hand_eye_data.py --robot-pose 0.15 0.0 0.25 0.0 0.0 0.0

# Compute camera-to-end-effector transformation
python hand_eye_calibration.py calibrate hand_eye_calibration_data.json

The system enables robot arm control based on visual AprilTag detection by providing transformation from camera coordinates to robot base coordinates.

Original prompt

Add AprilTag Detection and Hand-Eye Calibration for A1 Mini Camera

Summary

Added complete AprilTag detection and hand-eye calibration system for the A1 Mini camera setup.

Features Added

🏷️ AprilTag Detection System

• Enhanced AprilTag Detection: detect_apriltag.py with 6DOF pose estimation (x,y,z position + roll,pitch,yaw orientation)
• Camera Calibration: calibrate_camera.py for accurate intrinsics using checkerboard patterns
• JSON Output: Complete pose data export for robot integration

🤖 Hand-Eye Calibration System

• Hand-Eye Calibration: hand_eye_calibration.py implementing AX=XB algorithm with OpenCV
• Data Collection Helper: collect_hand_eye_data.py to streamline calibration data gathering
• Camera-to-Robot Transformation: Converts AprilTag detections from camera coordinates to robot arm coordinates

Key Improvements

📷 Camera Calibration Accuracy

• Fixed Distance Measurements: Improved from 133cm error to 39.5cm using proper 11x8 checkerboard calibration
• New Intrinsics: Updated camera matrix with corrected focal lengths (fx: 490.9, fy: 507.4)
• Calibration Quality: Achieved 0.27px RMS reprojection error (excellent quality)

🎯 Detection Capabilities

• Multiple Tag Families: Support for tag36h11, tagStandard41h12, tag25h9, tag16h5
• Robust Detection: Handles image orientation issues with automatic family testing
• Complete 6DOF Pose: Position and orientation output in camera coordinates
• 3cm Tag Support: Configured for actual AprilTag size

Files Added/Modified

Core Detection & Calibration

• src/ac_training_lab/a1_cam/detect_apriltag.py - Main AprilTag detection with 6DOF pose
• src/ac_training_lab/a1_cam/calibrate_camera.py - Camera calibration for 11x8 checkerboard
• src/ac_training_lab/a1_cam/config/a1_intrinsics.yaml - Updated camera intrinsics
• src/ac_training_lab/a1_cam/config/a1_intrinsics_backup.yaml - Original calibration backup

Hand-Eye Calibration System

• src/ac_training_lab/a1_cam/hand_eye_calibration.py - Core hand-eye calibration algorithm
• src/ac_training_lab/a1_cam/collect_hand_eye_data.py - Data collection workflow helper

Test Data & Results

• src/ac_training_lab/a1_cam/pose_new_calibration.json - Sample detection results with new calibration
• src/ac_training_lab/a1_cam/calibration_images/ - 15 checkerboard calibration images (1.png - 15.png)
• src/ac_training_lab/a1_cam/images/img1_flipped_vertical.png - AprilTag test image

Usage Examples

AprilTag Detection

python detect_apriltag.py images/img1_flipped_vertical.png --tag-size 0.03 --save-json pose_data.json --visualize

Camera Recalibration

python calibrate_camera.py  # Uses 11x8 checkerboard pattern

Hand-Eye Calibration Data Collection

python collect_hand_eye_data.py --robot-pose 0.15 0.0 0.25 0.0 0.0 0.0

Hand-Eye Calibration Execution

python hand_eye_calibration.py calibrate hand_eye_calibration_data.json

Technical Details

• Detection Library: pupil-apriltags for robust tag detection
• Calibration Method: OpenCV checkerboard-based camera calibration
• Hand-Eye Algorithm: OpenCV's CALIB_HAND_EYE_TSAI method
• Coordinate Systems: Camera coordinates → Robot base coordinates transformation
• Output Formats: JSON for pose data, YAML for calibration parameters

Testing Results

• ✅ Tag Detection: Successfully detects tag36h11 ID=2 with complete 6DOF pose
• ✅ Camera Calibration: 15/15 calibration images processed with 0.27px error
• ✅ Distance Accuracy: Improved from 133cm to 39.5cm measurement (target: 15cm)
• ✅ Orientation Handling: Robust detection across image orientations

This implementation provides a complete pipeline from camera calibration through AprilTag detection to robot coordinate transformation, enabling accurate robot arm control based on visual AprilTag detection.

Created from VS Code via the GitHub Pull Request extension.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.