Apriltag_calibration #542
Apriltag_calibration #542
Conversation
- Created a dummy package (`dummy_pkg`) to facilitate testing on non-RPi OS. - Added `README.md`, `pyproject.toml`, `requirements.txt`, and `setup.cfg` for package configuration. - Implemented mock classes for `libcamera` and `picamera2` to simulate camera functionality. - Developed tests for the dummy package to ensure basic functionality. - Introduced `detect_apriltag.py` script for detecting AprilTags in images using camera intrinsics. - Added YAML configuration for camera intrinsics and updated requirements for additional dependencies. - Included example images and JSON output for pose corrections.
- Implemented `collect_hand_eye_data.py` for collecting calibration data by detecting AprilTags and saving robot poses. - Created `hand_eye_calibration.py` to perform hand-eye calibration using collected data, including saving and transforming poses. - Added functionality for loading, saving, and managing calibration points in JSON format. - Included example data generation for ease of use and testing.
There was a problem hiding this comment.
Pull request overview
This PR implements AprilTag-based hand-eye calibration functionality for the A1 Mini robot arm camera system. It provides tools for camera intrinsic calibration, AprilTag detection with 6DOF pose estimation, and hand-eye calibration to determine the transformation between camera and end-effector coordinates.
Key Changes
- Hand-eye calibration pipeline using OpenCV's
calibrateHandEyemethod - AprilTag detection with pupil-apriltags library for 6DOF pose estimation
- Camera intrinsic calibration using checkerboard patterns
- Data collection utilities for gathering calibration point pairs
Reviewed changes
Copilot reviewed 10 out of 69 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
hand_eye_calibration.py |
Main calibration logic with AX=XB formulation, data persistence, and pose transformation |
detect_apriltag.py |
AprilTag detection with camera intrinsics, pose estimation, and JSON output |
collect_hand_eye_data.py |
Helper script to pair robot poses with AprilTag detections for calibration |
calibrate_camera.py |
Camera intrinsic calibration using checkerboard images |
requirements.txt |
Added opencv-python, PyYAML, and numpy dependencies |
config/a1_intrinsics*.yaml |
Camera calibration parameter files |
pose_new_calibration.json |
Example AprilTag detection output data |
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.
| if __name__ == "__main__": | ||
| main() No newline at end of file |
There was a problem hiding this comment.
The if __name__ == "__main__" pattern should be avoided in package code according to the custom coding guidelines. Consider removing this block and leaving the main function call as a top-level script, or moving this logic to a separate script file.
| if __name__ == "__main__": | ||
| main() No newline at end of file |
There was a problem hiding this comment.
The if __name__ == "__main__" pattern should be avoided in package code according to the custom coding guidelines. Consider removing this block and leaving the main function call as a top-level script.
| if __name__ == "__main__": | ||
| main() No newline at end of file |
There was a problem hiding this comment.
The if __name__ == "__main__" pattern should be avoided in package code according to the custom coding guidelines. Consider removing this block and leaving the main function call as a top-level script.
| if __name__ == "__main__": | ||
| main() No newline at end of file |
There was a problem hiding this comment.
The if __name__ == "__main__" pattern should be avoided in package code according to the custom coding guidelines. Consider removing this block and leaving the main function call as a top-level script.
| def save_pose_to_json(detections, output_path): | ||
| """Save 6DOF pose data to JSON file""" | ||
| pose_data = { | ||
| "timestamp": str(pd.Timestamp.now()) if 'pd' in globals() else str(datetime.now()), |
There was a problem hiding this comment.
The variable pd is checked in globals() but is never imported. This will always fall back to using datetime.now(). If pandas timestamp is desired, add import pandas as pd at the top of the file. Otherwise, simplify this to just use str(datetime.now()).
| } | ||
|
|
||
| self.calibration_data.append(calibration_point) | ||
| print(f"✅ Added calibration point {len(self.calibration_data)}") |
There was a problem hiding this comment.
Print statements should be used sparingly according to the custom coding guidelines. Consider using a proper logging framework instead of print statements for production code, or remove unnecessary print statements if they're only for debugging.
| import json | ||
| import yaml | ||
| from scipy.spatial.transform import Rotation as R | ||
| from scipy.optimize import least_squares |
There was a problem hiding this comment.
Import of 'least_squares' is not used.
sgbaird
left a comment
sgbaird
left a comment
There was a problem hiding this comment.
@copilot Lots of fallbacks that should be removed. Use your perplexity mcp tool to look up guides on how to do apriltag pose detection. cross-check your math against other sources. We want to make sure we're not missing something basic with the math
| try: | ||
| from pupil_apriltags import Detector | ||
| PUPIL_APRILTAGS_AVAILABLE = True | ||
| except ImportError: |
| with open(config_path, 'r') as f: | ||
| calib = yaml.safe_load(f) | ||
| return calib | ||
| except FileNotFoundError: |
| Detect AprilTag using the pupil-apriltags library. | ||
| """ | ||
| if not PUPIL_APRILTAGS_AVAILABLE: | ||
| return None |
| img = cv2.imread(image_path) | ||
| if img is None: | ||
| print(f"❌ Could not load image: {image_path}") | ||
| return None |
| camera_params = [camera_matrix[0,0], camera_matrix[1,1], camera_matrix[0,2], camera_matrix[1,2]] | ||
|
|
||
| # Try different tag families | ||
| families_to_try = ['tag36h11', 'tagStandard41h12', 'tag25h9', 'tag16h5'] |
There was a problem hiding this comment.
I'd rather just specify the family in advance
- Enhanced detect_apriltag.py with improved detection and coordinate handling - Added transform_coordinates.py for coordinate system transformations - Added camera calibration configuration file for AC lab setup
- Added img1.png as main test image - Updated img1_flipped_vertical.png - Added img1_detection_corrected.json with corrected detection results - Removed old img2 test files and reports for cleanup
collect_hand_eye_data.pyfor collecting calibration data by detecting AprilTags and saving robot poses.hand_eye_calibration.pyto perform hand-eye calibration using collected data, including saving and transforming poses.