-
Couldn't load subscription status.
- Fork 2.9k
feat: bi-so101, v3.0->v2.1->v2.0, xlerobot, teleops (ar, joycon, gamepads), mjpeg, #2248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Co-authored-by: Copilot <[email protected]> Signed-off-by: Yihao Liu <[email protected]>
Initial supports for ar teleop
V3 0 to v2 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request adds comprehensive support for bimanual SO-101 robot configurations, teleoperators (including AR, Joy-Con, and gamepad controllers), and the XLerobot combined platform. It also includes schema downgrade utilities from v3.0 to v2.1 for compatibility reasons, MJPEG camera support, and training configuration optimizations.
Key changes:
- Adds bimanual SO-101 follower and leader support for dual-arm robotic setups
- Introduces XLerobot platform combining SO-101 arms with LeKiwi mobile base
- Implements multiple teleoperation options: AR controller, Joy-Con controllers, and gamepad support
- Includes dataset schema conversion utilities for v3.0 to v2.1 compatibility
Reviewed Changes
Copilot reviewed 67 out of 67 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/lerobot/teleoperators/xlerobot_mount_idle/ | Static teleoperator for XLerobot mount joints |
| src/lerobot/teleoperators/xlerobot_leader_gamepad/ | Composite teleoperator combining SO-101 arms with gamepad base control |
| src/lerobot/teleoperators/xlerobot_joycon_gamepad/ | Composite teleoperator combining Joy-Con arms with gamepad base control |
| src/lerobot/teleoperators/lekiwi_base_joycon/ | Joy-Con controller for LeKiwi base movement |
| src/lerobot/teleoperators/lekiwi_base_gamepad/ | Gamepad controller for LeKiwi base movement |
| src/lerobot/teleoperators/bi_so101_leader/ | Bimanual SO-101 leader arm configuration |
| src/lerobot/teleoperators/bi_joycon/ | Bimanual Joy-Con teleoperation support |
| src/lerobot/teleoperators/bi_gamepad/ | Bimanual gamepad teleoperation support |
| src/lerobot/teleoperators/ar_controller/ | AR-based teleoperation controller |
| src/lerobot/robots/xlerobot/ | Combined XLerobot platform implementation |
| src/lerobot/robots/xlerobot_mount/ | XLerobot camera mount robot implementation |
| src/lerobot/robots/lekiwi_base/ | LeKiwi mobile base robot implementation |
| src/lerobot/robots/bi_so101_follower/ | Bimanual SO-101 follower robot implementation |
| src/lerobot/datasets/v30/convert_dataset_v30_to_v21.py | Dataset schema conversion utility |
| src/lerobot/cameras/opencv/ | MJPEG camera capture support |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| for cam in self.cameras.values(): | ||
| try: | ||
| cam.disconnect() | ||
| except Exception: | ||
| logger.warning("Failed to disconnect camera", exc_info=True) |
Copilot
AI
Oct 18, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duplicate camera disconnection loop. The second loop (lines 190-194) is identical to the first (lines 185-189) and should be removed.
| for cam in self.cameras.values(): | |
| try: | |
| cam.disconnect() | |
| except Exception: | |
| logger.warning("Failed to disconnect camera", exc_info=True) |
| # if self._left_joycon is not None: | ||
| # try: | ||
| # self._left_joycon.disconnect() | ||
| # except Exception: # pragma: no cover - hardware cleanup | ||
| # pass | ||
| # self._left_joycon = None | ||
|
|
||
| # if self._right_joycon is not None: | ||
| # try: | ||
| # self._right_joycon.disconnect() | ||
| # except Exception: # pragma: no cover - hardware cleanup | ||
| # pass | ||
| # self._right_joycon = None | ||
|
|
||
| # self._last_update = None | ||
| # logger.info("JoyCon teleoperator disconnected.") | ||
| pass |
Copilot
AI
Oct 18, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The disconnect method is commented out and only contains 'pass'. This leaves resources potentially uncleaned. Either implement proper cleanup or document why it's intentionally disabled.
| # if self._left_joycon is not None: | |
| # try: | |
| # self._left_joycon.disconnect() | |
| # except Exception: # pragma: no cover - hardware cleanup | |
| # pass | |
| # self._left_joycon = None | |
| # if self._right_joycon is not None: | |
| # try: | |
| # self._right_joycon.disconnect() | |
| # except Exception: # pragma: no cover - hardware cleanup | |
| # pass | |
| # self._right_joycon = None | |
| # self._last_update = None | |
| # logger.info("JoyCon teleoperator disconnected.") | |
| pass | |
| if self._left_joycon is not None: | |
| try: | |
| self._left_joycon.disconnect() | |
| except Exception: # pragma: no cover - hardware cleanup | |
| pass | |
| self._left_joycon = None | |
| if self._right_joycon is not None: | |
| try: | |
| self._right_joycon.disconnect() | |
| except Exception: # pragma: no cover - hardware cleanup | |
| pass | |
| self._right_joycon = None | |
| self._last_update = None | |
| logging.info("JoyCon teleoperator disconnected.") |
| # if self._joycon is not None: | ||
| # try: | ||
| # self._joycon.disconnect() | ||
| # except Exception: # pragma: no cover - hardware cleanup | ||
| # logger.debug("Failed to disconnect JoyCon cleanly.", exc_info=True) | ||
| # self._joycon = None | ||
| pass | ||
|
|
Copilot
AI
Oct 18, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The disconnect method is commented out and only contains 'pass'. This leaves JoyCon resources potentially uncleaned. Either implement proper cleanup or document why it's intentionally disabled.
| # if self._joycon is not None: | |
| # try: | |
| # self._joycon.disconnect() | |
| # except Exception: # pragma: no cover - hardware cleanup | |
| # logger.debug("Failed to disconnect JoyCon cleanly.", exc_info=True) | |
| # self._joycon = None | |
| pass | |
| if self._joycon is not None: | |
| try: | |
| self._joycon.disconnect() | |
| except Exception: # pragma: no cover - hardware cleanup | |
| logger.debug("Failed to disconnect JoyCon cleanly.", exc_info=True) | |
| self._joycon = None |
| result = subprocess.run( | ||
| cmd, | ||
| check=True, | ||
| timeout=300, # 5 minute timeout | ||
| capture_output=True, | ||
| text=True | ||
| ) |
Copilot
AI
Oct 18, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The subprocess call executes ffmpeg with user-controlled file paths. While validation exists in _validate_video_paths(), consider using shlex.quote() on the file path arguments to prevent potential shell injection if paths contain special characters.
Feat/xlerobot mount hardware
Mount and exit handling
What this does