Skip to content

Conversation

@bingogome
Copy link

What this does

  • Supports bimanual SO-101 configurations out-of-the-box
  • Adds XLerobot combined platform (bi_so101 arms + LeKiwi base)
  • Uses dataset schema v2.1 (downshifted from upstream v3.0 for compatibility)
    • If you need to convert to v2.0 for GR00T, check out a fork of any4lerobot where I have an implementation for v2.1 to v2.0
  • Enables fast MJPEG capture via camera fourcc=mjpeg
  • Verified with the AR teleoperation controller
  • Verified with Nintendo Switch Joy-Con controller integration
  • Verified with Quest 3 AR control

Copilot AI review requested due to automatic review settings October 18, 2025 23:47
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request adds comprehensive support for bimanual SO-101 robot configurations, teleoperators (including AR, Joy-Con, and gamepad controllers), and the XLerobot combined platform. It also includes schema downgrade utilities from v3.0 to v2.1 for compatibility reasons, MJPEG camera support, and training configuration optimizations.

Key changes:

  • Adds bimanual SO-101 follower and leader support for dual-arm robotic setups
  • Introduces XLerobot platform combining SO-101 arms with LeKiwi mobile base
  • Implements multiple teleoperation options: AR controller, Joy-Con controllers, and gamepad support
  • Includes dataset schema conversion utilities for v3.0 to v2.1 compatibility

Reviewed Changes

Copilot reviewed 67 out of 67 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/lerobot/teleoperators/xlerobot_mount_idle/ Static teleoperator for XLerobot mount joints
src/lerobot/teleoperators/xlerobot_leader_gamepad/ Composite teleoperator combining SO-101 arms with gamepad base control
src/lerobot/teleoperators/xlerobot_joycon_gamepad/ Composite teleoperator combining Joy-Con arms with gamepad base control
src/lerobot/teleoperators/lekiwi_base_joycon/ Joy-Con controller for LeKiwi base movement
src/lerobot/teleoperators/lekiwi_base_gamepad/ Gamepad controller for LeKiwi base movement
src/lerobot/teleoperators/bi_so101_leader/ Bimanual SO-101 leader arm configuration
src/lerobot/teleoperators/bi_joycon/ Bimanual Joy-Con teleoperation support
src/lerobot/teleoperators/bi_gamepad/ Bimanual gamepad teleoperation support
src/lerobot/teleoperators/ar_controller/ AR-based teleoperation controller
src/lerobot/robots/xlerobot/ Combined XLerobot platform implementation
src/lerobot/robots/xlerobot_mount/ XLerobot camera mount robot implementation
src/lerobot/robots/lekiwi_base/ LeKiwi mobile base robot implementation
src/lerobot/robots/bi_so101_follower/ Bimanual SO-101 follower robot implementation
src/lerobot/datasets/v30/convert_dataset_v30_to_v21.py Dataset schema conversion utility
src/lerobot/cameras/opencv/ MJPEG camera capture support

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines 190 to 194
for cam in self.cameras.values():
try:
cam.disconnect()
except Exception:
logger.warning("Failed to disconnect camera", exc_info=True)
Copy link

Copilot AI Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate camera disconnection loop. The second loop (lines 190-194) is identical to the first (lines 185-189) and should be removed.

Suggested change
for cam in self.cameras.values():
try:
cam.disconnect()
except Exception:
logger.warning("Failed to disconnect camera", exc_info=True)

Copilot uses AI. Check for mistakes.
Comment on lines +167 to +183
# if self._left_joycon is not None:
# try:
# self._left_joycon.disconnect()
# except Exception: # pragma: no cover - hardware cleanup
# pass
# self._left_joycon = None

# if self._right_joycon is not None:
# try:
# self._right_joycon.disconnect()
# except Exception: # pragma: no cover - hardware cleanup
# pass
# self._right_joycon = None

# self._last_update = None
# logger.info("JoyCon teleoperator disconnected.")
pass
Copy link

Copilot AI Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The disconnect method is commented out and only contains 'pass'. This leaves resources potentially uncleaned. Either implement proper cleanup or document why it's intentionally disabled.

Suggested change
# if self._left_joycon is not None:
# try:
# self._left_joycon.disconnect()
# except Exception: # pragma: no cover - hardware cleanup
# pass
# self._left_joycon = None
# if self._right_joycon is not None:
# try:
# self._right_joycon.disconnect()
# except Exception: # pragma: no cover - hardware cleanup
# pass
# self._right_joycon = None
# self._last_update = None
# logger.info("JoyCon teleoperator disconnected.")
pass
if self._left_joycon is not None:
try:
self._left_joycon.disconnect()
except Exception: # pragma: no cover - hardware cleanup
pass
self._left_joycon = None
if self._right_joycon is not None:
try:
self._right_joycon.disconnect()
except Exception: # pragma: no cover - hardware cleanup
pass
self._right_joycon = None
self._last_update = None
logging.info("JoyCon teleoperator disconnected.")

Copilot uses AI. Check for mistakes.
Comment on lines +133 to +140
# if self._joycon is not None:
# try:
# self._joycon.disconnect()
# except Exception: # pragma: no cover - hardware cleanup
# logger.debug("Failed to disconnect JoyCon cleanly.", exc_info=True)
# self._joycon = None
pass

Copy link

Copilot AI Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The disconnect method is commented out and only contains 'pass'. This leaves JoyCon resources potentially uncleaned. Either implement proper cleanup or document why it's intentionally disabled.

Suggested change
# if self._joycon is not None:
# try:
# self._joycon.disconnect()
# except Exception: # pragma: no cover - hardware cleanup
# logger.debug("Failed to disconnect JoyCon cleanly.", exc_info=True)
# self._joycon = None
pass
if self._joycon is not None:
try:
self._joycon.disconnect()
except Exception: # pragma: no cover - hardware cleanup
logger.debug("Failed to disconnect JoyCon cleanly.", exc_info=True)
self._joycon = None

Copilot uses AI. Check for mistakes.
Comment on lines +328 to +334
result = subprocess.run(
cmd,
check=True,
timeout=300, # 5 minute timeout
capture_output=True,
text=True
)
Copy link

Copilot AI Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The subprocess call executes ffmpeg with user-controlled file paths. While validation exists in _validate_video_paths(), consider using shlex.quote() on the file path arguments to prevent potential shell injection if paths contain special characters.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants