Adds visual-based tactile sensor with shape sensing example #3420
Conversation
clean up license and configs; use full import path; add doc for visuo_tactile_sensor
JuanaDd
requested review from
jtigue-bdai,
kellyguo11,
Mayankm96 and
ooctipus
as code owners
|
|
||
| References | ||
| ~~~~~~~~~~ | ||
|
|
There was a problem hiding this comment.
@iakinola23 Could you help view this documentation here? Thanks!
There was a problem hiding this comment.
Hi @iakinola23 updated the documentation with your edits.
There was a problem hiding this comment.
@kellyguo11 Hi Kelly, I will upload the npz binaries, and the USD files to nucleus once the licensing concerns are resolved. Thanks!
Mayankm96
changed the title
github-actions
bot
added
documentation
| taxim_gelsight = gelsightRender("gelsight_r15") | ||
| import ipdb | ||
|
|
||
| ipdb.set_trace() |
There was a problem hiding this comment.
why is there a breakpoint here?
| self._camera_sensor = TiledCamera(self.cfg.camera_cfg) | ||
|
|
||
| # Initialize camera if not already done | ||
| # TODO: Juana: this is a hack to initialize the camera sensor. Should camera_sensor be managed by TacSL sensor or InteractiveScene? |
There was a problem hiding this comment.
maybe we can do something similar as the Raycaster camera?
There was a problem hiding this comment.
Hi Kelly, I checked the RayCasterCamera implementation, and it differs slightly from the tacsl sensor logic. The tacsl_sensor includes a camera sensor that can be enabled or disabled via the cfg, while RayCasterCamera only uses CameraData without having a camera or other sensor as a member. I’ve kept the current implementation and cleaned up the comments. Please let me know if you have any further concerns or suggestions.
There was a problem hiding this comment.
all changed to jpg format and documentation is updated accordingly.
| .. code-block:: bash | ||
|
|
||
| conda activate env_isaaclab | ||
| pip install opencv-python==4.11.0 trimesh==4.5.1 imageio==2.37.0 |
There was a problem hiding this comment.
do we need to add these to setup.py?
There was a problem hiding this comment.
Added opencv-python to setup.py; trimesh is already included, and imageio's dependency is not required in the newest commit.
| license agreement from NVIDIA CORPORATION is strictly prohibited. | ||
|
|
||
| ---------------- | ||
| Tensorized implementation of RGB rendering of gelsight-style visuo-tactile sensors |
There was a problem hiding this comment.
can you also add any licenses required for gelsight under docs/licenses?
There was a problem hiding this comment.
Updated the license header to be consistent with other files.
|
please also make sure to run ./isaaclab.sh -f for the linter checks |
| return dzdx, dzdy | ||
|
|
||
|
|
||
| def generate_normals(height_map): |
There was a problem hiding this comment.
generate_normals not needed and deprecated. only the tensorized version is used.
| self.background_tensor = torch.tensor(self.background, device=self.device) | ||
| print("Gelsight initialization done!") | ||
|
|
||
| def render(self, heightMap): |
There was a problem hiding this comment.
Delete render as it is not used. Only the tensorized version (render_tensorized ) is used.
There was a problem hiding this comment.
deleted render , generate_normals, and compute_image_gradient , padding as well.
|
Thank you for making TacSL available on IsaacLab! I have two quick questions:
Thanks! |
Description
This is an implementation of TacSL integrated with Isaac Lab, which demonstrates how to properly configure and use tactile sensors to obtain realistic sensor outputs including tactile RGB images, force fields, and other relevant tactile measurements.
Fixes # (issue)
Type of change
Screenshots
The screenshots of added documentation and simulation outputs.
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there