TorchForge and the future of TorchRL

[xiaotian]

Hi all,

In the recent Pytorch Conference 2025 in SF, Meta announced TorchForge, which seems to have fairly significant overlap with TorchRL. Just wondering, if moving forward, Meta will consolidate these 2 libraries or deprecate TorchRL in favor of the new framework?

Thanks!

cc: @vmoens
-X.T.

[vmoens]

Hi!

Great question — and it’s true that there’s some conceptual overlap between TorchForge and TorchRL, especially around components like environments, data collection, and replay buffers. That’s largely because both projects share similar design principles that have proven effective for reinforcement learning workflows. I was involved early on in some of the design discussions around TorchForge and had a hand in shaping some of its core ideas. It’s always exciting to see new initiatives exploring similar spaces — maintaining an RL framework is no small task, so I’m genuinely curious to see how it evolves.

TorchRL remains an actively developed and well-supported library within the PyTorch ecosystem. We’re continuing to ship new features, performance improvements, and integrations with other PyTorch tools (including potential interop points with projects like TorchForge where it makes sense). I'll make a new major soon with more support for environments with composable tools, global services in distributed frameworks, and SOTA training algorithms such as CISPO, DAPO etc.

I’d see the two projects as exploring different angles of the same space — TorchForge with a broader focus on experiment orchestration and infra integration, and TorchRL as a mature, specialized library for RL research and production pipelines.

In short: TorchRL isn’t going anywhere 🙂 and we’re making sure that users can benefit from advances across both initiatives as they evolve.

[xiaotian]

Thank you Vincent your comment on this, and your excellent work on this fantastic RL library!