forward performance tuning for MI350

[liligwu]

bwd performance optimization for ROCm.
Fix numerical issues

[netlify]

✅ Deploy Preview for pytorch-fbgemm-docs ready!

Name	Link
🔨 Latest commit	2c4a497
🔍 Latest deploy log	https://app.netlify.com/projects/pytorch-fbgemm-docs/deploys/68ed6512a0f9200008a224cd
😎 Deploy Preview	https://deploy-preview-4925--pytorch-fbgemm-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[facebook-github-bot]

@haoyuz has imported this pull request. If you are a Meta employee, you can view this in D83116315.

[q10]

@liligwu we're seeing

OSError: libtbb.so.12: cannot open shared object file: No such file or directory

We already install tbb here, so it might just be an issue of updating the build scripts to put libtbb in the LD_LIBRARY_PATH

[meta-codesync]

@q10 has imported this pull request. If you are a Meta employee, you can view this in D83116315.

[liligwu]

@liligwu we're seeing

OSError: libtbb.so.12: cannot open shared object file: No such file or directory

We already install tbb here, so it might just be an issue of updating the build scripts to put libtbb in the LD_LIBRARY_PATH

Hi @q10 , sorry I missed your message.
I actually have this commit 4d2bfdd that links tbb explicitly, it works in your container. Do you have any suggestions for fixing this issue in CI, please?

BTW, we discovered a numerical issue in 986cceb and reverted it in 85417b4. It unblocks merging the bwd optimization first.

Thank you.

[q10]

@liligwu we're seeing

OSError: libtbb.so.12: cannot open shared object file: No such file or directory

We already install tbb here, so it might just be an issue of updating the build scripts to put libtbb in the LD_LIBRARY_PATH

Hi @q10 , sorry I missed your message. I actually have this commit 4d2bfdd that links tbb explicitly, it works in your container. Do you have any suggestions for fixing this issue in CI, please?

BTW, we discovered a numerical issue in 986cceb and reverted it in 85417b4. It unblocks merging the bwd optimization first.

Thank you.

I think this commit only addresses the build step, where we need to link to tbb. However, for runtime, you might need to do a find in $CONDA_PREFIX from inside the container, and manually update LD_LIBRARY_PATH, or create a symlink (something like

FBGEMM/.github/scripts/utils_build.bash

Line 383 in 0d49628

(print_exec ln -s "${conda_prefix}/lib/librhash.so" "${conda_prefix}/lib/librhash.so.0") || return 1

).