Skip to content

NIXLBENCH: Add GPU tests to CI #780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 52 commits into from
Oct 3, 2025
Merged

NIXLBENCH: Add GPU tests to CI #780

merged 52 commits into from
Oct 3, 2025

Conversation

ovidiusm
Copy link
Contributor

@ovidiusm ovidiusm commented Sep 5, 2025 •
edited
Loading

What?

Add nixlbench GPU tests to CI. Also enables CUDA support in all unit tests executed on GPU workers.

Why?

To catch regressions related to VRAM/CUDA support

How?

  • gtest-parallel introduced to fix issues where tests do not clean up completely registered CUDA memory regions. gtest-parallel spawns a separate process per test, so it prevents interference between tests.
  • ucx_mo_backend_test skipped when a GPU is present. VRAM tests are broken. Confirmed with manual runs. We should fix it separately.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 5, 2025

👋 Hi ovidiusm! Thank you for contributing to ai-dynamo/nixl.

Your PR reviewers will review your contribution then trigger the CI to test your changes.

🚀

@ovidiusm
Copy link
Contributor Author

ovidiusm commented Sep 5, 2025

/build

@ovidiusm
Use two workers in UCX tests
a7cdcde 
Signed-off-by: Ovidiu Mara <[email protected]>
@ovidiusm
Copy link
Contributor Author

ovidiusm commented Sep 5, 2025

/build

@ovidiusm
Remove CUDA compat paths
49305eb 
Signed-off-by: Ovidiu Mara <[email protected]>
@ovidiusm
Copy link
Contributor Author

ovidiusm commented Sep 5, 2025

/build

@ovidiusm
Remove redundant tests
7ba49db 
Signed-off-by: Ovidiu Mara <[email protected]>
@ovidiusm
Remove peer in place
d60cbf4 
Signed-off-by: Ovidiu Mara <[email protected]>
@ovidiusm
Copy link
Contributor Author

ovidiusm commented Oct 2, 2025

/build

@ovidiusm
Revert "Remove peer in place"
154b398 
This reverts commit d60cbf4.

Signed-off-by: Ovidiu Mara <[email protected]>
@ovidiusm
Revert "Fix for std::runtime_error exception on connection close"
99749f5 
This reverts commit d0b3178.

Signed-off-by: Ovidiu Mara <[email protected]>
@pull-request-size pull-request-size bot added size/M and removed size/L labels Oct 2, 2025
@ovidiusm
Copy link
Contributor Author

ovidiusm commented Oct 2, 2025

/build

@ovidiusm ovidiusm merged commit 2a86446 into ai-dynamo:main Oct 3, 2025
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yosefe yosefe yosefe left review comments

@saturley-hall saturley-hall saturley-hall approved these changes

@brminich brminich brminich approved these changes

@mkhazraee mkhazraee Awaiting requested review from mkhazraee

@roiedanino roiedanino Awaiting requested review from roiedanino

Assignees
No one assigned
Labels
external-contribution size/M
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ovidiusm @dpressle @saturley-hall @yosefe @brminich