Skip to content

Conversation

@Sunidhi-Gaonkar1
Copy link

What does the PR do?

Checklist

  • I have read the Contribution guidelines and signed the Contributor License
    Agreement
  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Related issues are referenced.
  • Populated github labels field
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • I ran pre-commit locally (pre-commit install, pre-commit run --all)
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging ref.
  • All template sections are filled out.
  • Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

  • build
  • ci
  • docs
  • feat
  • fix
  • perf
  • refactor
  • revert
  • style
  • test

Related PRs:

#8329

Where should the reviewer start?

Test plan:

Used below commands to create triton server docker instance and starting up the triton server:
1.sudo docker run --privileged -it -p8000:8000 --volume /model_repository:/var/models --name triton_server_test tritonserver
2../bin/tritonserver --model-repository=/var/models

Tested the simple identity model using below commands in a different session:
1.curl -X POST localhost:8000/v2/repository/index | jq

curl -X POST localhost:8000/v2/repository/index | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    58  100    58    0     0  29000      0 --:--:-- --:--:-- --:--:-- 29000
[
  {
    "name": "simple_identity",
    "version": "1",
    "state": "READY"
  }
]

2.curl localhost:8000/v2/models/simple_identity | jq

curl localhost:8000/v2/models/simple_identity | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   193  100   193    0     0   188k      0 --:--:-- --:--:-- --:--:--  188k
{
  "name": "simple_identity",
  "versions": [
    "1"
  ],
  "platform": "python",
  "inputs": [
    {
      "name": "INPUT0",
      "datatype": "BYTES",
      "shape": [
        -1,
        -1
      ]
    }
  ],
  "outputs": [
    {
      "name": "OUTPUT0",
      "datatype": "BYTES",
      "shape": [
        -1,
        -1
      ]
    }
  ]
}

Caveats:

Background

Hi @whoisj , as mentioned in #8329 (comment), this PR adds changes to skip the libnvshmem3-cuda-13 library for cpu only build.
Please review/merge this PR.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@whoisj whoisj requested a review from mc-nv November 20, 2025 16:48
@alhad-deshpande
Copy link

Hi @mc-nv
Can you please review/merge this PR?

df += f"""
if FLAGS.triton_wheels_dependencies_group != "cpu" :
repo_arch = "sbsa" if target_machine == "aarch64" else "x86_64"
df += f"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason behind this installation is that PyTorch backend is looking for that library, and PyTorch is part of the CPU build process.

In the same time, using flag --triton-wheels-dependencies-group is not applicable here that flag is designed for different purpose, it's for wheel dependency group options.

Copy link

@alhad-deshpande alhad-deshpande Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mc-nv
Ok. Thanks for your review. We will change the code as per your suggestion and were thinking to introduce cuda specific flag 'disable_cuda' to disable this library. The default value be false. The same cuda flag 'disable_cuda' can be used in pytorch_backend as well to disable all cuda related libraries like libtorch_cuda.so, libcublasLt.so etc.
Please do let me know if this approach sounds ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants