[https://nvbugs/5481087][fix] fix bug of ci when we use mocker #7332
Conversation
📝 WalkthroughWalkthroughMove GSM8K patching from a shared evaluator kwargs dict to per-test attribute/dict patches and remove four SKIP entries in the waives list to re-enable several TestGPTOSS w4-cutlass variants. (48 words) Changes
Sequence Diagram(s)(No sequence diagram — changes are test configuration/waiver edits, not runtime control-flow or new features.) Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
tests/integration/defs/accuracy/test_llm_api_pytorch.py(3 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
**/*.py: Code must target Python 3.8+
Indent with 4 spaces; do not use tabs
Preserve module namespaces in imports: import the subpackage/module, not the symbol (from package.subpackage import foo; foo.SomeClass())
Naming: files snake_case; classes PascalCase; functions/methods snake_case; local variables snake_case (k_ prefix if starting with a number); globals G_ + UPPER_SNAKE_CASE; constants UPPER_SNAKE_CASE
Avoid shadowing outer-scope variables; initialize all externally visible members in init
Prefer docstrings for interfaces used outside a file; reserve comments for function-internal or file-local interfaces
Use Google-style docstrings for classes and functions; inline docstrings for attributes/variables are allowed
Avoid reflection when straightforward code suffices (e.g., prefer explicit parameters over dict(**locals()))
Use narrow except clauses (e.g., catch FileNotFoundError instead of bare except)
For duck-typing try/except, keep try body minimal and use else for the main logic
Files:
tests/integration/defs/accuracy/test_llm_api_pytorch.py
**/*.{cpp,cc,cxx,cu,h,hpp,hh,hxx,cuh,py}
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
Prepend NVIDIA copyright header with current year to all source files
Files:
tests/integration/defs/accuracy/test_llm_api_pytorch.py
🧬 Code graph analysis (1)
tests/integration/defs/accuracy/test_llm_api_pytorch.py (2)
tensorrt_llm/evaluate/lm_eval.py (1)
GSM8K(440-488)tests/integration/defs/accuracy/accuracy_core.py (1)
GSM8K(293-308)
🪛 Ruff (0.12.2)
tests/integration/defs/accuracy/test_llm_api_pytorch.py
2739-2739: SyntaxError: Expected ',', found ':'
2779-2779: SyntaxError: Expected ',', found ':'
2810-2810: SyntaxError: Expected ',', found ':'
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Pre-commit Check
|
/bot run |
byshiue
changed the title
|
PR_Github #16876 [ run ] triggered by Bot |
|
PR_Github #16876 [ run ] completed with state |
|
/bot run |
|
PR_Github #16881 [ run ] triggered by Bot |
|
PR_Github #16881 [ run ] completed with state |
|
/bot run |
|
PR_Github #16916 [ run ] triggered by Bot |
|
PR_Github #16916 [ run ] completed with state |
Signed-off-by: bhsueh <[email protected]>
Signed-off-by: bhsueh <[email protected]>
Signed-off-by: bhsueh <[email protected]>
Signed-off-by: bhsueh <[email protected]>
|
/bot run --disable-fail-fast |
|
PR_Github #17032 [ run ] triggered by Bot |
|
PR_Github #17032 [ run ] completed with state |
|
/bot run |
|
PR_Github #17088 [ run ] triggered by Bot |
|
PR_Github #17088 [ run ] completed with state |
Signed-off-by: bhsueh <[email protected]>
|
/bot run --stage_list "B200_PCIe-PyTorch-1" |
|
PR_Github #17127 Bot args parsing error: usage: /bot [-h] |
|
/bot help |
GitHub Bot Help
Provide a user friendly way for developers to interact with a Jenkins server. Run See details below for each supported subcommand.
Launch build/test pipelines. All previously running jobs will be killed.
kill
Kill all running builds associated with pull request. skip
Skip testing for latest commit on pull request. reuse-pipeline
Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break. |
|
/bot run --stage-list "B200_PCIe-PyTorch-1" |
|
PR_Github #17131 [ run ] triggered by Bot |
|
PR_Github #17131 [ run ] completed with state |
|
/bot skip --comment "only test the unwaive CI tests" |
|
PR_Github #17166 [ skip ] triggered by Bot |
|
PR_Github #17166 [ skip ] completed with state |
…A#7332) Signed-off-by: bhsueh <[email protected]>
…A#7332) Signed-off-by: bhsueh <[email protected]> Signed-off-by: Dongfeng Yu <[email protected]>
…A#7332) Signed-off-by: bhsueh <[email protected]> Signed-off-by: Dongfeng Yu <[email protected]>
Summary by CodeRabbit
Tests
Chores