Skip to content

Commit f61b74f

Browse files
crazydemocrazydemo
authored
[None][test] add l20 specific qa test list (#7067)
Signed-off-by: Ivy Zhang <[email protected]>
1 parent 630e67b commit f61b74f

File tree

2 files changed

+64
-2
lines changed

2 files changed

+64
-2
lines changed

‎tests/integration/test_lists/qa/README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ Triton backend tests validate the integration with NVIDIA Triton Inference Serve
3232
The following Python packages are required for running QA tests:
3333

3434
```bash
35-
pip install mako oyaml rouge_score lm_eval
35+
pip3 install -r ${TensorRT-LLM_PATH}/requirements-dev.txt
3636
```
3737

3838
### Dependency Details
@@ -52,7 +52,8 @@ This directory contains various test configuration files:
5252
- `llm_function_nim.txt` - NIM-specific functional test cases
5353
- `llm_function_multinode.txt` - Multi-node functional test cases
5454
- `llm_function_gb20x.txt` - GB20X release test cases
55-
- `llm_function_rtx6kd.txt` - RTX 6000 Ada specific tests
55+
- `llm_function_rtx6kd.txt` - RTX 6000 series specific tests
56+
- `llm_function_l20.txt` - L20 specific tests, only contains single gpu cases
5657

5758
### Performance Test Files
5859
- `llm_perf_full.yml` - Main performance test configuration

‎tests/integration/test_lists/qa/llm_function_l20.txt

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# only covers single gpu cases
2+
accuracy/test_llm_api.py::TestQwen2_7BInstruct::test_auto_dtype
3+
accuracy/test_llm_api.py::TestQwen2_7BInstruct::test_weight_only
4+
accuracy/test_llm_api.py::TestLlama3_1_8BInstruct::test_gather_generation_logits_cuda_graph
5+
accuracy/test_llm_api.py::TestLlama3_1_8BInstruct::test_logprobs
6+
accuracy/test_llm_api.py::TestPhi4MiniInstruct::test_auto_dtype
7+
accuracy/test_llm_api.py::TestPhi4MiniInstruct::test_fp8
8+
accuracy/test_llm_api.py::TestQwen2_5_1_5BInstruct::test_auto_dtype
9+
accuracy/test_llm_api.py::TestQwen2_5_1_5BInstruct::test_weight_only
10+
accuracy/test_llm_api.py::TestLlama3_1_8B::test_fp8_rowwise
11+
accuracy/test_llm_api.py::TestQwen2_7BInstruct::test_fp8
12+
accuracy/test_llm_api.py::TestQwen2_5_0_5BInstruct::test_fp8
13+
accuracy/test_llm_api.py::TestQwen2_5_1_5BInstruct::test_fp8
14+
accuracy/test_llm_api.py::TestQwen2_5_7BInstruct::test_fp8
15+
accuracy/test_llm_api.py::TestQwen2_5_7BInstruct::test_fp8_kvcache
16+
accuracy/test_llm_api.py::TestMistralNemo12B::test_auto_dtype
17+
accuracy/test_llm_api.py::TestMistralNemo12B::test_fp8
18+
19+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_chunked_prefill[attn_backend=FLASHINFER]
20+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_chunked_prefill[attn_backend=TRTLLM]
21+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_fp8_llm_sampler
22+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_fp8_beam_search
23+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_eagle3[eagle3_one_model=True-overlap_scheduler=True]
24+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_eagle3[eagle3_one_model=False-overlap_scheduler=False]
25+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_ngram
26+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_guided_decoding[xgrammar]
27+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_guided_decoding[llguidance]
28+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[xgrammar]
29+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_guided_decoding_4gpus[llguidance]
30+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[xgrammar]
31+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[llguidance]
32+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_ngram[xgrammar]
33+
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_ngram[llguidance]
34+
35+
accuracy/test_llm_api_pytorch.py::TestKanana_Instruct::test_auto_dtype
36+
accuracy/test_llm_api_pytorch.py::TestBielik11BInstruct::test_auto_dtype
37+
accuracy/test_llm_api_pytorch.py::TestBielik11BInstruct::test_fp8
38+
accuracy/test_llm_api_pytorch.py::TestMinistral8BInstruct::test_auto_dtype
39+
accuracy/test_llm_api_pytorch.py::TestMinistral8BInstruct::test_fp8
40+
accuracy/test_llm_api_pytorch.py::TestPhi4MM::test_auto_dtype
41+
accuracy/test_llm_api_pytorch.py::TestPhi4MM::test_auto_dtype_long_rope
42+
accuracy/test_llm_api_pytorch.py::TestPhi4MiniInstruct::test_auto_dtype
43+
accuracy/test_llm_api_pytorch.py::TestMistralNemo12B::test_auto_dtype
44+
45+
test_e2e.py::test_ptp_quickstart_multimodal[NVILA-8B-FP16-vila/NVILA-8B-image-False]
46+
test_e2e.py::test_ptp_quickstart_multimodal[NVILA-8B-FP16-vila/NVILA-8B-video-False]
47+
test_e2e.py::test_ptp_quickstart_multimodal[llava-v1.6-mistral-7b-llava-v1.6-mistral-7b-hf-image-False]
48+
test_e2e.py::test_ptp_quickstart_multimodal[qwen2-vl-7b-instruct-Qwen2-VL-7B-Instruct-image-False]
49+
test_e2e.py::test_ptp_quickstart_multimodal[qwen2-vl-7b-instruct-Qwen2-VL-7B-Instruct-video-False]
50+
test_e2e.py::test_ptp_quickstart_multimodal[qwen2.5-vl-7b-instruct-Qwen2.5-VL-7B-Instruct-image-False]
51+
test_e2e.py::test_ptp_quickstart_multimodal[qwen2.5-vl-7b-instruct-Qwen2.5-VL-7B-Instruct-image-True]
52+
test_e2e.py::test_ptp_quickstart_multimodal[qwen2.5-vl-7b-instruct-Qwen2.5-VL-7B-Instruct-video-False]
53+
test_e2e.py::test_ptp_quickstart_multimodal[qwen2.5-vl-7b-instruct-Qwen2.5-VL-7B-Instruct-video-True]
54+
test_e2e.py::test_ptp_quickstart_multimodal_phi4mm[audio]
55+
test_e2e.py::test_ptp_quickstart_multimodal_phi4mm[image]
56+
test_e2e.py::test_ptp_quickstart_multimodal_phi4mm[image_audio]
57+
test_e2e.py::test_ptp_quickstart_bert[VANILLA-BertForSequenceClassification-bert/bert-base-uncased-yelp-polarity]
58+
test_e2e.py::test_ptp_quickstart_bert[TRTLLM-BertForSequenceClassification-bert/bert-base-uncased-yelp-polarity]
59+
test_e2e.py::test_ptp_star_attention_example[Llama3.1-8B-BF16-llama-3.1-model/Meta-Llama-3.1-8B]
60+
test_e2e.py::test_trtllm_bench_pytorch_backend_sanity[meta-llama/Llama-3.1-8B-llama-3.1-8b-hf-nvfp4-False-False]
61+
test_e2e.py::test_ptp_scaffolding[DeepSeek-R1-Distill-Qwen-7B-DeepSeek-R1/DeepSeek-R1-Distill-Qwen-7B]

0 commit comments

Comments
 (0)