Skip to content

Commit f7a4465

Browse files
committed
set apply_chat_template as false for gsm8k
Signed-off-by: bhsueh <[email protected]>
1 parent 2d98747 commit f7a4465

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

tests/integration/defs/accuracy/accuracy_core.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -303,7 +303,8 @@ class GSM8K(AccuracyTask):
303303
MAX_OUTPUT_LEN = 256
304304

305305
EVALUATOR_CLS = tensorrt_llm.evaluate.GSM8K
306-
EVALUATOR_KWARGS = dict(dataset_path=DATASET_DIR, random_seed=0)
306+
EVALUATOR_KWARGS = dict(dataset_path=DATASET_DIR, random_seed=0,
307+
apply_chat_template=False)
307308

308309
EVALUATE_KWARGS = dict(scores_filter=None)
309310

0 commit comments

Comments
 (0)