[P1] The results on Qwen2.5-7B-Instruct

[mrsempress]

Thank you very much for your work. May I ask if the author can provide the experimental results of LoReFT on qwen2.5-7B instruction. I used the original code and the command

python examples/loreft/train.py -task gsm8k -model ../models/Qwen2.5-7B-Instruct/ -seed 42 -l all -r 8 -p f7+l7 -e 12 -lr 9e-4 -type LoreftIntervention -gradient_accumulation_steps 8 -batch_size 4 -eval_batch_size 4 --dropout 0.05 --test_split test --use_normalized_template --share_weights --greedy_decoding --warmup_ratio 0 --weight_decay 0.06 -max_length 1024 -save_model -output_dir official_results/Qwen

The accuracy is 46.10, but I'm not sure if this value is normal. As the public accuracy of GSM8K(4-shot) on is 85.4.

[mrsempress]

From the perspective of loss, the result of ReFT on Qwen2.5-7B-Instruct is normal.

But I tested it directly on the Qwen2.5-7B-Instruct directly and found that the performance was 75.51.

[frankaging]

hey, i think the script under examples/loreft/train.py is specifically for reproducing experiments in the paper.

for running it on new models, you need to adapt the script. see the following line for instance where we adapt the script for llama-3.1:
https://github.com/stanfordnlp/pyreft/blob/main/examples/loreft/dataset.py#L142

you may need to check other places.