[P1] Not supportive to all Hugging Face models.

[krishnardt]

Hi,

I tried to train below models with LorEFT for instruction fine tuning of text2SQL-

1. Qwen/Qwen2-7B-Instruct-AWQ
2. microsoft/phi-2

But these models are not able to load to GPU.
I tried this code base to load the model and finetune: https://github.com/stanfordnlp/pyreft/blob/main/examples/loreft/train.py

Command to run the script in above mentioned link:
python train.py --task "instruct" --model "meta-llama/Llama-2-7b-hf" --data_dir /datasets --train_dataset /datasets/text_2_sql/train_final.json --seed 42 --layers 'all' --rank 4 --position "f10+l10" --epochs 16 --save_model --intervention_type "LoreftIntervention" --gradient_accumulation_steps 4 --batch_size 4 --output_dir /finetuned_models/reft_models/test_layers_all_rank4_fl1010_epochs_16_lr_5e3 --lr 5e-3 --schedule "cosine" --warmup_ratio 0.03 --logging_steps 10

Am I missing anything here?
Could someone look into this issue for further model support?

[frankaging]

Hey, it is because the script is for Llama models! Not because of the library, i think. For all other types, please add them to this dict:
https://github.com/stanfordnlp/pyreft/blob/main/examples/loreft/train.py#L57

[cnmoro]

I also had a similar issue with qwen0.5b:

...

reft_model.set_device("cpu") # send back to cpu before saving.
reft_model.save(
    save_directory="./qwen2.5-0.5b-reft-text-compression", 
    save_to_hf_hub=True, 
    hf_repo_name="qwen2.5-0.5b-reft-text-compression"
)

...

import torch, transformers, pyreft
device = "cuda"

model_name_or_path = "Qwen/Qwen2.5-0.5B-Instruct"
model = transformers.AutoModelForCausalLM.from_pretrained(
    model_name_or_path, torch_dtype=torch.bfloat16, device_map=device)

reft_model = pyreft.ReftModel.load(
    "qwen2.5-0.5b-reft-text-compression", model
)

TypeError: 'type' object is not iterable
File ~/miniconda3/envs/geral/lib/python3.11/site-packages/pyreft/reft_model.py:26, in ReftModel.load(*args, **kwargs)
     [24](.../lib/python3.11/site-packages/pyreft/reft_model.py:24) @staticmethod
     [25](.../lib/python3.11/site-packages/pyreft/reft_model.py:25) def load(*args, **kwargs):
---> [26](.../lib/python3.11/site-packages/pyreft/reft_model.py:26)     model = pv.IntervenableModel.load(*args, **kwargs)
     [27](.../lib/python3.11/site-packages/pyreft/reft_model.py:27)     return ReftModel._convert_to_reft_model(model)

File ~/miniconda3/envs/geral/lib/python3.11/site-packages/pyvene/models/intervenable_base.py:1284, in IntervenableModel.load(load_directory, model, local_directory, from_huggingface_hub, include_model)
   [1281](.../lib/python3.11/site-packages/pyvene/models/intervenable_base.py:1281) saving_config = IntervenableConfig.from_pretrained(load_directory)
   [1282](.../lib/python3.11/site-packages/pyvene/models/intervenable_base.py:1282) casted_intervention_types = []
-> [1284](.../lib/python3.11/site-packages/pyvene/models/intervenable_base.py:1284) for type_str in saving_config.intervention_types:
   [1285](.../lib/python3.11/site-packages/pyvene/models/intervenable_base.py:1285)     casted_intervention_types += [get_type_from_string(type_str)]
   [1286](.../lib/python3.11/site-packages/pyvene/models/intervenable_base.py:1286) saving_config.intervention_types = (
   [1287](.../lib/python3.11/site-packages/pyvene/models/intervenable_base.py:1287)     casted_intervention_types
   [1288](.../lib/python3.11/site-packages/pyvene/models/intervenable_base.py:1288) )