[Compression] Remove legacy compression and decompression pathways

[kylesayrs]

No description provided.

[dsikka]

I don’t think we want to remove these until the new methods can compress / decompress when starting with a checkpoint that isn’t already in memory.

[kylesayrs]

@dsikka To be clear, are you referring to compressing/ decompressing from disk? Is there a remaining use case for this?

[dsikka]

@dsikka To be clear, are you referring to compressing/ decompressing from disk? Is there a remaining use case for this?

Yeah for anyone who wants to use compressed-tensors independent of the transformers pathway / is using ct as a standalone

I certainly think we can improve these functions but from disk is something we should have some way to support.

[kylesayrs]

Yeah for anyone who wants to use compressed-tensors independent of the transformers pathway / is using ct as a standalone

@dsikka There has never been a "from disk" compression pathway. In order to load any compressed model, you must use transformers from_pretrained.

In terms of the from disk decompression pathway, there has also never been a "from disk" pathway that doesn't also rely on transformers from_pretrained. The old disk pathway would use an already compressed model loaded from_pretrained, then load again from disk and replace the weights

[dsikka]

Yeah for anyone who wants to use compressed-tensors independent of the transformers pathway / is using ct as a standalone

@dsikka There has never been a "from disk" compression pathway. In order to load any compressed model, you must use transformers from_pretrained.

In terms of the from disk decompression pathway, there has also never been a "from disk" pathway that doesn't also rely on transformers from_pretrained. The old disk pathway would use an already compressed model loaded from_pretrained, then load again from disk and replace the weights

This is not true:

Afrom transformers import AutoModelForCausalLM, AutoTokenizer
from llmcompressor.utils import dispatch_for_generation


MODEL_ID = "nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A8_tensor_weight_static_per_tensor_act-e2e"
model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

from compressed_tensors.compressors import ModelCompressor

compressor = ModelCompressor.from_pretrained(MODEL_ID)
compressor.decompress(MODEL_ID, model)

print("\n\n")
print("========== SAMPLE GENERATION ==============")

dispatch_for_generation(model)
input_ids = tokenizer("Hello my name is", return_tensors="pt").input_ids.to(
    model.device
)
output = model.generate(input_ids, max_new_tokens=100)
print(tokenizer.decode(output[0]))
print("==========================================\n\n")

The compressed weights are decompressed after being read from disk. from_pretrained loads the skeleton for the model however the compressed weights are never read through the from_pretrained pathway. This enables decompression without relying on our transformers integration

This also gives us independent compression functionality, such as

compressed-tensors/src/compressed_tensors/compressors/helpers.py

Line 34 in 0e5df88

def save_compressed(

allowing a pathway to checkpoint while maintaining the original state dict in memory for further use

While the default pathway makes sense to be in memory compression / decompression, these are useful tools we still should maintain

[brian-dellabetta]

I think part of the motivation for this is that decompress/compress seem like the two main public methods that users should use, which led myself and this user astray. If we don't want to deprecate these, could we at least rename them (maybe to decompress_from_disk/compress_from_disk?) so users don't think they are the preferred methods to be used?