GitHub - krishkc5/energy_aware_quantization: End-to-end framework for evaluating accuracy, latency, and energy of quantized LLMs (FP32, FP16, INT8). Includes pre-tokenized SST-2 datasets, reproducible inference harness, GPU power logging via nvidia-smi, and analysis tools for energy-aware model deployment.

krishkc5 / energy_aware_quantization Public

Notifications You must be signed in to change notification settings
Fork 1
Star 1

End-to-end framework for evaluating accuracy, latency, and energy of quantized LLMs (FP32, FP16, INT8). Includes pre-tokenized SST-2 datasets, reproducible inference harness, GPU power logging via nvidia-smi, and analysis tools for energy-aware model deployment.