Support DeepSeekV3-style block FP8 quantization (clean)

[shanjiaz]

SUMMARY:
Fixes 1475
This was originally pr #1607, the commit history got messy. I cherry picked Michael's original commit 451219a and updated from there.

TEST PLAN:
Tested locally and generated the model.

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[gemini-code-assist]

Summary of Changes

Hello @shanjiaz, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces comprehensive support for DeepSeekV3-style block FP8 quantization, a technique aimed at further compressing large language models for improved efficiency. The changes encompass the implementation of the core block-wise quantization algorithm, enhancements to the calibration process to manage block-shaped quantization parameters, and the addition of an example script and documentation to facilitate its usage. This work resolves a previously identified issue and represents a cleaned-up re-submission of earlier development efforts.

Highlights

• New Quantization Scheme: Introduced support for W8A8-FP8_BLOCK quantization, a DeepSeekV3-style scheme that applies block-wise FP8 compression to weights (e.g., 128x128 tiles) and dynamic per-token-group FP8 quantization for activations. This scheme does not require a calibration dataset.
• Block-wise Quantization Logic Implementation: Implemented the core logic within the Observer class to calculate quantization scales and zero points on a per-block basis, replacing a NotImplementedError. This enables the system to derive quantization parameters for specific blocks of a tensor.
• Dynamic Parameter Handling for Block Shapes: Updated the calibration process to correctly handle and store block-shaped quantization parameters (scales and zero points). The system now dynamically registers or updates offload_parameter instances, ensuring that parameter shapes can adapt to the varying dimensions required by block-wise quantization.
• Example and Documentation: Added a new example script (examples/quantization_w8a8_fp8/fp8_block_example.py) demonstrating how to apply the FP8_BLOCK scheme. Corresponding documentation was added to docs/guides/compression_schemes.md to describe the new quantization method.
• Test Coverage: Added a new test case (test_block_strategy_parsing) to verify that the GPTQModifier correctly parses and resolves the block quantization strategy and its associated block_structure configuration.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces support for DeepSeekV3-style block FP8 quantization. The review includes suggestions to improve clarity in the documentation and example code, enhance maintainability by addressing variable shadowing, and fix a potential data type issue in the observer logic.

[brian-dellabetta]

Things are looking good, but agree with comments left by kyle and gemini

[kylesayrs]

Thanks for seeing this through

[brian-dellabetta]

🚀