Skip to content

Conversation

Xia-Weiwen
Copy link
Collaborator

@Xia-Weiwen Xia-Weiwen commented Sep 22, 2025

Summary
This PR merges the two definitions of get_block_size in ao/torchao/quantization/pt2e/observer.py and torchao/quantization/observer.py and put it in torchao/quantization/utils.py
It also removes duplicate definitions of granularities in the pt2e path.

Test plan
This util function is used in many places so it should have been covered by all kinds of UT.

Copy link

pytorch-bot bot commented Sep 22, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3039

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit f07ab47 with merge base eadead5 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 22, 2025
@Xia-Weiwen Xia-Weiwen added the topic: not user facing Use this tag if you don't want this PR to show up in release notes label Sep 22, 2025
@Xia-Weiwen Xia-Weiwen marked this pull request as ready for review September 23, 2025 01:19
@Xia-Weiwen Xia-Weiwen requested review from Copilot, andrewor14 and jerryzh168 and removed request for Copilot and andrewor14 September 23, 2025 01:19
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR consolidates the get_block_size function by moving it from two separate locations (torchao/quantization/pt2e/observer.py and torchao/quantization/observer.py) to a centralized location in torchao/quantization/utils.py. The PR also removes duplicate granularity class definitions from the pt2e path, centralizing granularity-related functionality.

  • Moves get_block_size function to a central utils.py location
  • Removes duplicate granularity class definitions from pt2e/observer.py
  • Updates import statements across multiple files to use the centralized functions

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
torchao/quantization/utils.py Adds the unified get_block_size function and imports granularity classes
torchao/quantization/quantize_/workflows/float8/float8_tensor.py Updates import to use get_block_size from utils instead of observer
torchao/quantization/quant_api.py Updates import to use get_block_size from utils instead of observer
torchao/quantization/qat/fake_quantizer.py Updates import to use get_block_size from utils instead of observer
torchao/quantization/pt2e/observer.py Removes duplicate granularity classes and get_block_size function
torchao/quantization/pt2e/_affine_quantization.py Updates import to use get_block_size from utils
torchao/quantization/pt2e/init.py Removes exports for granularity classes and get_block_size
torchao/quantization/observer.py Removes original get_block_size function and unused import
torchao/quantization/init.py Adds export for Granularity base class
test/quantization/pt2e/test_quantize_pt2e.py Updates imports to use granularity classes from main quantization module

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines +710 to +713
assert input_shape[-1] % granularity.group_size == 0, (
f"Group size {granularity.group_size} does not divide input shape {input_shape}"
)
return (1,) * (len(input_shape) - 1) + (granularity.group_size,)
Copy link
Preview

Copilot AI Sep 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unified get_block_size function has inconsistent behavior for PerGroup compared to the original pt2e version. The original pt2e version enforced a 2D input shape constraint and returned (1, granularity.group_size), while this version allows any dimensional input and returns (1,) * (len(input_shape) - 1) + (granularity.group_size,). This could break existing code that depends on the 2D constraint.

Suggested change
assert input_shape[-1] % granularity.group_size == 0, (
f"Group size {granularity.group_size} does not divide input shape {input_shape}"
)
return (1,) * (len(input_shape) - 1) + (granularity.group_size,)
assert len(input_shape) == 2, (
f"PerGroup granularity requires 2D input shape, got {input_shape}"
)
assert input_shape[-1] % granularity.group_size == 0, (
f"Group size {granularity.group_size} does not divide input shape {input_shape}"
)
return (1, granularity.group_size)

Copilot uses AI. Check for mistakes.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should allow any dimensional input instead of requiring 2D.

@Xia-Weiwen
Copy link
Collaborator Author

Hi @jerryzh168 Could you please review this PR? Thanks.

@Xia-Weiwen Xia-Weiwen closed this Sep 23, 2025
@Xia-Weiwen Xia-Weiwen reopened this Sep 23, 2025
@jerryzh168
Copy link
Contributor

@Xia-Weiwen thanks, can you fix ruff?

Copy link
Contributor

@jerryzh168 jerryzh168 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, please make sure CI passes

@Xia-Weiwen Xia-Weiwen merged commit 8e2ca35 into pytorch:main Sep 23, 2025
18 checks passed
@cccclai
Copy link
Contributor

cccclai commented Sep 24, 2025

How about PerBlock?

jerryzh168 added a commit that referenced this pull request Sep 24, 2025
@jerryzh168
Copy link
Contributor

sorry reverting this first, I think we should keep BC

jerryzh168 added a commit that referenced this pull request Sep 24, 2025
jerryzh168 added a commit that referenced this pull request Sep 24, 2025
Revert "Unify get_block_size (#3039)"

This reverts commit 8e2ca35.
@Xia-Weiwen
Copy link
Collaborator Author

How about PerBlock?

PerBlock is added in the reopened PR #3059. However, you can easily get block size by granularity.block_size if granularity is PerBlock.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. topic: not user facing Use this tag if you don't want this PR to show up in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants