Skip to content

[Transform] [Utils] Canonical matching utilities #392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 22, 2025

Conversation

kylesayrs
Copy link
Contributor

@kylesayrs kylesayrs commented Jul 12, 2025

Purpose

  • There currently exist 23 different functions for matching modules/parameters. These utils consolidate them to just 6.
  • Used by transforms for module matching and layer norm matching
Function Used for/by Maps to
get_layer_by_name awq attrgetter(layer_name)(module)
get_layers_params pruning modifiers match_named_parameters
get_param None Delete
get_params get_layers_params , get_param Delete
set_layer Distillation modifier Module.set_submodule
get_layer None Delete
get_layers Lots match_named_modules
match_layers_params get_layers Delete
get_default_params match_layers_params Delete
match_targets Many match_named_modules
get_quantizable_layers match_layers_params Delete, use explicit targets
get_prunable_layers match_layers_params Delete, use explicit targets
get_terminal_layers match_layers_params Delete, use explicit targets
get_matching_layer Smoothquant match_modules_set
AWQ.set_resolved_mappings AWQ match_modules_set
expand_target_names Compressor match_named_parameters
is_target Quantization is_match
find_name_or_class_matches Quantization match_name or match_class
_find_matches Quantization Delete
get_nested_weight_mappings Compressor match_named_parameters
match_param_name get_nested_weight_mappings Delete,match_name
match_modules sequential pipeline match_named_modules
get_linear_layers infer_sparsity_structure_from_model match_named_modules

get_linear_layers

Changes

  • Implement 3 commonly-used functions
    • match_named_modules returns modules
    • match_named_parameters returns parameters (and their parent modules)
    • match_modules_set returns modules in predefined groups
  • Implements 3 helper functions which are publicly exposed
    • match_name returns True if target regex matches the name
    • match_class returns True if target matches any parent classes
    • is_match returns True if either match_name or match_class is True

Integration Plan

  • This PR utilizes these matching utilities for transforms. Future PRs will replace existing functions with these function as described in the table

Testing

  • Added comprehensive tests generated by Claude, which I have manually reviewed and fixed any issues with

Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
@kylesayrs kylesayrs changed the title [Utils] Canonical matching utilities [Transform] [Utils] Canonical matching utilities Jul 14, 2025
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
@kylesayrs kylesayrs marked this pull request as ready for review July 16, 2025 15:53
Copy link
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need llm-comrpessor or vLLM changes or any tests for backwards compatibility?
Should we test this with the nightly given how many things we're touching here?

Copy link
Contributor

@shanjiaz shanjiaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding tests! Only thing is that I see a lot of repeated logic across match_named_modules, match_named_parameters, and match_modules_set. Would be great if they can be refactored. Otherwise looks good to me!

@kylesayrs
Copy link
Contributor Author

Confirmed that vLLM does not use any CT matching utilities
https://github.com/search?q=repo%3Avllm-project%2Fvllm+%22from+compressed_tensors%22&type=code

@kylesayrs
Copy link
Contributor Author

kylesayrs commented Jul 22, 2025

@dsikka

Do we need llm-comrpessor or vLLM changes or any tests for backwards compatibility?
Should we test this with the nightly given how many things we're touching here?

Not right now, since this PR only adds new utilities. The other utilities will be removed in follow ups. Once this lands I can create an issue which users can contribute to

@dsikka dsikka merged commit 7e0dc32 into main Jul 22, 2025
1 check passed
@dsikka dsikka deleted the kylesayrs/canonical-matching-utilities branch July 22, 2025 22:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants