-
Notifications
You must be signed in to change notification settings - Fork 21
[Transform] [Utils] Canonical matching utilities #392
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
Signed-off-by: Kyle Sayers <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need llm-comrpessor or vLLM changes or any tests for backwards compatibility?
Should we test this with the nightly given how many things we're touching here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding tests! Only thing is that I see a lot of repeated logic across match_named_modules, match_named_parameters, and match_modules_set. Would be great if they can be refactored. Otherwise looks good to me!
Confirmed that vLLM does not use any CT matching utilities |
Not right now, since this PR only adds new utilities. The other utilities will be removed in follow ups. Once this lands I can create an issue which users can contribute to |
Purpose
get_linear_layers
Changes
match_named_modules
returns modulesmatch_named_parameters
returns parameters (and their parent modules)match_modules_set
returns modules in predefined groupsmatch_name
returns True if target regex matches the namematch_class
returns True if target matches any parent classesis_match
returns True if eithermatch_name
ormatch_class
is TrueIntegration Plan
Testing