Skip to content

Conversation

kylesayrs
Copy link
Collaborator

Purpose

  • When [Tracing] Skip non-ancestors of sequential targets #1389 landed, modules being skipped by ignore were no longer being skipped. However, this requires that the sequential targets list be correct. Mllama defaults to targeting vision layers, and hence the vision tower was being traced, leading to errors.
_no_split_modules = [
    "MllamaVisionEncoderLayer",
    "MllamaCrossAttentionDecoderLayer",
    "MllamaSelfAttentionDecoderLayer",
]

Changes

  • Only target text decoder layers, not vision decoder layers

Testing

Signed-off-by: Kyle Sayers <[email protected]>
Copy link

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

@kylesayrs kylesayrs added the ready When a PR is ready for review label Apr 29, 2025
Copy link
Collaborator

@shanjiaz shanjiaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@dsikka dsikka enabled auto-merge (squash) April 30, 2025 14:03
@dsikka dsikka merged commit d783c26 into main Apr 30, 2025
8 checks passed
@dsikka dsikka deleted the kylesayrs/fix-mllama branch April 30, 2025 14:04
kylesayrs added a commit that referenced this pull request May 5, 2025
## Purpose ##
* Add regression testing to model tracing beyond example tests
* These tests complete in ~1 min and can be run at a quicker cadence
than example tests
* These can also be used to test tracing capabilities beyond those in
the examples, for example tracing into linear layers

## Prerequisites ##
* #1334
* #1402

## Changes ##
* Fix function signature of peoples speech dataset
* Add `trust_remote_code` argument to debugger
* Add `tests/llmcompressor/transformers/tracing/models.py`
* I did not include phi3 because it's a very difficult model to work
with programmatically. I will revisit once the major tracing
improvements have landed

---------

Signed-off-by: Kyle Sayers <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready When a PR is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants