You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Implement NormalizableMixin for activation normalization and enhance BatchTopKSAE and CrossCoder classes
This commit introduces the `NormalizableMixin` class, which provides methods for normalizing and denormalizing activations using an `ActivationNormalizer`. The `BatchTopKSAE` and `CrossCoder` classes are updated to inherit from this mixin, allowing them to utilize the new normalization functionality.
Key changes include:
- Addition of `normalize_activations` and `denormalize_activations` methods in the mixin.
- Updates to the constructors of `BatchTopKSAE` and `CrossCoder` to accept an optional `activation_normalizer`.
- Modifications to the `encode` and `decode` methods in both classes to incorporate normalization and denormalization processes.
These enhancements improve the flexibility and robustness of activation management during training and evaluation, ensuring that activations can be appropriately normalized based on the provided normalizer.
0 commit comments