Skip to content

Commit d50bc88

Browse files
committed
Implement NormalizableMixin for activation normalization and enhance BatchTopKSAE and CrossCoder classes
This commit introduces the `NormalizableMixin` class, which provides methods for normalizing and denormalizing activations using an `ActivationNormalizer`. The `BatchTopKSAE` and `CrossCoder` classes are updated to inherit from this mixin, allowing them to utilize the new normalization functionality. Key changes include: - Addition of `normalize_activations` and `denormalize_activations` methods in the mixin. - Updates to the constructors of `BatchTopKSAE` and `CrossCoder` to accept an optional `activation_normalizer`. - Modifications to the `encode` and `decode` methods in both classes to incorporate normalization and denormalization processes. These enhancements improve the flexibility and robustness of activation management during training and evaluation, ensuring that activations can be appropriately normalized based on the provided normalizer.
1 parent e3b8993 commit d50bc88

File tree

4 files changed

+423
-108
lines changed

4 files changed

+423
-108
lines changed

0 commit comments

Comments
 (0)