Set is_binarized to false for valid dataloaders.

[mattsetz]

What does this PR do? Please describe:
Currently is_binarized is set for all data splits. But in practice we only use it for train and not valid splits (bc these are generally small). As a result, the pretrain parity run fails to run on H100s without this change.

Fixes #{issue number}

Does your PR introduce any breaking changes? If yes, please list them:
List of all backwards-incompatible changes.

Check list:

• Was the content of this PR discussed and approved via a GitHub issue? (no need for typos or documentation improvements)
• Did you read the contributor guideline?
• Did you make sure that your PR does only one thing instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• Did you update the CHANGELOG? (no need for typos, documentation, or minor internal changes)