-
Notifications
You must be signed in to change notification settings - Fork 293
Open
Labels
scoping requiredFeatures that need significant design and planning before being actionableFeatures that need significant design and planning before being actionabletype:featureNew feature or requestNew feature or request
Description
Currently our preprocessors require a static sequence length for all batches (example). However, on average max sequence length in a batch can be much smaller than max sequence length possible. Choosing a smaller static sequence_length
can give speedups but affects accuracy.
Let's allow a "dynamic"
option where each batch is padded to the minimum amount possible. This would likely interfere with XLA compilation so this would have to be scoped and benchmarked.
Metadata
Metadata
Assignees
Labels
scoping requiredFeatures that need significant design and planning before being actionableFeatures that need significant design and planning before being actionabletype:featureNew feature or requestNew feature or request