Skip to content

Allow dynamic padding in preprocessor #662

@jbischof

Description

@jbischof

Currently our preprocessors require a static sequence length for all batches (example). However, on average max sequence length in a batch can be much smaller than max sequence length possible. Choosing a smaller static sequence_length can give speedups but affects accuracy.

Let's allow a "dynamic" option where each batch is padded to the minimum amount possible. This would likely interfere with XLA compilation so this would have to be scoped and benchmarked.

Metadata

Metadata

Assignees

No one assigned

    Labels

    scoping requiredFeatures that need significant design and planning before being actionabletype:featureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions