This PR implements #459, i.e., it adds the models and taskmodules implemented originally in [pie-modules](https://github.com/ArneBinder/pie-modules) (except for QA and span-pair based RE, see potential follow-ups below). - added models: - `SequenceClassificationModelWithPooler` - `SequencePairSimilarityModelWithPooler` - `SimpleTokenClassificationModel` - `SimpleGenerativeModel` - `SimpleSequenceClassificationModel` - `TokenClassificationModelWithSeq2SeqEncoderAndCrf` - added taskmodules: - `RETextClassificationWithIndicesTaskModule` - `TextToTextTaskModule` - `LabeledSpanExtractionByTokenClassificationTaskModule` - `PointerNetworkTaskModuleForEnd2EndRE` - `CrossTextBinaryCorefTaskModule` **IMPORTANT: This restricts the version of transformers to `>=4.35.0,<4.37.0`! So, this is breaking.** requires: - #482 - #499 Additional changes: - add `tabulate`, and `pytorch-crf` to dev dependencies - set dependence `torchmetrics[text] >=1.5, <2` to solve conflicts with `nltk` (`text` loads the required additional dependencies and `>=1.5` ensures that no deprecated nltk models are loaded. Note that we already use the modern nltk models in [`pie_documents.document.processing.NltkSentenceSplitter`](https://github.com/ArneBinder/pie-documents/blob/main/src/pie_documents/document/processing/sentence_splitter.py)) - add `SpanNotAlignedWithTokenException` and `get_aligned_token_span` to `utils.document` - add `RequiresMaxInputLength` and `RequiresTaskmoduleConfig` to `models.interface` potential follow-ups: - [ ] add remaining models (SimpleExtractiveQuestionAnsweringModel and SpanTupleClassificationModel) - [ ] add remaining taskmodules (ExtractiveQuestionAnsweringTaskModule, and RESpanPairClassificationTaskModule) --------- Co-authored-by: Danylo Mysak <RainbowRivey@hotmail.com>

add taskmodules and models from pie-modules #459

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions