PARSeq Model #2089
PARSeq Model #2089
Conversation
|
@sineeli - which parts of the PR are ready for review? Asking because it's still marked as draft |
|
Sure @abheesht17 First preprocessing and tokenizer these parts I think are good for reviewing, as they are the primary steps.
|
| self.char_to_id = tf.lookup.StaticHashTable( | ||
| initializer=tf.lookup.KeyValueTensorInitializer( | ||
| keys=list(self._stoi.keys()), | ||
| values=list(self._stoi.values()), | ||
| key_dtype=tf.string, | ||
| value_dtype=tf.int32, | ||
| ), | ||
| default_value=0, | ||
| ) | ||
| self.id_to_char = tf.lookup.StaticHashTable( | ||
| initializer=tf.lookup.KeyValueTensorInitializer( | ||
| keys=list(self._stoi.values()), | ||
| values=list(self._stoi.keys()), | ||
| key_dtype=tf.int32, | ||
| value_dtype=tf.string, | ||
| ), | ||
| default_value=self.pad_token, | ||
| ) |
There was a problem hiding this comment.
The defaults don't match. EOS is the 0th token, and pad is the len(vocabulary) - 1th token
There was a problem hiding this comment.
I recognized the same in the original code, but seems they are using EOS -> 0, BOS->len(vocabulary), but while padding they are doing BOS first and then EOS at the end.
| label = tf.strings.upper(label) | ||
|
|
||
| label = tf.strings.regex_replace(label, self.unsupported_regex, "") | ||
| label = tf.strings.substr(label, 0, self.max_label_length) |
There was a problem hiding this comment.
Why are we truncating the input to 25 characters?
There was a problem hiding this comment.
While preparing the dataset in the preprocessing itself if the label is above 25 they jus ignore that datapoint itself. Instead I truncated and we can start and end tokens instead.
There was a problem hiding this comment.
Thanks, added some comments,
could you please add a PR description by following the recent PR description template which includes Colab notebook link with end to end working demo and numerics verification.
Also add the original implementation reference in the PR description.
sachinprasadhs
added
kokoro:force-run
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces the PARSeq model, a ViT-based OCR model, to KerasHub. I've identified a few issues, including two critical bugs related to model serialization and tokenizer functionality that must be addressed. I've also found a couple of medium-severity issues regarding a typo in a layer name and a docstring example that should be corrected for clarity and maintainability.
PARSeq Model
Description of the Change
This PR adds an end-to-end scene text recognition model, PARSeq, to KerasHub. PARSeq is a ViT-based OCR model that enables iterative decoding for robust text recognition in natural scenes.
Closes the first half of #<issue_number>
Reference
For details, see Scene Text Recognition with Permuted Autoregressive Sequence Models (PARSeq paper). The model and configuration are based on the official paper and open-source implementation
Colab Notebook
Usage and numerics matching Colab:
Checklist