Using -l <language>+equ frequently causes floating point exceptions (core dumped)

[callegar]

Environment

• Tesseract Version: 5.0.0
• Commit Number: from version tag
• Platform: Linux 5.15.7-1-MANJARO #1 SMP PREEMPT Wed Dec 8 10:09:19 UTC 2021 x86_64 GNU/Linux

Issue is also occurring with Ubuntu linux 20.04.

Current Behavior:

It is my understanding that for documents containing math, one should use the 'equ' language, as in -l eng+equ. However, (on large documents?) this frequently causes the program to terminate with a floating point exception after some pages.

Expected Behavior:

The program should not terminate with floating point exceptions

Additional notes

Since I have experienced the behavior with documents that I cannot share, I do not have a test case so far. As soon as I can find one, I will attach it here. In the meantime, I have nevertheless opened this issue just in case someone else is encountering the same problem and has a test case ready to attach.

[amitdo]

-l <language>+equ

This is wrong usage.

Drop the +equ

-l <language>

and to turn on the equation detection feature add this to the command:

-c textord_equation_detect=true

In any case, the equation detection in Tesseract is known to be weak.

[amitdo]

tesseract-ocr/tessdata_fast#4 (comment)

(referring to the omission of equ.traineddata from the tessdata_fast repo)

jbreiden commented on 9 Jan 2020

I just noticed this issue. The equation recognizer was left out deliberately from Tesseract 4. The Tesseract 3 version works very poorly. And it was not easy to create a new one for Tesseract 4. So Ray decided to abandon it. Maybe one day it will be possible to remove all Tesseract 3 code and data. (This is hard and maybe impossible because OSD is so critical) But I think that that holding onto the old equation recognizer is not worth it.

[callegar]

Sorry for the noise, my mistake then... but may I suggest:

• that trained data files that are not meant to be used with -l result in an error rather than unpredictable behavior such as the floating point exception that I saw?
• that --list-langs does not list data sets that are not meant to be used with -l?

[amitdo]

You can still try to detect equations using the correct usage. At least with --oem 0 (the old ocr engine) and a lang.traineddata from the tessdata repo, it might sometimes produce something somewhat useful.

but may I suggest:

• that trained data files that are not meant to be used with -l result in an error rather than unpredictable behavior such as the floating point exception that I saw?

A reasonable suggestion.

• that --list-langs does not list data sets that are not meant to be used with -l?

Not sure what's the 'right' thing to do.

list-langs and -l (short for language) are misleading options. The traineddata could be a script like Latin with multiple languages or a special file: osd or equ.

So IMO, it should be --list-models and -m <model>.

[amitdo]

@callegar, I accidentally overrode your comment with my reply. Sorry!

I fixed it. Your comment was restored.

[amitdo]

Related issue: #3513.

[Paul-Taylor-EU]

Continuation of my "duplicate" issue "Floating exception with -l equ #3732"

Following the suggestions on this page, I tried:

tesseract Stolz3.tiff Stolz -l deu -c textord_equation_detect=true
Detected 41 diacritics
Segmentation fault

tesseract Stolz3.tiff Stolz -l deu --oem 0
Error: Tesseract (legacy) engine requested, but components are not present in /usr/share/tesseract-ocr/5/tessdata/deu.traineddata!!
Failed loading language 'deu'
Tesseract couldn't load any languages!
Could not initialize tesseract.

tesseract Stolz3.tiff Stolz --oem 0
Error: Tesseract (legacy) engine requested, but components are not present in /usr/share/tesseract-ocr/5/tessdata/eng.traineddata!!
Failed loading language 'eng'
Tesseract couldn't load any languages!
Could not initialize tesseract.

My test file is http://paultaylor.eu/drafts/Stolz3.tiff (C)1888

[amitdo]

You can try with a model that contains lstm and legacy data:
https://github.com/tesseract-ocr/tessdata/raw/main/deu.traineddata

[Paul-Taylor-EU]

My post on MathOverflow gives an introduction and references for the technology of the 1950s for printing mathematics, ie long before LaTeX. This would be a valuable and feasible goal for a project to extend tesseract to recognising mathematics.

[Paul-Taylor-EU]

Sorry, amitdo, I don't understand what you're suggesting that I do with this testdata file. As I said, the program runs to completion with -l deu, it's equ that's the problem. I see that several people have already picked up my test file. I am still very new to tesseract and playing with it is already a displacement activity, so I will leave it to you.

[amitdo]

Error: Tesseract (legacy) engine requested, but components are not present in /usr/share/tesseract-ocr/5/tessdata/deu.traineddata!!

You told Tesseract to use the legacy engine but the traineddata model in your tessedata directory does not contain legacy data.

I gave you a link to a model that contains both lstm and legacy data.

Your tessdata directory should include the linked model and the equ model.

The right command is:

tesseract image.tiff image -l deu -c textord_equation_detect=true

[Paul-Taylor-EU]

tesseract Stolz3.tiff Stolz -l deu -c textord_equation_detect=true
Detected 41 diacritics
Segmentation fault