Growing number of suggester files in Tomcat temp directory

[anishk25]

The tomcat temp directory for my opengrok server (/opt/flatiron/mounts/var/lib/tomcat8/temp/) is filling up with opengrok_suggester_wfst_input_XX.tmp and opengrok_suggester_wfst_sort_XX.tmp files. I have been tracking the number of files of this type being created by the suggester and also the amount of space being taken up. Both of them have been steadily growing. Is there a way to configure opengrok so that these files get automatically cleaned up ?

[vladak]

There was similar issue filed however I cannot find it right now.

[vladak]

Definitely, the temporary files should be cleaned up automatically.

[tulinkry]

It is still the same issue even if you can't find it

[anishk25]

I think you might be referring to this issue: #2240

[vladak]

Yep, that's the one. What version are you running ? Dne út 9. 7. 2019 16:58 uživatel Anish Khattar <notifications@github.com> napsal:

…

I think you might be referring to this issue: #2240 <#2240> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2857?email_source=notifications&email_token=AAWMMDENVLFYPXO7RL2DRBDP6SRPNA5CNFSM4H66ESL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQRBOY#issuecomment-509677755>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWMMDH5ZH2XECQ7BQAZD6LP6SRPNANCNFSM4H66ESLQ> .

[anishk25]

I am running version 1.2.7

[anishk25]

I am guessing I need to update the a later version to fix this issue.

[vladak]

It may be worth trying. Dne čt 11. 7. 2019 0:13 uživatel Anish Khattar <notifications@github.com> napsal:

…

I am guessing I need to update the a later version to fix this issue. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2857?email_source=notifications&email_token=AAWMMDE4IOE5KZUKHERH3GTP6ZNG3A5CNFSM4H66ESL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZU43SA#issuecomment-510250440>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWMMDCYP7RQNF6UAWD72B3P6ZNG3ANCNFSM4H66ESLQ> .

[ahornace]

Those files are completely in Lucene's control.

https://github.com/apache/lucene-solr/blob/master/lucene/suggest/src/java/org/apache/lucene/search/suggest/SortedInputIterator.java#L176
https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/util/OfflineSorter.java#L317

The best approach would probably be to create the issue on Lucene's side.

However, we might be able to explicitly delete all the files in temp directory with opengrok_suggester_wfst prefix after the suggester builds its data structures. I don't like it but there is not much else we can do. What do you guys think?

[unhipzippo]

FYI, problem is still present in 1.3.6.

[vladak]

@ahornace as long as the files are no longer needed that could be a workaround.

[vladak]

Checking our production instance (runs 1.3.6, Tomcat uses -Djava.io.tmpdir=/var/tomcat8/temp) I don't see the issue. Also, the Tomcat process does not currently have any of these files opened.

[unhipzippo]

What version of Tomcat?

We're using 9.0.27 with AdoptOpenJDK 11.0.3+7 and with a systemd script that sets up (among other things):

Environment=CATALINA_HOME=/opt/opengrok/apache-tomcat-9.0.27
Environment=CATALINA_BASE=/var/opt/opengrok/hydra/tomcat
Environment='JAVA_OPTS=\
   -Xms1g -Xmx1g \
   -Djava.awt.headless=true \
   -Dport.shutdown=8005 \
   -Dport.ajp=8009 \
   -Dport.http=8080 \
   -Dldap.group=thwur \
   -Dcom.sun.jndi.ldap.connect.pool.timeout=1200000 \
   -Djava.util.logging.config.file=/var/opt/opengrok/hydra/tomcat/conf/logging.properties \
   -Dlog.dir=/var/opt/opengrok/hydra/log \
   -Dconfig.xml=/var/opt/opengrok/hydra/etc/configuration.xml'

ExecStart=/opt/opengrok/java/jdk/bin/java \
   -classpath ${CLASSPATH} \
   -Dcatalina.home=${CATALINA_HOME} \
   -Dcatalina.base=${CATALINA_BASE} \
   -Djava.endorsed.dirs=${JAVA_ENDORSED_DIRS} \
   -Djava.io.tmpdir=${CATALINA_BASE}/temp \
   -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
   $JAVA_OPTS $CATALINA_OPTS \
   org.apache.catalina.startup.Bootstrap \
   start

Suggester tempfiles seem to be getting touched around midnight local time and 3am local time (we have a regular cron indexing job that runs daily at 3am -- not sure what's touching the files at midnight)