thread leakage?

[nicferrier]

I'm writing something quite complex with jet and I'm getting quite serious thread leakage. About 100 threads across a particular test run.

I tried to make a simple thing to see whether that suffered thread leakage and it did! Here it is:

(defn fake-jetty-handlers []
  [(fn [{:keys [in out ctrl ws] :as opts}]
     (timbre/debug "fake-jetty-handlers ws-request" (:uri opts))
     (a/go-loop [data (a/<! in)]
       (a/>! out (str/upper-case data))))
   (fn [request-msg]
     (timbre/debug "fake-jetty-handlers http-request" (:uri request-msg))
     {:status 200
      :header {:content-type "text/html"}
      :body (format "<h1>%s</h1>" (str/upper-case (:uri request-msg)))})])

(defn test-fake-jetty-server []
  (-init-logging true)
  (let [[ws-handler http-handler] (fake-jetty-handlers)
        server (jet/run-jetty
                {:port 8020
                 :join? false
                 :websocket-handler ws-handler
                 :ring-handler http-handler})
        client (http/client)]
    (threads-log)
    (async-do-then
     100
     ;; First an HTTP request ...
     (let [con-ch (http/get client "http://localhost:8020/blah/blah")
           response (a/<! con-ch)
           body-ch (:body response)
           body (a/<! body-ch)]
       (threads-log)
       ;; ... and then a WS request
       (websoc/connect!
        "ws://localhost:8020/blah2"
        (fn [{:keys [in out ctrl ws]}]
          (threads-log)
          (a/go
            (threads-log)
            (a/>! out "some data")
            (let [raw-in (a/<! in)]
              (RETURN [body raw-in]))))))
     (then
      (threads-log)
      (.stop server)
      (threads-log)
      (let [[body ws-sent] (RECV)]
        (timbre/debug "test-fake-jetty> body-response is" body)
        (timbre/debug "test-fake-jetty> raw-response is" ws-sent)
        (assert (= ws-sent "SOME DATA"))
        (assert (= body "<h1>/BLAH/BLAH</h1>"))
        (threads-log))))))

Hopefully that makes sense... when I run it I get a leakage of 2 threads.

Anyone else come across this? Is there a good way to debug it?

Here's the results of a run:

2016-01-04 09:57:33.824:INFO:oejs.Server:nREPL-worker-5: jetty-9.3.6.v20151106
2016-01-04 09:57:33.839:INFO:oejs.ServerConnector:nREPL-worker-5: Started ServerConnector@a7f7d{HTTP/1.1,[http/1.1]}{0.0.0.0:8020}
2016-01-04 09:57:33.840:INFO:oejs.Server:nREPL-worker-5: Started @549761ms
[09:57:33] fake-jetty-handlers http-request /blah/blah
[09:57:33] threadcount: 580
[09:57:33] threadcount: 582
[09:57:33] fake-jetty-handlers ws-request /blah2
[09:57:33] threadcount: 591
[09:57:33] threadcount: 591
[09:57:33] threadcount: 591
2016-01-04 09:57:33.910:INFO:oejs.ServerConnector:nREPL-worker-5: Stopped ServerConnector@a7f7d{HTTP/1.1,[http/1.1]}{0.0.0.0:8020}
[09:57:33] threadcount: 582
[09:57:33] test-fake-jetty> body-response is <h1>/BLAH/BLAH</h1>
[09:57:33] test-fake-jetty> raw-response is SOME DATA
[09:57:33] threadcount: 582

[mpenet]

Hi,

This could be from the client manager, try running http/stop-client! at the end of your test. Clients instance are meant to be reused and they have to be stopped at some point if you need to reclaim their ressources.

I am currently traveling, I cant look into it before a couple of days unfortunately.

[nicferrier]

Awesome. Will give it a try.

[mpenet]

Did it help in any way ? Feel free to close yourself if it did. (I am still on the move)

[mpenet]

That said I should write some documentation for the clients parts and make this obvious.

[mpenet]

Something else I noticed, you never close the socket in your Websocket code (both client/server):
(async/close! out)

[mpenet]

any news on this?

[nicferrier]

Nothing yet. I have rewritten a lot of my code to not use mult/tap - I am still getting thread leaks though but only a few. Seems like a manageable thing to track down. I'll be on it this week at some point.

[mpenet]

Alright. Just a heads up about mult, might or might not be relevant to your usage (unrelated to this issue): http://dev.clojure.org/jira/browse/ASYNC-90

[nicferrier]

It still happened even when I unsubed everything.