Rate Limiting #107
Description
I've been playing around with building a UI on top of ui-be, during my development today none of my queries worked. I realized that most of my requests were getting 500 or gateway timeouts or bad requests...
For example:
Error posting query: 'Error: ERROR: status: 502; msg: 'Bad Gateway'' [{"message":{"query_graph":{"nodes":{"sn":{"categories":["biolink:ChemicalEntity"],"ids":["PUBCHEM.COMPOUND:3386"]},"on":{"categories":["biolink:Gene"]}},"edges":{"t_edge":{"subject":"sn","object":"on","predicates":["biolink:affects"],"knowledge_type":"inferred","qualifier_constraints":[{"qualifier_set":[{"qualifier_type_id":"biolink:object_aspect_qualifier","qualifier_value":"activity_or_abundance"},{"qualifier_type_id":"biolink:object_direction_qualifier","qualifier_value":"increased"}]}]}}}}}]
Here area few more logs...
{"level":30,"time":1693259489186,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":46,"method":"GET","url":"/static/css/main.54375581.css.map","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","pragma":"no-cache","cache-control":"no-cache","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"empty","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53132},"res":{"statusCode":200,"headers":{"x-powered-by":"Express","accept-ranges":"bytes","cache-control":"public, max-age=0","last-modified":"Sat, 19 Aug 2023 04:40:11 GMT","etag":"W/\"173b5-18a0c17f5f2\"","content-type":"application/json; charset=UTF-8","content-length":95157}},"responseTime":2,"msg":"request completed"}
{"level":30,"time":1693259489233,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":47,"method":"GET","url":"/static/js/main.d5262822.js.map","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","pragma":"no-cache","cache-control":"no-cache","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"empty","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53132},"res":{"statusCode":200,"headers":{"x-powered-by":"Express","accept-ranges":"bytes","cache-control":"public, max-age=0","last-modified":"Sat, 19 Aug 2023 04:40:11 GMT","etag":"W/\"3344b1-18a0c17f5f5\"","content-type":"application/json; charset=UTF-8","content-length":3359921}},"responseTime":17,"msg":"request completed"}
{"level":30,"time":1693259489258,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":50,"method":"GET","url":"/config","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","pragma":"no-cache","cache-control":"no-cache","sec-ch-ua":"\"Chromium\";v=\"116\", \"Not)A;Brand\";v=\"24\", \"Google Chrome\";v=\"116\"","sec-ch-ua-platform":"\"macOS\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","content-type":"application/json","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"http://localhost:8386/results?l=Melancholia&i=MONDO:0002444&t=0&q=Internal%20Server%20Error","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53130},"res":{"statusCode":200,"headers":{"x-powered-by":"Express","content-type":"application/json; charset=utf-8","content-length":"469","etag":"W/\"1d5-trqbNK4X5az4PQvQyngdpEsFuHg\""}},"responseTime":1,"msg":"request completed"}
{"level":30,"time":1693259489258,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":48,"method":"GET","url":"/static/media/Property%201%20Down.be3348dfeadb653a3fbe.svg","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","pragma":"no-cache","cache-control":"no-cache","sec-ch-ua":"\"Chromium\";v=\"116\", \"Not)A;Brand\";v=\"24\", \"Google Chrome\";v=\"116\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"macOS\"","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"http://localhost:8386/static/css/main.54375581.css","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53132},"res":{"statusCode":200,"headers":{"x-powered-by":"Express","accept-ranges":"bytes","cache-control":"public, max-age=0","last-modified":"Sat, 19 Aug 2023 04:40:11 GMT","etag":"W/\"d8-18a0c17f5ff\"","content-type":"image/svg+xml","content-length":216}},"responseTime":2,"msg":"request completed"}
Error querying status for Internal Server Error: 'Error: ERROR: status: 502; msg: 'Bad Gateway''
{"level":50,"time":1693259489320,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":49,"method":"POST","url":"/api/creative_status","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","content-length":"31","pragma":"no-cache","cache-control":"no-cache","sec-ch-ua":"\"Chromium\";v=\"116\", \"Not)A;Brand\";v=\"24\", \"Google Chrome\";v=\"116\"","sec-ch-ua-platform":"\"macOS\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","content-type":"application/json","accept":"*/*","origin":"http://localhost:8386","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"http://localhost:8386/results?l=Melancholia&i=MONDO:0002444&t=0&q=Internal%20Server%20Error","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53129},"msg":"Internal Server Error: Error: Error: ERROR: status: 502; msg: 'Bad Gateway'"}
{"level":30,"time":1693259489321,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":49,"method":"POST","url":"/api/creative_status","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","content-length":"31","pragma":"no-cache","cache-control":"no-cache","sec-ch-ua":"\"Chromium\";v=\"116\", \"Not)A;Brand\";v=\"24\", \"Google Chrome\";v=\"116\"","sec-ch-ua-platform":"\"macOS\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","content-type":"application/json","accept":"*/*","origin":"http://localhost:8386","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"http://localhost:8386/results?l=Melancholia&i=MONDO:0002444&t=0&q=Internal%20Server%20Error","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53129},"res":{"statusCode":500,"headers":{"x-powered-by":"Express","content-type":"application/json; charset=utf-8","content-length":"49","etag":"W/\"31-A8BspA1XWgbyLNywUZO4qtY92XM\""}},"err":{"type":"Error","message":"failed with status code 500","stack":"Error: failed with status code 500\n at ServerResponse.onResFinished (/Users/merrickchristensen/Developer/biomedical-translator-official/ui-be/node_modules/pino-http/logger.js:113:40)\n at ServerResponse.emit (node:events:525:35)\n at onFinish (node:_http_outgoing:969:10)\n at callback (node:internal/streams/writable:555:21)\n at afterWrite (node:internal/streams/writable:500:5)\n at afterWriteTick (node:internal/streams/writable:487:10)\n at process.processTicksAndRejections (node:internal/process/task_queues:81:21)"},"responseTime":65,"msg":"request errored"}
{"level":30,"time":1693259489629,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":51,"method":"GET","url":"/favicon.ico","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","pragma":"no-cache","cache-control":"no-cache","sec-ch-ua":"\"Chromium\";v=\"116\", \"Not)A;Brand\";v=\"24\", \"Google Chrome\";v=\"116\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"macOS\"","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"http://localhost:8386/results?l=Melancholia&i=MONDO:0002444&t=0&q=Internal%20Server%20Error","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53129},"res":{"statusCode":200,"headers":{"x-powered-by":"Express","accept-ranges":"bytes","cache-control":"public, max-age=0","last-modified":"Sat, 19 Aug 2023 04:40:11 GMT","etag":"W/\"47e-18a0c17f5ef\"","content-type":"image/x-icon","content-length":1150}},"responseTime":1,"msg":"request completed"}
{"level":30,"time":1693259489629,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":52,"method":"GET","url":"/manifest.json","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","pragma":"no-cache","cache-control":"no-cache","sec-ch-ua":"\"Chromium\";v=\"116\", \"Not)A;Brand\";v=\"24\", \"Google Chrome\";v=\"116\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"macOS\"","accept":"*/*","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"manifest","referer":"http://localhost:8386/results?l=Melancholia&i=MONDO:0002444&t=0&q=Internal%20Server%20Error","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53138},"res":{"statusCode":200,"headers":{"x-powered-by":"Express","accept-ranges":"bytes","cache-control":"public, max-age=0","last-modified":"Sat, 19 Aug 2023 04:40:11 GMT","etag":"W/\"1f8-18a0c17f604\"","content-type":"application/json; charset=UTF-8","content-length":504}},"responseTime":1,"msg":"request completed"}
{"level":30,"time":1693259489635,"pid":25040,"hostname":"Merrick-4DY0Q9","req":{"id":53,"method":"GET","url":"/logo192.png","query":{},"params":{},"headers":{"host":"localhost:8386","connection":"keep-alive","pragma":"no-cache","cache-control":"no-cache","sec-ch-ua":"\"Chromium\";v=\"116\", \"Not)A;Brand\";v=\"24\", \"Google Chrome\";v=\"116\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36","sec-ch-ua-platform":"\"macOS\"","accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"image","referer":"http://localhost:8386/results?l=Melancholia&i=MONDO:0002444&t=0&q=Internal%20Server%20Error","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":53129},"res":{"statusCode":200,"headers":{"x-powered-by":"Express","accept-ranges":"bytes","cache-control":"public, max-age=0","last-modified":"Sat, 19 Aug 2023 04:40:11 GMT","etag":"W/\"14e3-18a0c17f605\"","content-type":"image/png","content-length":5347}},"responseTime":0,"msg":"request completed"}
I was wondering if maybe the services that power ui-be have some rate limiting? Or worse, considered me an adversarial party and blocked me? I certainly hope I didn't cause any trouble poking around with this code! Thanks for sharing this awesome work!