Web Assets don't cache DNS, causing excessive DNS queries can lead to failures

[laundmo]

Bevy version and features

Bevy 0.17.2, features: bevy_dev_tools, dynamic_linking, http, https, track_location

[Optional] Relevant system information

Os: EndeavourOS
Using: NetworkManger with dnsmasq resolver

DNS Server: Unbound with local, stub and forward zones

What you did

Tried loading a grid of 1040 different PNGs from the same https domain.

What went wrong

A good number of the PNGs never finished loading with the error:

ERROR bevy_asset::server: Encountered an I/O error while loading asset: unexpected error while loading asset https://example.com/img329.png: io: failed to lookup address information: Temporary failure in name resolution

Additional information

Since this seemed odd, I checked the DNS server where I noticed a surprising amount of requests. That caused me to look at tcpdump using:
,
sudo tcpdump -l -nn port 53 and src host <my_local_ip> | grep '[+] A[?] example.com' | nl

Which prints all outgoing A record dns queries querying for example.com with line numbers. And indeed:

  1046	15:33:03.429826 IP 192.168.178.111.51068 > 192.168.178.2.53: 4799+ A? example.com. (34)
  1047	15:33:03.430279 IP 192.168.178.111.38595 > 192.168.178.2.53: 63990+ A? example.com. (34)

Bevy makes 1047 DNS requests to load 1040 tiles. I'm not sure where the extra 7 come from, but that's hardly relevant. This is an excessive number of DNS queries for a single domain. And loading many assets from a single domain is likely a very common use case for web assets.

This is why I strongly suspect that the amount of queries just overloads the DNS server, causing the errors.

(I'll have to have a look at my Unbound config tho, as it should be able to handle even this amount.)

[laundmo]

I've had a look at this, and it seems ureq uses rusts ToSocketAddrs trait, which in turn relies on libc::gtaddrinfo. That doesn't include result TTL.

That sadly means the lightweight option for fixing this, implementing a copy of that default ureq Resolver which wraps a cache around it, won't be compliant with the dns query TTL.

Of course, it could still make sense to implement a relatively short cache to avoid bursts overwhelming some upstream resolver.

[laundmo]

Note: Most OSs should cache DNS internally, which is the stated reason a dns cache was not implemented in either Rusts std or in ureq

• getaddrinfo may be slow on OS X rust-lang/rust#31665 (comment)
• implement dns cache algesten/ureq#401