Retry requests when status code is 429 (#410)

This code is what HTTP uses to indicate the client has sent too many
requests in a given amount of time. It may sound counter-intuitive to
retry in this situation, but that's why we use an exponential backoff
mechanism. It gives the server the opportunity to recover, without
failing the requests immediately.

The retry mechanism already works in object stores like S3 because they
return a server error. But without this change, we are not handling GCS
properly. GCS returns a client error `429 Too Many Requests` instead.
This change enables retries on this response too.

A more advanced retry mechanism would use the optional response header
`Retry-After`, but that is beyond the scope of this PR.

Closes: #309

Author: paraseba

src/client/retry.rs

@@ -402,6 +402,7 @@ impl RetryableRequest {
                         let status = r.status();
                         if ctx.exhausted()
                             || !(status.is_server_error()
+                                || status == StatusCode::TOO_MANY_REQUESTS
                                 || (self.retry_on_conflict && status == StatusCode::CONFLICT))
                         {
                             let source = match status.is_client_error() {