Using S3 api with Cloudflare R2 fails out of the box

[dazld]

I've been migrating some storage to Cloudflare's R2, which has an S3-compatible API.

Out of the box, most of the options already available worked well - I initialised a client with the following options:

(def s3 (aws/client {:api                  :s3
                     :region               "us-east-1"
                     :endpoint-override    {:hostname (format "%s.r2.cloudflarestorage.com" BUCKET-ID)
                                            :protocol :https}
                     :credentials-provider (credentials/basic-credentials-provider
                                             {:access-key-id     (config/cfg-key :cloudflare :key)
                                              :secret-access-key (config/cfg-key :cloudflare :secret)})}))

..and the API was able to connect, although it then threw an error when parsing the XML response. Being able to inspect the response via meta was incredibly useful - thank you for implementing this pattern.

After a bit of debugging, found that the :skip-whitespace option in the XML parsing utility caused it to fail.

(ns cognitect.aws.util ....)

...

(defn xml-read
  "Parse the UTF-8 XML string."
  [s]
  (xml/parse (ByteArrayInputStream. (.getBytes ^String s "UTF-8"))
             :namespace-aware false
             ;; option provided here
             :skip-whitespace true))

I'm not sure exactly why this caused it to fail - but just wanted to share back that getting it to work required a bit of monkey patching as below:

(defn invoke [& args]
  (let [fixed-parse (fn
                      [s]
                      (xml/parse (ByteArrayInputStream. (.getBytes ^String s "UTF-8"))
                                 :namespace-aware false))]
    (with-redefs [cognitect-aws-util/xml-read fixed-parse]
      (apply aws/invoke s3 args))))

With this in place, the client behaves as normal, although it's a little annoying.

Two questions:

• Are there any gotchas with this approach that you can see? (beyond the obvious ick of monkey patching an external library)
• If not, would making the parsing options configurable be a possibility?

Thanks!