fix: add support for compressed hydrator. Closes #988

[mvanderlee]

Oathkeeper doesn't restrict the encoding it accepts when making the hydrator HTTP call. If the hydrator is configured with compression it will try to parse the compressed string directly as JSON and fail.

This PR restricts the accepted encoding types to gzip and deflate, and add the ability to handle those encodings.
I experimented with brotli as well, but that would require CGO, which seemed too big of a change.

Related issue(s)

#988

Checklist

• I have read the contributing guidelines.
• I have referenced an issue containing the design document if my change
  introduces a new feature.
• I am following the
  contributing code guidelines.
• I have read the security policy.
• I confirm that this pull request does not address a security
  vulnerability. If this pull request addresses a security. vulnerability, I
  confirm that I got green light (please contact
  [email protected]) from the maintainers to push
  the changes.
• I have added tests that prove my fix is effective or that my feature
  works.
• I have added or changed the documentation.

Further Comments

I've already deployed this fix in our production environment and it work.
Don't think we need to update any docs as this fixes a low level bug.

[codecov]

Codecov Report

Merging #1018 (24aab25) into master (b5a13cc) will decrease coverage by 0.09%.
Report is 135 commits behind head on master.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master    #1018      +/-   ##
==========================================
- Coverage   78.06%   77.98%   -0.09%     
==========================================
  Files          83       83              
  Lines        3999     4010      +11     
==========================================
+ Hits         3122     3127       +5     
- Misses        598      604       +6     
  Partials      279      279              

Files	Coverage Δ
pipeline/mutate/mutator_hydrator.go	64.28% <50.00%> (-1.81%)	⬇️

[aeneasr]

The go stdlib HTTP client should do that automatically, does it not?

[mvanderlee]

From the stdlib HTTP docs:

// If the Transport requests gzip on
// its own and gets a gzipped response, it's transparently
// decoded in the Response.Body. However, if the user
// explicitly requested gzip it is not automatically
// uncompressed.

So because we set the Accept-Encoding header, we must decompress ourselves.
Based on what I and Juan131 observed, the transport was not decoding the response prior to that either.

It's been over 2 years since I touched Go and we've been running this code in production all this time.
However, I have not yet tested again with the latest version of Go.