This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.
Azure Front Door is upgrading its PoP node (point of presence) infrastructure to improve performance and resiliency. While most customers will not notice any changes to the way Front Door proxies HTTP(S) traffic, there are some differences. One of those is stricter implementation of the HTTP Range-Request protocol (RFC 7233). In this article I outline some of these changes, show how to test to see if your origins will be impacted, and provide advice on how to fix origins that are not responding to range requests properly.
Background
In December Microsoft sent a Service Health advisory to Azure Front Door customers via the Azure Portal. The advisory included the following information:
We recently pushed an upgrade causing more range requests to be sent to customer origins (to have additional efficiency in distributing the caching of large customer resources in Azure Front Door). Unfortunately, certain customer origins didn't properly respond to range requests when the “Accept-Encoding: gzip” header was present and returned an invalid “Content-Range” header value, resulting in failed client requests...
...One thing you should verify works properly on the origin/backend is the handling of HTTP range requests. If the origin doesn't have proper support for range requests, it should simply ignore the Range header and return a regular, non-range response (e.g., with status code 200). But if the origin returns a response with status code 206 it implies support for range requests. In that case, it must send a valid Content-Range header. If compression is used, the Content-Range header values must be based on the compressed (encoded) resource size.
For example, suppose an uncompressed resource on the origin is 100 KB; but with gzip compression, it's only 20 KB. Suppose the origin receives a range request for "bytes=0-1048575" (that is, the first 1 MB) and suppose that the header “Accept-Encoding: gzip” is also present. If the origin chooses to return a range response (status code 206) and it chooses to compress the resource (“Content-Encoding: gzip”), it should set the value of the Content-Range to “bytes 0-20479/20480”, indicating that the response contains 20 KB of data (not 100 KB of data).
Read the full Service Health advisory here (requires Azure Portal login to a subscription with Azure Front Door deployed).
This behavior is also documented in Azure Front Door Caching - Delivery of large files.
The stricter protocol implementation policy changes are being gradually deployed to Azure Front Door (AFD) PoP's (point of presence infrastructure nodes). When a request is made to an origin that does not respond properly to range requests, via an upgraded AFD PoP, the behavior I have observed is that the PoP will wait for approximately 75 seconds for the correct response (the correct amount of data as specified in the response header to be returned). When the PoP fails to receive more data from the origin it times-out and returns all data it has received as an HTTP 206 Partial response. The response in this case will never be cached. This may also cause an HTTP 499 error to appear in AFD logs.
Advice & recommendations
Accepting range requests is optional. Origin servers should be configured to either accept range requests (and respond correctly) or ignore range request headers and respond with a full (200 Ok) response.
Nodejs/express
It looks like compression and range requests are incompatible in the nodejs/express/compression module. The compression module won't stream the entire file into RAM because for large files because this could cause out of memory issues. Therefore, the Content-Range header byte-content-range value is not updated to match the actual body data size (the total amount of body data sent once the response has finished). This mismatch is values is invalid according to RFC 7233.
You can reproduce this behavior by building and running the Nodejs/express server and Mocha tests in the companion repo for this article: DanielLarsenNZ/nodejs-express-range-headers.
The solution for Nodejs/express servers is to either support range requests, or compression (not both at the same time).
- Compression: Best performance for scripts, stylesheets, JSON and HTML files, any files that compress well. The express compression module will not compress files that don't benefit from (Gzip) compression (like JPEG files which are already compressed).
- Ranges: Good for very large media files that need to seek or be pre-fetched in shards. Most media formats (like MPEG) are already compressed.
Testing origins
You can manually test origins for correct range-request behavior using curl. The resource being requested should be more than 1kB in size. This command will request the first 1024 bytes of data from that resource as a range request and save the data to a fileoutput.txt.
The standard output from `curl` should include the following headers:
And the size of the body data (written to output.txt) should be 1024 bytes. curl will also write the data size to standard out:
If the data size return is not exactly 1024 bytes, the response is invalid.
Conclusion
I hope that this article has helped to explain why responding correctly to HTTP range-requests is important for optimum performance when using Azure Front Door, and how to test and fix any issues. If you have questions feel free to comment below or tweet @DanielLarsenNZ.
You can also talk to FastTrack for Azure engineers like me in the next Ask FastTrack for Azure event on Microsoft Reactor. Register here.