Request collapsing
Request collapsing was born out of the need to protect a busy origin server with changing content. With many visitors on a site, a cache miss can amplify requests to an origin server when content expires. This behaviour is commonly referred to as a cache stampede failure or dog-piling.
When request collapsing is enabled, only a single request is sent to an origin server for a given resource, and then using the resulting body to satisfy pending requests.
This can prevent a highly demanded resource in the cache causing a flood of requests to an origin server when it expires. Enabling this feature appropriately on high traffic sites can level out requests and traffic to origin servers.
How it works
Request collapsing is implemented internally using cache keys and queues. The cache keys are used as keys to a map, with client requests tracked using a queue. Responses secondary keys are then used to match waiting client requests for origin responses. Matching requests can then be fulfilled using the same response.
Failed requests are retried in the same manner.
If a response to an active request is marked as private Cache-Control: private or Set-Cookie then all queued
and future matching requests will go directly to the origin without trying to match them.
This implementation allows Peakour to serialise requests to an origin, so its important to only enable the feature on resources that you know will be cacheable, either through cache-control headers or Peakhour configuration.
How do you know its working?
Request collapsing can be verified with the Debug enabled and the cache-status header in the response. A collapsed request will look like the below, noting the 'collapsed' at the end.
cache-status: peakhour.io; fwd=uri-miss; key="https://website.com/home-eco.jpg"; ttl=86400; collapsed
How its used in practice?
Image optimisation
Peakhour.IO uses request collapsing internally for image transforms. The feature ensures we only transform a single image when a resources expires on the edge, saving latency sensitive image transformations, client transform costs and a better all round experience to end users.
Dynamic page caching
Request collapsing is an excellent candidate when enabling caching of expensive dynamically written pages. Peakhour.IO can cache server heavy Wordpress, Magento, Presta Shop, Drupal and other platforms via our caching plugin. The plugin enables seamless operation of the CDN by notifying us when content changes.
Of course, content always needs to change during busy periods such as sales or item purchases. During these periods servers can be overwhelmed when pages incurr a cache miss. Request collapsing can help smooth out this traffic ensuring optimal response times for all users.
Think you could benefit?
Contact us now! We're hiring!