caching strategy

[robjtede]

Related to #66 but covers overall caching policies instead of details about just cargo index caching.

We need to consider what our stale-ness tolerance is, especially for the SVG badges (worst case), since I gather than plan will be running a caching CDN in-front of the service. The maximum staleness time will always be the sum of all cache TTLs.

I think ideally, it should be no longer than 1 minute from the time the live crates index is updated to the time a badge is updated.

An sample proposal:

Cargo index cache update time:          20 seconds
Crate query (from cached index) TTL:	10 seconds
SVG Max-Age / CDN TTL:                  30 seconds
==================================================
Total (worst case update time):         60 seconds

[Feliix42]

Thanks for the write-up! Looks like a good approach to me. I guess one minute is a good goal, because otherwise we strain the server more than necessary.
In theory, it would be nice if we could ensure that error badges are not cached, but that may be too much overhead and is not worth it with a cache time of max. 1 minute.

[paolobarbolini]

In theory, it would be nice if we could ensure that error badges are not cached, but that may be too much overhead and is not worth it with a cache time of max. 1 minute.

The Cache-Control header would get set by us dynamically for every response, so that shouldn't be an issue.

Total (worst case update time): 60 seconds

It would be more like 90 seconds for GitHub users, since they proxy every external image through their reverse proxy