Consider allowing caching endpoint discovery #113
Description
Right now, the spec doesn't allow webmention 658B senders to cache discovered endpoints:
3.1.2 Sender discovers receiver Webmention endpoint
The sender MUST fetch the target URL (and follow redirects [FETCH]) and check for an HTTP Link header [RFC5988] with a rel value of webmention. If the content type of the document is HTML, then the sender MUST look for an HTML and element with a rel value of webmention.
We've been talking about relaxing this, eg by changing MUST fetch to SHOULD fetch, and maybe including more explicit language around what kind of caching might be ok.
This generally won't matter to small/medium individual web sites, but it definitely does for large services like Bridgy, which currently sends 1-2 webmentions per minute.
Bridgy currently caches endpoints for 2h, per domain, with home pages as a special case since they sometimes don't advertise an endpoint. Specifically, Bridgy's cache key is [domain, http or https, home page or not].
The first small, hopefully non-controversial change we could make here would be to allow caching at all, for individual target URLs. After that, there's a larger conversation around whether and how to allow more broad caching, eg by domain. The counterargument is per-URL webmention endpoints, which the community has experimented with, eg to expire them after a brief time window, but that idea didn't pan out, and I don't know that we've found a compelling need for them yet otherwise.