Description
Standardize Profile Extensions Now; Deal with Comprehensive Extensions Later
Extensions can be usefully divided into two basic types—those that fit under the definition of a profile and those that make more drastic changes, which I'm calling "comprehensive extensions".[1] I think we should release the system for handling profile extensions ASAP, and then figure out the comprehensive extension mechanisms later.
Releasing a mechanism for comprehensive extensions needs to be done with some care,[2] but officially supporting profile extensions is relatively easy and it would immediately "unblock" the community, since there are a ton of issues that could be addressed by profile extensions. Allowing the community to develop extensions for these features, and to start using them right away in their apps in a compliant fashion, would be a huge win, and it doesn't preclude us from standardizing any of the things they develop if those ideas pan out and when we have more bandwidth.
Finally, we've already kept people waiting far too long for some official extension system, which is my fault, so let's just get something out the door!
(Note: focusing on profile extensions first doesn't block the operations extension, which we could release in a backwards-compliant way even if the comprehensive extension mechanism isn't ready.[3])
Profile Extension Design Goals
Negotiation for and discovery of profile extensions is dead simple, because it's largely unnecessary.[4] Instead, I think the interesting design problems are around:
- preventing an extension from conflicting with another extension, or with a future base spec addition, used in the same request/response document (optionally, while allowing data provided from multiple extensions to be interspersed with each other);
- surfacing recommended extensions and discouraging use of ones that have known issues (since setting good conventions is a huge part of the spec's goal);
- figuring out if/how a published profile extension can be changed/added to over time;
- making it easy for someone to quickly make an extension and, if need be, get it registered/approved, so that people trying to innovate/experiment don't feel like they're banging they're head against unneeded restrictions;
- making sure that, if we have some kind of review process (e.g. to control quality), we aren't blocking peoples' projects during that time.
An open question is whether we want to incorporate successful extensions into the base spec over time and, if so, how we can do this more-or-less seamlessly after the extension has been widely adopted with the extension syntax. Tentatively, I'm leaning towards "no, an extension should always remain an extension (and we should just make the extension syntax nice)". Under this approach, we could require extension registration and then just have a very prominent list of "recommended extensions" on the site. This should give us almost the same convention-setting power as formally incorporating an extension into the base spec, while giving us much more flexibility (we can replace extensions on that recommended list over time, if necessary) and saving us the design headache of creating a syntax that supports a transition from usage as an extension to usage as a standardized feature (which is harder than you'd think; see here).
Design Proposal
To satisfy these goals, I've been considering the following design:
Syntax/Definition
A profile extension would be applied to a JSON API document by adding a new key–value pair at the top-level of: the document, a resource object, a relationship object, a link, a resource identifier object, or more than one of these places. (Profile extension data could not be added within "attributes" or directly under "links" or "meta".) Every place that a given profile extension is applied, it would have the same key (but, of course, the value could change). Also, each applied extension would be listed in a top-level profile link (details below). So, for example, a hypothetical "last-modified" profile extension might be applied as follows:
{
"links": {
"profile": [/* details later in this proposal */]
},
"last-modified": "2015-10-01",
"data": [{
"type": "articles",
"id": "2",
"attributes": { /* */ },
"relationships": {
"tags": {
"links": { /* */ },
"last-modified": "2015-09-01"
}
},
"last-modified": {"date": "2015-10-01", "changed-fields": ["title", "tags"] }
}]
}This is just to show that the extension could add data in multiple places; that it would always add that data under the same key name; but that the type/meaning of data would be different in different places (e.g. "changed-field" may only make sense in a resource object) and that these semantics would be defined entirely by the profile extension.
Registration Requirement
All profile extensions would need to be registered, but registration would be granted very quickly. The criteria for approval wouldn't involve whether the proposed design is good, but just that the registration request:
- includes a suggested a key name at which the extension's data should be placed. (For the example above, this would be
"last-modified".) This key name should be the same as the key name used by other registered extensions that do very similar things, or the same thing with a different syntax (if any) and different from any registered extensions that this extension expects it could be used with. - includes a specification, detailed enough to allow for interoperable implementations, of the meanings/types of values that can occur as this extension's data, at the different places where that data can occur. Like the base specification, each extension's specification should include must-ignore rules to make it extensible over time, but this isn't a requirement.
- includes some basic contact info for the registrant (name and email, plus perhaps some optional and/or private fields like website and phone number).
- defines an extension that fits the definition of a profile extension (i.e. doesn't try to override the meaning of base spec members or define new data in invalid places)
- reuses objects/patterns from the base json api spec where appropriate. For example, if the extension defines a field that points to another resource, that field should hold a resource identifier object. Or, if an extension uses the key
"type"it should only use that key as the base spec does (i.e. to hold a string indicating a resource's type). Additionally, if the extension includes a mechanism for doing something "architectural" that other extensions may need to do too (e.g. if it defines a syntax to point to another member of the document), and we don't have a standardized mechanism for doing such a thing, the extension may be held a bit for special consideration to check that its proposed mechanism, were it to become a convention, wouldn't have any majorly bad architectural ramifications. - (Optional) includes the minimum base spec version that the extension requires, if not 1.0.
- (Optional) includes a URI by which to identify this extension, that should dereference to the extension's specification document. If no URI is provided, the extension's specification would be hosted permanently on jsonapi.org, and the extension's identifier would be a jsonapi.org URI.
After a registration request is received, there would be some very brief window (no more than a week) for one of the spec editors to check that it satisfies the three conditions above. This period might also allow for public comments, which could be useful just in case someone catches a major problem with the submission that the author didn't know about (e.g. that an extension already exists that fits their needs). But again, the point of required registration isn't to do a design review—it's to prevent certain conflicts (see below) and support extension discovery—so the author wouldn't be bound to act on these public comments.
Before submitting a registration request, authors would be encouraged to see if an existing extension could be extended to fit their needs. And they would be advised, if so, to ask the author of the existing extension to consider tweaking it to cover their case. The existing extension's author may reject the request, of course, but should respond to it within a reasonable timeframe (2 weeks). If the author of the existing extension had died, moved out of contact, or otherwise couldn't make changes that were important to the community, the JSON API editors could reassign responsibility for the extension, to allow it to continue to evolve.
(For truly experimental features whose behavior is too unstable to be standardized, the "meta" key would continue to be an immediately-usable, api-specific playground.)
Using Profile Extensions
-
The profile extensions in use on a given document would be listed in the
Content-Typeheader (as theprofileparameter of the JSON API media type) and in the document's top-level"links"(see below). -
Clients could request particular profile extensions by listing all those they support in the
profileparameter of the JSON API media type that they include in their request'sAcceptheader, as described in Extensions Overhaul #650. -
The
"profile"key in the document's top-level"links"would hold the URIs identifying any profile extensions used in the document. If a profile link is given in "link object" form (i.e. not just a string), that link object would have an extra allowed key, called"key". The value of"key"is the name of the key throughout the document at which the extension will put its data. If"key"is absent, its default value is the extension's suggested key (from its registration). So, consider the following:"profile": ["http://jsonapi.org/ext/last-modified", "http://jsonapi.org/ext/actions"]Here, the suggested key for the
"http://jsonapi.org/ext/last-modified"extension is "last-modified" and the suggested key for the"http://jsonapi.org/ext/actions"extension is "actions". Therefore, the above is equivalent to the more explicit:"profile": [ {"href": "http://jsonapi.org/ext/last-modified", "key": "last-modified"}, {"href": "http://jsonapi.org/ext/actions", "key": "actions"} ]Now, suppose a server wants to send a document with both the
http://jsonapi.org/ext/actionsextension and thehttp://jsonapi.org/ext/transitionsextension, and both were registered with the same suggested key, "actions". To apply both to the document, the server must change one or both of their"key"values, so that they're distinct. Whenever a sender manually sets an extension's key, the key's value must begin with an "_" followed by the extension's suggested key. So the server'sprofilelinks might look like so:"profile": [ "http://jsonapi.org/ext/actions", {"href": "http://jsonapi.org/ext/transitions", "key": "_actions2"} ]The payload would then have two keys,
"actions"and"_actions2", holding the extensions' respective data. Usually, though, extension negotiation and the registration requirements would combine such that two extensions that have the same suggested key would not both be needed in the same document. So this"key": "_{name}"provision is more of a fallback/escape hatch for special cases.
Future (Base) Spec Updates
- Because each extension would only be able to appear under its suggested key or an underscore-prefixed version of the same, the base spec editors could freely add any keys that no extensions have registered with (and that don't start with an underscore), without risk of conflict. After such a key was added to the base spec, future extensions would not be allowed to register with it as their suggested key.
- A profile extension's specification could also be updated, by submitting an updated registration request with the new definition, to accept an expanded set of values, as long as those new values wouldn't break existing clients.
Rationale
I didn't preface my proposal above with any rationale because I want to know if, unexplained, it strikes people as way to complicated. If that's how you felt, let me know! Or, even better, I'd love suggestions for how to simplify the design, while ensuring it still meets the goals listed earlier.
If you're curious about how I got to this particular strategy for avoiding extension conflicts, the various iterations I went through are here (listed from top to bottom).
Footnotes
- See this comment and the rest of that thread for background on the utility of this distinction and the definition of a profile extension.
- Figuring out the constraints and negotiation rules for comprehensive extensions is much harder than for profile extensions, since comprehensive extensions are less likely to compose and be future proof by default. Hence considerations/approaches like this, which would now also need to consider this.
- We'd do this by making operations an optional part of the base spec. This would be totally backwards compatible because: clients that send
"operations"data would be opting into the extension by virtue of forming that request, and servers would signal their support by returning a 2xx, whereas as current/non-supporting servers would return a 4xx on account of the missing"data". Likewise, if clients want to retrieve operations data (e.g. a change list), they can simply append a new query parameter (e.g.?operations) to their request to opt-in and, again, a server understanding the request would send a 2xx while one that doesn't (including all existing servers) would send a 4xx—a consequence of our rule about rejecting unknown query parameters. Moreover, such a design for operations would not be inconsistent, as it's already how the optional sorting and include requests work. - Either party (the client or the server) can send extended data and, if the recipient doesn't know how to interpret some or all of the extensions, it'll just ignore the ones it doesn't understand. Beautiful! If a server wants to minimize unnecessary data transfer, it can choose to send data for a profile extension only if the client indicates that extension in its
Acceptheader; if a client wants to check to do the same, it can perform an OPTIONS request first (as defined in Extensions Overhaul #650) and only send data for supported extensions. Again, really simple.