Decentralising the Social Web
(and other stories)

The previous two chapters had social-science leanings, in both the literature surveyed and the studies carried out. This chapter marks the beginning of the more technology-focused half of the thesis. It is necessary to define jargon, discuss software specifications, and talk about the details of specific technologies used in particular systems. This is where I try to align my theoretical findings with outcomes that are practical and meaningful for software architects and developers.

First of all, this chapter presents a survey of decentralised social systems: systems which store data about a person, their attributes and activities, and encourage sharing of this data and interactions with others in the network. These systems are analysed along various axes to determine their intended function, fitness for purpose and how they are or were ultimately used and to what extent. The core focus is on treatment of individuals as users of or participants in a system, and I classify systems according to their approach to handling user identities and profile data. I derive modules of decentralised social systems and the contexts in which they are likely to work well and benefit users; I also attempt to identify gaps and common pitfalls in existing theoretical and practical work.

I finish by describing the most recent work (still ongoing at the time of writing) of the W3C Social Web Working Group, which is producing standards for decentralised social interactions on the Web.

Identity and reference are a critical part of broader Web architecture [websci, halpinthesis], and in some systems built on top of the Web, this distinction matters. This topic was the subject of the infamous 'httpRange-14' issue debate, a lengthy discussion (over a decade of W3C mailing lists exist) and a supposed resolution about the meaning of an HTTP URI.

Q: When is a person not a person?

A: When they're a document.

HTTP URIs can be used to represent both Web-based documents (information resources, IRs) - which can be transported over HTTP - and everything else (non-information resources, NIRs, like people and places) [http14]. The latter, needless to say, cannot be transported over HTTP (yet). In this case, some systems separate people from the documents about them by using fragment URIs (fragments aren't passed to servers, so the server can only return the resource represented by the URI up to and not including the fragment) or 303 redirects (the server says "sorry this URI identifies a NIR which I can't return, so I have to send you to an IR about it instead").

Not all systems make this distinction however. For some, conflating metadata about a webpage and metadata about a person is simply not a problem, or not one worth solving. However, when we are thinking about federated systems, architectural differences in data models like this may cause interoperability problems. The representation feature of profile-hosting systems which was elicited in the first study of the previous chapter demonstrates some variations in how people and profiles are modeled in various contemporary social systems.

Other ways systems vary their data models are by using different vocabularies and syntaxes. In many cases, open data models for representing people and their activities within social networks have been created over the years independently of software implementations that use them, and are sometimes picked up by separate projects. Common vocabularies, or mappings between them, significantly improves interoperability options. The following are well known, and may be referred to as in-use by other protocols or implementations throughout the remainder of this chapter:

• FOAF: An RDF vocabulary with various properties for representing attributes of people, as well as the links between them using the knows predicate. A cornerstone of any Semantic Web based social projects [foaf14].
• Relationships: An extension to FOAF to provide more granular relationships than knows [foafship].
• SIOC: An RDF vocabulary for representing online discussions, such as blogs, forums and mailing lists. Complements FOAF [Passant2010].
• vCard:A standard for representing attributes of a person that might be found on a business card. A common export format from email clients and mobile address books, and the core of many 'contacts' type applications [vcard].
• Atom: An XML based standard for representing streams of content from blogs [atom].
• ActivityStreams 1.0: An XML or JSON syntax for representing social interactions as activities, which consist of actor, verb, object, and target, as well as other more domain-specific properties; widely deployed, in particular as part of the OStatus stack. Predecessor to ActivityStreams 2.0 [as1].
• microformats2: A set of classes which can be used with HTML elements to provide semantics about the value of the element. The classes cover types and properties for a core set of social objects (eg. people, organizations, events, locations, blog posts, products, reviews, resumes, recipes) as well as a specification for how to parse them. Built on vCard. [mf].
• XHTML Friends Network (XFN): A set of values for the HTML4 rel attribute which indicate a relationship between the person who created the link and the person being linked to (eg. "met", "coworker", "parent") [xfn]. Works with microformats2. This one is not bothered by httpRange-14, ie. my homepage can be friends with your homepage.

Certain more modular (ie. only do one thing) open standards are used by multiple projects. I describe them here for convenience (references are therefore not inline when they are mentioned again):

• Webfinger: is a discovery mechanism for user profiles using the .well-known URL pattern and user@domain.tld style identifiers [webfinger].
• WebID+TLS: an authentication mechanism that uses FOAF files and browser certificates [foaftls].
• OAuth 2: a commonly used authentication mechanisms which uses bearer tokens [oauth2].
• JSON-LD: a JSON syntax for RDF [jsonld].
• Semantic Pingback: a federation protocol which alerts a resource when it has been linked to on the Web [sp].
• PubSubHubbub: a federation protocol for subscribing to publishers and delivering content to subscribers [push].
• Salmon: a federation protocol for passing responses to decentralised threaded conversations 'upstream' so the originator of the conversation is notified [salmon].

Some projects publish the specifications of their systems as protocols that anyone can implement. Projects which publish their protocols and state their intent to have interoperability across multiple independent projects, but in practice have few distinct implementations, include Tent [tent] and RemoteStorage [rs]. The latter bundles Webfinger, OAuth 2.0, JSON-LD and HTTP REST, together with a thin layer of additional requirements, and is published as an IETF draft [rsietf]. Hubzilla [hubz] (a fork? of Friendica) publish the Zot! protocol [zot], but it isn't clear there is uptake outside of Hubzilla implementations.

Implementations may be assembled from an existing set of more modular protocols. SMOB (Semantic Microblog) [smob] and DSSN [dssn] are based on Semantic Web technologies, and use overlapping vocabularies for their data, but different mechanisms for content creation and federation. SMOB uses RDFa in published content, and propagates updates throughout the network with SPARQL/Update and HTTP POST requests. DSSN and OpenLink Open Data Spaces (ODS) [ods], both use WebID, FOAF, Semantic Pingback and PubSubHubbub. ODS is additionally LDP [ldp] (for content reading and writing) and LDN [ldn] (for notifications) compatible.

In some cases, existing sets of protocols have been identified and documented as complimentary. One example is OStatus [ostatus], which combines Atom, ActivityStreams, PubSubHubbub, Salmon and Webfinger. OStatus has several independent implementations listed on its Wikipedia page at the time of writing, including GNU Social (previously StatusNet) and Mastodon, both of which there are hundreds to thousands of instances. Another is Solid [solid], which uses LDP, WebID+TLS, FOAF and Web Access Control [wac].

Conversely, some projects cultivate monocultures, either by not publishing their protocols at all or by gaining little adoption outside of a single core implementation. In computing, a monoculture is when a group of computers all run the same software. Several FOSS Social Web projects meet this description, with large and active developer communities who all contribute to the same codebase (Pump.io [pump], NextCloud [nextc], Friendica [friendica], Hubzilla). One particular piece of software is expected to be downloaded and installed on multiple different servers, and different instances can interoperate with each other so it runs in a decentralised manner. This has the advantage that, assuming a straightforward setup process, new instances can be set up quickly and easily, aiding adoption. However, a security vulnerability in the core code base would be present across all instances; there is overhead for developers working on the project to agree its direction and realisation; and focusing on this type of architecture can mean that optimisations for same-software interoperability come at the expense of potentially more open protocols, as is the case with diaspora* [dcent]. Monocultures can emerge when one implementation of an open protocol is particularly successful; an example is Known [known], which implements the open standards Micropub [mp] and Webmention [wm] but also has an easy setup process, good community support, and a for-profit company behind it which has driven wide adoption through managed and white-labelled instances for educational institutions, massively increasing the dependence upon a single codebase.

Friendica is based on DFRN [zot], but has code which bridges to a wide variety of decentralised protocols, including OStatus and diaspora*, as well as the APIs of centralised services. Systems like diaspora* and GNU Social are moving in this direction too, desiring to federate with instances of software implementing totally different (or, sometimes overlapping) protocols.

Research into Personal Data Stores (PDS) spans back a long way in the Personal Information Management Systems (PIMS) field. Once a PDS can communicate with another PDS, grant access to a third-party, or PDS owners are able to spread their data around across multiple separate systems or instances, we find many parallels with decentralised social networking [crit12]. MyDex [mydex] is a commercial offering which provides software to organisations who wish to securely hold data on behalf of their customers. With an emphasis on consent-based data sharing through their proprietary API, datastore owners can authorise trusted third-parties to access their data in exchange for a useful service. indx (previously known as WebBox) [vkpds, vkpds14, indx] is similar, but based on Web standards rather than proprietary technology; it is expected that individual users will host their own instances rather than relying on a provider (and an academic rather than commercial project). PrPl [prpl] on the other hand indexes personal data aggregated from other services, and provides a 'Personal Cloud Butler' to negotiate data sharing with other parties who may be able to provide something in return. Data is indexed with RDF, but they created a new query language (SociaLite) for federating and searching across instances. The team behind PrPl outline a deployment plan which uses devices already existing in people's homes such as media centers to allow self-hosting with low setup cost.

NextCloud (the actively maintained fork of ownCloud) is a free and open source PDS which lets users manage files and media as well as data such as calendars and contacts. Social features such as file sharing, tagging, and commenting are possible across separate instances of NextCloud through the OCS (Open Collaboration Services) federation API. NextCloud provides hosted services, as well support for self-hosted instances. OpenLink Open Data Spaces (ODS) is similarly hosted and for similar purposes; it is developed by a commercial organisation, and instances federate using existing open standards.

The RemoteStorage protocol provides users with the opportunity to trust their data to a third-party provider, or to set up an instance of the software on their own server. It is a grassroots rather than commercial effort, and intended as a generic personal data store. Solid is an open source project with its roots in academia; multiple server implementations exist, and are known as PODS (Personal Online Data Stores). The data stores are generic, with little logic built into the servers, and use a combination of existing open standards with small specialisations to communicate with other instances by means of smart client applications and access control rules.

The data stored in a PDS may be distinctly personal, shared with third-parties only in specific contexts for a particular purpose (eg. sharing medical data with a doctor's office); or it may be inherently social, expected to be broadcast on creation to at least one other person and possibly subject to further reshaping, sharing, and propagation throughout a network of people. Many datastores are optimised for one or the other of these scenarios and may place constraints on their users accordingly. Others, in particular RemoteStorage and Solid, attempt to meet requirements for both types of use and encounter different implementation and usability challenges as a result.

Many systems are designed with the emphasis on social networking over personal data storage. Even systems which emphasis privacy or access control are still inherently about sharing content through a network. Several of these attempt to replicate the functionality of centralised SNS, and a common focus was around the 'status update' feature of the likes of Facebook. Twitter, with its continuous feeds of short posts, changed the discourse somewhat to 'microblogging'. SMOB augments ordinary text publishing with machine-readable metadata, tags as links to other concepts on the Web, and data portability through RDF. [smob] describes microblogging as a "hybrid of blogging, instant messaging, and status notifications."

The original 'Twitter clone' was StatusNet, an implementation of the OStatus protocol which is now known as GNU Social. Mastodon is another OStatus implementation, and pump.io has the same origins. Friendica, Hubzilla, diaspora* [diaspora] and Known provide similar user interfaces for posting to and reading from a timeline of notes, articles and messages from the profiles you friend or follow, but each use different (combinations of) protocols to pass content between instances. The architecture of Tent is also based on passing around 'posts' (pieces of text with various metadata).

Now, many centralised services offer different types of updates, like check-ins, or multimedia sharing experiences, and decentralised social networks are somewhat lagging behind on that front. The most advanced in terms of media is MediaGoblin, an implementation of pump.io specifically oriented around sharing images and video. Known also provides different 'post types' such as audio and location. DSSN is similar to SMOB in that it is based on Semantic Web technologies, however the reference implementation is integrated into Semantic Media Wiki, so users see changes to wiki pages federated between instances, rather than status updates.

The types of content which social networks enable users to create is pertinent to understanding the self-presentation possibilities of profiles within certain systems. Decentralised systems which cannot accommodate a wide variety of types of content or interaction may in the end not meet users' needs for expressiveness. Leaning on extensible technologies and protocols could be one way to ease customisation of basic systems by communities with particular needs.

A common focus of decentralised Social Web projects is decoupling application logic from data storage. Whilst most of the protocols have client-to-server APIs which would technically allow this architecture (like Pump.io), they don't necessarily focus on it as a strength; others however are designed specifically around this model.

RemoteStorage servers are developed by the same community behind Unhosted web apps [unhosted], at the core of which is the idea that applications should run entirely in the browser so that the entire source code of the application is made available to the user; they can download it and run it locally if desired, and have no need to depend on a third party to run the service. This alone has limited use as data cannot be persisted, so coupling these kinds of Web apps with RemoteStorage servers allows users to authenticate with their own datastore and have the application operate on their data, without the application developer processing that data at a third-party, potentially untrusted, server. The Unhosted community provide several sample applications and a JavaScript library to help with app development.

Solid works in a similar manner, with the explicit aim to reduce the logic of a server, making them simple storage devices which can write data and serve it up again. Applications are expected to be smart, including managing access control for resources. Data is stored on the servers as RDF, so applications using shared vocabularies can reuse the same data, giving users even more freedom of choice. Again, the core Solid team are developing sample applications and helper libraries to encourage development of applications by others. OpenLink ODS has several protocols in common with Solid, and there is at least one clientside application (dokieli) which works with both Solid PODS and ODS datastores [do-ods, solid]. PrPl is similar, describing the potential of empowering users through choice of applications.

NextCloud encourages community development of applications which operate on user data, but these exist as plugins to the NextCloud architecture so they're a little more tightly coupled. SMOB, indx and Tent also purport to prefer this kind decoupled-application of architecture, however I was unable to find samples or galleries of applications which have been developed to work with the accompanying servers, and these projects are now largely retired.

There is some overlap between the mindset behind the PDS-type systems and the decoupled-application architectures. Where client-to-server APIs exist for systems like Pump.io, diaspora* and GNU Social, these are typically employed to build applications on different platforms (ie. desktop and native mobile clients for different operating systems) rather than with the idea that multiple Web apps should exist to do different things with the same data stored on someone's instance. This is perhaps due to the broader notion of what kinds of data are stored on a server for PDSs compared to more focussed microblogging and social sharing sites.

[crit12] advocates for open standards, not as the solution to decentralisation problems, but as a prerequisite for interoperability success. Standards "needs serious developer commitment as well as the involvement of standards bodies with significant authority."

Ultimately, the Social Web Working Group has produced the following recommendations:

• ActivityStreams 2.0 Core and Vocabulary: A JSON data model and syntax for social content and interactions, with extensibility based on JSON-LD.
• ActivityPub: An API for creating content, delivering notifications, and managing common side-effects of interactions within social networks, based on ActivityStreams 2.0.
• Linked Data Notifications: An API for delivering and reusing generic notifications, based on JSON-LD.
• Micropub: An API for creating content, based on microformats2.
• Webmention: An API for delivering notifications when a resource on the Web refers to the URL of another.
• WebSub: An API for managing and fulfilling subscription requests to Web content (formally known as PubSubHubbub).

And the following Working Group notes:

• JF2: An alternative JSON syntax and vocabulary for representing social objects, based on microformats2.
• Post-type Discovery: An algorithm for converting between implicitly typed social objects (like JF2) to explicitly typed ones (like ActivityStreams 2.0).
• Social Web Protocols: A guide to the specifications produced by the group, including how they overlap or complement each other.

Here we uncover common modules of decentralised social systems from previous discussion, and look at where there are overlaps and things that stand out.

Identifiers: Most of the systems mentioned use URIs to identify and locate individual resources (like blog posts). Users of the systems, or their accounts, are identified by a combination of Webfinger URIs and HTTP URLs. A useful point of alignment between different systems might be a standard for mapping between Webfinger URIs and HTTP URLs for user profiles.

Data storage: Users of decentralised systems must be acutely more aware of the location(s) of their data. When faced with choices between multiple possible third-party providers (datastore providers or particular hosted software instances), or self-hosting (either on one's own local hardware, or a rented webhosting service), ones personal data gains a sense of concreteness, somewhat divorced from activities which might create the data, of interacting and socialising online.

Trust: In most cases, instances federate by passing data from one server or PDS to another. Unless the data is end-to-end encrypted and able to be interpreted by authorised clients only, users ultimately need to trust the servers of their entire social network in the cases where their posts are broadcast to all of their followers, and their followers responses are broadcast to all of their followers. This creeps into the territory of imagined vs. actual audience, which individuals might not be fully aware of.

Content: Content and activities being passed around between different instances and implementations of protocols means that it is likely to be seen in different user interfaces, and as part of different streams of other data, and even perhaps with additional affordances or missing features. Systems which mix and match functionality of different protocols, and are extended or otherwise customised, will have a significant impact on how people interact with and consume content from others, and in a way that will be fairly unpredictable to the content publisher.

In the next chapter we take a deeper look at the Social Web Protocols and the context in which they were developed.