A Matter of Semantics

Structuring client-server communications with hypermedia messages

Messages on the Web carry three levels of information: Structure Semantics, Protocol Semantics, and Application Semantics. No matter the implementation style, all three of these are needed for any successful communication between client and server. This threesome (S-P-A) forms the essentials of communication over distributed networks.

Most of the time, though, these levels are obscured or muddled at implementation time. For example, both Protocol Semantics (how we create valid network requests) and Application Semantics (domain names like users, customers, orders, etc) are often mixed together in conversation ("You POST new users to this URL") and both of these are usually only defined in human-readable documentation and implemented in the source code of the client application itself. In other words, the protocol-level and application-level semantics are tightly coupled. An easy way to discover this is to see if you can take the same message format and implement your API using a protocol other than HTTP (e.g. WebSockets or FTP). I illustrated this "protocol-agnostic" design pattern back in 2010 ("A RESTful Hypermedia API in Three Easy Steps").

But there is a way to keep these separate from each other and view each of these aspects in their own light. In doing so, you’ll strengthen the quality and value of your message design while increasing flexibility and choices.

Structure Semantics

Structure Semantics provides the set of rules regarding how to create a well-formed message. XML has rather simple structure semantics. JSON's rules for well-formedness are a bit more vague but reachable since JSON.parse(…) turns out to be the ultimate arbiter of such things. Determining well-formedness of other, more complex media types (HTML, Atom, HAL, Collection+JSON) is tougher, but do-able even if external validators are not always available.

Building a successful web implementation that does not contain structure semantics is difficult—and that's a good thing.

Protocol Semantics

Protocol Semantics is the set of rules regarding how to fashion valid requests and responses using one or more message protocols. HTTP, FTP, WebSockets, etc. are all message protocols. Most implementations for the Web use human-readable documentation to describe these semantics. For example, typical RPC-style implementations document URIs, associated request rules (HTTP methods and headers), and any possible body content.

Hypermedia-style implementations express the protocol semantics directly in the message (usually in the body, but sometimes in HTTP headers). The most common way this is done is to include rules for links and 'forms' that may appear in server responses (think HTML).

Registered media types don't often include hypermedia controls. For example, CSV, XML, and JSON have no definitions for hypermedia expression within the message. However some media types such as HTML, Atom, HAL, Collection+JSON (and others), do include the predefined ability to express protocol-level information within the message sent from the server.

Application Semantics

Application Semantics govern the way in which application-specific information is shared between client and server. This includes names of domain data elements (users, customers, products, etc.) and specific actions within the domain (adding a user, calculating tax on purchases, etc.). Well-formed requests and responses are ones that contain valid expressions of these domain-specific concepts. There are very few registered media-type designs that contain this level of specificity and the ones that do contain it are usually aimed at an industry-wide domain (e.g. VoiceXML and Maze+XML).

There are some machine-readable examples of application semantics that can be used in conjunction with existing structurally semantic forms (XML, JSON, etc.). XSD, JSON-Schema, and other schema-based document types do a good job of expressing application-specific information. There are other examples such as WSDL for SOAP and its cousin WADL.

Most of the application-level semantics exist as stand-alone content with no standardized way to apply that information to a message. Some media type designs are better than others at supporting application-level semantics. HTML, for example, has several attributes (id, name, class, rel) with which you can apply application-level semantic information to a message.

Semantic Profiles

An alternative approach to application-level semantics is the use of a "Profile." Tantek Celik has been championing this approach since 2003 with his XHTML Meta Data Profiles pattern for HTML. The "profile" link relation was recently registered with the IANA and there has been some renewed interest in this model. The POWDER spec uses the "describedby" link relation in a similar way, too.

The Profile pattern lets you outline the application semantics independent of the protocol and structure. You can collect up all the data points (userName, address, email, phone, etc.) and all the possible actions (addUser, editUser, approveUser) into a single place. The profile provides a clear, stand-alone description of the problem domain that can be kept up-to-date without mixing in protocol (HTTP, WS, etc.) or format (XML, Atom, HTML, etc.) details.

Even better, you can create a machine-readable version of this profile: one that can be referred to at run time by both client and server implementations. The Application-Level Semantic Profile (ALPS) format was designed for this purpose. This is an early attempt at a specification that supports profile descriptions that can be shared and applied to multiple media types (structures) and protocols. It's too early to know if this kind of approach will become widely adopted, but I think it is worth pursuing.

Design Hallmarks

Designing a message format requires solving for all three levels (structure, protocol, and application). The simplest approach is to select an existing structural format (XML, JSON, etc.) and then use human-readable documentation to apply the other two (protocol and application). This requires added implementation coordination between client and server, but is quite common.

Selecting (or designing) media types that include protocol semantics and/or support applying application semantics directly within the message puts added responsibility on the message designer(s) and standardizes much more of the way client and servers interact. This can increase the likelihood that compatible implementations can be done at a distance (both in space and time) and that implementors (client and server developers) have an opportunity to safely evolve communications over time including adding new interactions to the domain-space without the need for prior coordination with the initial message designer.

These two aspects of the Web, support for distance and evolvability, are a hallmark of hypermedia-style designs.

tags: , , , ,