How to Achieve Interoperability with Different AMQP Versions

At first, one could assume that the protocols AMQP 0.9.1 and AMQP 1.0 are very similar and that the differences between them cannot be that severe. Unfortunately, the naming could lead to confusion for those who are not familiar with the concepts of both protocols. In fact, AMQP 0.9.1 and AMQP 1.0 should be treated like completely different specifications. The Advanced Message Queuing Protocol (AMQP) was released for communication between clients and messaging brokers. It is widely used and supported by several broker implementations: RabbitMQ, ActiveMQ and SwiftMQ, to name just a few examples. The architecture that is assumed by AMQP 0.9.1 relies mainly on queues and exchanges.
In contrast to that, AMQP 1.0 was developed to be more universally applicable and therefore does not know concepts like these entities. By assuming less, the protocol should be more flexible to use.
As also stated on the AMQP website, the protocol was introduced “to become the standard protocol for interoperability between all messaging middleware”. Using the RabbitMQ broker as an example, we want to take a closer look at the extent to which AMQP is supported and how different versions interoperate with each other. Therefore, in the following, we will compare the use of three Java client libraries: RabbitMQ, SwiftMQ and Qpid JMS. The RabbitMQ broker requires an activated AMQP 1.0 plugin for this.
The competing consumer and publish-subscribe patterns were implemented using these clients within this repository. Feel free to have a look at the implementations and how they differ from each other! Additionally, there were different interoperability examples implemented which build the basis for this blog post.

Most of the established brokers only or mainly support AMQP 1.0 since this protocol version was introduced as an OASIS standard. In contrast to that, RabbitMQ focuses on AMQP 0.9.1, and AMQP 1.0 support can only be added via a plugin. Nevertheless, since SwiftMQ, ActiveMQ and Qpid are JMS-based, there are fewer AMQP 1.0 client libraries available for programming languages other than Java.
In case a broker is needed which supports AMQP 0.9.1 and AMQP 1.0 on equal levels, the choice will remain limited:

Coming back to RabbitMQ as an example, their Java client only supports sending and receiving AMQP 0.9.1 messages. So another library is needed to use the AMQP 1.0 plugin. At the moment of writing this article and as also stated in the documentation of the plugin, there are only a few clients for AMQP 1.0, especially for Java. The following clients are also partly mentioned on Github:

As already mentioned, AMQP 1.0 is not based on queues or topics. The target (or source) of a message has to be specified using the address field of an AMQP 1.0 message like it is described in the plugin documentation.

Since the concepts of AMQP 0.9.1 do not exist in AMQP 1.0, the Java clients do not offer the possibility to manage queues, topics and bindings. Therefore, these entities need to already exist when the application is started or can be created using RabbitMQ’s REST API. However, the REST API is only available with the help of the management plugin that needs to be enabled.

This approach of managing queues comes along with some constraints in comparison to AMQP 0.9.1 clients: The entities that are created with HTTP requests are not client-bound and therefore cannot be created as “exclusive”. Automatic deletion needs to be achieved through other queue properties like auto-delete and TTL. In addition, the REST API does not offer to create queues without names to let the broker choose a (temporary) unique queue name. This leads to the consequence that unique names have to be managed by the application.
In most cases, these requirements will only be needed in specific scenarios. Nevertheless, it should be clear that RabbitMQ in conjunction with AMQP 1.0 can only be used with partially limited functionality.

As we already know, AMQP 0.9.1 and AMQP 1.0 should be considered as completely different specifications which leads to the questions: Is interoperability between these two protocols possible “out-of-the-box”? And if not, what has to be considered to achieve interoperability? What about interoperability between different clients speaking the same protocol?
These points are especially important when we think again about an (evolving) event-driven architecture. The components of a system should be able to exchange events and data independently of the protocol they are using.
As also described within RabbitMQ’s plugin documentation, interoperability between AMQP 0.9.1 and AMQP 1.0 is provided with some constraints:

As denoted in the table above, when testing the different clients, it was generally possible to send and receive messages using AMQP 1.0 and AMQP 0.9.1.

For AMQP 1.0 messages, it should be considered that AMQP 1.0 uses the so-called AMQP-encoding. This means that all messages that are sent with AMQP 1.0 are encoded beforehand. The grammar of this encoding is described within the AMQP 1.0 specification. However, this grammar can be implemented differently by each client.
The consequences will become clear when looking at an example:

The examples show that there are some additional bytes appended in front of the messages. The reason for this is the AMQP-encoding that is used by the message producers. Qpid and SwiftMQ both implement the AMQP specification correctly, but use a slightly different interpretation. It becomes more clear when looking at the hexadecimal representation of the exchanged message: SwiftMQ uses 0x80 as a descriptor code which represents an “unsigned long with 8 bytes”. In contrast to that, Qpid uses 0x53 as descriptor code which symbolizes “small unsigned long”.
In terms of the transferred bytes, Qpid JMS is much more efficient. However, both interpretations are valid and work correctly.
Nevertheless, the question remains if it is a good approach to allow different possibilities to basically express the same.

If AMQP 1.0 messages should be readable for AMQP 0.9.1 clients, there are some constraints like it is also described in the table. AMQP 1.0 offers the possibility to send data in different fields of the message body: there can be one or more data sections, one or more amqp-sequence sections or a single-value section. The data sections are only allowed to contain binary data while the other sections can contain any primitive or composite type. The RabbitMQ AMQP 1.0 plugin can only convert version 1.0 messages to version 0.9.1 messages if they are sent within a data section. If this is the case, the RabbitMQ plugin can decode the binary data to send it to the 0.9.1 consumer. If messages are sent as part of other message sections, the plugin does not know how to transform them because information about the target format would be needed to do this. Of course, this should not be the responsibility of the plugin. Therefore, those messages are directly forwarded in AMQP 1.0 format without any conversion.

As already mentioned, AMQP 1.0 allows multiple data sections with binary data. However, if the messages should be received by an AMQP 0.9.1 consumer, the RabbitMQ plugin can only handle messages with a single data section. The reason for this is that AMQP 0.9.1 only specifies a single data value. The plugin does therefore not know how to translate multiple data sections correctly.
If you want to dig into the details, you can find the actual implementation of the RabbitMQ plugin here.

To send messages as data sections and therefore readable for 0.9.1 clients, it needs to be explicitly implemented within the code. For this reason, it needs to be known at the time of the implementation if there are or will be communication partners who do not speak AMQP 1.0.
For a Qpid client sending messages to a RabbitMQ client (line 5 in the table), this implementation could look like the following:

Qpid JMS Producer

A Qpid JMS consumer can also receive AMQP 0.9.1 messages (line 6 in the table). Nevertheless, it needs to be checked for the original type of the message for appropriate conversion. Messages of the type TextMessage can only be sent by an AMQP 1.0 producer because the amqp-value field needs to be used for sending. A message of the type JMSBytesMessage can be sent by AMQP 0.9.1 or AMQP 1.0 clients. While JMSBytesMessages of AMQP 0.9.1 producers arrive within the data field, byte messages of AMQP 1.0 producers arrive in data, amqp-value or amqp-sequence sections:

Qpid JMS Consumer

Like it can also be seen in the code, the Qpid consumer does not have to check explicitly in which message section the data was sent. This check for data, amqp-sequence or amqp-value section is done internally by the client library. In contrast to that, this needs to be implemented by yourself when using SwiftMQ.

As an additional side-note: Maybe you also read about the RabbitMQ Shovel plugin which was developed to move messages from a source in one cluster to a destination in another cluster. This plugin also supports AMQP 0.9.1 and AMQP 1.0. Since the same implementations for conversion are used internally, also the same constraints in terms of interoperability apply.

When using RabbitMQ as a broker, interoperability with AMQP 0.9.1 and AMQP 1.0 is possible. Nevertheless, this can only be done when some constraints are considered at the time of the implementation. Therefore, when adding new components to existing systems, the question remains if these considerations can always be taken into account. And if this can be taken into account, it may still be easier to stay with one AMQP specification.
At this point, it can also be added that at least for RabbitMQ, AMQP 0.9.1 remains supported in the long term.

Instead of using the AMQP 1.0 encoding, it can also be considered to use an established encoding like Avro, Protobuf or JSON. These formats are much more widely used and provide more flexibility than the AMQP 1.0 message format. As an example, Avro allows generating classes from schema definitions and managing schema evolution.
AMQP 1.0 was introduced to become a new standard in interoperability between messaging middleware. Still, some systems are relying on AMQP 0.9.1, and interoperability with these systems cannot be guaranteed. In terms of this, it might probably be a better choice to use a more established encoding. Additionally, the grammar for AMQP 1.0 encoding leaves room for ambiguity.

To summarize, the advantages of AMQP 1.0 in comparison to other widely-used encoding formats became not clear to me. Perhaps several usage scenarios need to be considered to make a final judgment. Feel free to leave a comment with your thoughts about this!