Magento 2 Message Queue Framework (Enterprise)

Today, we are going to shed light on feature that is available only in Enterprise Edition. Meet the Magento 2 Message Queue Framework – a system that provides modules with the ability to send messages to various queues. Besides, it defines consumers that receive messages asynchronously.

The Magento 2 Message Queue Framework is based on RabbitMQ that acts as a messaging broker, providing a platform for a message exchange. You can find the official installation guide for RabbitMQ here: RabbitMQ on Dev Doc. Furthermore, the system incorporates a special mechanism that provides the ability to store undelivered messages. Let’s describe how the Message Queue Framework works. The following diagram illustrates key components of the framework:

Where:

• A publisher is the first major component responsible for sending messages to an exchange. It understands several important aspects. First of all, the format of the messages. Besides, the publisher knows which exchange to publish to.
• An exchange is incorporated into a broker – the intermediate component based on RabbitMQ. It receives messages from the publisher sending them to queues. Among multiple exchange types supported by RabbitMQ, Magento 2 utilizes only topic exchanges. Every topic consists of a routing key based on several text strings separated by dots. As a result, we have the following format for a topic name: string1.string2.string3. Use “*” to replace one string and “#” to replace several strings.
• A queue is another part of the broker. It is a buffer for storing messages.
• A consumer the last component of the Magento 2 Message Queue Framework. It receives messages and knows which queues to consume. Besides, the consumer is able to map processors of the message to specific queues.

If RabbitMQ seems too complicated or unreliable, you can utilize a MySQL adapter instead of it, since the adapter can use database to store messages. As a result, you will get a simple message queue system with three database tables and the usage of cron jobs that ensures that consumers can receive messages. We do not recommend to use this replacement for RabbitMQ, because it shows low scalability. In its turn, the Magento 2 Message Queue Framework can be utilized whenever you wish.

Magento 2 Message Queue Framework Configuration

If you want your module to leverage the Magento 2 Message Queue Framework, it is necessary to build a <module>/etc/queue.xml file as well as define such components as:

• publisher
• consumers
• exchanges
• bindings

While such elements as publisher, topic, consumer, and binding can be incorporated within the queue.xml file, there are also required elements. The following code lines must be included in each queue.xml file:

As for other elements, they are described below.

The publisher element

The publisher element performs two vital actions: iit configures the type of connection and the same time sets up the exchange to publish to. Magento 2 uses only one exchange out of the box, but multiple exchanges are possible. As for the name of every exchange, it is a part of publisher configuration. The publisher element incorporates the following parameters:

• name – a unique identifier related to the publisher. A topic element specifies its value. Note that the default system publisher is named default.
• connection. In case when RabbitMQ is used for the queue management, the value of this parameter must be rabbitmq. Besides, it can be db; alternatively, the name of a custom adapter can be used.
• exchange illustrates the name of the exchange to publish to. The bind element gives its value. magento is its default name.

The topic element

The topic element determines the interface necessary for processing the message and assigning the publisher. It incorporates the following parameters:

• name is a parameter assigned to the topic. It utilizes the following format: object.action + .subaction. It is necessary to use verbs in the form of the past tense for indicating that an event has happened. As for the value of the name parameter, it is specified in a bind element.
• schema describes the message structure. Use the Data Interface format from the Service Contracts. Besides, there is another way to specify a service method signature: Magento\Customer\Api\CustomerRepositoryInterface::save. Please note that you should format your message as an array of all available service method parameters in this case, like in a situation with a call_user_func_array call. The callback of a consumer should expect each part of the message to be passed in a form of a separate parameter.
• publisher illustrates publisher’s name.

The consumer element

As for consumer elements, they map the message receiver to a specific queue. Such parameters as class and method indicate what elements of the Magento 2 Message Queue Framework receive and process the message. The consumer element incorporates the following parameters:

• name shows the name of the consumer. Its value should be exact the same as the appropriate magic method utilized as a callback.
• queue shows where to send the message by introducing the queue name. Its value is utilized to define a bind element.
• connection has one strict requirement: it must be rabbitmq; other value from the publisher’s connection parameter is also possible.
• class shows the path to a class which consumes the message.
• method illustrates the method from the specified class utilized in message processing.
• max_messages sets the limit for messages to be consumed.

The bind element

The bind element of the Magento 2 Message Queue Framework defines the message queue topology by connecting topics with exchanges and queues. You can send every topic to an unlimited number of queues. The bind element incorporates the following parameters:

• queue is the name of a queue previously defined via the consumer element.
• exchange is the name of an exchange previously defined via the publisher element.
• topic is the name of a topic previously defined via the topic element. Supports asterisk and pound sign.

The example of the queue.xml file utilized in the Magento 2 Message Queue Framework is provided below:

Source