How to Integrate Automation Data with Industrial Messaging Protocols

The protocol that went to the cloud and back is here to bring your data along for the ride.

I Stock 1157557928

In spite of recent economic setbacks, the manufacturing industry continues to pursue digital transformation. Users want more data. More than merely supporting production, they want automation systems that deliver actionable insights and easily integrate into a cohesive data network extending from the manufacturing floor to the executive office. 

However, there are many obstacles to creating the level of integration required to fulfill that vision. Moving a single I/O signal from the factory floor to the cloud requires a technology stack that involves many layers and many players. Each layer adds complexity, which negatively impacts the overall security and scalability of these systems and increases labor and cost (Figure 1). 

Figure 1: Providing even basic equipment information to central business applications involves a complex hierarchy of software and hardware systems.Figure 1: Providing even basic equipment information to central business applications involves a complex hierarchy of software and hardware systems.Opto 22

Fortunately, new technologies are coming to the fore that provide an alternative to the traditional technology stack. This article looks at one of these: a lightweight, open-source, publish-subscribe communications protocol designed for large-scale industrial device integration and the internet of things (IoT) called MQTT. 

A Quick History of MQTT 

MQTT, formerly known as MQ Telemetry Transport, has gained popularity in recent years with everyone from Raspberry Pi enthusiasts to developers in smart energy and even consumer messaging applications. The roots of its development reach back much further, however.

In the 1990s, ConocoPhillips (now Phillips 66) was looking for a way to improve telemetry reporting over their low-bandwidth dial-up and costly VSAT (small satellite dish) SCADA network. IBM partnered with system integrator Arcom Control Systems (now Cirrus Link Solutions) to develop a minimalist communication protocol that could gracefully handle intermittent network outages and high latency among many distributed devices over TCP/IP. 

IBM abandoned the traditional poll-response communication model in favor of a brokered publish-subscribe model. In that scheme, a central server (the broker) manages data delivery for the entire network (Figure 2). MQTT-enabled field devices and gateways publish data to the broker when they detect a change in a monitored value, rather than in response to cyclic update requests. The broker then routes published data to any registered subscribers, which can be back-end software applications or even other field devices (think: Twitter for machines). 

Figure 2: An MQTT broker serves as the communication hub for industrial data publishers and subscribers, each communicating through a secured connection.Figure 2: An MQTT broker serves as the communication hub for industrial data publishers and subscribers, each communicating through a secured connection.Opto 22

Scaling up with MQTT 

Combined with a streamlined payload—as small as two bytes—the communications model used by MQTT results in an 80-90% reduction in overall bandwidth consumption compared to poll-response protocols, according to Cirrus Link. This efficiency creates significant room for existing networks to grow.

But it’s the decoupled nature of MQTT data exchange that unlocks the potential scalability of industrial networks. 

Legacy communications protocols require some form of addressing between senders and receivers. Each new data consumer separately addresses and interrogates field devices, and if an address changes, every consumer must be updated. 

In an MQTT network, the relationship is flipped. Available data are identified as unique topic strings, for example, “CellA/Oven/Temperature.” Any client that wants access to published data—a maintenance database, ERP, or SCADA system, for instance—can simply point to the common MQTT broker and subscribe to the desired topics without needing any details of the publishing source. Network traffic between publishers and the broker remains constant regardless of how many subscribers receive updates. 

This also has important implications for security. The broker alone manages user authentication, access rights, and message delivery, so each client can remain anonymous to other network members. Additionally, each MQTT connection is device-originating, so no firewall exceptions or complex port forwarding rules are required. Combined with SSL/TLS encryption and certification, secure site-to-site communication is feasible even over public networks. 

Enter Sparkplug B 

After being nurtured for years at IBM, MQTT was released on a royalty-free license in the early 2010s and became an OASIS/ISO standard soon after. The community recognized its potential as an open messaging protocol for the internet of things, and it was quickly adopted by all the major cloud platforms. MQTT is now managed by the Eclipse Foundation.

As MQTT caught on in the enterprise sector, Cirrus Link continued development targeting industrial applications, eventually developing the Sparkplug MQTT Topic and Payload Definition. The current version, Sparkplug B, adds three requirements to MQTT implementations for mission-critical applications: 

  • Enforced state awareness: A device or application that conforms to the Sparkplug spec will always publish a birth message on connection, defining all the topics it publishes and announcing its online status to subscribers. It will also register a death message with the broker to be distributed in case it unexpectedly disconnects from the network. 
  • Payload definition: MQTT is data agnostic, which puts the burden on subscribers to interpret data correctly. Sparkplug packages the MQTT payload with structured metadata so the format can be identified by another Sparkplug client. The full payload is then encoded to maintain a small footprint. 
  • Standard topic format: MQTT’s flexible topic structure can complicate interoperability at scale, since devices from different vendors may use very different naming patterns. Sparkplug defines a standard topic format, reducing variability and enabling clients to automatically discover available data. 

Figure 3: MQTT/Sparkplug B devices and gateways, like Opto 22’s groov RIO, connect the manufacturing floor to a reliable MQTT infrastructure.Figure 3: MQTT/Sparkplug B devices and gateways, like Opto 22’s groov RIO, connect the manufacturing floor to a reliable MQTT infrastructure.Opto 22

With Sparkplug B, an MQTT network is guaranteed to be state-aware and interoperable without compromising flexibility and efficiency. Devices from different vendors can securely share and use data without needing to know the details of where it originated, host applications can discover published data quickly, and all clients are notified in real time when data becomes stale. 

Simplify, simplify, simplify 

Obviously, there is still an investment involved in introducing MQTT to the factory floor. In particular, it needs to be supported by low-level automation and networking devices to move data into the MQTT infrastructure. But MQTT is already being adopted by a variety of vendors, and its open-source nature makes it easy to experiment with.

When we look at the current manufacturing landscape and consider how to get where we need to go, we find many obstacles to digital transformation: a general lack of security, incompatible OT and IT protocols, and complex architectures that are costly to scale. 

MQTT with Sparkplug B makes possible a new, more efficient architecture wherein industrial devices and IT systems can share data securely using a common protocol. And MQTT’s ability to support millions of clients provides exactly the kind of scalability needed if industrial IoT is to become more than a daydream. 

Josh Eastburn is the Director of Technical Marketing at Opto 22. After 12 years as an automation engineer working in the semiconductor, petrochemical, food and beverage, and life sciences industries, Josh Eastburn works with the engineers at Opto 22 to understand the needs of tomorrow's customers. He is a contributing writer at

More in Industry 4.0