Everyone has heard the great “investment tip” of the 21st century, right? “Data is the new gold.” But while gold is relatively easy to buy for some money, you can’t do the same with data (except on the dark web, illegally and in a highly condemnable way). For some years now, the availability of private data has been subject to strict rules (GDPR), and B2C providers must pay particular attention to compliance throughout their sales&marketing processes. Only the use of data for which the user has previously given his/her consent is allowed. In an industrial context, however, the approach to obtaining data is quite different…
Resource savings, less downtime
The fact that data is the new gold was recognised years ago by the most up-to-date players in the manufacturing industry, the frontrunners of Industry 4.0. By now, if they are/were on the right track, they are reaping the benefits. Smart factories have already been implemented around the world and, by analysing data and re-engineering processes, they have reduced manufacturing waste by up to 17%, water consumption by 64% or unplanned machine downtime by 48%.
There are countless modern data analytics platforms around the world to achieve similar goals. Resources are scalable thanks to cloud-based technologies and the widespread adoption of learning algorithms.
Although, the basic assumption of all these technologies is that the production data to be analysed is already available – mostly in a nicely structured and optimised database (big data) on which “only” the analyses need to be run.
Legal machine data mining in an industrial environment
Among the modern data analytics platforms, WaMeWo is trying to stand out from the market noise with its own telemetric (from the Greek téle = ‘remote, far’ and metreó = ‘measure’ – Wikipedia) layer. Our system is capable of acquiring the necessary data, without which it is inconceivable to add value to any other, higher layer. In the recent years we have focused on developing this feature of WaMeWo, which is the essence of our bottom-up approach.
The basic concept of WaMeWo was originally born out of the problem of how to extract valuable information from various industrial controllers (robots, PLCs, etc.) without having to make costly hardware or software additions. This requires a thorough domain knowledge of industrial controls and their communication.
Wide range of manufacturers, uniform data
It is very difficult to mine, preformat and standardise data from controllers from many different manufacturers in a consistent way. On the one hand, manufacturers have not prepared their tools for platform-independent data collection in the past, due to a lack of demand, and on the other hand, business interests are nowadays getting in the way: machine manufacturers want to sell their own IIoT solutions for their own tools. However, end-user needs are not pointing in the same direction.
The 6 key requirements that a data collection application must meet
We have compiled a list of the functional requirements that a modern data collection application must meet:
#1 Connectivity
In order to fit into modern IT-OT network infrastructures, it must be able to implement some kind of Ethernet-based protocol (typically TCP/IP), which can be achieved either over a wired or wireless connection. Alternative methods include mobile internet (4G/5G) connectivity, MESH (e.g. LoRaWan) or even BLE (bluetooth low energy), but these are only partially deployed in industry.
#2 Sampling
It should be able to read data from controls via requests or receive data on an event basis with the highest sampling rate available (as allowed by the protocol and physical devices). For example, in the case of an extremely short cycle time device (typical in the food industry), if the cycle rate is 60 parts/minute, then in order not to miss any events, a sampling rate faster than 120 Hz is required by Shannon’s law. The data acquisition is usually done by so-called client or adapter applications, which transmit it to the server or agent application.
#3 Receive
On the server side, data arriving at a high frequency, especially if it comes from hundreds of devices simultaneously/sequentially, must be received and processed without loss. This can mean tens of thousands of data points in a second. Several methods are already available on the market to overcome this, one of our preferred applications is Telegraf, which can collect data for the InfluxDB time-series database, among others.
#4 Unification
The collected data should be entered into an appropriate database (databases suitable for IIoT tasks will be discussed later) by the server application, preferably following standardised schemas. Consolidated data models are the basis for advanced analytics and reporting, and will greatly facilitate the work of the upper layers if proper attention is paid to this.
#5 Configuration
A universal configuration schema should be available to describe, for example, which memory addresses, tags, and frequency of data should be read from a given device, and which tag names should be used to transmit the data to the database. By developing the appropriate schema, the amount and parameters of the data points to be accessed can be easily defined and scaled.
#6 Buffering
Last but not least: unfortunately, what can go wrong, will go wrong. Since the client and server are typically located remotely from each other (a typical setup is that the client application is running on an edge device next to the machine and the data is stored on a cloud server), a network problem can occur between the two at any time. To avoid this, the client device/application should be able to implement buffering, i.e. store data locally in memory until the connection to the server is restored. It is important to pre-size the edge device for this!
Machine data collection methodologies
Beyond defining the general operational preferences, let’s take a look at the methods that can be used to access machine data. (The list is not exhaustive.) WaMeWo is compatible with these methods.
Serial
Although fortunately rare, we can still find legacy equipment that does not have an ethernet interface, but only some classic serial implementation (e.g. RS-232/485). This is typically a slow and cumbersome way to get the data you need, and if you are thinking of networking, it is definitely worth connecting a serial to ethernet converter to the device.
Retrieving and parsing HTTP pages
Even this is not a trivial data retrieval solution, but our experience shows that it does occur for industrial devices (e.g. certain types of robots). These controllers run a web server in the background and are accessible from a browser at the IP address of the device. The result is basically a web page with a simple design that summarizes information about the device (physical characteristics, variable values, memory status, error messages, etc.) in text files. By downloading these and running searches on them, for example using regular expressions, you can collect the data you need.
REST(API)
If we climb one step further up the ladder of modernisation, you will find the world of REST (Representational State Transfer) requests. REST is a connectivity solution that builds on certain principles of current web fundamentals and usually runs on HTTP. It implements stateful communication. REST ignores the details of component implementation and protocol syntax in order to focus on the role of components, the constraints of interaction with other components, and the interpretation of meaningful data elements. REST is also used to develop web services as an alternative to other distributed communication types such as SOAP. REST is often used in mobile applications, social networking sites, mashup tools and automated business processes.
Memory reading with standard protocols
PLC vendors have already implemented protocols that can, for example, share memory space between two PLCs, i.e. allow an external actor (node) to read and write (the entire) memory space (usually via a TCP/IP connection). There are different solutions for this from manufacturer to manufacturer, but perhaps the best known are
- Snap7 (Siemens),
- FINS (Omron) and
- MC protocol (Mitsubishi).
These network services ensure that an appropriate telemetry client application can read or, if necessary, write data from a given address in the PLC memory.
Standardised solutions
Fortunately, there is nowadays a growing number of initiatives to enable modern industrial controllers (be it robots, PLCs, servos, machining centres, etc.) to provide data streams to IIoT platforms using standardised methodologies and formats. Several industrial vendors have “industry-quadrupled” their portfolios by implementing the latest protocols in their next-generation controllers. Perhaps the most common of these are MQTT, OPC (UA) and MTConnect, which are specifically designed for such tasks. A detailed description of these would be well worth a separate post, but for now we will just mention their main advantages:
- they provide a standardised format for different devices from different manufacturers
- they are platform independent, with libraries available in multiple programming languages
- event-based notification of fresh data, no need to trigger timed queries
- this makes it extremely fast and efficient to collect large amounts of data
- they provide secure, authenticated communication between client and server
- data points and members are easily configurable and scalable
As these solutions become more widespread and the generations of controllers are updated, it is expected that many of the methodologies described above will become obsolete in the future.
Data collection with WaMeWo
A WaMeWo’s advanced telemetry layer, even independently of the complete production control system, provides high quality industrial data mining for both architecture and third-party systems, even via standard outputs (MQTT, REST, DB insert, etc.).
What is this series of articles about?
This series of articles has been compiled by our team in order to provide both technological and business know-how for professionals involved in the digitalisation of the industrial sector. The chapters follow two guiding principles:
- exploring the reasons behind the manufacturing digitisation software technologies used and
- demonstrating their potential for business use.
We trust that our content will provoke constructive professional discourse. We welcome your feedback, opinions or questions on info@indeveyes.com-on or our social media platforms!
- In a previous article in this series, we wrote about the architecture of Current IIoT architectures.
- In our next article, we will talk about downtime logs.
