Advanced human-machine interfaces in the Industry 4.0 era

24.08.2020
8 min.
title

Human-machine interfaces (HMI) have changed significantly over the last decades, driven by rapid technological advances. The HMI industry has witnessed the evolution from a simple push button to a modern touch-screen display. Still, more changes in the way people communicate with machines are expected to come. Such trends as enterprise IoT and Industry 4.0 have an essential role to play in this regard, being the foremost carriers of this HMI transformation.

4 stages of the Industrial Revolution

The term Industry 4.0 was first used in 2011 in regard to smart manufacturing, and later received a broader meaning spanning transportation, smart buildings, mining, oil and gas, healthcare and other areas. It appeared during the fourth stage of the Industrial Revolution, so to understand the notion of Industry 4.0 it is important to explore its predecessors.

4 stages of the Industrial Revolution

The First Industrial Revolution

In 1784, the first mechanical loom was invented, marking the beginning of the first stage of the Industrial Revolution and Industry 1.0. This was followed by the introduction of water- and steam-powered machines to give workers the ability to drive machinery day and night without the need for multiple operators. Mechanization was at the core of Industry 1.0 and gave rise to the modern manufacturing as we know it.

The Second Industrial Revolution

By the beginning of the 20th century, electricity appeared to replace water and steam, thus enabling manufacturers to focus their power sources on individual machines. The Industry 2.0 era is characterized by mass production of goods, using assembly lines and the division of labor to optimize workforce.

The Third Industrial Revolution

The third stage of the Industrial Revolution started in the late 1960s with the creation of the programmable logic controller (PLC). The broad application of electronics and information technology to achieve further automation and digitalization of manufacturing underlie the age of Industry 3.0. Technologies such as IoT, cloud computing, and big data analytics appeared among the key drivers of the third Industrial Revolution as well.

The Fourth Industrial Revolution

We are in the midst of the fourth Industrial Revolution—the next leap in digitization and automation of the manufacturing sector, promising to bring a sustained productivity growth and change the way factories work. Industry 4.0 introduced a major change in the interaction between workers and machines. In this new era, machines’ responsibilities go beyond manual manufacturing processes: they are starting to play an important role in manufacturing management, resource planning, data collection and analysis, and logistics.

Industry 4.0 components

Industry 4.0 provides new opportunities for advanced production with the help of technologies that complement and augment human labor. The set of these technologies is well-known, as these are the innovations that surfaced during the last decade:

  • Internet of things
  • 3D printing
  • Advanced human-machine interfaces
  • Big data
  • Augmented and virtual reality

However, the key component of Industry 4.0 is a cyber-physical system (CPS), which is a combination of computational and physical processes.

CPS architecture

We have just reached a tipping point in artificial intelligence and machine learning, whereby these technologies are starting to have real-world applicability. This is particularly encouraging for the Industry 4.0 vision because these technologies will be essential in providing the autonomy, adaptability and “smarts” envisioned in Industry 4.0

Diego Tamburini

CPS benefits and concerns

According to the Industry 4.0 paradigm, CPSs are increasingly autonomous and self-organizing: they can collect data, analyze it, and even make decisions based on it. The next logical question is “What will be the human’s role in the manufacturing process?”

Some people have concerns about workerless production facilities if smart machines replace humans. In fact, CPSs will not substitute operators but will only change the roles. While intelligent CPSs are continuously monitoring the manufacturing process, predict or detect shifts in normality, people intervene only for critical decision-making. Moreover, Industry 4.0 can introduce new jobs related to big data analysis, robotics, and mechanical engineering, as well as revolutionize the way humans communicate with engineering systems and improve processes significantly.

It is expected that some key benefits will arise sequentially from the replacement of today’s machines with CPSs:

  • CPS supports real-time data transfer between industrial machines, manufacturing supply chain, suppliers, and business systems for effective decision-making.
  • CPS enables self-monitoring, so when an industrial asset gets broken it sends an alert to support engineers to fix issues proactively. This saves time, money and efforts required to maintain systems functioning.
  • CPS allows everyday control over the entire production, which helps to adapt the latter to better satisfaction of consumers’ demands.
  • CPS provides a higher degree of visibility and control on supply chains, improving the traceability and security of goods.

New types of human-machine interface

As smart CPSs are transforming the human-machine communication process, they require new types of interfaces to ensure smooth interaction. New HMIs need to be more sophisticated for enhanced efficiency and remote service operations, especially when workers are interacting with technologies in dusty, humid or dark environments. Since operators become involved in the manufacturing process for critical decision-making, the HMI system should allow commands that are easily and rapidly entered in order to increase the accuracy, safety and speed of problem-solving.

With these requirements in mind, new types of HMIs are being implemented now by Industry 4.0 and IoT developers: enhanced touch interfaces, voice interfaces, gesture interfaces, and AR/VR tools.

Enhanced touch interfaces

Touch-screen displays have developed significantly since their introduction around 20 years ago, becoming more user-friendly and powerful in terms of visual data representation. Industrial interfaces enable IoT and M2M connectivity, so manufacturing companies can monitor and control on-site industrial operations both locally and remotely. Modern touch interfaces are sensitive and allow managing machines even in gloves, which brings additional comfort and safety to the operator.

Voice interfaces

Voice interfaces typically don’t have a screen to display information but facilitate data access through hands-free, intuitive and efficient interactions. In Industry 4.0, voice-activated interfaces become indispensable, especially in the conditions where remote operation of machines is needed.

Makino, one of the companies that pioneers industrial voice-activated HMI, has developed Athena, which uses basic voice commands to control production equipment on the shop floor. Athena’s operators can now not only remotely issue commands but also ask questions regarding machines’ critical metrics.

Gesture interfaces

Similar to voice interfaces, gesture control allows for touchless manipulation of industrial machines or computer systems. They recognize an operator’s hand or head movements and use special mathematical algorithms to control or interact with devices. Enabled through a variety of methods—wired gloves, depth-aware and stereo cameras, or hand-tracking controllers—gesture interfaces provide more accurate and faster human-machine interaction.

VR and AR tools

Currently, augmented and virtual reality are among the most prominent technologies for increasing efficiency, reducing operational costs, and making production more flexible across many manufacturing processes, from inventory management to employee training.

When it comes to assembly line tasks, operators often need to be extremely focused in order to precisely put a plethora of components together. AR tools break down these components by superimposing key information, which significantly eases an assembly operator’s job and reduces the risk of mistakes.

Now employees don’t need to check assembly manuals and can always be sure about their next move, which dramatically improves performance. For example, Lockheed Martin, the world’s largest defense contractor, has integrated AR for spacecraft assembly. Given that there is no repair shop in space, the company puts extra effort in ensuring accuracy and precision. In this case, some crucial parts like fasteners have to be placed with a 0.5’ tolerance and employees equipped with AR glasses can now reliably align those components. Lockheed reports saving $38 in touch labor for every fastener, which are ordered in millions every year.

Another scope of the AR and VR impact in manufacturing is workforce training. When a new employee is onboarding, many challenges arise, including equipment and coach availability. Moreover, it’s often costly to replicate some on-site scenarios, especially the ones concerning safety. Lockheed Martin has also implemented AR to provide its employees with animated manuals for assembling spacecraft components. The company reports a jaw-dropping 95% decrease in time needed for workers to interpret assembly instructions. At the end of the day, these technologies simply provide a more immersive on-the-job training, sufficiently enhancing employee satisfaction.

Lastly, AR and VR can also dramatically decrease maintenance and error detection times. For example, ThyssenKrupp, a large elevator manufacturer, has provided its technicians with AR tools for repairing operations. Now service technicians can more reliably identify problems and receive a real-time superimposed guidance. The company claims technicians can now finish a repair about four times faster than before.

Moving beyond the hype

It can be counterintuitive but to make these technologies bring real long-term benefits requires more than simply introducing them into manufacturing workflows.

In Deloitte’s recent report titled Industry 4.0: At the intersection of readiness and responsibility, two-thirds of the surveyed organizations said that they were taking ad-hoc approaches to Industry 4.0, while only 10% of the respondents among business leaders said they had longer-term strategies for extracting maximum value from various HMI systems. Unsurprisingly, the latter group has seen far more success across the board.

In this case, the long-term approach isn’t about generating continuous technology adoption plans and product configuration strategies, but about finding balance between profit and social responsibility. PwC’s Digital Product Development 2025 report found that the companies focused on product lifecycle management and enterprise resource planning expect to achieve a 31% increase in product development efficiency and a 28% reduction in time to market.

This shift of attention towards sustainability and corporate social responsibility largely stems from the increased pressure from customers and employees. While in the ideal world the focus on sustainability should be the result of business leaders’ inner social responsibility, today’s customers, employees and stakeholders are increasingly concerned with the environment. Many see their corporate focus on societal issues driven by board of directors’ and clients’ expectations:

CSR main drivers

This brings us to the point where those with comprehensive CSR policies will be able innovate faster, hire better specialists, and experience a steady growth.

HMI: implementation tips

No big news here, you need an implementation plan. As companies are aiming to reap the maximum benefits offered by Industry 4.0 and HMIs, it’s crucial to consider the following implementation steps:

  • First, organizations need to start conducting audits in order to assess their current position on the Industry 4.0 maturity scale, followed by establishing a long-term R&D plan.
  • Next, it’s crucial to create new leadership roles that will be responsible for adopting and leveraging the power of HMIs and identifying new investment opportunities. Later on, such leaders will also need to take care of assigning teams that will be committed to innovation.
  • In preparation for the change, it’s also recommended to create sufficient workforce training and development programs. However, identifying which skills exactly your workforce needs remains a significant challenge. This is why it’s important to have a long-term R&D strategy implied by the first step above.

Final thoughts

With voice and gesture controls, enhanced touch interfaces, and VR/AR tools, human-machine interfaces between human and machine are shifting to full-on digital automation in this Industry 4.0 evolution. The new types of HMIs can bring many benefits to smart factories by improving their efficiency and decreasing downtime. Moreover, used for repetitive activities or in hazardous environments, they can facilitate operations and ensure extra safety in the workplace.