SyncSight

The growth of industries is closely linked to the adoption of new technologies. As industries expand, the need for real-time monitoring, analytics, and user-friendly interfaces becomes crucial. The drive for better operations, increased safety, and higher productivity has led to the exploration of technologies that can blend well with the existing industrial setup, marking the onset of Industry 4.0. Augmented Reality (AR) stands out among these technologies, finding applications across various sectors like healthcare, education, retail, and notably, industrial operations.

Figure 1. Industrial use of Augmented Reality Technology

AR, with its capability to display digital information in the real world, opens many possibilities for industrial applications. It offers a way to gather and display data from various sources in an intuitive and actionable manner. The real-time display of machine parameters, analytics, and operational metrics through AR helps close the information gap and provides a clear path towards data-driven decision-making and improved operational efficiency.

The solution discussed in the following sections showcases the significant potential of AR in transforming industrial operations. By combining AR technology with a customizable and interactive user interface, a new level of operational excellence and real-time monitoring is achieved. This solution aims to change the way industrial personnel interact with machinery and process data, setting the stage for a new era of industrial operations.

Background, motivation and problem statement

Background

In recent years, the integration of Augmented Reality (AR) in industrial settings has transitioned from experimental to practical, offering tangible benefits in various operational aspects. The ability of AR to meld digital information with the physical environment provides a unique solution to longstanding industrial challenges. The solution leverages this technology through AR glasses, paired with an Android phone, utilizing the Snapdragon Spaces framework for machine recognition, hand tracking, and interactive functionalities. This framework facilitates accurate machine identification and the rendering of a customizable 3D user interface (UI) that provides real-time machine parameter visualization, thereby enhancing the interactive experience between personnel and machinery on the factory floor.

Figure 2. Snapdragon Spaces AR Ecosystem

Motivation

The motivation driving this solution emanates from the following key observations:

• Quick Responses: In factories, quick actions can prevent big problems. But without real- time monitoring, it’s hard to react fast. Delays can lead to lost production time or even safety risks. AR can show important information instantly, helping staff act fast.
• Data Overload: Factories produce a lot of data. Going through all of it can be overwhelming. What’s needed is a way to see the most important information easily. AR can highlight key data points, making it simple to understand what’s going on.
• Flexibility Need: Factory tools and displays are often set in their ways. They don’t change easily for different jobs or machines. With AR, we can have flexible displays that adjust based on the job or the machine being looked at.
• Ease of Access for Everyone: Right now, not everyone in a factory can easily see all the data they need. Managers might see one thing, while workers see another. AR can give everyone a clear view. Whether you’re a manager or a worker, you can get the information you need, right away, just by looking.

Problem Statement

The core challenges this solution seeks to address encompasses:

• Machine Recognition: Creating a robust system capable of identifying different machines and rendering the corresponding UI based on the machine type and the user’s role.
• UI Customization: Developing a browser-based tool for role-specific and machine- specific UI customization to ensure the right data is displayed to the right personnel.
• Interactive Experience: Designing an interactive UI that can be manipulated using hand gestures for a user-friendly experience, adaptable across different machine types and industrial environments.
• Performance Metrics: Establishing clear performance metrics to evaluate the system’s effectiveness in enhancing operational efficiency, safety, and decision-making processes.

Through addressing these challenges, the solution endeavors to bridge the information gap, enhance operational efficiency, and foster a culture of data-driven decision-making within industrial settings.

GOALS & BENEFITS

Goals

The central aspiration of this solution is to enable real-time monitoring of machine parameters, a crucial step towards prompt decision-making and effective problem-solving in industrial settings. By offering a tailored user interface (UI) for different roles, the system ensures that the right information reaches the right personnel at the right time, thereby fine-tuning the operational workflow. The introduction of an interactive UI, which can be manipulated using hand gestures, is aimed at fostering a user-friendly experience for personnel across the board. Furthermore, the solution places a strong emphasis on flexible adaptation, allowing for UI customization through a web browser to cater to various machines and industrial settings.

Benefits

Using this AR solution has clear and direct benefits for those in factories and industrial settings. One of the main advantages is quick access to data. Both workers and managers can immediately see important machine information, enabling them to make swift and informed decisions. If a problem arises, they can address it without delay, ensuring smooth operations.

A standout feature of the solution is the ease with which the AR display can be modified. Through a simple web browser interface, the appearance and content of the AR display can be adjusted. Once changes are made, they are instantly reflected across all AR devices, eliminating the need for complicated coding, or waiting periods. This ease of use extends to individuals without programming expertise; the system is designed to be intuitive, allowing even those unfamiliar with coding to effectively customize the display.

The accuracy of the data is another crucial benefit. The AR display presents information directly sourced from the machine, ensuring that users receive up-to-date and reliable insights. This direct connection to the machine negates the need for additional tools or systems, streamlining the data visualization process.

Safety, a paramount concern in industrial settings, is also enhanced. The real-time nature of the data means potential safety issues can be identified and addressed promptly, fostering a safer working environment. Moreover, the system’s efficiency translates to time and cost savings. By facilitating quicker access to and comprehension of machine data, workers can allocate more time to other essential tasks, and early problem detection can prevent costly future complications.

Lastly, the system’s design emphasizes adaptability. It’s tailored to integrate seamlessly with a variety of machines and setups. As a factory’s needs evolve or expand, the AR solution is well- equipped to adapt, making it a forward-thinking investment for the industry.

In essence, this AR solution simplifies the task of accessing and understanding machine data, offering a user-friendly interface that bolsters efficiency and safety in industrial operations.

USE-CASES

1. Machine Parameter Visualization for Factory Workers
   • Scenario: A factory worker is on a routine inspection of the manufacturing floor and approaches a CNC milling machine.
   • Action: As the worker views the machine through the AR glasses, the system recognizes the machine type and instantly overlays a UI displaying real-time metrics like machine temperature, spindle speed, and operation cycle status.
   • Outcome: The worker identifies that the machine temperature is nearing its threshold and immediately escalates this to the maintenance team, preventing potential overheating and machine downtime.
2. Customized Dashboard for Managers
   • Scenario: A manager is interested in getting an overview of the productivity of the production line.
   • Action: As the manager scans the factory floor with the AR glasses, a customized dashboard appears, showcasing a summary of machine efficiencies, ongoing maintenance tasks, and hourly production metrics.
   • Outcome: The manager quickly identifies an underperforming machine and coordinates with the relevant team to optimize its output.
3. Maintenance and Troubleshooting
   • Scenario: During a peak production cycle, a crucial conveyor belt system unexpectedly malfunctions, threatening to severely disrupt operations.
   • Action: A maintenance engineer, equipped with AR glasses, quickly approaches the malfunctioning conveyor system. Upon activation, the AR interface not only highlights the potential fault areas but also provides a step-by-step, real-time troubleshooting guide. Simultaneously, the system offers a live feed to off-site expert engineers, allowing them to view the issue remotely and provide additional guidance or potential fixes directly through the AR interface.
   • Outcome: Leveraging both the AR system’s guidance and real-time input from remote experts, the on-site engineer swiftly identifies and rectifies the issue. This collaborative and technology-driven approach drastically reduces downtime, ensuring that the production schedule remains largely unaffected. Moreover, the ability to pull in remote expertise on-demand showcases the system’s potential in drastically reducing the need for physical interventions and site visits, leading to significant cost and time savings.

CHALLENGES

Successfully integrating Augmented Reality (AR) into an industrial setting to provide real-time machine parameter visualization and interaction presents several substantial challenges:

• Machine Recognition & Data Mapping: Establishing an efficient and accurate system to recognize a diverse range of industrial machines and subsequently map the correct data and UI based on both the machine type and the user’s role.
• UI Interactivity & Customization: Designing a universally intuitive yet deeply customizable UI that can cater to the varying needs of different roles in the industrial setup, from factory workers to plant managers, while being adaptable to different machines and their specific parameters.
• Integration with Web-Based Tools: Seamlessly integrating the AR interface with browser-based tools to allow real-time UI customization, ensuring that changes made in the browser are accurately and promptly reflected in the AR environment.
• Integration with External Frameworks: Harmonizing the AR system with the Snapdragon Spaces framework for functionalities like machine recognition, hand tracking, and interactions.

ARCHITECTURE

Figure 3. System Architecture of Industrial AR solution

The foundation of this solution is anchored by a WebGL-based Unity application, a pivotal component that functions both as a web server and an interactive web application interface. The architecture is modular, emphasizing scalability and responsiveness, and is structured around the following key components:

WebGL-based Unity Server: This server not only hosts the web application but also manages and stores the various UI configurations. Leveraging the capabilities of Unity ensures high- performance rendering, essential for real-time data visualization.

RESTful API Integration: The Unity server communicates via a RESTful API, a standard that ensures efficient and standardized data exchange. This API handles the storage, retrieval, and updates of the JSON configuration files, optimizing data flow between the server and the AR glasses.

Figure 4. Visual Representation of the Industrial AR solution

JSON Configuration Storage: Each machine-role UI configuration is encapsulated as a JSON file. These files are dynamically generated based on user preferences and are centrally stored on the Unity server, ensuring data consistency and easy accessibility.

AR Client Interface: The AR glasses operate as clients, pulling the latest UI configurations based on the detected machine and the pre-configured user role. These glasses utilize the Snapdragon Spaces framework, facilitating accurate machine recognition, seamless hand tracking, and intuitive XR interactions.

Dynamic UI Customization: When accessing the Unity server via a browser, users are presented with a dynamic UI customization toolkit. This toolkit fetches available UI themes, a diverse set of UI elements (ranging from pie charts to text labels), and real-time data parameters, all retrieved in real-time from the server. This ensures that users always have the most up-to-date tools and data at their fingertips.

In a typical use scenario, a user, upon logging into the web interface, selects a machine type and role, and then crafts a custom UI using the provided toolkit. Once saved, a JSON configuration is generated and stored. When the AR glasses are activated in the vicinity of the specified machine, they query the server via the RESTful API, download the pertinent JSON configuration, and render the custom UI, offering the user a tailored AR experience.

This architecture, with its emphasis on modularity, real-time data flow, and user-centric design, positions the system at the forefront of industrial AR solutions, bridging the gap between complex data and actionable insights.