Handbuch2AR

What it is about

Maintenance processes in an industrial environment are characterized by their high level of complexity. It is necessary to disassemble components, maintain them under precisely defined conditions and then reassemble them. A large number of manuals, standards, guidelines and procedural instructions are available to manage these tasks. The aim of this project is to make these extensive documents and sources of information digitally accessible. Augmented reality (AR) technology is to be used to present the content to operating and maintenance staff during their respective activities in a way that is appropriate to the situation and phase.

The information sources are developed using artificial intelligence (AI) and machine processing in combination with a domain-specific ontology. This ontology describes, for example, the specific characteristics of a “chemical plant”. The heterogeneous textual and graphical information (2D, 3D) is then converted into suitable visualization forms for the AR application.

Prototypes for various industrial applications are being developed as part of the project. These should have at least one webAR interface to support operating and maintenance tasks and to prepare specific information visually and appropriately for the situation. The long-term goal is to create an AR support solution for operation and maintenance that can be transferred to any technical device or system. This should make previously complex user instructions, manuals and similar documents obsolete.

Aims of the research project

The technology designed aims to overcome the existing lack of clarity in the area of digital operating instructions and manuals by introducing standards. For this purpose, an ad hoc ontology is first created, which is developed in the context of business process modeling. By analyzing use cases in the course of the project, a generic approach is to be formulated that is not only applicable to different use cases, but should also be transferable to completely different applications. In addition, text information, 2D and 3D graphic elements and step-by-step instructions are to be prepared in an augmented reality interface and made interactively searchable. We are aiming for a prototype solution that can serve as the basis for a broader standard.

The specific objectives of the project include

• The development of a workflow for capturing and detailed modeling of operations, maintenance and repair processes.
• The development of a methodology and technology for the automated analysis of text and graphic documents and their linkage to the corresponding work processes.
• The derivation and development of descriptive ontologies for industrial applications and their integration with existing ontologies.

Motivation

To date, there are no machine-readable operating instructions and manuals for the operation and maintenance of technical equipment. Current standards and directives, such as DIRECTIVE 2006/42/EC, focus primarily on the generation and general structure of technical documentation, but not on the structure and content of operating instructions. The lack of a controlled vocabulary is a significant problem.

The identification of a user’s task could be made possible by integrating it into business process models. Here, either a technical workflow or interaction with the user determines which business process and role they are working in. Linking process model elements with other content allows manual navigation to relevant documents.

AI methods from the field of Natural Language Processing (NLP) are required for the application of text analysis methods and machine processing based on a domain-specific ontology. To date, there are no methods that can semantically analyze specialist texts and map the central terms to an ontology.

In the field of augmented reality (AR), there is currently no generic solution for the integration of text content, 2D and 3D graphic objects. A major challenge here is the development of an intuitive user interface that systematically records the user’s information requirements and maps them to the available information sources. Determining the granularity of such information units will be an important research task. WebAR represents the gold standard for the development of generic and device-independent AR applications.