The experimental facility represents a small-scale factory for producing plastic cans closed with plastic lids. It is a sort of a toy product. The setup is equipped with 2 assembly modules (modules that perform cap on can assembly – module 1 puts red or blue caps, module 2 puts red or white caps on cans), 1 feeding station (that puts cans into production and can optionally insert a metal inlay in the product), 1 sorting station (which sorts the final products into a customer or user preference), and, finally, 1 disassembly station (which consists of 2 parts – a robotic manipulator that takes products and moves them to a holding place where it disassembles it afterwards). This setup used for demonstrating how the proposed AI methods in industry perform, what it means to add flexibility and transparency to the system, and how we can scale them to large facilities.

@ SIEMENS AG - use with permission

The problem we want to solve is divided into 2 parts: adding flexibility and adding transparency to manufacturing. The AI4Industry pilot consists of 3 main research areas, that can be correlated to 3 approaches to solve the problem: Skill Matching and Explanations, Planning, and Time Prediction.

Skill matching and explanations. To understand what skill matching is, we first need to understand the notion of a skill. A skill represents a description of a machine’s capabilities. In other words, a skill represents an action or a process that the machine can perform with respect to certain restrictions it has. As an example, consider an assembling action, that represents assembling of caps on cans. We know the type of action and we know the 2 types of materials used (caps and cans). We can also know certain restrictions that the skill has (e.g. the machine can only assemble caps on cans if the cap has a diameter 20mm and the can has diameter 20mm). This a description of a skill and it can be modeled in various ways. How we represent skills is through ontologies, specifically model them in the programming package Protégé. Ontologies represent formal naming's, definitions of concepts, relations between concepts, data properties, etc. They are based on Description Logics (DLs), which are an expressive and decidable part of First Order Logic (FOL). This gives the intelligent part of the model by allowing for reasoning capabilities, such as checking whether a logical expression is satisfiable with respect to the domain knowledge, checking whether everything is consistent in the model, listing of hierarchies and relations between concepts, etc. Protégé is an easy to use tool that offers all these features, which is why we use it.  Figure 1 represents a skill offered by a machine modeled in Protégé.

Now that we know the definition of a skill and how we represent it, we can move onto skill matching. Skill matching in the scope of industry represents matching a skill that a machine offers to a requirement that a product has. For e.g., consider the skill that is shown on Figure 1. The skill represents an assembly process, where a robot presses red or blue caps with width up to 180mm (adjustable gripper for different caps – if a can with larger width comes the machine can be reconfigured to work with it) on cans with width of 20mm. A requirement, on the other hand, can be considered also a type of a skill description, except this is from the product side. A skill requirement would ask for a specific process (i.e. action/operation/skill) and machines that are in production will need to answer to these requirements. Skill matching also indicates whether a machine that has a certain skill can answer to requirements of a product that has come into production. This is done by utilizing reasoning techniques in ontologies to ask for a contradiction between the skill offer and skill requirement.

@ SIEMENS AG - use with permission

Upon receiving the output from skill matching, we have a list of machines with certain skills that can or cannot answer to a product’s requirements. The explanations part represents generating explanations for why there is a negative match between a machine’s skill and a product’s requirements. This gives understanding why a product cannot be manufactured and users in the facility can quickly remedy the situation, whether that is reconfiguring a specific machine or replacing it with another one that can answer to the product’s requirements. To exemplify this, consider again the skill offer from Figure 1, and a product requirement with specific properties of the parts used in Figure 2. An explanation consists of presenting the reasons why this is a negative match. To generate explanations, we utilize something that is called justifications in ontologies. Justifications represent a minimal subset of ontology axioms, for which an entailment holds (i.e. it logically follows from them). An explanation for a negative match, between the skill offer in Figure 1 and skill requirement in Figure 2 (top left) is shown on Figure 3. From Figure 3, we can interpret the statements of the explanation. Starting from the top statement 1, we see that involving an initial material is the same as involving a material in the process AssemblyBR, which involves 2 things (statement line 2). Statements 5, 3, and 7 show us that one of the involved materials of the requirement is a physical object named blueCap_1, which is a type of Cap and has 200mm width. Statement 6 is what we call a closure statement, stating the main reason why there is a negative match between the offer and request, that is the width. If we compare statements 3 and 8, we can extract that the requested cap is 200mm, but the offered one is up to 180mm and that they are not compatible. This is a raw output from the explanation mechanism. Currently, we are adjusting the presentation of explanations according to users familiar with the production facility.

@ SIEMENS AG - used with permission

Action planning is an AI method for selecting actions that achieve a desired goal from a desired initial state. As a demonstration of Integrative AI, we use Answer Set Programming with External Computations to integrate Answer Set Planning with ontological reasoning over an existing industrial OWL ontology that is in use at Siemens and evosoft. This permits us to perform expressive reasoning than would be impossible within the ontology while it also permits us to reuse the existing ontology without converting either from ASP to OWL or vice versa. The resulting planning prototype can generate plans for producing certain sequences of configurable products and it can provide alternative solutions with small modifications of the goal or the time horizon if the goal as it is stated is not achievable. The solver tool that is used in this pilot is called Hexlite[1]. During the work in this pilot was developed a plugin for integrating OWL ontologies into action planning with Hexlite. The solver has already been published to the AI4EU platform as a component and we plan to publish the OWL ontology integration plugin in future milestones.

The time prediction is concerned with estimating the production time from a given production plan. We use a neural network to predict the time that a batch of production items (bases that are capped in the production process) will take to be finished. The production plan, which is the input to the neural network, consists of the order and time of feeding of the bases.

Since the available data could not be used for this scenario we have written a simulation of the process and trained the neural networks on the simulated data. We tried different architectures, loss functions and components for the neural network. The attention architecture, which could have provided explainability, did not train well. We ended up with an architecture of stacked LSTMs with temporal dropout. In the case that the feeding time of the bases is variable, we have developed the special preprocessing step which enables the training of the neural networks. Using a quantile loss for training the network reduced even slightly the absolute mean error.

The time prediction part of the pilot is connected with the research topic “Verifiable AI” of the work package 7.2. To rely on a machine learning approach one needs to have an idea of the weaknesses of the system. Uncertainty estimation is one method to determine the variance of a model and forecast a fault. The aim of the experiments is to estimate the uncertainty of the model to distinguish situation where the model is confident and those where the prediction of the model has a high variance and therefore is less reliable. Therefore, different approaches to measure uncertainty are evaluated on the use-case described above.

AI4EU platform exploitation

Siemens AG are currently identifying assets that can be presented to get better insights of the method and assessing the IPR situation.

The solver tool that is used in this pilot by TU Wien is called Hexlite[1]. During the work in this pilot was developed a plugin for integrating OWL ontologies into action planning with Hexlite. The solver has already been published to the AI4EU platform as a component and we plan to publish the OWL ontology integration plugin in future milestones.

Fraunhofer IAIS are planning to publish the time prediction part (or at least parts of it) on the AI4EU platform. The implementation and the upload are pending on approval due to IP issues of the involved partners.

Collaboration of the organizations

Our final step is providing a showcase of the methods in this scenario, presenting how they aid flexible manufacturing in an interactive manner. In close collaboration the three parties work on a video demonstration to present the outcomes from the AI4Industry pilot. Furthermore, we plan on publishing the theoretical and practical aspects of the methods developed by all partners in a joint use-case publication in the following months, as a part of the assessment phase of the project.

Timeline and achievements

Within the scope of the project, March 2019 was the starting month of the project and in the following 3 months, until June 2019, the pilot was initializing the necessary assets to work on the pilot. This resembles acquiring the necessary data, both knowledge and sensor data, to construct the models and develop the previously described methods.

From June 2019 until October 2020, with an extension of approximately 4-5 weeks due to the Covid-19 situation, all 3 partners actively engaged in discussions and hands on research and development of the symbolic AI and Machine Learning methods that are utilized to achieve and aid flexible manufacturing in a scalable experimental facility.

As of the current moment the AI4Industry pilot is finalizing the execution phase and entering the assessment phase. In this time the parties acquired the results from the developed functions and work on constructing a video presentation to demonstrate the benefits of the AI methods in an industrial scenario.