Freelancers
Cancel
 Sign In
Join Now

This is the 5th blog post of the blog series about Building Intelligent and Sustainability Scenarios on SAP BTP with AI and Planning by Gianluigi BAGNOLI, Yatsea Li, Alice Magnani, Cesare Calabria, Stuart Clarke, Dayanand Karalkar and Jacob Tan, which showcases our SAP Partners how to create industry cloud solutions for end-to-end industry-specific business processes that embeds intelligence and sustainability.

Previously, in the first blog post of this series, my colleague Gianluigi BAGNOLI has explained you the storyline about a traditional Milan-based Light Guide Plates (LGP) manufacturer BAGNOLI & CO was transformed into a sustainable smart factory with the help of SAP AI Core, SAP AI Launchpad and SAP Analytics Cloud for Planning. Using Artificial Intelligence deep learning for computer vision based automatic defect detection, and condition (sound) monitoring based predictive maintenance in production lines, the company has improved the efficiency of production and energy, and reduced waste.

In the last blog posts written by  Cesare Calabria and Alice Magnani, we have learnt about the end-to-end MLops with SAP AI Core, and how to build your own model with TensorFlow about product image segmentation for auto. defect detection and equipment sound anomaly classification for predictive maintenance with SAP AI Core and SAP AI Launchpad.

Overview

Now, in this blog post, let us put the pieces together into a custom app named Sustainable Smart Factory App, which can:

  • Monitor the plant daily operation in real-time about plant status, production yield, defective rate, equipment conditions, as well as a new sustainability KPIs (Key Performance Indicators) about energy consumption and calculated CO2 emission.
  • Ingest and score the real-time IoT (Internet of Things) streaming data against the AI Models
    • Image Segmentation model on LGP product images streaming from camera for automated defect detection
    • Sound Anomaly Classification model on equipment audio data from microphones.
  • Trigger a proactive maintenance order in SAP S/4HANA Cloud once the occurrences of a certain type of anomalies detected exceed the threshold within a period, which extends the Maintenance Management of SAP S/4HANA Cloud with Predictive Maintenance.

Solution%20Architecture%20of%20Sustainable%20Smart%20Factory%20Solution

Solution Architecture of Sustainable Smart Factory Solution

Introduction to the Sustainable Smart Factory App

Now, let us have a look at the Sustainable Smart Factory App from a functional perspective, which consists of the following four modules as listed below.

User%20interface%20of%20Sustaintable%20Smart%20Factory%20App

User interface of Sustainable Smart Factory App

  1. My Home (Target Persona: Plant Manager)
    • Tile to navigate to Plant 360 Analytics story in SAP Analytics Cloud.
    • Tile to navigate to Maintenance Order & Sustainability Planning story in SAP Analytics Cloud.

Plant%20360%20Analytics

Plant 360 Analytics story in SAP Analytics Cloud

  1. Auto. Defect Detection (Target Persona: Quality Inspector or Quality Manager)
    • Quality Record via Computer Vision: A product quality record with its image and quality classification predicted by AI Model.
    • Manage Defective Product Price: Define the pricing rule for the defective product based on the size of the defective area.

Auto.%20Defect%20Detection

Auto. Defect Detection

  1. Plant Master Data (Target Persona: Plant Manager)
    • Manage Equipment: Equipment master data maintained in the Sustainable Smart Factory App, which is synchronized as Technical Object in SAP S/4HANA Cloud, including general data such as equipment id, name, description, class etc., organization data such as company code, plant id, function location etc.
    • Configure Plant Status: Configuration of plant status, such as Breakdown, Fault, Maintenance, and Normal etc.
    • Configure of Anomaly Types: An Anomaly type is mapped into an Anomaly Class in the AI model, such as Slow Sound or Damage Noise.
  1. Predictive Maintenance (Target Persona: Plant Manager or Maintenance Technician/Manager)
    • Plant Condition Visualization: Real-time Monitoring of plant operation namely plant condition explained below, as well as Visualization of Plant Floor Plan about equipment conditions.
    • Manage Plant Condition: A plant condition is a historical snapshot of the plant condition including production yield, defective product number, energy consumption and its equipment conditions within a time period, such as a morning shift of today starting from 0:00 am to 7:59 am.
    • Manage Equipment Condition: Equipment Condition record within time period of a shift, including a list of detected anomalies associated to the equipment. Trigger the creation Maintenance order in SAP S/4HANA Cloud based on rule configuration.
    • Manage Equipment Anomalies: When an anomaly is detected on an equipment, an Anomaly record will be created including equipment, detected At, sound data, and sound anomaly classification by inferencing the AI model etc.

Sound-based%20Predictive%20Maintenance

Sound-based Predictive Maintenance

Now, you may be interested on how we built the Sustainable Smart Factory App. Let us have a look at its technical details.

Technical Details of Sustainable Smart Factory App

High-level Solution Architecture

The high-level solution architecture of the Sustainable Smart Factory App as below.

Now let us zoom into the architecture of Bagnoli&Co’s Sustainable Smart Factory App, which is built on top of SAP Cloud Application Programming model (CAP) with

  • SAP Fiori Element as its main UI technology
  • NodeJS as its application and service layer
  • SAP HANA Cloud as its database for persistence.

The app glues all the pieces of Bagnoli&Co’s Sustainable Smart Factory into an end-to-end solution to achieve its sustainable goals.

As you have already seen in screenshot of the app above, it provides an user interface as a  SAP Fiori app for the end users such as Quality Inspector, Quality Manager, Plant Manager, Maintenance Manager, Maintenance Technician etc.

It inferences the AI Models with real-time IoT data ingestion for Defect Detection and Sound Anomaly Detection through REST API deployed in SAP AI Core, which automates the quality inspection, and eliminates the human errors.

Furthermore, it extends the Maintenance Management of SAP S/4HANA Cloud side by side with sound-based Predictive Management by rule-based trigger of proactive maintenance order in SAP S/4HANA Cloud via SAP Cloud SDK when the occurrences of anomaly detected on an equipment exceed the configured threshold, which can minimize the disruption of production line, and maximize the lifespan and efficiency of the equipment, and reduce the maintenance cost.

And, all the API integrations above are through the destination and xsuaa service in SAP BTP for security reasons.

With data persistence in SAP HANA Cloud, some calculation views are custom developed for plant-condition and equipment-insight queries, which provide live connection to SAP Analytics Cloud for a Plant 360 Analytics story in real-time.

 

Detail Design

#1: Auto. Defect Detection

Data Model (sqi-data-model.cds)

  • CVQualityRecords: Quality Record by Computer Vision
  • DefectiveProductPrices: Pricing rules of defective products based on the defective area percentage range.

data%20model%20of%20defect%20detection

Data Model of Defect Detection

Code snippets

  1. Connect to SAP BTP destination service, and authenticate to inference API of AI Models in SAP AI Core – getDestination(dest)
    https://github.com/SAP-samples/btp-ai-sustainability-bootcamp/blob/main/src/sustainable-smart-factory-app/srv/admin-service.js#L390-L436
  2. Inference product image segmentation AI Model for Auto. Defect Detection – inferenceImageCV()
    https://github.com/SAP-samples/btp-ai-sustainability-bootcamp/blob/main/src/sustainable-smart-factory-app/srv/admin-service.js#L248-L354

 

#2: Predictive Maintenance

Data Model (pdm-data-model.cds)

  • Equipments: Equipment Master Data synchronized as technical objects in SAP S/4HANA Cloud
  • AnomalyType: An anomaly type is mapped as a sound anomaly class in AI model.
  • PlantConditions: A historical record about the plant daily operation by shift within a time period, such as plant status, production yield, number of defective products and energy consumption, and its associated EquipmentConditions records within the same time period.
  • EquipmentConditions: A historical record about the condition of the equipment within a time period, such as the associated equipment, equipment status, the detected anomalies by the AI model. Creation of proactive maintenance order will be triggered in SAP S/4HANA Cloud if the occurrences of the detected anomalies in the time period exceed the threshold defined in the anomaly type. The associated maintenance order number and cost will also be recorded in the equipment condition record if any.
  • Anomalies: An anomaly is detected by the AI Model on about its associated equipment, including when it is detected, what is source type such as sound, image temperature, vibration etc, which can be used for any type of anomaly detected by different sensors. The raw value and raw measure unit depend on the source type as, for example, if the source type is sound, then raw value is the path to sound clip in cloud storage; if the source type is temperature, the raw value could the decimal string about the temperature captured by the sensor, and the raw measure unit could be Celsius to Fahrenheit.

Data%20Model%20of%20Predictive%20Maintenance

Data Model of Predictive Maintenance

Code snippets

  1. Inference sound anomaly classification AI model for Predictive Maintenance: inferenceSoundAnomaly()
    https://github.com/SAP-samples/btp-ai-sustainability-bootcamp/blob/main/src/sustainable-smart-factory-app/srv/admin-service.js#L163-L246
  2. Trigger of proactive maintenance order creation in SAP S/4HANA Cloud based on rule: createMO()
    https://github.com/SAP-samples/btp-ai-sustainability-bootcamp/blob/main/src/sustainable-smart-factory-app/srv/admin-service.js#L73-L161 

#3: Plant 360 Analytics

To enable real-time plant 360 analytics, the following calculation views have been created.

Deployment

Please follow this manual to deploy the Sustainable Smart Factory App (sample app).

More materials

If you would like to learn more about this topic, please check out materials below

Summary

Throughout this sample, we have seen how an SAP partner can embed the AI models and sustainability dimension data (energy consumption and CO2 emission etc.) into end-to-end cloud solution on SAP BTP for Bagnoli&Co., which automates defect defection and employs sound-based predictive maintenance. As a result, Bagnoli &Co has improved the efficiency of production and the safety of workplace and reduced the waste.

What’s next in this blog series

In the next blog of this series, we will see how to perform an end-to-end planning for maintenance cost and sustainability for Bagnoli&Co with SAP Analytics Cloud for Planning. As a wrap up, here are all the episodes of our blog series:

Tags: Artificial Intelligence Machine Learning SAP Business Technology Platform SAP Cloud Application Programming Model SAP S/4HANA Cloud Sustainability
Sara Sampaio

Sara Sampaio

Author Since: March 10, 2022

0 0 votes
Article Rating
Subscribe
Notify of
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x