In today’s fast-paced manufacturing environment, staying ahead of the competition requires embracing advanced technologies that can streamline processes, increase efficiency, and optimize production. One such technology is object detection, which has the potential to revolutionize the manufacturing sector.

In this blog post, we will use YOLOv8, an object detection algorithm, in conjunction with SAP Digital Manufacturing (Cloud) (DM). If you are not familiar with SAP DM take a look at this page. First, it detects me (as a person) and also the cup. After the cell phone was detected we will trigger our own build API which is triggering a message in SAP DM. This might not be your relevant use case but I will use it to simplify the scenario.

Another interesting use case is the much nicer-looking candy counter which I will walk you through (via OPC UA integration) in part 2 (see roboflow blog).

Lets start!

Step 1: Create your SAP DM production process and publish it as an API

Go to the “Design Production Processes” App and create your own process which has in my case two steps. But of course, you could think of any other process (see details in this blog).

My Inputs:

InArray (StructureArray / contextData)

InPlant (String)

InWorkCenter (String)

Script Task 1: Build a message for my POD (production operator dashboard):

var StructureArray = $input.InArray;

for (var i = 0; i < StructureArray.length; i++) {
  if (StructureArray[i].object_detected) {
    var message = StructureArray[i].object_detected + " detected!";
  }
}

In POD Message I use the message as a variable:

Assign also your production process to a POD:

Before we deploy the process select the three dots next to debug, to publish the process as an API:

Select Edit Header and activate “Publish to Service Registry”:

Click on “Quick Deploy” and confirm with “Deploy and Activate”:

Now we need to get the API endpoint. For that, we enter the App “Manage Service Registry” and search for your production process. Note down your URL/PATH: /pe/api/v1….

Now we need to note down our token URL, client id, and client secret in the service key of the DM (the same that you need for Postman (oauth2)). For that, you need to jump into your BTP Subaccount in which the DM is deployed:

Step 2: Install Python requirements and configure the API OAuth2 file

I assume that you have installed python and pip. Upfront load the code example in your project folder. The first step is to install the required python libraries with:

pip install requirements.txt

Now set up your OAuth 2.0 credentials and endpoints to obtain an access token. This is crucial for authenticating your API requests when connecting YOLOv8 to the SAP DMC. Store your client ID, client secret, and token endpoint as variables. I created an oauth2_request.py file which gets the access token:

Github Link

Step 3: Define the Main Function and create a Real-Time Detection

In the main function, start by configuring the webcam resolution settings.

Establish a while loop that reads frames from the webcam and processes them through the YOLOv8 model. The model will return a list of detections that includes the object’s class, label, and confidence score. Filter the results to only include objects with confidence scores above a certain threshold, e.g., 0.8, and “cell phone” as a relevant object.

Github Link

Step 4: Call the SAP DM API

If the detected object is a cell phone and its confidence score is above the threshold, call the API to send the data to the SAP DM. This data includes the object label, the number of detected objects, and the confidence score:

        data = {
            'InPlant': "--yourplant---",
            'InWorkCenter': "--yourworkcenter--",
            'InArray': [
                {
                    'object_detected': object_label,
                    'number': number
                }
            ]
        }

Step 5: Execute the Main Function & open your POD

Finally, run the main function to initiate real-time object detection and integration with SAP DM.

Show your cell phone and check the POD:

Conclusion

By integrating YOLOv8 object detection with SAP Digital Manufacturing (Cloud), you can enhance your manufacturing processes and improve production efficiency. This integration enables the system to detect custom objects in real-time, ultimately leading to a more streamlined and effective manufacturing operation.