This blog is the first of two I’m publishing and has the focus on how data from an on-premise S/4HANA system is extracted (for ECC it’s analogous), transformed and loaded (ETL) into SAP Signavio Process Intelligence for business process analysis. It reflects my personal learning journey and excitement working with SAP Signavio. It’s therefore not a best-practice document or recommendation in any way.

B. Signavio Overview

SAP Signavio consists out of several modules and it’s worth looking at them briefly before moving straight to the point of this blog.

SAP Signavio overview – Signavio Process Transformation Suite

Today we will look briefly into Process Manager and in more depth into Process Intelligence. We focus on how data can be extracted in relation to a defined business process. In another, upcoming blog I’ll explain how to analyze the data against the process definition to find improvement points.

In principle you use:

1. Process Explorer to organize the different artifacts you created or that were shared by others like: BPMN 2.0 diagrams, value maps, customer journey maps, UML diagrams and several more. The look-and-feel is similar to Windows Explorer or a file manager on Linux.
2. Collaboration Hub to share documents with the wider community in the company (ideally every employee should be able to look at the defined company process that impacts him/her). From there feedback can be provided.
3. Process Manager to define, build and manage process definitions.
4. Process Intelligence to extract, transform and load data from business systems or files that contain process data for measurement and to map them to process definitions.
5. Process Insights to continuously monitor what has been set up in Process Intelligence to identify bottlenecks, instabilities and compare across plants or company codes within a group.

C. On-premises extractor setup

To retrieve data from SAP S/4HANA on-premise you need the On-premises Extractor provided as part of Signavio Process Intelligence. Note that this tool can connect to a multitude of on-premises systems and S/4HANA is just one of them.

You can find it in the Process Intelligence → Manage Data menu.

Signavio Process Intelligence – Manage Data: Creating a new On-Premises extractor.

Through creation you obtain secrets that you have to store at the place where you run the extractor application.

Architecture of how Signavio On-Premises Extractor is integrated with S/4HANA.

Since the extractor is a component that must be run somewhere (e.g. a local server, notebook etc.) I decided to deploy it on SAP BTP Kyma, SAP’s Kubernetes offering.

For that you need to create a Docker image and in case you would like to replicate I share the Dockerfile below. Here are the detailed system requirements.

#Build SAP Signavio On-Premises Extractor Image
FROM openjdk:11.0.16-oraclelinux7

WORKDIR /usr/signavio

COPY . /home

RUN yum -y update && 
    yum -y install nano && 
    yum -y install unzip

EXPOSE 443/tcp

Dockerfile for the extractor. Put the downloaded and zipped on-premises extractor file in the same folder where the dockerfile resides. It will be copied into the image.

apiVersion: v1
kind: Namespace
metadata:
  name: dl-signavio
  labels:
    istio-injection: enabled
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: signavio-extractor-app
  namespace: dl-signavio
spec:
  replicas: 1
  selector:
    matchLabels:
      app: signavio-extractor-app
  template:
    metadata:
      labels:
        app: signavio-extractor-app
    spec:
      containers:
        - name: signavio-extractor-image
          image: <imagename from dockerhub>:1.0
          imagePullPolicy: Always  
          ports:
          - containerPort: 443
          resources:
            requests:
              memory: "2Gi"
              cpu: "250m"
            limits:
              memory: "3Gi"
              cpu: "500m"  
          volumeMounts:
          - mountPath: /home/config
            name: signavio-extractor-config-configmap
          command: ["/bin/sh"]
          args: ["-c", "if [ ! -f /usr/signavio/start.sh ]; then cd /home && unzip /home/*.zip && cd pi* && mv ./* /usr/signavio && rm /home/pi* -r && cp /home/config/config.env /usr/signavio/config.env; else echo Already installed, starting; fi; cd /usr/signavio && ./start.sh"]   
      volumes:
      - name: signavio-extractor-config-configmap
        configMap: 
          name: signavio-extractor-configfile
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: signavio-extractor-app
  name: signavio-extractor-service
  namespace: dl-signavio
spec:
  ports:
  - name: "adminport"
    port: 443
    targetPort: 443
  type: ClusterIP
  selector:
    app: sapcc-app
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: signavio-extractor-configfile
  namespace: dl-signavio
data:
  config.env: |
    ############ PLEASE INCLUDE YOUR EXTRACTOR ON-PREM CONFIGURATION BELOW #####################
    EXTRACTOR_ID=<my ID>
    EXTRACTOR_SECRET=<it's a secret! ;-)>
    WEBSOCKET_SERVER_ADDRESS=wss://<your signavio server>.suite-saas-prod.signav.io

You see the config.env file is added to kubernetes deployment yaml file. You’ll get the content for this file at the time you set up a new extractor in Signavio.

This yaml will extract the extractor 😀 (from its zip file) and then start it.

With that in place we can deploy it on SAP BTP Kyma. If you look for an introduction to Kyma, see the end of the blog.

We should check the log for the pod or – if we are bold go straight away to Signavio PI to check for the extractor status. After ca. 1 min of deployment it should come online.

D. Extraction method and details

The Signavio On-Premises Extractor uses an RFC connection  to the S/4HANA or ECC system based on the Java Connector (JCo). In the S/4HANA itself then, the well known RFC_READ_TABLE function module serves to generically read the table data.

This module has a limitation to 512 characters per table row in the TABLES parameter. There was a recommendation to create a Z-function module and specify that in the connection settings. It seems however, that with the recent addition of the ET_DATA export parameter using strings (and therefore removing the 512 characters limitation) that was removed. Please see the relevant note 2246160 in the references at the end of the blog.

One thing I found worth to mention – make sure that in the connection settings you have all tables defined that are part of the data pipeline! If you don’t there is only a java error raised that won’t be understandable easily:

com.signavio.pi.etl.extractor.shared.exception.EtlException: ExtractionJob failed | source-system: SAP
| exception: java.lang.IllegalStateException
| cause: null
| sql-query: [SELECT "MANDT" as "mandt", "EKGRP" as "ekgrp", "EKNAM" as "eknam" FROM T024
...
| Caused by: java.lang.IllegalStateException: Unable to obtain the schema for table: T024

See the screenshot below of the relevant settings.

Connection settings for S/4HANA: Unlock the tables for extraction here.

E. Process data pipeline and extraction

A process data pipeline is a chain of actions with a reference to a business process defined in SAP Signavio. It is that referenced process which is measured and later analyzed using Signavio Process Intelligence or monitored through Signavio Process Insights.

Often it’s a good idea to start with a template prepared – it saves time to assemble the S/4HANA tables needed and compiling the data model.

I’ve prepared a little video explaining the steps from template to extraction.

F. Outlook and next steps

I hope this first tutorial was useful to show how on-premise data is extracted and loaded into SAP Signavio. In an upcoming blog I will show how to analyze the procure-to-pay data inside SAP Signavio PI for optimization and value realization. Happy to receive your feedback!

References:

• Overview of connectors for SAP Signavio – link
• SAP Signavio Process Intelligence Overview – link
• SAP BTP Kyma: Build an Application in the Kyma Runtime – link
• SAP Note 2246160 – Enhancement RFC_READ_TABLE (7.40+) – link