This is going to be my first blog on SAP Data Intelligence and quite excited about sharing with all my SAP colleagues/members working on this tool. This document overviews about deployment life cycle management for SAP Data Intelligence using CI/CD methodology. Since SAP DI doesn’t provide any proprietary tools for deployments as of the current....

Table replication using SAP Data Intelligence and SAP Landscape Transformation Replication Server (SLT) can be leveraged using Generation 1 SLT operator and Generation 2 Read Data from SAP operator. Blog by Martin Boeckling and Replicating data using Gen2 operators blog by Pranchal Narang go into detail on how to setup pipelines for replicating data using the above-mentioned operators. In addition to that, SAP Data Intelligence Cloud provides....

For the CDS view extraction from SAP S/4HANA we are going to use ABAP CDS Reader operator in DI. It is a standard operator to read data from a CDS view. Problem description: Using this operator, you will not see any headers in the file by default. There are three versions of ABAP CDS Reader operator. The....

Custom Python operators are the most convenient way to develop non-standard integrations in SAP Data Intelligence. The vast range of Python libraries available is virtually unlimited. In this blog post, I will demonstrate how to proxy TCP traffic of third-party libraries through the SAP Cloud Connector, enabling the integration of on-premises systems to be as....

The Transmission Control Protocol (TCP) is a widely used protocol that provides a reliable and ordered delivery of data between applications running on different hosts. It serves as the foundation for many technologies and plays a crucial role in modern IT infrastructure. SAP Data Intelligence is a powerful platform that allows you to integrate various....

The (Secure) File Transfer Protocol is still a very common way to integrate files from different sources. SAP Data Intelligence supports many source systems for file operations out of the box. To allow for even more flexibility in the connection to SFTP servers, this blog post shows how to use the Python library Paramiko to....

Part 3: Steps to enable Address Validation Service in SAP Success Factors Employee Central This Blog is a collaboration work between SAP Business Technology Platform (BTP) Team, SAP Data Quality Management Team and SAP SuccessFactors Employee Central – Localization Team. Welcome to Part 3 blog series on enabling address validation service within Data Quality Management,....

First things first Regular ChatGPT uses a large language model and has a lot of general knowledge in training. I believe that if you are reading this blog, you already tried OpenAI´s ChatGPT and was impressed by it. However, things might get a little weird when we ask for insights about specific context that it....

GPT is a great technology when it comes to understanding the semantics and context of statements given by the user. But with prompt engineering, we can boost its assertiveness by providing clear and effective orders and also the context in which the answer should be based upon. It´s one of the most important parts of....

SAP Data Intelligence offers a powerful feature that allows users to create custom operators using Python. This blog post demonstrates how to access cloud connector resources directly from within the custom Python operator. By combining the Cloud Connector with SAP Data Intelligence Python Operators, users can effortlessly connect to an even wider range of on-premise....

Part 1: Setting up Address Validation Service in BTP This Blog is a collaboration work between SAP Business Technology Platform (BTP) Team, SAP Data Quality Management Team and SAP SuccessFactors Employee Central – Localization Team. In this article we want to share our hands-on experience on a recent project experience and learning journey on how....

Part 2: Creating Address Mapping Configurations This Blog is a collaboration work between SAP Business Technology Platform (BTP) Team, SAP Data Quality Management Team and SAP SuccessFactors Employee Central – Localization Team. Welcome back to part 2 of three-part series blog on Address Validation Service within Data Quality Management, Microservices for Location Data and SAP....

On a recent blog post, I discussed creating dynamic tables with Generation 2 graphs.  There was a question on how one can use Structure Table operators like Data Transform or Table Producer with a Kafka operator and Python operator using Generation 1 graphs.   This combination would require the graph to construct a table and use a....

In this blog, you will learn, detailed steps required to create an SAP Datasphere Service Instance in the SAP Business Technology Platform (BTP), If you’ve signed the Cloud Platform Enterprise Agreement (CPEA) or Pay-As-You-Go for the SAP Business Technology Platform. SAP Datasphere is the next generation of SAP Data Warehouse Cloud, which is a unified service for data integration, data cataloging, a....

Hello everyone, Again another request from one of my customer, actually from several of my customers. How can you extract data from an ECC system or an S/4HANA system into Google Big Query with delta load? As you’ll see, this is very simple and straightforward using SAP Data Intelligence, you can perform Intial load, delta....

SAP Landscape Transformation Replication Server (SLT) is a product that allows users to replicate data between systems. While there are a lot of blogs out there covering SLT in general, I will focus on how SLT can be used with SAP Data Intelligence (DI) for replication. SAP Data Intelligence is a product that is used....

Dear SAP-community, The start of a new year is the best time to fuel your brain with SAP integration knowledge! Picture Credit: DALL·E 2   To get an extensive overview of various SAP integration technologies, I highly recommend reading “SAP Interface Management Guide” by Adam Kiwon (me), Mark Lehmann, Manuel Männle, Martin Tieves which is available in German and English. You will learn about....

While data is a critical asset for modern businesses, the ability of technology to scale has resulted in a flood of big data. Data management and storage has evolved into a necessary component for modern operational processes. Snowflake, a cloud data warehouse that is lauded for its ability to support multi-cloud infrastructure environments, is one....

This is Article 1 in a three-part blog series by Daniel Howard (see bio below) on data management covering the then and now, the anatomy of a data management platform, and making data management work for you. Data management has always been in constant flux. While individual data technologies can, occasionally, become stagnant – or even die out –....

This is Article 2 in a three-part blog series by Daniel Howard (see bio below) on data management covering the then and now, the anatomy of a data management platform, and making data management work for you. Following on from our previous blog on the recent trends within the data management space, this blog will introduce some of the....

This is Article 3 in a three-part blog series by Daniel Howard (see bio below) on data management covering the then and now, the anatomy of a data management platform, and making data management work for you. Previous blogs in this series have discussed the state of data management and described the most important capabilities for a data management....

Intelligent Situation Automation (ISA) is an extension of Situation Handling in SAP S/4HANA Cloud on the SAP Business Technology Platform (SAP BTP). If you aren’t familiar with Situation Handling yet, read this blog post to find out more. Intelligent Situation Automation enables us to make Situation Handling even more intelligent and integrate cutting-edge technologies, such as Machine Learning and Knowledge Graphs.  In this blog post, I want to introduce you to the Situation Knowledge Graph – it’s the basis of the Explore Related Situations....

It is the purpose of this blog to describe the Git Terminal Application in SAP Data Intelligence Cloud that is included in release 2022.08 mid of August 2022 (see the What’s New Blog of my colleague Eduardo Schmidt Haussen) Here is the table of contents for this blog. Introduction Prerequisites and Resources The SAP Data Intelligence Cloud....

The DI (Data Intelligence) RMS (Replication Management Service) provides initial loads or initial  and delta, sometimes a scheduled delta is preferred. In this blog post I will describe how we can achieve a scheduled delta with an RMS Flow and/or RMS Tasks. At the time of writing (November 2022) the Data Intelligence scheduler doesn’t allow....

This blog describes the process to connect Google Cloud with SAP Data Intelligence, install the ODBC driver and create a pipeline to write data into a table in Google BigQuery. 1.- Create the connection between SAP Data Intelligence and Google Cloud Login to your Google Cloud Storage Account, navigate to IAM & Admin -> Service Accounts....

For those of you who enjoy using SAP Data Intelligence Cloud (SAP DI Cloud) for data management, integration and processing, and are wondering how it can be integrated with SAP Profitability and Performance Management Cloud (SAP PaPM Cloud). Well, you’re in luck because this blogpost will cover this topic, so just keep on reading!    Scenario: A source table is created as an input for the View function.  Note:  The source table....

SAP Analytics Cloud makes it easy for businesses to understand their data through its stories, dashboards and analytical applications. However, sometimes we might not be sure how we can leverage SAC to create these based on data from other applications For this worked example, we’re going to make use of SAP Data Intelligence Cloud to retrieve data from SAP....

This is Part Two of a blog series on Ariba Analytics using SAP Analytics Cloud, Data Intelligence Cloud and HANA DocStore. If you would like to start with Part One, please click here Recap SAP Analytics Cloud makes it easy for businesses to understand their data through its stories, dashboards and analytical applications. Our worked example is....

Since Release 2.3, I used to Leverage Ceph as persistent storage for SAP Data Hub. At the time, this had been based on the Mimic release, which has been archived mid-2020. It did its job and even came with an at the time read only Dashboard already. Intermittently, I leveraged Longhorn to Install Gardener on Rancher Update 2022. However, lately....

This is Part Three of a blog series on Ariba Analytics using SAP Analytics Cloud, Data Intelligence Cloud and HANA DocStore. If you would like to start with Part One, please click here Recap SAP Analytics Cloud makes it easy for businesses to understand their data through its stories, dashboards and analytical applications. Our worked example is....

In a previous blog post we examined how to add external Python packages that are not included in the default Python operator by using a custom dockerfile. In this blog we will detail how to load functions from a local Python file to be used in Data Intelligence (DI). Prerequisites: Basic Python knowledge Basic knowledge of the Data Intelligence....

Intro Once you start seriously implementing and orchestrating the pipelines in the SAP DI, question will come your way – how can I dynamically run a (sub)graph? How to pass parameters to it? The standard offers us two ways to solve this problem, but both are not ideal. Which is great, because we can solve....

In this tutorial, I will show you how to create additional python kernels on the SAP Data Intelligence to be used within the Jupyter Lab and the ML Scenario Manager. I, like many of you probably, really like working with the SAP Data Intelligence. What I probably use most, is the Jupyter Lab application, which....

Working with the SAP Data Intelligence is great, especially because you can create your own operators, suited exactly for your use case. But every time when I worked with custom operators, I asked myself about how to versionise those operators, e.g. in git (stay tuned for a follow up article on working with git repos....

Disclaimer: the paragraph “SAP Data Intelligence pipeline” has been updated on 2022, September 29th.   Welcome to the third episode of the series: Integrating SAP Signavio Process Intelligence and SAP Data Intelligence. In case you missed the two previous blog posts: In Integrating SAP Signavio Process Intelligence and SAP Data Intelligence Cloud: process mining streamlined across....

Since I did Install Gardener on Rancher, it has become my Kubernetes platform of choice. Lately, I ran into an issue creating new clusters leveraging Cloudflare though: To troubleshoot this, I install gardenctl in 4 steps: Package Manager kubectl kubelogin gardenctl Package Manager Working on Windows, I leverage Chocolatey: Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1')) kubectl To start with,....

Introduction Document Extraction Information is a BTP service, and helps you to process large amounts of business documents. The purpose of this blog post is to demonstrate how you can combine it with SAP Data Intelligence. The use case is simple: we want to upload an invoice document into Document Extraction Services using Data Intelligence, and....

I’ve written a bunch of Sample Scripts, in Postman, that demonstrate how the ‘export data’ API retrieves data from models stored inside SAP Analytics Cloud. These samples are now available. This blog first takes a step back and provides an overview of other means and ways to extract data before then introducing the sample scripts....

Previously, I Install Gardener on Rancher, wondering what Gardener might add to Prepare your SAP Data Intelligence installation with Rancher. Coming from Make your SAP Data Hub Distributed Runtime work on the SUSE CaaS Platform, I had been leveraging SUSE Linux Enterprise Server to start with. Lately, I have been having also good experience with Ubuntu though, resulting in some additional optimizations described in this blog....

📢Note: Before implementing data extraction or write back to/from SAP systems please verify your licensing agreement. Dear community, For many companies, SAP ERP is at the heart of daily business operations and therefore, there is a desire by many stakeholders to leverage SAP data for insights. Data often gets curated in shiny slide decks and....

Building deep learning model within SAP BTP Dear readers,  This is a blog series from the Meter Reading project and we attempted to discover the possibilities for ML models in a web browser. This long journey has provided many lessons learned in terms of deep learning and deployment in web browsers. As a beginner in....

In this blog post I will show how we can configure the SAP Analytics Cloud (SAC) Data Export API to access SAC planning data and integrate it with Data Warehouse Cloud (DWC) and SAP Data Intelligence (DI). The API provides OData and Cloud Data Integration (CDI) access for great integration with DWC and DI. CDI....

In a recent project, we wanted to present data in a SAP Analytics Cloud story. The usecase was to export data from Qualtrics to show it live in the SAC. Additionally we also wanted to include information coming from a Machine Learning model, together with the original data, so that the user can see all....

Before explaining how a specific SAP analytics or data management product can address a dedicated problem, it is quite important to understand the overall problem using a product agnostic approach. A reference architecture typically helps in the identification of areas and business pain points that need to be addressed. Together with a world wide team....

SAP Data Intelligence have some capabilities of extract data from many sources, like SAP ABAP based systems, traditional and nosql databases, streaming tools, API’s and even SAAS 3rd Party solutions, but if you need to extract data from Salesforce, there are no standard operator to achieve these requirement. Because of this, we will see in....

As an SAP BTP customer, you likely are aware that SAP Data Intelligence provides Data Management capabilities such as – data integration, data catalog, etc. Have you seen how SAP Data Intelligence empowers a Data Scientist to perform end-to-end data science jobs?  In this blog, we want to show you how you can enrich/transform the....

This blog is demonstrating how we can leverage the power of SAP Data Intelligence around DEVOPS. In this blog we are giving glimpse how to check the ATL CODE of SAP Data Services developed by user following naming standard of an Organization or not Using SAP Data Intelligence pipeline and metadata explorer we can proactively....

Quite a number of blogs have been written about how to use SAP Data Intelligence for Machine Learning scenarios. Have a look for example to excellent and very instructive blogs of Andreas Forster. With the 2nd generation operators of SAP Data Intelligence we have got some additional facilitations to create productive training and predication pipelines. In....

In this blog post I will demonstrate how to load Qualtrics Survey Data via a custom SAP Data Intelligence operator into a SAP HANA Cloud table for use in SAP Data Warehouse Cloud and SAP Analytics Cloud. I will show how solutions from unified data and analytics can be used together to get insights from....

During our SAP Community Call of SAP Integration Suite on Dec 9th, 2021, you asked us a few questions, which I would like to answer as Q&A in this blog post. In the community call “SAP Integration Suite – Overview, Recent Innovation and Roadmap,” my colleague Udo Paltzer tells you about SAP Integration Suite, which is....

Highlights from the latest conversation between Mohamed Abdel Hadi, VP Product Management & Strategy, SAP Data Warehouse & Miro Dimitrov, COO, NWO.ai Today’s most critical asset in helping companies make better strategic decisions? Data. And in order to be a successful data-driven company with a competitive edge, internal data just won’t cut it. Companies must....

Introduction: automation using Zapier I doubt it, you don’t know what Zapier is, but for a short recap: it’s the SaaS application that “automates apps in millions of ways”. You can choose ready to implement receipts that connect and automate the most popular applications into one workflow. Set it once – and forget. From my....

This post will guide you through the steps to connect your SAP Cloud for Real Estate API published by the SAP API Business Hub for Cloud for Real Estate to your SAP Cloud for Real Estate standard content available in SAP Data Warehouse Cloud via SAP Data Intelligence Cloud. ****************************************************************************************************** As long as SAP Data Warehouse Cloud does not support....

Integration is usually the roadblock where every promising business transformation initiative stumbles sooner or later, usually leading to higher complexity and effort and lower quality, agility and reliability. In my view, this is often due to the fact integration is misunderstood as a “bottom-up” technology topic, rather than as one of the key pillars of....

Introduction With SAP Data Intelligence release 2110  a new set of operators have been introduced: The generation 2 operators. It is more a leap than a next step that includes not only new features but the fundamental design has also changed. This is a first release of these new type of operators therefore not all properties of....

The easiest way to get data into SAP Data Warehouse Cloud is to setup a connection to your data sources and retrieve the data from there. Though this is certainly the most common scenario, there are a few other interesting options on data ingestion covering different use cases, which I want to highlight in the....

Each year the Business Application Research Center (BARC) is conducting The Data Management Survey: The new survey on data management tools (bi-survey.com) and is collecting the feedback from the end users and implementation or consulting partners of Data Management solutions to present their findings and their view on the overall solution offering and portfolio of the different....

A big thank you for all your participation and contribution during the four months of our challenge! We are very happy to share that the Featured Contributors list on the SAP Data Intelligence community topic page has just been updated! We are excited to announce that these are the current Featured Contributors: Agus Jamaludin Andy Yin Indu....

SAP Data Intelligence, cloud edition DI:2103 is now available. Within this blog post, you will find updates on the latest enhancements in DI:2103. We want to share and describe the new functions and features of SAP Data Intelligence for the Q2 2021 release. If you would like to review what was made available in the....

Introduction This blog post refers to a series of blogs around SAP’s Business Technology Platform which aim to demonstrate an end-to-end scenario across the SAP HANA Database & Analytics Portfolio. It is part three out of five and it aims to demonstrate how to model geospatial data coming from SAP HANA Cloud in SAP Data Warehouse Cloud.....

Introduction This is the 1st of 5 blog posts which are part of the “SAP BTP Data & Analytics Showcase” series of blog posts. We recommend you to look into our overall blog post to gain a better understanding of the end-to-end story and context which involve multiple SAP HANA Database & Analytics Solutions in the cloud. In this....

A podcast on this topic is also available here. Let us now continue into our series and understand the details behind consuming the machine learning services from SAP Cloud Platform. In the earlier blog, we discussed briefly the various use cases around embedding and consuming machine learning models with SAP S/4HANA by explaining the way these use cases are....

Data is an asset that can be monetized in different ways that yield a high ROI. From consumable data sets and analysis results, businesses can go beyond reporting by incorporating the intelligence into existing products, digital platforms, or new business models. Take, for example, the SAP Analytics Cloud solution. When embedded in process and application workflows, the solution....

In this blog post I leverage the Iris flower data set provided by scikit-learn that contains three classes of fifty instances each, where each class refers to a type of iris plant: from sklearn.datasets import load_iris df = load_iris() df.target_names array(['setosa', 'versicolor', 'virginica'], dtype='<U10') One class (setosa) is linearly separable from the other two, but the latter....