Are you curious how to maintain your Value Mapping objects in Cloud Integration in a non-manual and automatic way with data from external systems, applications or remote locations?

Look no more! You have found the right place! 

But let’s start from the beginning…

Introduction

Value Mapping (VM) is a very well-known and useful integration artifact that is consumed in most Integration Scenarios. Experienced Integration Developers are familiar and often use this object for mappings and conversions. If you are the lucky one and already have experience with VM objects you may want to skip this unit and jump directly to the core and Scenario unit.

Value Mapping in SAP Cloud Integration similarly to how it was in SAP PI/PO provides functionality to store bidirectional value mapping entries stored in containers identified with a unique pair of Source/Target Agency and Identifiers. VM artifact can be then used in the Message Mapping step or in Groovy Script to retrieve target values based on source value of the field. Simple example of Value Mapping conversion might be Country ISO code conversion between ISO alpha-3 and ISO alpha-2 code.

Example:

From the source system we receive Country code as “POL” (alpha-3 ISO code) but for integration purposes we need to convert it to “PL” (alpha-2 ISO code) as target system process and understand only alpha-2 ISO codes. For that purpose we can implement Message Mapping or Groovy Script in our iFlow that will call VM artifact and convert into target value based on provided source value. Such VM objects can be maintained with Country code conversion for reusability purposes by multiple iFlows at the same time.

In this way we will be able to use beforehand prepared value mapping entries. Information stored in a VM object is providing bidirectional mapping functionality, it means we can convert both ways from source to target and from target to source values.

Value Mapping objects might be used for more complex scenarios and use cases for example to store some Integration Flow (iFlow) related attributes needed in runtime for dynamic configuration, determination or for message routing purposes.

For detailed documentation on Value Mapping object in Cloud Integration please refer to official SAP documentation:

Creating Value Mapping | SAP Help Portal

Configuring Value Mappings | SAP Help Portal

Above documentation provides detailed procedure on how to maintain Value Mapping artifacts and VM entries through WebUI of Cloud Integration.

If you want to check how to integrate and maintain Cloud Integration VM objects with data coming from 3rd party systems using SAP Cloud Integration OData API you are in the right place! Please continue with reading this blog – I promise it will not be (so) long reading.

Scenario

In this use case scenario let’s assume we want to replicate Customer master data from SAP backend system to 3rd party system using SAP Cloud Integration as middleware. Due to the nature and capabilities of the target system this had to be a custom iFlow as no standard content was available.

As part of Customer master data entity on the target system side there are several fields that require conversion but not only to different readable values but also into system specific GUID format (for example 825f1c22-162e-e011-8836-1200c2992223). These target system GUID values could unexpectedly change on the target system, therefore automatic updating of value mapping is the main prerequisite for the orchestration. I was looking for a generic and standard approach as it is required to convert multiple fields in such a way. To handle this requirement in the best way and to not impact performance of the replication process I’ve chosen a Value Mapping object in SAP Cloud Integration. This object fits well with the requirement of storing source to target field values conversions.

Ultimate goal is to be able to automatically update and deploy VM artifacts with new/updated value mapping entries coming from 3rd party systems so it can be then used in the Customer master data replication process. In this Blog I will focus on one field that is converted in such a way and will present the approach to maintain a VM object with external values using a separate scheduled process.

Implementation

Currently SAP Cloud Integration provides an already impressive number of internal OData API’s to handle objects and internal operations on Cloud Integration artifacts. You can find always up to date list of Integration Content API here: Overview | Integration Content | SAP API Business Hub

New features are coming on a regular basis so if you want to stay up to date with the new Cloud Integration API’s and features coming in then follow Release Notes for Cloud Integration.

In this blog I will focus on Integration Content – Value Mapping related operations. API Reference contains an already long list of API’s ready to be used for VM artifacts maintenance. You can read whole value mapping artifacts or specific value mapping entries. You can also update VM artifacts with new entries as well change version of VM artifact and deploy it remotely on Cloud Integration tenant. You can also pull technical details of value mapping entries and set default values.

This time I will deep dive into Value Mapping API’s that are necessary to implement reliable end to end process of VM maintenance:

• Getting Value Mapping artifact details
• Getting Value Mapping entries details
• Updating Value Mapping entries (Insert)
• Setting default values for Value Mapping entries
• Creating version of Value Mapping artifact
• Deploying Value Mapping artifact on the Cloud Integration tenant

This Blog is based on SAP Integration Suite Cloud Foundry implementation, for Neo some aspects may be different.

Preparation

To be prepared for such automatic handling of Value Mapping artifacts I will present below recommended initial steps that worked well in my case.

1. Establish Service Key OAuth 2.0 Client Credentials in “api” plan and with proper roles on your BTP tenant. This will not be described in this Blog in detail. Please follow standard documentation for this. Detailed guideline can be found in official SAP Help Portal Creating Service Keys in Cloud Foundry | SAP Help Portal Roles to be added to the Instance of this Service Key for Value Mapping API are following: “WorkspacePackagesConfigure”, “WorkspacePackagesEdit”, “WorkspaceArtifactsDeploy”.
2. Identify proper endpoint to be used with your Cloud Integration Tenant for internal OData API’s. This comes together with Service Key from BTP cockpit. Addition /api/v1/ is needed in the URL to use described API’s in this Blog.
3. Establish a way to provide external VM entries to your Cloud Integration Tenant by either pushing or pulling data, this part is completely up to the source system capabilities and needs to be designed accordingly with connectivity testing. In case of this Blog I used an HTTPS adapter to pull data from an external system.
4. Create a VM object and maintain Source and Target Agencies as well the Identifiers needed by your scenario. Please note that it is required to maintain the first default entry manually as it’s not possible to save an object without at least one value mapping entry. Deploy your initial VM to your tenant.
5. My recommendation is also to set up Postman requests in line with your SAP Cloud Integration Process to test all GET/POST calls independently. This will definitely speed up the development process.

Hint: Make sure you are not in Edit mode of VM while upserting new entries through OData API as you will get HTTP 500 exception during runtime.

iFlow steps

Below picture shows an example of the main Integration Process that connects Local Integration Processes together. First step is preparation of all required Headers and Properties, if any. Second Local Integration Process refers to processing VM entries and updating VM artifact. Last Local Integration process will trigger deployment of VM artifact after all updates are done, you can split the process into more readable building blocks as you prefer, this is just an example.

It’s recommended to call below API’s in below sequence to successfully update and deploy VM objects.

Fetch Value Mapping entries from external system

First step is to fetch Value Mapping entries from external systems, applications, files or any other source. You could also expose such integration as a REST or SOAP service and expose it to your counterpart.To be precise both pull and push integration patterns are available for this scenario.

You may want to use this also for A2A integration or expose the possibility to maintain Value Mapping entries for Integration Process to Functional or Business Key Users through user friendly corporate portal. All these scenarios can be implemented following guidelines in this blog. Depending on your scenario you need to choose an appropriate trigger (for example Timer) and Sender adapter in your Cloud Integration iFlow. You could also support full or delta loads depending on specific requirements or technical limitations of the source system.

In my case I’ve implemented a REST call to source system to pull all details about Country GUIDs value mapping. The most important information to process from VM perspective was source system abbreviation for Country and GUID. In my case it was also possible to focus on delta mode and pull only new or changed entries on the source system side. Please remember to consider initial full load at first before running delta updates.

Hint: Log payload groovy script step to store payload as MPL attachment is used only for testing purposes, it is not recommended to use in productive scenarios.

Below is sample XML file pulled from the source system that will be used in updating of Cloud Integration Value Mapping.

<?xml version='1.0' encoding='UTF-8'?>
<response>
    <entry>
        <value>
            <modifiedon>2022-11-22T16:33:37Z</modifiedon>
            <countryid>28173567-836a-ed11-9561-000d3adf702f</countryid>
            <name>Poland</name>
            <abbreviation>PL</abbreviation>
            <createdon>2022-11-22T16:33:37Z</createdon>
        </value>
    </entry>
</response>

The fields that are crucial for this update are the abbreviation field with country code and the contryid field with Country GUID. Fields modifiedon and createdon from this source system might be used for delta processing of VM entries.

Get current VM version

First step is to get the current version of Value Mapping artifact so it can be increased while update of new VM entries is being done. This can be retrieved from Cloud Integration using the following OData API ValueMappingDesigntimeArtifacts as below.

Integration Content API’s on Cloud Integration are implemented as OData but in this case we call API without any body in GET request therefore HTTPS adapter is used.

Hint: If you want to retrieve the latest version id of a VM object (without knowing it) use ‘active’ in Version URL field value. You can also specify the intended version id and provide it in the query, for example as ‘1.10.1’.

Payload sent back from Cloud Integration with details of current active version of VM artifact will look like below and will contain current latest active version id of this object, it can be stored as property and used to calculate next version.

<?xml version='1.0' encoding='utf-8'?>
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xml:base="<CPI_host>:443/api/v1/">
    <id>https://<CPI_host>:443/api/v1/ValueMappingDesigntimeArtifacts(Id='Country_GUID_VM',Version='1.0.1')</id>
    <title type="text">ValueMappingDesigntimeArtifacts</title>
    <updated>2023-01-11T08:13:58.138Z</updated>
    <category term="com.sap.hci.api.ValueMappingDesigntimeArtifact" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"/>
    <link href="ValueMappingDesigntimeArtifacts(Id='Country_GUID_VM',Version='1.0.1')" rel="edit" title="ValueMappingDesigntimeArtifact"/>
    <link href="ValueMappingDesigntimeArtifacts(Id='Country_GUID_VM',Version='1.0.1')/$value" rel="edit-media" type="application/octet-stream"/>
    <link href="ValueMappingDesigntimeArtifacts(Id='Country_GUID_VM',Version='1.0.1')/ValMapSchema" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ValMapSchema" title="ValMapSchema" type="application/atom+xml;type=feed"/>
    <content type="application/octet-stream" src="ValueMappingDesigntimeArtifacts(Id='Country_GUID_VM',Version='1.0.1')/$value"/>
    <m:properties>
        <d:Id>Country_GUID_VM</d:Id>
        <d:Version>1.0.1</d:Version>
        <d:PackageId>PRPOC</d:PackageId>
        <d:Name>Country GUID VM</d:Name>
        <d:Description></d:Description>
        <d:ArtifactContent m:null="true"/>
    </m:properties>
</entry>

Upsert Value Mapping in Cloud Integration

After retrieving value mapping entries from the source system and pulling the latest version of the VM object we can start processing new/updated VM entries and upsert them to the VM object. For that purpose you can use Splitter step and process it one by one to VM entries.

In my case I also implemented Router step to check if source system VM entry contain mandatory fields (this could be also implemented with XSLT or Groovy or XSD/XML validation step).

Assignment of source/target values to properties.

Once both source/target values are stored as properties we can call another Cloud Integration API for Inserting new VM entries that is called UpsertValMaps. With this API you can create new entries in VM object with values coming from external system or process. Make sure your POST call doesn’t contain any body as you may get Bad Request exception.

Below are query parameters used in this POST call, there is no body of message in this request, therefore make sure to set body as initial to avoid Bad Request errors.

After successful Upsert of an entry into a VM object you should receive HTTP 202 code without any response message body.

Hint: Be careful with using IsConfigured as true, once VM is in status “Configured” it cannot be Updated with VM API.

Update default Value Mapping entry

In case of initial or second update of the same Value Mapping source to target identifier below steps are not required. Default values are needed if there is only one unique source/target value entry. Default values are set automatically for a second update of the same source value.

2 VM entries for the same source value and default value is correct.

3rd entry for the same source value comes in and default value is not updated.

In such a situation, if the source system is providing subsequent updates for the same source value then the expected default value needs to be enforced. This can be done with another API called UpdateDefaultValMap.

Before that, you will need one additional call to be able to use UpdateDefaulValMap as it requires ValMapId, which is GUID of a specific Value Mapping entry. Unfortunately it’s not provided back from Cloud Integration in headers while you insert an entry therefore after successful update of VM entry you need to pull Value Mapping ID using again following ValueMappingDesigntimeArtifacts API.

This is how GET HTTPS call is configured in this case:

You should get back similar payload where you can easily locate Value Mapping ID in field “Id” and use it in the next step.

<?xml version='1.0' encoding='utf-8'?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xml:base="https://<CPI_Host>:443/api/v1/">
    <id>https://<CPI_Host>:443/api/v1/ValMaps</id>
    <title type="text">ValMaps</title>
    <updated>2023-01-11T10:04:46.547Z</updated>
    <author>
        <name/>
    </author>
    <link href="ValMaps" rel="self" title="ValMaps"/>
    <entry>
    <id>https://<CPI_Host>:443/api/v1/ValMaps('bc11b4c7e938f3d1116751de93b0456f')</id>
        <title type="text">ValMaps</title>
        <updated>2023-01-11T10:04:46.547Z</updated>
        <category term="com.sap.hci.api.ValMap" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"/>
        <link href="ValMaps('bc11b4c7e938f3d1116751de93b0456f')" rel="edit" title="ValMap"/>
        <content type="application/xml">
            <m:properties>
                <d:Id>bc11b4c7e938f3d1116751de93b0456f</d:Id>
                <d:Value m:type="com.sap.hci.api.Value">
                    <d:SrcValue>PL</d:SrcValue>
                    <d:TgtValue>44480a51-ef5f-eb11-8117-005056b72e26</d:TgtValue>
                </d:Value>
            </m:properties>
        </content>
    </entry>
</feed>

Having a Value Mapping ID GUID you can proceed with setting default value for this value mapping source/target value pair.

This is how POST HTTPS call is configured in this case:

After successful HTTPS POST to this API you should get HTTP 202 response code and no body in response message.

Save new Value Mapping version

After all new entries are updated in the VM object you may want to create a new version of the VM artifact in version history to track changes. You can do this with ValueMappingDesigntimeArtifactSaveAsVersion API.

This API is required to provide Value Mapping technical ID and new version id to be created. Below are HTTPS POST Call details for this step.

After a successful update you should get HTTP 200 response code and the VM version should be updated accordingly. There is no way right now to put comments on this version.

You are ready to deploy this new version!

Deploy Value Mapping object

After all updates are done in VM object and saved as a version you expect to be, then VM object can be deployed on Cloud Integration so it can be used by iFlow in runtime.

For that purpose use DeployValueMappingDesigntimeArtifact API, this call need to be done with Value Mapping Artifact ID and Version id that you want to deploy it might be specific version that you just created in previous step after all updates (for example 1.1.0) or “active” version. In case you choose the active version, the latest draft version will be picked up and deployed with changes.

As in the previous steps, also in this case it’s a pure REST call so the OData V2 adapter cannot be used as we will not provide any body of request message and all parameters will be in the URL. I used the HTTPS adapter as in previous steps with the same authentication concept. To avoid Bad Request errors you can clean up message body in the Content Modifier step.

Hint: In this case it’s recommended to set ‘Throw Exception on Failure’ to capture problems with deployment and react on it.

If deployment is triggered successfully based on the above request you should get HTTP 202 response code.

Result

If you follow this Blog in detail you should be able now to update Cloud Integration Value mapping artifact with value mapping entries coming from external system or location. For the purpose of my Demo I have updated below value mapping with Country GUIDs from external system.

For updated entries you should also get default values maintained in case of multiple target values for single source value.

Hope this helps to implement a similar scenario! Let me know in the comments how it goes.

Wrap up

Value Mapping artifacts maintained in such a way, automatically, as background process from 3rd party system, application or file will bring huge benefits to integration processes where regular updates of VM entries are required for smooth and reliable data transfer and will reduce mapping issues to minimum. This approach will also reduce the burden of manual updates in VM object performed by Integration Consultant.

It’s important to mention that it’s also possible to read value mapping entries directly in integration flow using Groovy Script but this blog is not focusing on this functionality and the whole process implemented for this demo was done without a single line of Groovy Script code.