Have you wondered if Mathematical Optimization software like MIP Codes from FICO-Xpress or Gurobi can be easily deployed on SAP BTP Runtimes, just as ML loads. The answer is yes, you can.In this 2-part blog we will walk through quick and easy steps to deploy on 2 of the BTP Runtimes 1. SAP’s AI Core via the AI Launchpad 2. SAP Kyma (another blog post).

The steps to create the Docker image are the same for both deployment options in the two runtimes. Infact once you have created the docker and pushed to docker repo, you can use the same image for AI Core or Kyma.

AI Core has some set of requirements in terms of naming end points and ports that are exposable, however these restrictions serve useful in having a prescribed format and do not cause any un–necessary overhead hence I prefer to have the same Docker image which can be easily used by either runtime.

Now which deployment option is best suited depends very much on the use-case and how the optimization results results need to be integrated in the business process and the overall technology landscape. I will provide some guidelines at the end of the blog but the choice of deployment option would need a wider consideration depending on the use-case you have in mind.

Background

Mathematical Optimization techniques are widely needed to solve a wide range of problems in decision making and finding optimal solution(s) when many exist.Besides the many real world complex problems that depend on Mathematical Optimization, these techniques are also increasingly being used to enhance Machine Learning solutions to provide explainability and interpretability as shown by for example

• Interpretable AI and develop-more-accurate-machine-learning-models-with-mip
• Use-cases on finding counterfactuals to provide options which could make ML Classifier classify differently Counterfactual Analysis for Functional Data

These techniques are useful when for example it is not sufficient to classify credit scores or health scores into good or bad but also provide what can be changed to make the classification go from bad to good.

SAP Products like SAP Integrated Business Planning incorporate optimization in the supply chain solutions. For other problems where you need to extend the solutions and embed ML or Optimization, this blog post walks through the steps of easily installing third-party libraries on BTP so these solutions can be integrated with the enterprise solutions which needs these capabilities.

Prerequisites

To go through the steps described in the blog make sure you already have the following

• SAP AI Core and AI Launchpad enabled on your BTP account : Details for this can be found in the official SAP Help Document which provides links to helpful tutorials on developers.sap.com
• Docker Desktop for building docker images : Tutorial from SAP AI Core on how to get started It is useful to go through this tutorial if you are not yet familiar with SAP AI Core and AI Launchpad.
• Access to a Github repo and considerations for SAP AI Core

SAP AI Core provides many equivalent capabilities via SAP AI Launchpad or programmatically via SAP AI Core SDK.In this blog post I will primarily use SAP AI Launchpad as it is easier to get started with minimal coding effort.

The subsequent step is to include the development artifacts into an existing project structure, for which the SAP AI Core SDK way might be better suited. Since this step is highly project dependent it is not covered in this blog post.

Create the Docker Image

We will create a docker image to solve a simple problem of coin changing using one of three ways:

1. FICO-Xpress
2. Gurobi
3. Pyomo

We do not need all the 3 libraries but we add them to show the deployment method is the same. In the example provided here you can choose to install either only 1 or only 2 or 2 and 3 as pyomo does not come with a solver of its own.

The library you choose would depend on your use-case and which of these is your preferred way of working. There are of course other optimization libraries, for this blog post I chose the above 3 for their ease on installation as they are all on pypi and come with community licenses which can help get started. For productive use you would need to obtain commercial licenses for FICO-Xpress or Gurobi. Pyomo is BSD but does not come with an optimization solver of its own, and is useful if you have not yet decided on the solver itself. In my example since we already install Gurobi we would also use it via the Pyomo interface as the solver.

If you have highly non-linear problems(higher than Quadratic) as in the chemical industry, FICO-Xpress would be a good option with its SLP and Knitro solvers.

Optimization Problem Description

The aim of this blog post is not to solve large and challenging optimization problems, but to facilitate deployment of optimization solutions in productive environments using SAP BTP Runtimes. We will solve the problem of change making using one of the 3 libraries above. Real world problems would be harder and longer lines of code, this problem is chosen as it has solutions which can be checked easily which is handy when you are getting started. The real problems would have their own version of optimizer.py but the deployment mechanism would be very much the same as this sample.

The change-making problem addresses the question of finding the minimum number of coins (of certain denominations) that add up to a given amount of money. For the example we will use pennies, nickels, dimes and quarters. So for example for amount of 101 cents the optimal coins would be 4 quarters and 1 penny.

Here are code snippets which solve this for the 3 libraries, the input is the amount that needs to be converted. Its in a file called optimizer.py

You can choose only one the 3 options if you do not want all 3 library options.

### Choose the right set of functions if you dont want to try all 3 options: Xpress/Gurobi/Pyomo
import xpress as xp
def make_change_xpress(amount, coins):
    model = xp.problem('coin-changer-xpress')
    x = {(c): xp.var(vartype=xp.integer,name='x(' + c + ')') for c in coins.keys()}
    model.addVariable(x)
    model.addConstraint(sum(x[c]*coins[c] for c in coins.keys()) == amount)
    model.setObjective(sum(x[c] for c in coins.keys()), sense=xp.minimize)
    
    model.solve()
    return {c: int(model.getSolution(x[c])) for c in coins.keys()} 

import gurobipy as gp
def make_change_gurobi(amount, coins):
    model = gp.Model('coin-changer-gurobi')
    x = model.addVars(coins.keys(), vtype=gp.GRB.INTEGER,name='x')
    model.addConstr(sum(x[c]*coins[c] for c in coins.keys()) == amount)
    model.setObjective(sum(x[c] for c in coins.keys()), sense=gp.GRB.MINIMIZE)
   
    model.optimize()
    return {c: int(x[c].x) for c in coins.keys()} 

import pyomo
from pyomo.environ import *
def make_change_pyomo(amount, coins):
    model = ConcreteModel()
    model.x = Var(coins.keys(), domain=NonNegativeIntegers)
    
    model.amount = Constraint(expr = sum(model.x[c]*coins[c] for c in coins.keys()) == amount)
    
    model.total = Objective(expr = sum(model.x[c] for c in coins.keys()), sense=minimize)

    SolverFactory('gurobi').solve(model)
    return {c: int(model.x[c]()) for c in coins.keys()} 

Now we will need a main app to call the above functions, we put it in file call app.py

SAP AI Core requires the endpoints to have a format with /v<number>/ but the name is upto you. Here our main entry function we call callOptimizer with route /v1/callOptimizer. We will need this later to test.

from flask import Flask
from flask import request
import os

app = Flask(__name__)


@app.route('/v1/status')
def status():
    print("/ called")

    return 'Container is up & running'

# problem data
coins = {
    'penny': 1,
    'nickel': 5,
    'dime': 10,
    'quarter': 25
}

import optimizer
@app.route('/v1/callGurobi')
def callGurobi():
    print("/ called callGurobi")
    amount = request.args.get('amount')
    if amount is None:
        amount = 1034
    print("amount=" + str(amount))
    return optimizer.make_change_gurobi(int(amount),coins)

@app.route('/v1/callXpress')
def callXpress():
    print("/ called callXpress")
    amount = request.args.get('amount')
    if amount is None:
        amount = 1034
    print("amount=" + str(amount))
    return optimizer.make_change_xpress(int(amount),coins)

@app.route('/v1/callPyomo')
def callPyomo():
    print("/ called callPyomo")
    amount = request.args.get('amount')
    if amount is None:
        amount = 1034
    print("amount=" + str(amount))
    return optimizer.make_change_pyomo(int(amount),coins)

@app.route('/v1/callOptimizer')
def callOptimizer():
    try:
        optOption = os.environ["optimizer"]
        print(f'{optOption}')
        amount = request.args.get('amount')
        if amount is None:
            amount = 1034
        print("amount=" + str(amount))
        if optOption == 'Pyomo':
            return optimizer.make_change_pyomo(int(amount),coins)
        if optOption == 'Gurobi':
            return  optimizer.make_change_gurobi(int(amount),coins)
        if optOption == 'Xpress':
            return optimizer.make_change_xpress(int(amount),coins)
        
        return {'status':'No Optimizer chosen'}
    except Exception as e:
        print(repr(e))
        return(repr(e))

   

if __name__ == '__main__':
    print("Serving Started")
    app.run(debug=True, host='0.0.0.0',port=9001)

Docker Image

Now that we have the code ready, we can create a docker file Dockerfile. All the installation dependencies for the Docker image are placed in requirements.txt

Flask==2.0.1
gunicorn==20.1.0
pyomo
gurobipy
xpress
hana_ml ## Not Needed for the example. Provided as code evolves to use SAP HANA for data access

Dockerfile

FROM python:3.9

COPY ./requirements.txt /app/requirements.txt
RUN pip3 install -r /app/requirements.txt

# Copy app.py and optimizer.py 
COPY ./ app 

# Required to execute script for SAP AI Core
RUN chgrp -R nogroup /app && 
    chmod -R 770 /app

Now we build and push the docker image for example like this.

### Build the docker image
docker build --tag docker.io/<YOUR DOCKER REPO USERNAME>/sap-optiml-example:1.0 ./

### Push the docker image
docker push docker.io/<YOUR DOCKER REPO USERNAME>/sap-optiml-example:1.0

Create the Deployment Script on GitHub

Now we are ready to create the deployment. Depending on the usecase we can deploy the optimization code as a WorkflowTemplate or a ServingTemplate. For this example since the problem solving time is sub-second I will deploy it as a ServingTemplate and send the amount as a parameter to the API call. This enables quick testing. For larger optimization problems which require more resources and done as batch jobs they should be deployed as a Workflow and then we would not have functions in app.py which expect request objects.

The workflow is in a file callcall_optimizer.yaml which needs to be in a Git Repo.

apiVersion: ai.sap.com/v1alpha1
kind: ServingTemplate
metadata:
  name: sap-optiml-example-deployment # executable ID, must be unique across your SAP AI Core instance, for example use `server-pipeline-yourname-1234`
  annotations:  
    scenarios.ai.sap.com/description: "Optimization Libraries (Example)"
    scenarios.ai.sap.com/name: "sap-optiml-example-deployment"
    executables.ai.sap.com/description: "Create online server to call optimization libraries"
    executables.ai.sap.com/name: "sap-optim-model-deployment"
  
  labels:
    scenarios.ai.sap.com/id: "sap-optiml-example-deployment"
    ai.sap.com/version: "1.0"
spec:
  inputs:
    parameters:
      - name: optimizer # (options are: 'Xpress','Gurobi','Pyomo')
        type: string 
  template:
    apiVersion: "serving.kubeflow.org/v1beta1"
    metadata:
      annotations: |
        autoscaling.knative.dev/metric: concurrency   # condition when to scale
        autoscaling.knative.dev/target: 1
        autoscaling.knative.dev/targetBurstCapacity: 0
      labels: |
        ai.sap.com/resourcePlan: starter # computing power
    spec: |
      predictor:
        imagePullSecrets:
          - name: docker-registry-secret   # your docker registry secret
        minReplicas: 1
        maxReplicas: 5    # how much to scale
        containers:
        - name: kfserving-container
          image: "docker.io/sapgcoe/sap-optiml-example:1.0"
          ports:
            - containerPort: 9001   # customizable port
              protocol: TCP
          command: ["/bin/sh", "-c"]
          args:
            - >
              set -e && echo "Starting" && gunicorn --chdir /app app:app -b 0.0.0.0:9001 # Port same as above in line containerPort
          env:
            - name: optimizer 
              value: "{{inputs.parameters.optimizer}}"

Some of the key differences for an Optimization deployment vs an ML Inference Deployment is that the Optimization deployment does not refer to a pre-trained model or have storage artifacts related to it.The only configuration parameter above is choice of optimizer. This is again not mandatory but shows how you can add your specific parameters you would want to configure when running the model.

Here are few configuration changes that are key to understand as they describe what the deployment can do.The yaml defines the following

1. The scenario that you will see in the AI Launchpad
2. The parameters that can be configured at deployment when you create a config in AI Launchpad
3. The docker that will be used and the corresponding secret
4. How the docker image should run at start, here we use gunicorn to run the flask app, here is it important that the name of the file is same as you name the python code above, in our example app is the name of the file app.py and also the name of the flask app
5. 

For larger optimization problems that we do as WorkflowTemplates we can save results for an optimization run and then use these to warm-start subsequent runs. In this case we can use the same mechanism as ML Models for persistency, here we save solutions that can be improved upon and not models.

Create the Deployment on SAP AI Core via AI Launchpad

Onboard GitHub with deployment yaml to SAP AI Core

Onboard the git repo with the code above to SAP AI Core as described in Steps 1-4 of Start for Your First AI Project Using SAP AI Core

Once the repo is onboarded now we can follow all the steps in AI Launchpad. All the specifics for deployment optimization libraries are covered in the code snippets for Dockerfile and deployment yaml above, now we can follow the standard procedure in AI Launchpad to create a deployment.

In the Launchpad the Git repo in the SAP AI Core Administration looks something like this, the values of course depend on your git repo

Create an application to sync workflow and access the scenario in AI Launchpad

Now we will add a corresponding Application for this workflow. Go to Applications on AI Launchpad and click Create. For reference these steps are described in Step 5 of Quick Start for Your First AI Project Using SAP AI Core.

Change the entries based on your setup

Run the AI Workflow for Optimization

Next step is to create a configuration for this scenario. The scenario gets created from the yaml file without any user input. Now we just need a configuration that can be used for deployment. The only parameter that we had in the yaml file was to specify which optimizer we want to use by default, so that is the only parameter we need to specify and we are ready to deploy the optimizer.

The version shown will depend on what is in the yaml file in the field: ai.sap.com/version

Optimizer Selection: Enter one of Pyomo/Gurobi/Xpress

Once you have created the config you can directly create Deployment

Create Optimizer Deployment

Test the Deployment

Once you have the deployment it will go from state Pending to Running in a few minutes

Deployment Pending

Deployment Ready

The logs will show the status of the deployment, incase there are any errors these would show here too.

The above also provides the url we will need for testing. The steps to test are same as testing any AI Deployment as described in Step 5 of Tutorial on AI Core Deployment

For this step you would the service key for your SAP AI Core instance as described here SAP Help Document for Creating a Service Key

If you save the service key in a file called say aic_service_key.json and have installed ai-api-client-sdk (which is available via  pip install ai-api-client-sdk). The steps here are for python notebook but you can follow the Postman route if you prefer

Setup the ai_api_client

resource_group = "<your resource group from AI Launchpad>"

aic_service_key = 'aic_service_key.json'
with open(aic_service_key) as ask:
    aic_s_k = json.load(ask)

ai_api_v2_client = AIAPIV2Client(
    base_url=aic_s_k["serviceurls"]["AI_API_URL"] + "/v2",
    auth_url=aic_s_k["url"] + "/oauth/token",
    client_id=aic_s_k['clientid'],
    client_secret=aic_s_k['clientsecret'],
    resource_group=resource_group)

We will use the endpoint /v1/callOptimizer (note this is the function you had created in the app.py)

deployment = ai_api_v2_client.deployment.get(<your deployment id from AI Lanuchpad>)

endpoint = f"{deployment.deployment_url}/v1/callOptimizer"
headers = {"Authorization": ai_api_v2_client.rest_client.get_token(),
           'ai-resource-group': resource_group,
           "Content-Type": "application/json"}
response = requests.get(endpoint, headers=headers,params={"amount":42})

response
if (response.status_code == 200):
    print(response.text)

Optimizer Result

In the AI Launchpad you can see the detail logs for this call. The optimal objective is 5 (seen in the logs below) as we used 5 coins to make 42 cents, corresponding to the API call above for amount of 42 with the optimal change being 1 Quarter, 1 Dime, 1 Nickel and 2 Pennies as seen above.

Optimizer Logs from AI Launchpad

When to use SAP AI Core for Optimization Deployment

SAP AI Core is well-suited if you have Machine Learning models running on SAP AI Core and it is desirable to consolidate all intelligence enabling applications to be managed from a central place.

In some cases input for the Optimization models comes from Machine Learning models, so for such scenarios too its helpful to have the ML and Optimization applications be close together and possibly in the same scenario if they are related.

In addition SAP AI Core also provides capabilities for tracking metrics which are useful when you want to benchmark different optimization runs either for different solvers or parameter tuning. For more details on enabling metrics tracking in your code refer to Tutorial on SAP AI Core Metrics.