Background:
Large-scale distributed data has become the foundation for analytics and informed decision-making processes in most businesses. A large amount of this data is also utilized for predictive modeling and building machine learning models.
There has been a rise in the number and variety of hyperscaler platforms providing machine learning and modeling capabilities, along with data storage and processing. Businesses that use these platforms for data storage can now seamlessly utilize them for efficient training and deployment of machine learning models.
Training machine learning models on most of the hyperscaler platforms is relatively smoother if the training data resides in their respective hyperscaler-native data storages. The tight coupling of the Machine Learning services with the native data storage results in the need to migrate or replicate the data from non-native data storages. Migrating the data from non-native data storage is a complex and time-consuming process. Moreover, replicating the data to the hyperscaler-native data storage to access its Machine learning capabilities leads to redundant storage, thereby incurring storage costs with the overhead of ensuring data consistency.
Proposed Solution:
Federated-ML or FedML is a library built to address these challenges. The library applies the data federation architecture with SAP Data Warehouse Cloud for intelligently sourcing the data in real-time from data storages. The library provides functionality that enables businesses and data scientists to build, train and deploy machine learning models on hyperscalers, without the hassle of replicating or migrating the data from the original data storage. The library also provides the capabilities to build machine learning models by sourcing the data stored across multiple data storages in real-time.
By abstracting the data connection, data load, model training and deployment on these hyperscalers, the FedML library provides end to end integration with just a few lines of code.
FedML Solution Diagram
In this blog, we use the FedML Azure library to train the model with the data from SAP Data Warehouse Cloud and deploy the model to Azure Compute Targets and SAP BTP Kyma Environment. We also inference the deployed model and store the inference data back to SAP Data Warehouse Cloud for further analysis.
The data can be federated to SAP Data Warehouse Cloud from numerous data sources including SAP and non-SAP data sources. The data from various data sources can also be merged to create a View, which can be used for the FedML experiment. Please ensure that the View used for the FedML experiment has consumption turned on.
Pre-requisites for the Federated ML Library for Azure ML
- Create a new Azure subscription by referring to the link or access the existing subscription information from the Azure portal.
- Create an AzureML workspace by referring to the guide. Once the workspace is created, take a note of the workspace configurations present in the config.json file that can be downloaded by referring this guide.
- Create an AzureML Notebook and compute by referring to the guide.
Train and deploy the model using FedML Azure Library:
For information on the classes and methods in the FedML Azure library, please refer the documentation.
1. Install the FedML Azure library.
pip install fedml-azure --force-reinstall2. Import the libraries needed
from fedml_azure import create_workspace,create_compute,create_environment
from fedml_azure import DwcAzureTrain
from fedml_azure import DbConnection
from fedml_azure import register_model,deploy,predictFedML Azure Library Version 1 Features:
3. Initialization of AzureML resources required for training:
3.1. Initialize the AzureML Workspace.
workspace=create_workspace(workspace_args={
"subscription_id": "<subscription_id>",
"resource_group": "<resource_group>",
"workspace_name": "<workspace_name>"
})
In the above cell, you need to replace the ‘subscription_id’, ‘resource_group’ and ‘workspace_name’ with the workspace configuration obtained from step 2 of the pre-requisites.
3.2 Create the desired Compute Target for running the training job.
You can create different types of Azure Compute Targets using the FedML Azure library. Here is a generic syntax for the creation of Compute Target.
compute=create_compute(workspace=workspace,
compute_type='<compute_type>', # Type of compute to be created
compute_args={<compute_configurations>} # Configurations for compute
) # provisioning
Details on how to create different types of Compute Targets with examples can be found here.
3.3 Create an Environment specifying the dependencies required for the training job.
You can create an environment specifying the dependencies by passing either the packages, whl files or a conda dependency file. Here is a generic syntax for creation of an environment.
environment=create_environment(workspace=workspace,
environment_type='<way_to_create_the_environment>',
environment_args={<environment_arguments>} # dependencies required for
) # the training job
Remember to pass the pip package ‘fedml-azure’ while creating an environment.
Details on different ways of creating an environment with examples can be found here.
4. Train the model:
Instantiate the training class which assigns the resources required for the training job.
train=DwcAzureTrain(workspace=workspace,
environment=environment,
experiment_args={'name':'<experiment_name>'},
compute=compute)Here, a new experiment with the name specified in ‘experiment_name’ is started, by using the above created resources.
Documentation on how to use the DWCAzureTrain class with examples can be found here.
5. Read the data from SAP Data Warehouse Cloud for the training job.
You need to create a training folder in the Azure Machine Learning Studio and place the training script inside the training folder, which contains the code to train the model. Here is a sample training script.
Then, you need to create a config.json file in the Azure Machine Learning Studio, containing the credentials to access the SAP Data Warehouse Cloud View. Please refer the documentation to setup a config.json file.
Once the config.json file is setup, use the below code snippet inside the training script to read the training data from SAP Data Warehouse Cloud:
import pandas as pd
dwc = DbConnection()
data = dwc.get_data_with_headers(table_name='<view_name>',size=<data_size_in_float>)
training_data = pd.DataFrame(data[0], columns=data[1])The following code snippet can be used for more flexibility on querying the data from SAP Data Warehouse Cloud:
import pandas as pd
dwc = DbConnection()
data = dwc.execute_query('<query_to_fetch_the_data>')
training_data = pd.DataFrame(data[0], columns=data[1])6. Specify the configurations required for the training job and start the training run.
src=train.generate_run_config(
config_file_path='<path_to_config.json_file>',# Path of config.json file to connect to
# DWC
config_args={
'source_directory':'<path_to_training_folder>',
'script':'<training_script_name>',
'arguments':[
'--model_file_name','<model_file_name>.pkl', # Name of the pickle file
# to be created
'--table_name','<view_to_be_queried_from dwc>'
]})Specify the config.json file path created in step 4, and the configurations for the training run in the above cell.
run=train.submit_run(src)The above cell starts the training job.
Detailed information on how to generate the run config can be found here and more information on how to submit the training run can be found here.
7. Register the model/models for deployment
7.1 Register the trained model/models for deployment.
model=train.register_model(
run=run,
model_args={
'model_name':'<model_name>', # Name of the model to register
'model_path':'<model_path>' # Path where the model pickle file is
# stored
},
resource_config_args={'cpu':1, 'memory_in_gb':0.5}, # Optional Resource config
# in case of deployment to ACI
is_sklearn_model=True) # Indicates a sklearn model and model framework and
# framework version is setDetails on registering a trained model for deployment with examples can be found here.
FedML Azure Version 2 Features:
7.2 In case you need to register model/models not trained by the training run, you can use the below code snippet for registering the model/models for deployment.
model=register_model(
model_args={'workspace':workspace, # Workspace for model registration
'model_name':'<model_name>', # Name of the model to register
'model_path':'<model_path>'},# Path where the model pickle file is stored
resource_config_args={'cpu':<cpu-cores>, 'memory_in_gb':<memory-in-gb>},# Optional Resource
# config in case of deployment to ACI
is_sklearn_model=True # Indicates a sklearn model and model framework and framework version
# is set
)Detailed information on documentation and examples can be found here.
8. Deploy the model/models as a webservice.
8.1. Deploy the model/models as a webservice to Azure Compute Targets.
You can deploy model/models as a webservice to various Azure Compute Targets such as Azure Container Instances (ACI), Azure Kubernetes Service (AKS) or deploy the model locally for testing/debugging purposes using the FedML Azure Library.
Remember to create the appropriate Azure Compute for deployment before you start the deployment process.
You will need to pass an inference script containing the code for inferencing to the ‘entry_script’ parameter in the cell below. Here is a sample inference script.
The following code snippet shows the generic syntax to deploy the model/models to Azure Compute Targets.
endpoint,api_key,service=deploy(compute_type='<compute_type_for_deployment>',
inference_config_args={'entry_script':'<entry-script-path>', 'environment':
environment},
deploy_config_args={<optional_deploy_configurations_for_computes>},
deploy_args={'workspace':workspace,'name':'<service-name>','models':<list-of-
models>}
)Detailed documentation for deployment to Azure Compute Targets with examples can be found here.
8.2 Deploy the model/models as a webservice to SAP BTP Kyma Environment
You can deploy model/models as a webservice to SAP BTP Kyma Environment.
Before starting the deployment process, you need to create a Service Principal to authenticate to Azure Container Registry (ACR). The steps to create a Service Principal can be found here.
Store the Service Principal credentials in a json file and pass the json file path to ‘sp_config_path’ parameter in the below cell. The library assigns an ACR Pull role to the Service Principal to pull the container image from ACR to SAP BTP Kyma Kubernetes.
You also need to create a kubeconfig.yaml to authenticate to the SAP BTP Kyma Environment. The steps to create a kubeconfig.yaml can be found here. Provide the kubeconfig.yaml file path to ‘kubeconfig_path’ parameter in the below cell.
Now, we are set to deploy the model/models as a webservice to SAP BTP Kyma Kubernetes. Use the following code snippet:
kyma_endpoint=deploy(compute_type='Kyma',
inference_config_args={'entry_script':'<entry-script-path>', 'environment':
environment},
deploy_args={'workspace':workspace,
'name':'<service-name>',
'models':<list-of-models>,
'num_replicas':<num-replicas>,
'kubeconfig_path':'<path-to-kubeconfig.yaml>',
'sp_config_path':'<path-to-sp_config.json>'
})The inference script provided to the ‘entry_script’ parameter will be used for inferencing. Here is a sample inference script.
You can deploy multiple models as a single webservice by registering the models and passing the list of models to the ‘models’ parameter.
If you wish to update or replace the deployed webservice with the updated model, set the ‘overwrite_service’ parameter to True. You can also control the number of Kyma Kubernetes pod replicas with ‘num_replicas’ parameter.
If you wish to copy the entire source directory containing additional files required for inferencing, to the container, you can provide the path of the source directory to the ‘source_directory’ parameter. Remember, in this case the entry script (inference script) path must be relative to the source directory.
Detailed documentation for deployment to SAP BTP Kyma Kubernetes with examples can be found here.
9. Inference the deployed webservice endpoint
As the model is deployed to an Azure Compute Target or SAP BTP Kyma Kubernetes, you can now perform inferencing on the webservice endpoint by passing the test data.
Use the below code snippet to perform inferencing by passing the endpoint url obtained from the deploy method, test data and the desired Compute Target type.
import json
test_data = json.dumps({
'data': <pandas-dataframe-containing-inferencing-data>.values.tolist()
}) #test data in the form of a serialized json format
inference_result=predict(endpoint_url='<endpoint-url>',
data=test_data,compute_type='<compute_type>')For the model/models deployed on Azure Compute Targets, you can also perform inferencing by passing the webservice object obtained from deploy method and test data.
import json
test_data = json.dumps({
'data': <pandas-dataframe-containing-inferencing-data>.values.tolist()
}) # test data in the form of a serialized json format
inference_result=predict(service=<webservice-object>,data=<test-data>)Detailed information on how to perform inferencing with examples can be found here.
10. Store the inference data in SAP Data Warehouse Cloud.
Now that you have run the predictions, you can store the inference data in SAP Data Warehouse Cloud to perform further analysis.
Firstly, create a table in SAP Data Warehouse Cloud to store the data.
dwc.create_table("CREATE TABLE <table_name> (ID INTEGER PRIMARY KEY, <column_name> <data_type>,..)")You can now restructure the data to write back to SAP Data Warehouse Cloud in your desired format. Then, you can insert the data in the table created by using the below code snippet
dwc.insert_into_table('<table_name>',<pandas_dataframe_containing_dwc_data>)Now, that the data is inserted into the local table in SAP Data Warehouse Cloud, you can create a View and deploy it in SAP Data Warehouse Cloud. You can then use the View to perform further analysis using SAC.
You can also drop the table using the code snippet below:
dwc.drop_table('<table_name>')More information on the use of the library and sample notebooks with the corresponding training and inferencing scripts can be found here.
In summary, the Federated Machine Learning library provides an effective and convenient way to federate the data from multiple data storages, perform cross-platform ETL’s, train and deploy machine learning models, without the overhead of any data migration or replication. With Version 2 of the library, you can effectively deploy models to Azure Compute Targets and SAP BTP Kyma Environment, perform inferencing on the deployed webservice and store the inference data back to SAP Data Warehouse Cloud to perform further analysis.
If you have any questions, please leave a comment below or contact us at paa@sap.com
