This blog demonstrates an approach of collecting information about applications, service instances and other entities from SAP HANA Extended Application Services, Advanced Model (XSA) using XS Controller APIs.

TL;DR: The blog provided an overview of a command-line tool that utilizes XS Controller APIs to simplify some of maintenance operations. The tool is based on a reusable Python module that represents various entities of the XS Controller, such as applications, service instances and others, and methods to work with them. Besides the given examples, you can extend the functionality to cover your own use cases. The comprehensive documentation for this example is provided in the Git repository https://github.com/nklinked/otter.

The data samples provided in Example Use Cases are wide tables that might be quite complicated to read. Nevertheless, providing them here is a better approach than just describing their content without an example.

Introduction

On your road to cloud-native applications running in SAP Business Technology Platform and following the evolution of the on-premise HANA Native Development, you may face a requirement to analyze a current state of your SAP HANA XS Advanced system and establish some maintenance operations improving it. Some examples of those tasks are:

• Collect general statistics about applications and service instances deployed in all organizations and spaces. Preserving this information may be an important step when running an operation changing the global state of the system, e.g., while doing a system upgrade or a system copy

• Collect the information about all User-Provided Services

• Collect mappings between HANA service instances and their HDI container schemas. Not critically important but sometimes a nice feature, when you need to identify a respective service instance of the problematic HDI container

• Identify orphaned applications and service instances created temporarily by SAP Web IDE for HANA to clean them up. Such objects are very intensively produced in an actively used development system

• Identify STOPPED and CRASHED application instances to clean them up massively. Those instances are usually used for the post-mortem analysis, but deleting them for every single application can be a time-consuming task

• Identify outdated service keys

• Identify if count of the di-builder service bindings is coming closer to the threshold (should be <= 120)

• Identify application API calls demonstrating decreased response time

• Collect Organization and Space Roles and Role Collection assigned to users

• Establish a foundation to collectively manage the respective entities

The complexity of those operations directly correlates with the number of the objects they should be perform on:

Motivation

Fulfilling the aforementioned tasks you may need to interactively run XS CLI commands and manually collect the required information, e.g., to collect information about MTA objects we need to iterate over all spaces and run:

• xs mtas to fetch all deployed MTA applications
• xs mta <mta_name> to fetch all applications and service instances belonging to the MTA project
• xs app <app_name> to fetch information about the specific application belonging to the MTA project
• xs service <service_instance_name> to fetch information about the specific service instance belonging to the MTA project

Automating such a scenario based on XS CLI, you may need to run the shell script performing all those steps iteratively. Even though this approach is feasible, it requires some further optimization.

Thanks to Thomas Jung and his great post about XS Controller interactions, implementation of those requirements received a solid foundation.

Addressing the basic requirements, following assumptions are taken:

• Python can be used to prototype and to implement the required functionality faster
• The user interface is not required, a command-line interface is absolutely sufficient
• All collected information can be stored in csv files for further analysis. Specific conclusions will be done based on those reports
• Operations changing state of the system should be marked as experimental and should be run with all required precautions

Brief Implementation Overview

The provided example is a Python-based application decoupling two layers:

• The reusable module representing the XS Controller entities, their metadata and applied methods. Files of the module are located in components/controller
• Specific functions consuming XS Controller APIs are located in components/tools. This folder contains tools implementing use cases provided above

Specific client configuration can be passed in the config.yaml, e.g., lists of objects that are prohibited for modifications.

otter
├── components
│   ├── controller
│   │   ├── application.py
│   │   ├── controller.py
│   │   ├── database.py
│   │   ├── organization.py
│   │   ├── service.py
│   │   ├── session.py
│   │   └── space.py
│   └── tools
│       ├── cleaner.py
│       ├── client.py
│       ├── collector.py
│       └── utils.py
├── config.yaml
├── otter.py
└── requirements.txt

The application follows the standard UAA authentication flow and requires the XS Advanced user with Role Collections XS_CONTROLLER_USER and XS_CONTROLLER_ADMIN assigned.

All commands run by the example application can be triggered by the respective command-line arguments, that follows the method similar to XS CLI. Every command requires the XS Controller API endpoint, user credentials and selective criteria to be provided.

Example Use Cases

Following examples demonstrate possible reports that can be generated from the system for further analysis.

Collecting information about mapped Tenant Databases

Using this command we can collect information about database tenants known by XS Advanced in the CSV file.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org --report-databases

The operation is the system wide and ignores any selective argument. The resulting CSV file will be stored in <output_dir>/databases/databases.csv

The produced output will contain following fields:

Collecting information about Organization Roles assigned to users

Using this command we can collect information about Organization roles (OrgManager, OrgAuditor) assigned to users in the CSV file.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org --report-org-roles-assignment

The operation is the system wide and ignores any selective argument. The resulting CSV file will be stored in <output_dir>/users/org_roles_assignment.csv

The produced output will contain following fields:

Collecting information about Space Roles assigned to users

Using this command we can collect information about Space roles (SpaceManager, SpaceAuditor, SpaceDeveloper) assigned to users in the CSV file.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org --report-space-roles-assignment

The operation is the system wide and ignores any selective argument. The resulting CSV file will be stored in <output_dir>/users/space_roles_assignment.csv

The produced output will contain following fields:

Collecting information about Role Collections assigned to users

Using this command we can collect information about Role Collections assigned to users in the CSV file.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org --report-space-roles-assignment

The operation is the system wide and ignores any selective argument. The resulting CSV file will be stored in <output_dir>/users/role_collections_assignment.csv

The produced output will contain following fields:

Collecting information about Applications

Using this command we can collect information about applications in the CSV file.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org --report-application-instances

The operation can be limited to a single space by providing argument -s, –space <SPACE>. In case argument -s, –space <SPACE> is not given the command will be executed for all spaces within the given organization.

The resulting CSV file will be stored in <output_dir>/apps/<org>/<space>/apps.csv

The produced output will contain following fields:

Collecting information about Service Instances

Using this command we can collect information about service instances in the CSV file.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org --report-service-instances

The operation can be limited to a single space by providing argument -s, –space <SPACE>. In case argument -s, –space <SPACE> is not given the command will be executed for all spaces within the given organization.

The resulting CSV file will be stored in <output_dir>/services/<org>/<space>/service_instances.csv

The produced output will contain following fields:

Collecting information about User-Provided Service Instances

Using this command we can collect information about user-provided service instances in the CSV file.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org --report-user-provided-service-instances

The operation can be limited to a single space by providing argument -s, –space <SPACE>. In case argument -s, –space <SPACE> is not given the command will be executed for all spaces within the given organization.

The resulting CSV file will be stored in <output_dir>/services/<org>/<space>/user_provided_service_instances.csv

The produced output will contain following fields:

Collecting information about Service Keys

Using this command we can collect information about service instances and keys in the CSV file.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org --report-service-keys

The operation can be limited to a single space by providing argument -s, –space <SPACE>. In case argument -s, –space <SPACE> is not given the command will be executed for all spaces within the given organization.

The resulting CSV file will be stored in <output_dir>/services/<org>/<space>/service_instance_keys.csv

The produced output will contain following fields:

Collecting information about Applications and Service Instances not belonging to any of the deployed MTA Projects

Using this command we can collect detailed information about applications and service instances, having no MTA information in the CSV file. Most frequently applications and service instances having no Multi-Target Application attribution are created by the SAP Web IDE and are not cleaned up properly after the development. Nevertheless, the exceptions are possible, e.g., the manually created service instances and applications pushed to XS Advanced.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org -s some_space --report-non-mta-objects

The produced outputs will contain the same fields as -rai (–report-application-instances) and -rsi (–report-service-instances).

The operation can be limited to a single space by providing argument -s, –space <SPACE>. In case argument -s, –space <SPACE> is not given the command will be executed for all spaces within the given organization.

The resulting CSV files will be stored in <output_dir>/apps/<org>/<space>/non_mta_apps.csv and in <output_dir>/services/<org>/<space>/non_mta_service_instances.csv

Collect parsed Router Logs of applications

Using this command we can collect the parsed application log, having only RTR entries, in the CSV file. Such an application log contains only the entries representing the API call routed to the application. The log is useful to identify the long running API calls within the monitored timeframe.

otter -a https://some.hostname:30033 -u some_user -p SomePassword12 -o some_org -s some_space -app some_app --report-parsed-app-log

The operation can be limited to a single app by providing argument -app, –application <APPLICATION> coupled with argument -s, –space <SPACE> .

The operation can be limited to a single space as well by providing argument -s, –space <SPACE> . In case argument -s, –space <SPACE> is not given the command will be executed for all spaces within the given organization.

The resulting CSV files will be stored in <output_dir>/apps/<org>/<space>/<app>/router_log.csv

The produced output will contain following fields:

Closing

The blog provided a brief overview of a Python-based module utilizing XS Controller APIs. Please refer to the complete example and more comprehensive documentation in the repository https://github.com/nklinked/otter.

Please note that running massive operations with the tool, e.g., deleting all STOPPED and CRASHED application instances, you need to specify the Exclusion List to avoid any influence on the critical applications. Current list of massive operations:

• Identify and stop continuously crashing applications still having a small portion of uptime
• Delete all STOPPED and CRASHED application instances
• Delete applications and service instances not belonging to MTA projects (highly experimental and requiring the Exclusion Lists to be properly set)

Reference

• Product Documentation: The XS Command-Line Interface Reference
• The example solution for Node.js: SAP HANA XSA Controller API Interaction
• The API documentation: Cloud Foundry /v2 APIs
• The example solution for Python: Otter