In this post I describe how to scale a microservice by deploying it in multiple apps on SAP BTP, Cloud Foundry runtime. I describe my experience with achieving this using multiple modules in a multitarget application (mtar). SAP BTP, Cloud Foundry runtime allows scaling of apps by adding more instances and more memory. It was possible always to run more apps too. I present a case here for why this might be a simpler solution to some problems.

Scaling solutions to meet the demand is an important consideration in microservices design. But when the boundaries of a microservice were decided, this may not have been a consideration. This decision might have been dictated more by team velocity and paved paths from tools. Resource and scaling requirements of workloads for a microservice may not have been even apparent at the time the decision was made. It might not have been a deliberate choice or the choice may not have been apparent. Whatever the reason, is restructuring the microservice the only choice now?

Solution

It was always possible to deploy the same build results  for a microservice as several applications.

a micro service deployed as multiple applications

Here is an example of deploying the same same build results as multiple apps:

cf push cf-bl-app-1 -path gen/srv --no-route
# We will explcitly bind to nicely named route

cf push cf-bl-app-2 -path gen/srv --no-route
# We will explcitly bind to nicely named route

Note that deployment is usually described in an App manifest or a Multitarget Application Development Descriptor where scaling and routes are also specified. I am splitting these into multiple commands for illustration. There are commands for copying package of an app to another. I let these pass too for simplicity in illustration.

Then each application can be scaled independently.

cf scale cf-bl-app-1 -i 10 -k 1G -m 1G
#   -i             Number of instances
#   -k             Disk limit (e.g. 256M, 1024M, 1G)
#   -m             Memory limit (e.g. 256M, 1024M, 1G)

Each application can be a mapped to a different route and so workloads can be executed on the dedicated application when the clients choose the right host.

cf create-route cfapps.us10.hana.ondemand.com 
    --hostname risk-risk-management-mkjy
cf create-route cfapps.us10.hana.ondemand.com 
    --hostname mitigation-risk-management-mkjy

cf map-route cf-bl-app-1 cfapps.us10.hana.ondemand.com 
    --hostname risk-risk-management-mkjy
cf map-route cf-bl-app-2 cfapps.us10.hana.ondemand.com 
    --hostname miti-risk-management-mkjy

For example Application Router configuration could be (for illustrating) as follows:

"routes": [
  {
    "source": "^/service/risk(.*)$",
    "destination": "risk-service"
  } ,
  { 
    "source": "^/service/mitigation(.*)$",
    "destination": "mitigation-service"
  } 
]

with destinations configured as follows:

destinations:  
[
  {"name":"risk-service", 
   "url":"https://risk-risk-management-mkjy.cfapps.us10.hana.ondemand.com"},
  {"name":"mitigation-service", 
   "url":"https://miti-risk-management-mkjy.cfapps.us10.hana.ondemand.com"}
]

When different workloads are attached to different paths segments, routes can be created with the same hostname but different paths. The different applications can be mapped to these different routes. For example the commands shown below could be used for this.

cf create-route cfapps.us10.hana.ondemand.com 
    --hostname risk-management-mkjy 
    --path service/risk
cf create-route cfapps.us10.hana.ondemand.com 
    --hostname risk-management-mkjy 
    --path service/mitigation

cf map-route cf-bl-app-1 cfapps.us10.hana.ondemand.com 
    --hostname risk-management-mkjy 
    --path service/risk
cf map-route cf-bl-app-2 cfapps.us10.hana.ondemand.com 
    --hostname risk-management-mkjy 
    --path service/mitigation

Then the clients would not need to know the different hosts to be used for different services. Instead they can use the microservice deployed as multiple applications without the knowledge how it is deployed using the same hostname. This is illustrated with configuration for Application Router as a client below.

"routes": [
  {
    "source": "^/service/risk(.*)$",
    "destination": "business-logic-app"
  } ,
  { 
    "source": "^/service/mitigation(.*)$",
    "destination": "business-logic-app"
  } 
]

destinations:  
[
  {"name":"business-logic-app", 
   "url":"https://risk-management-mkjy.cfapps.us10.hana.ondemand.com"}
]

Multitarget Applications

Most applications are deployed as multitarget applications (MTA) in SAP BTP, Cloud Foundry environment. Once deployed, modules in an MTA manifest as applications. We need to have multiple modules in an MTA to have multiple applications in SAP BTP Cloud Foundry, runtime like described earlier. But the modules can share the same content or build result. This is illustrated in the following snippet from an MTA descriptor.

modules:
  - name: risk-management-risk
    type: nodejs
    path: gen/srv
    parameters:
      buildpack: nodejs_buildpack
      command: cds serve RiskService
      instances: 2
      memory: 112M
      disk-quota: 400M
      routes:
      - route: ${myhostname}/service/risk
    requires:
      - name: risk-management-db
      - name: risk-management-uaa

  - name: risk-management-mitigation
    type: nodejs
    path: gen/srv
    parameters:
      buildpack: nodejs_buildpack
      command: cds serve MitigationService
      instances: 4
      memory: 96M
      disk-quota: 400M
      routes:
      - route: ${myhostname}/service/mitigation
    requires:
      - name: risk-management-db
      - name: risk-management-uaa

Both the modules refer to the same source (referred to by path) for the deployed content. Each module may have its own command so as to activate only the functionality it serves. This is not mandatory as the requests for other paths are not routed to the app at all.  But the modules refers to the same shared resources. Environment variables could be used to have different database pool sizes or control workload specifications for used services if the microservice makes use of these features.

The build process can be optimized by adding build dependencies so that the source code is not built multiple times redundantly.

  - name: risk-management-mitigation
....
    build-parameters:
        requires:
          - name: risk-management-risk
        builder: custom
        commands: []
....

Shared Module Binary

Even though we now have multiple modules using the same content, the content is packaged twice (or as many times as there are modules) in the generated archive. It is possible to avoid duplicate content in an MTA archive by sharing the content among modules. Shared Module Binaries is supported in SAP BTP for deploying Multitarget Applications. The support described uses mtad.yaml and MANIFEST.MF files But in a typical workflow, these files are generated by Cloud MTA Build Tool from mta.yaml file. We can achieve the desired result with the following changes to path and build-parameters. Path now points to the previous module and the build-parameters indicate that there is no source to be bundled for this module.

  - name: risk-management-mitigation
    path: risk-management-risk
...
    build-parameters:
      no-source: true
      requires:
        - name: risk-management-risk
...

With these changes, even though the generated mtad.yaml file is as specified for shared module binary, the MANIFEST.MF is not. The generated MANIFEST.MF contains an entry like the following for the module that includes the content:

Name: risk-management-risk/data.zip
MTA-Module: risk-management-risk
Content-Type: application/zip

It contains no entries for the modules with no content (no-source: true). We need to correct the above entry to be the following:

Name: risk-management-risk/data.zip
MTA-Module: risk-management-risk, risk-management-mitigation
Content-Type: application/zip

So this workflow, currently, requires post build patching to correct the generated file MANIFEST.MF and enable this feature. I used the below script for the same. It relies on a copy of MANIFEST.MF I changed and saved and its original (MANIFEST_org.MF) before the change in a directory named META-INF.bak.

mtar='risk-management.mtar'
mbt build -t . --mtar $mtar
jar -xvf $mtar META-INF/MANIFEST.MF
diff3  -am META-INF/MANIFEST.MF META-INF.bak/MANIFEST_org.MF 
      META-INF.bak/MANIFEST.MF > META-INF/MANIFEST.MF
jar -uMvf $mtar META-INF/MANIFEST.MF

This worked so far. The file MANIFEST.MF does not change unless new modules are added (or deleted or renamed). So one could just have a good copy and update the archive with it.

In this article I showed a method of scaling microservices deployed as applications in SAP BTP Cloud Foundry, runtime that was always possible but not frequently used. I present the case that this is the easiest way to partition workloads in a microservice.

Have you used this in your microservices? How did that work for you. Please let me know in the comments.