In this blog post we will discuss about one of the major pain areas while loading big tables in your cFiN migration project :  “Long load times“. We will see how can we leverage parallel jobs for performing initial loads for big tables. We will discuss about how can we reduce load time and also the precautions to take while doing so. We will also talk about how we can make sure there is minimal or no impact on the SAP source system while running the initial loads in parallel jobs.

Background

We know that initial load consists of 2 main phases – Access Plan Calculation (APC) and the Actual Load. Both these phases run as background jobs in systems. While APC job runs simultaneously in SLT and SAP source systems, the load job runs only in the SLT system.

Roughly 70-80 % of the actual load time is spent in the APC job. The remaining is spent on the actual load. By default APC is done via a single background job only which can be a problem in loading big tables like COBK, AUFK which may have millions of records. With a single job for APC the calculation job may take days to finish thereby increasing the actual load time. It is therefore imperative that we do run the APC job in multiple background jobs in parallel.

• After creating the MTID in SLT open LTRS settings in SLT for that same MTID.

• Now add the table (e.g.COBK) which has let’s say 10 million records in the SAP source system.

• In the “Initial Load Options” Tab change the reading type from “1 Range Calculation” to “5 Sender Queue”. This reading type is for Performance Optimized Initial load mode of Transparent tables  (like cFiN related tables COBK, AUFK, etc.).

With the below settings, 10 million records in the table COBK will be divided into 10 parallel jobs with 1 million record each.

This means that 10 jobs each will run in the SAP source system and the SLT system in parallel.

It is important to note here that SAP recommends  – If the access plan is calculated based on the primary key, only use a maximum of 8 parallel jobs for one table.

But, in my personal experience I have used 25-30 parallel jobs as well with no negative impact on my initial loads. Of course while doing so we have to agree with source system owners to provide a dedicated application server for SLT loads or at least provide some non-busy time slots for loads (like nights or weekends).

• After saving the LTRS settings we will be setting up the load for COBK in SLT. Details about the setup I will cover in other blog.

• Before applying the parallel job settings in LTRS we need to make sure that SLT and more importantly the source system has enough background work processes available. We don’t want our cFiN loads to negatively impact the source systems. Otherwise, the core value of cFiN would fail,  that is, ” Non-Disruptive with no negative effect on the source system” .

There is an  “Advanced Job Settings” option that we can use in SLT configurations that will make sure that only a particular application server from the source would be used for load / replication jobs related to SLT. From the “Administration Data” tab of our MTID we can hard-code a particular application server in “Source System” settings.

So, basically we need to ask the source system owners that we will need additional app server in the SAP source systems to make sure that there is no negative performance impact due to SLT transfers on the running source system.

• Parallelization can only be done for a table in source which has more than 50k records. Anyways there won’t be any tangible load time reductions in case we have 50k or less records in any table.

Conclusion

Running loads for big tables should always be done in parallel processes to reduce the overall load times. e.g. Load running for a table with 10 parallel work process will finish approximately 10 times faster. An experienced SLT consultant must know and suggest to client the benefits of parallel loading . Especially for larger tables this technique is vital to achieve lesser load times. I have seen loads running for 8-10 days that after parallelisation got completed in 3-4 hours.

SLT Performance Optimization guide

I hope this blog post will help you to understand more about the importance performing parallel loads in SLT. Thank you for spending time reading it. Please do provide your valuable feedbacks and if you have any questions, please contact me.

Please follow me on the forum for more SLT-cFIN related contents : Abhishek Singh

Follow this space for related contents: SAP Landscape Transformation replication server | SAP | SAP Blogs