Autostop HANA database when reboot a linux system (with systemd integration)

This is my first Blog so please conisder;-)

As a technical sap consultant working for large companies we are facing several challenges concerning big SAP- and HANA-landscapes. There are a lot of automated processes for administrating of these landscapes and this leads to problems in connection with HANA databases. One of these challenges I wan’t to share in the following blog post.

Initial situation

Because of security reason there is a monthly linux patching timeframe which is during night and there is no one that can stop the HANA databases before patching and rebooting the linux servers. Currently the reboot is triggered although HANA is running, so the database has always to recover after restart because there is no ordinary shutdown of the HANA database. When analyzing the traces we always see entries like the following in the startup process after a reboot (without shutdown HANA database before):

[4464]{-1}[-1/-1] 2023-01-04 11:16:03.399284 i Logger RecoveryHandlerImpl.cpp(00417) : Scan of open segment found log end position 0x17bb2598c0 within segment LogSegment[0x0:0x17bb234c80/Writing,ts=2023-01-04 11:02:27.351677][GUID=d0aa515f-003e-20230104-100159-223000000c/PrevGUID=e4e1a93f-003e-20230104-093146-34d000000b/PersGUID=6f3c30be-003e-20220628-080019-068000000e/RestoreGUID=f6039e5c-00af-20200513-081833-074000000f] at file position 0x931000, unused rest 0x3f6cf000

With the systemd integration there seems to be (not official SAP document founded) the possibility to have an ordinary shutdown process of the HANA database during reboot of Linux server.

Activate systemd integration

Prerequisites

SUSE Linux Enterprise Server 15 (systemd version at least 234)
```
zypper info systemd
```
Installation of the polkit package is mandatory (see note 3139184)
```
zypper info polkit
```
HANA 2.0 SPS06 Revision 66 which is shipped with sapstartsrv 7.53 PL 1030 (at least PL 1011 is required for native systemd integration). At the time of blog writing, the HANA 2.0 SPS05 Revision 59.6 has still sapstartsrv 7.53 PL 1010 which not meet the prerequisites.

Step-by-step guide

Stop HANA database and the service

sapcontrol -nr <NR> -function Stop
sapcontrol -nr <NR> -function StopService

with user root make a new registration of sapstartsrv (you can find these Information in file /usr/sap/sapservices)

export LD_LIBRARY_PATH=/usr/sap/<SID>/HDB<NR>/exe; /usr/sap/<SID>/HDB<NR>/exe/sapstartsrv pf=/usr/sap/<SID>/SYS/profile/<SID>_HDB<NR>_<hostname> -reg

start new systemd Service

systemctl start SAP<SID>_<NR>.service

Start HANA Database (with <sid>adm)
```
HDB start
```

As a result, there is a modified entrie for the HANA instance in sapservices-File:

before:

LD_LIBRARY_PATH=/usr/sap/<SID>/HDB<NR>/exe:$LD_LIBRARY_PATH;export LD_LIBRARY_PATH;/usr/sap/<SID>/HDB<NR>/exe/sapstartsrv pf=/usr/sap/<SID>/SYS/profile/<SID>_HDB<NR>_<HOSTNAME> -D -u <sid>adm

after:

systemctl --no-ask-password start SAP<SID>_<NR> # sapstartsrv pf=/usr/sap/<SID>/SYS/profile/<SID>_HDB<NR>_<hostname>

There is also a new systemd service which is running in SAP.slice:

systemctl cat SAP<SID>_<NR>.service

systemd%20service

Now when you trigger a reboot of the linux system, firstly it will ordinary shutdown the HANA database before rebooting the system. When analyzing the traces after such a reboot of the system, I can se an ordinary shutdown of the HANA database (this I couldn’t find before):

[4187]{-1}[-1/-1] 2023-01-04 10:26:35.677882 i Basis HDBConsListener.cpp(00186) : Stopping HDBCons listener
[4270]{-1}[-1/-1] 2023-01-04 10:26:35.677898 i Basis HDBConsListener.cpp(00168) : HDBCons listener stopped
[4187]{-1}[-1/-1] 2023-01-04 10:26:35.685287 i Service_Shutdown TrexService.cpp(00725) : System down

Conclusion / Disclaimer

This procedure was only testing with a small HANA database. There were no other SAP instances on this server. When there is also a SAP instance on the server you have to consider the dependencies while stopping/starting. When you are a Linux-Expert (I’m not;-)) you can handle this dependencies within systemd with custom-settings.conf file.

Before use this, test it on a sandbox or development system. You have to be aware of the timeout (TimeoutStopSec) in the systemd service. If the ordinary shutdown process takes longer than the default timeout (360s) you have to adjust it otherwise the reboot will take place before the ordinary shutdown of the HANA database.