Security-Enhanced Linux is a Linux Security Model (LSM) that allows defining security policies to implement mandatory access controls (MAC), providing a very granular layer to strengthen the OS against attacks.

Despite the obvious benefits of using SELinux, it has been historically advised not to use it on Enforcing mode (we will see what this is later on) for servers hosting SAP HANA because of some isolated customer cases where the performance of the DB seemed to be affected by that configuration. SAP HANA is very resource demanding and it spawns a high number of threads during its normal operation so adding the policy checks for all of them was deemed by SAP to be the cause of those isolated performance issues and they did not consider carrying out further tests.

Let’s look in more detail into SELinux.

The underlying idea is to regulate or constrain the possible actions of a subject (usually a process) on a target (files, memory, I/O devices, network resources, etc.). The policies contain rules that are evaluated every time a subject tries to access a target and is therefore allowed or not. Apart from MAC, SELinux is also based on and implements role-based access control (RBAC).

In SELinux, processes run in domains so they are separated from each other. Each user (process) is assigned a username, a role, and a domain, and each target is labeled with a name, role, and a type. In the policies it is specified the domains users need to belong in order to execute an action on a target, for example to bind to a listening port. If a process is compromised, the attacker will only have access to the resources in the domain that process has been assigned to, avoiding the danger of that attacker having access to system files and gaining control of the whole OS.

It is also a very useful feature for applications deployed in containers as it provides an additional layer of isolation between the containers and the hosts they run on.

SELinux has 3 operation modes:

• Enforcing. The policies are active and enforced.
• Permissive. The system uses the policies but it does not deny access to the targets, it just writes the approval and denial messages in the system logs (this mode is normally used to test policies before rolling them out to production).
• Disabled.

Using SELinux in Enforcing mode and with custom defined policies can override security issues that might be present in the kernel of the OS that is being used, and is highly recommended.

Red Hat Enterprise Linux has included SELinux since the release of version 4 in 2005 (it was already available in Fedora since version 2).

Last year the conversation around this topic between Red Hat’s and SAP’s engineering teams in SAP’s headquarters in Walldorf (Germany) was revived and a commitment was made to perform a thorough testing of SAP HANA on RHEL hosts with SELinux in Enforcing mode.

The good news is that Red Hat’s Engineering team in Walldorf has successfully run the “SAP HANA validation test suite” (the one used by SAP to determine if a system is able to run and process SAP HANA DB) without almost any impact to the DB performance (the decrease was only about 2%). The tests have been carried out on RHEL 8.2, RHEL 8.4, RHEL 8.6 and RHEL 9 so far and with minimal package installation, which is a security best practice to minimize the number of processes and applications that could be potential targets of attacks. This is the link to the published KB.

The OSS note with the recommendations for RHEL 8 on hosts running HANA has been already modified saying that ‘SELinux can allow SAP HANA to run within “unconfined mode”’ (meaning unconfined domain), which means that the SAP HANA processes will run with little or no interference from SELinux (but the system can be protected against remote attacks by having the network processes run in confined domains).

This is a very important result since as of now customers can run their SAP HANA DBs according to the highest security standards and with peace of mind knowing that their configuration is allowed by both SAP and Red Hat.

The SELinux tests are only one of the parts of a bigger security assessment for RHEL hosts running SAP HANA that will soon be published as a whitepaper.