Drovorub’s Ability to Conceal C2 Traffic And Its Implications For Docker Containers

As you may have heard already, the National Security Agency (NSA) and the Federal Bureau of Investigation (FBI) released a joint Cybersecurity Advisory about previously undisclosed Russian malware called Drovorub.

According to the report, the malware is designed for Linux systems as part of its cyber espionage operations.

Drovorub is a Linux malware toolset that consists of an implant coupled with a kernel module rootkit, a file transfer and port forwarding tool, and a Command and Control (C2) server.

The name Drovorub originates from the Russian language.

It is a complex word that consists of 2 roots (not the full words): “drov” and “rub”. The “o” in between is used to join both roots together. The root “drov” forms a noun “drova”, which translates to “firewood”, or “wood”. The root “rub” /ˈruːb/ forms a verb “rubit”, which translates to “to fell”, or “to chop”. Hence, the original meaning of this word is indeed a “woodcutter”.

What the report omits, however, is that apart from the classic interpretation, there is also slang. In the Russian computer slang, the word “drova” is widely used to denote “drivers”. The word “rubit” also has other meanings in Russian. It may mean to kill, to disable, to switch off. In the Russian slang, “rubit” also means to understand something very well, to be professional in a specific field. It resonates with the English word “sharp” – to be able to cut through the problem.

Hence, we have 3 possible interpretations of ‘Drovorub‘:

• someone who chops wood – “дроворуб”
• someone who disables other kernel-mode drivers – “тот, кто отрубает / рубит драйвера”
• someone who understands kernel-mode drivers very well – “тот, кто (хорошо) рубит в драйверах”

Given that Drovorub does not disable other drivers, the last interpretation could be the intended one. In that case, “Drovorub” could be a code name of the project or even someone’s nickname.

Let’s put aside the intricacies of the Russian translations and get a closer look into the report.

DISCLAIMER

Before we dive into some of the Drovorub analysis aspects, we need to make clear that neither FBI nor NSA has shared any hashes or any samples of Drovorub. Without the samples, it’s impossible to conduct a full reverse engineering analysis of the malware.

Netfilter Hiding

According to the report, the Drovorub-kernel module registers a Netfilter hook. A network packet filter with a Netfilter hook (NF_INET_LOCAL_IN and NF_INET_LOCAL_OUT) is a common malware technique. It allows a backdoor to watch passively for certain magic packets or series of packets, to extract C2 traffic.

What is interesting though, is that the driver also hooks the kernel’s nf_register_hook() function. The hook handler will register the original Netfilter hook, then un-register it, then re-register the kernel’s own Netfilter hook.

According to the nf_register_hook() function in the Netfilter’s source, if two hooks have the same protocol family (e.g., PF_INET), and the same hook identifier (e.g., NF_IP_INPUT), the hook execution sequence is determined by priority.

The hook list enumerator breaks at the position of an existing hook with a priority number elem->priority higher than the new hook’s priority number reg->priority:

int nf_register_hook(struct nf_hook_ops *reg)
{
    struct nf_hook_ops *elem;
    int err;
    err = mutex_lock_interruptible(&nf_hook_mutex);
    if (err < 0)
        return err;
    list_for_each_entry(elem, &nf_hooks[reg->pf][reg->hooknum], list) {
        if (reg->priority < elem->priority)
            break;
    }
    list_add_rcu(&reg->list, elem->list.prev);
    mutex_unlock(&nf_hook_mutex);
    ...
    return 0;
}

In that case, the new hook is inserted into the list, so that the higher-priority hook’s PREVIOUS link would point into the newly inserted hook.

What happens if the new hook’s priority is also the same, such as NF_IP_PRI_FIRST – the maximum hook priority? In that case, the break condition will not be met, the list iterator list_for_each_entry will slide past the existing hook, and the new hook will be inserted after it as if the new hook’s priority was higher.

By re-inserting its Netfilter hook in the hook handler of the nf_register_hook() function, the driver makes sure the Drovorub’s Netfilter hook will beat any other registered hook at the same hook number and with the same (maximum) priority.

If the intercepted TCP packet does not belong to the hidden TCP connection, or if it’s destined to or originates from another process, hidden by Drovorub’s kernel-mode driver, the hook will return 5 (NF_STOP). Doing so will prevent other hooks from being called to process the same packet.

Security Implications For Docker Containers

Given that Drovorub toolset targets Linux and contains a port forwarding tool to route network traffic to other hosts on the compromised network, it would not be entirely unreasonable to assume that this toolset was detected in a client’s cloud infrastructure.

According to Gartner’s prediction, in just two years, more than 75% of global organizations will be running cloud-native containerized applications in production, up from less than 30% today.

Would the Drovorub toolset survive, if the client’s cloud infrastructure was running containerized applications? Would that facilitate the attack or would it disrupt it? Would it make the breach stealthier?

To answer these questions, we have tested a different malicious toolset, CloudSnooper, reported earlier this year by Sophos.

Just like Drovorub, CloudSnooper’s kernel-mode driver also relies on a Netfilter hook (NF_INET_LOCAL_IN and NF_INET_LOCAL_OUT) to extract C2 traffic from the intercepted TCP packets.

As seen in the FBI/NSA report, the Volatility framework was used to carve the Drovorub kernel module out of the host, running CentOS. In our little lab experiment, let’s also use CentOS host.

To build a new Docker container image, let’s construct the following Dockerfile:

FROM scratch
ADD centos-7.4.1708-docker.tar.xz /
ADD rootkit.ko /
CMD [“/bin/bash”]

The new image, built from scratch, will have the CentOS 7.4 installed. The kernel-mode rootkit will be added to its root directory.

Let’s build an image from our Dockerfile, and call it ‘test’:

[root@localhost 1]# docker build . -t test
Sending build context to Docker daemon   43.6MB
Step 1/4 : FROM scratch
—>
Step 2/4 : ADD centos-7.4.1708-docker.tar.xz /
—> 0c3c322f2e28
Step 3/4 : ADD rootkit.ko /
—> 5aaa26212769
Step 4/4 : CMD [“/bin/bash”]
—> Running in 8e34940342a2
Removing intermediate container 8e34940342a2
—> 575e3875cdab
Successfully built 575e3875cdab
Successfully tagged test:latest

Next, let’s execute our image interactively (with pseudo-TTY and STDIN):

docker run -it test

The executed image will be waiting for our commands:

[root@8921e4c7d45e /]#

Next, let’s try to load the malicious kernel module:

[root@8921e4c7d45e /]# insmod rootkit.ko

The output of this command is:

insmod: ERROR: could not insert module rootkit.ko: Operation not permitted

The reason why it failed is that by default, Docker containers are ‘unprivileged’. Loading a kernel module from a docker container requires a special privilege that allows it doing so.

Let’s repeat our experiment. This time, let’s execute our image either in a fully privileged mode or by enabling only one capability – a capability to load and unload kernel modules (SYS_MODULE).

docker run -it –privileged test

or

docker run -it –cap-add SYS_MODULE test

Let’s load our driver again:

[root@547451b8bf87 /]# insmod rootkit.ko

This time, the command is executed silently. Running lsmod command allows us to enlist the driver and to prove it was loaded just fine.

A little magic here is to quit the docker container and then delete its image:

docker rmi -f test

Next, let’s execute lsmod again, only this time on the host. The output produced by lsmod will confirm the rootkit module is loaded on the host even after the container image is fully unloaded from memory and deleted!

Let’s see what ports are open on the host:

[root@localhost 1]# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1044/sshd

With the SSH server running on port 22, let’s send a C2 ‘ping’ command to the rootkit over port 22:

[root@localhost 1]# python client.py 127.0.0.1 22 8080
rrootkit-negotiation: hello

The ‘hello’ response from the rootkit proves it’s fully operational. The Netfilter hook detects a command concealed in a TCP packet transferred over port 22, even though the host runs SSH server on port 22.

How was it possible that a rootkit loaded from a docker container ended up loaded on the host?

The answer is simple: a docker container is not a virtual machine. Despite the namespace and ‘control groups’ isolation, it still relies on the same kernel as the host. Therefore, a kernel-mode rootkit loaded from inside a Docker container instantly compromises the host, thus allowing the attackers to compromise other containers that reside on the same host.

It is true that by default, a Docker container is ‘unprivileged’ and hence, may not load kernel-mode drivers. However, if a host is compromised, or if a trojanized container image detects the presence of the SYS_MODULE capability (as required by many legitimate Docker containers), loading a kernel-mode rootkit on a host from inside a container becomes a trivial task.

Detecting the SYS_MODULE capability (cap_sys_module) from inside the container:

[root@80402f9c2e4c /]# capsh –print
Current: = cap_chown, … cap_sys_module, …

 

Conclusion

This post is drawing a parallel between the recently reported Drovorub rootkit and CloudSnooper, a rootkit reported earlier this year.

Allegedly built by different teams, both of these Linux rootkits have one mechanism in common: a Netfilter hook (NF_INET_LOCAL_IN and NF_INET_LOCAL_OUT) and a toolset that enables tunneling of the traffic to other hosts within the same compromised cloud infrastructure.

We are still hunting for the hashes and samples of Drovorub.

Unfortunately, the YARA rules published by FBI/NSA cause False Positives.

For example, the “Rule to detect Drovorub-server, Drovorub-agent, and Drovorub-client binaries based on unique strings and strings indicating statically linked libraries” enlists the following strings:

• “Poco”
• “Json”
• “OpenSSL”
• “clientid”
• “—–BEGIN”
• “—–END”
• “tunnel”

The string “Poco” comes from the POCO C++ Libraries that are used for over 15 years. It is w-a-a-a-a-y too generic, even in combination with other generic strings.

As a result, all these strings, along with the ELF header and a file size between 1MB and 10MB, produce a false hit on legitimate ARM libraries, such as a library used for GPS navigation on Android devices:

• f058ebb581f22882290b27725df94bb302b89504
• 56c36bfd4bbb1e3084e8e87657f02dbc4ba87755

Nevertheless, based on the information available today, our interest is naturally drawn to the security implications of these Linux rootkits for the Docker containers.

Regardless of what security mechanisms may have been compromised, Docker containers contribute an additional attack surface, another opportunity for the attackers to compromise the hosts and other containers within the same organization.

The scenario outlined in this post is purely hypothetical. There is no evidence that supports that Drovorub may have affected any containers. However, an increase in volume and sophistication of attacks against Linux-based cloud-native production environments, coupled with the increased proliferation of containers, suggests that such a scenario may, in fact, be plausible.