Container Deep Diving: Part 1
Containers serve 1 purpose. Selling kubernetes certificates! \s
But of course there is more to it then just hype.
Over the next few posts on my blog I would like to dig into the container world, see how we got to the current state and show what the current state actually is.
So what is the real purpose of containers?
Isolate different families of processes on 1 computer from each other, so that if a process is ever compromised it does not affect any other processes on the same machine.
In order to get to a good understanding of all the techniques involved I will split up this series into the following sections:
- Part 1: chroot history
- Part 2: containers and how they are made
- Part 3: the current container ecosystem
Enough intro. Lets kick it off with
Chroot (change root)
chroot is used to give a process and all the sub-processes it spawns a new root directory to work inside and it is implemented as a system call in the kernel.
What this means is that the process will not be able to access any files that are not insde the provided root directory or any descendants.
This technique is usually called a chroot jail.
It is a basic concept but pretty powerful.
Lets start a new
bash shell inside a
chroot jail directory to see
chroot in action.
This example will not use the system-call directly but rather use the GNU chroot binary that will ship with your modern linux distribution (Ubuntu 21.10 inside a VM in my case).
# Make sure you are logged in as the root user on your machine. echo host >> host cat host mkdir test-root mkdir test-root/bin cp /bin/bash test-root/bin/bash cp -a /usr /lib /lib64 test-root chroot test-root echo chroot >> chroot cat chroot cat ../host pwd exit cat test-root/chroot
As you can see there is no way to see the files on the real root once changing inside of the jail.
This is also the reason why
bash and all its dependencies are copied inside the jail folder before starting the process. They would not be there otherwise and without them there is no binary to execute.
Now while the jailed process can not read anything on the real root, the same is not true in the other direction. Any files inside the jail folder are accessible from the outside.
So far so good. But why are we not just using chroot jails everywhere then?
Lets look at some of the shortcommings of using
chroot to secure your system.
Lets check the network interfaces first.
As you can see the interfaces on the host and inside the
chroot are exactly the same.
Having access to the interface means that packets could potentially be sniffed out.
- start long running process on host
- get the process id
- chroot and kill the process
This should demonstrate the the root user inside the chroot still has access to all the processes running on the host.
They are not isolated and the root user inside the
chroot can manipulate all running processes on the machine at will.
This can be especially concerning if the user in the jail starts attaching itself to processes outside the jail to gain privileges on the host machine.
The best place to learn about the tools are usually their manuals.
With the small intro
man chroot.2 should be easier to decipher now.
DESCRIPTION chroot() changes the root directory of the calling process to that specified in path. This directory will be used for pathnames beginning with /. The root directory is inherited by all children of the calling process. Only a privileged process (Linux: one with the CAP_SYS_CHROOT capability in its user namespace) may call chroot(). This call changes an ingredient in the pathname resolution process and does nothing else. In particular, it is not intended to be used for any kind of security purpose, neither to fully sandbox a process nor to restrict filesystem system calls. In the past, chroot() has been used by daemons to restrict themselves prior to passing paths supplied by untrusted users to system calls such as open(2). However, if a folder is moved out of the chroot directory, an attacker can exploit that to get out of the chroot directory as well. The easiest way to do that is to chdir(2) to the to-be-moved directory, wait for it to be moved out, then open a path like ../../../etc/passwd. A slightly trickier variation also works under some circumstances if chdir(2) is not permitted. If a daemon allows a "chroot directory" to be specified, that usually means that if you want to prevent remote users from accessing files outside the chroot directory, you must ensure that folders are never moved out of it. This call does not change the current working directory, so that after the call '.' can be outside the tree rooted at '/'. In particular, the superuser can escape from a "chroot jail" by doing: mkdir foo; chroot foo; cd .. This call does not close open file descriptors, and such file descriptors may allow access to files outside the chroot tree. Linux 2021-03-22 CHROOT(2)
Chrooting is a basic building block for process isolation on a single machine but should not be used for the purpose these days.
The missing process and network isolation are the biggest concerns besides the jail escapes described in the man pages, which I was not able to execute on a Ubuntu 21.10 Server VM.
Now that the basic building block is covered Part 2 will jump into the
container stack and the mechanisms used to provide the isolation that
chroot is missing.
FreeBSD Jails will not be covered but deserve a shoutout. They are the first improvement on top of
chroot before containers became the hot new thing.
Please let me know in the comments if you’d like me to go into more detail on certain aspects in future posts.