general question: container isolation-level
 

Hi altogether,

I´ve got a general question about Linux containers, in particlar with regard to LXD/LXC.

When running programms/processes within an LXC-container: to what extent are they isolated from the host system?

I mean I can run programs within a virtual machine (with a certain amount of overhead) or run programs within a sandbox (like firejail).
In those cases isolation from the host seems to be quite effective, at least to my knowledge.

How does the isolation-level of those compare to LXD/LXC?

Thanks in advance for any information.

Greetings
Rosika :scratch:

P.S.:
system: Linux/Lubuntu 16.04.3 LTS, 64 bit

I have no idea what do you mean by quite effective, but I think the isolation level is quite effective on LXD/LXC too.
Do you have any special use case, issue or example to discuss?

Virtualbox uses hardware-assisted virtualization, and creates a new environment that will boot a full operating system.
That is, it will boot the Linux kernel and then start the rest of the operating system.
For usability, Virtualbox offers a trick to share the graphics card with the host operating system. This is required
if we want hardware acceleration for graphics. If we enable 3D acceleration, we can even play 3D games in Virtualbox.
In addition, when you install the Guest Additions, you let more communication between the VM and your host.
This is good for usability, because you can, for example, copy/paste between the host and the VM.
However, it can lead to security vulnerabilities such as those described at https://www.techrepublic.com/article...in-virtualbox/
As always with security vulnerabilities, you install the updates and you are OK.

LXC 1.0, LXD and firejail use security features (namespaces, cgroups) of the Linux kernel
in order to run processes isolated from the rest of the system. The end result is different,
and you have the choice to select which one is better for you.

The big difference that LXD gives you, is that you create machine containers.
A machine container is like a virtual machine. You start the machine container
and it boots up, gets an IP address and is ready to use. The machine container shares the same
Linux kernel as the host, therefore it does not boot a separate Linux kernel.
Because of that, you can have ten times more machine containers on the same server than virtual machines.
And if you create machine containers with distros like Alpine, you can fit even more machine containers.

The Linux kernel security features provide the protection and isolation of what is going on inside the container.
The container can have no access at all at the host, even no network connectivity.
If you use the LXD installation defaults, then the containers will get a private IP address.
The containers by default are not accessible from the local network, and they cannot access directly
computers from your local (i.e. home) network. They get Internet connectivity though.

However, to make the best use of LXD, you may want to relax a bit the restrictions.
For example, if you follow the guide at https://blog.simos.info/how-to-make-...from-your-lan/
you can make some of the containers to appear as standalone hosts on your local network.
That is, if you computer has IP 192.168.1.10, then you can get three containers exposed (and accessible) to the local network
with IP addresses 192.168.1.11, 192.168.1.12, 192.168.1.13.
No need for separate physical computers, you can just use machine containers.
It is up to use to make sure that these special machine containers are secure (do not install bad software).

Also, you can run GUI apps in a machine container according to the guide at
https://blog.simos.info/how-to-run-g...buntu-desktop/
For example, you can run Steam inside the container with full graphics acceleration.
The way it works, is that the machine container has also full access to the X session.
This means that you should not run unknown GUI programs in that way.
On the other hand, it's up to you to use Xephyr instead, which provides isolation.

Overall, LXD is in the same group with LXC 1.0, firejail and others. Depending on your needs, it may be suitable to select LXD over the others.

In my experience, host processes are very well hidden from the container, container processes are not well hidden from the host. It somewhat depends upon what access to the process information worries you.

Hi pan64,

thanks for your answer.
My question was rather of theoretical nature. I more or less wnated to know whether a process running within a container is as secure as running it in a sandbox like firejail.
Secure in the sense of providing a "shield" against unwanted changes of the host.

Greetings.
Rosika

Hi wpeckham,

thank you for the answer.
I have no particular scenario in mind. I just wanted to see if there are any differences in general as compared to a sandbox (firejail).

Cheerio.
Rosika

Hi simosx,

thank you so much for your very detailed answer and the interesting links.

I also read about the principle differences between virtual machines (based on a hypervisor etc) and containers.
But the way you pointed out the important points is very informative and good to read.

I already installed lxd within a virtual machine (xubuntu 16.04.3 LTS, 32 bit) and "played around" with it in order to gain some experience before installing it on my host (Lubuntu 16.04.3 LTS).

In a whitepaper provided by canonical ("For CTO’s: the no-nonsense way to accelerate your business with containers") it says on page 10:
This is the bit that worries me.
The author talks about about the possibility of the kernel being crashed "due to a vulnerability within an application which itself is insecure".

I´m not quite sure what he/she means by that. What application? The one running within the container or the one running on the host-system? I mean is it as easy as that for the containerized application to get out of its container and do damage to the kernel?

Therefore my question.

Greetings.
Rosika

containers use the kernel of their host, they have no "own" kernels. So if there was an app which can harm the system most probably it will crash the host (running several hosts), not only the container itself.
From the other hand if there was a kernel vulnerability you need to upgrade all of your hosts, including VMs, therefore you need to reboot/restart all of them (including bare metals, VMs and containers).

Hi pan64,

thanks again for your reply.
That´s a bit of a shame. But that´s what I already feared.
So is it safe to say that running an "untrusted" application in a virtual machine or even a sandbox for that matter provides a higher degree of safety?

Rosika

Hi Rosika,

Finding a way to crash the Linux kernel is not so easy. We are talking here about the Linux kernel code sans device drivers.

The quote refers more to mission-critical applications, or if you are researching malware samples.
In that case, you would probably get hardware that supports both VT-x and VT-d,
and definitely use a VM.

With LXD, you can set up restrictions to the amount of computing power, memory and disk space for a container.
In that way, you can control a container that happens to be accessible from outsiders.
For example, have a look at https://linuxcontainers.org/lxd/try-it/
There, you can get a shell into an LXD container and using for free for 30 minutes.
In this LXD container, you get your own nested LXD container that you can use to launch your own containers.
You can give it a go and see how well it works.
The source code of this demo server is also available, https://insights.ubuntu.com/2016/04/...xd-in-lxd-812/

I like to use the analogy that "containers are a very clever illusion." :)

They bring together several different Linux kernel facilities to create an environment in which a process can believe that it knows what the filesystem looks like, and it believes that it knows what its user-id is, and it believes that it can become root when it wants to. And so on. But, none of this is actually the truth.

The container occupant's "rose-colored glasses" view of the world is actually mapped onto the actual environment of the Linux host, but the container occupants can't see that.

Processes running outside of the container environment can perceive the processes that are running in "container mode," but not the other way around.

So what this gives you is ... "good isolation, cheaply." You don't have the overhead of a virtual machine. You do have the isolation that you need. Although it isn't the same kind of isolation that a VM provides, it is very often good enough. (The overhead of a virtual-machine environment is quite noticeable when you don't have it.)

As a good for-instance, I often build (or, re-build) websites and such that run on VMWare. At one time I would build-out a bunch of small virtual machines. But I since learned to use containers, instead, with very capacious virtual machines. The exposure to VMWare's behaviors – which I typically have little or no control over – is sharply curtailed. Now, I have control over the environment. There are, of course, now "container hosts" which do not use a visible VMWare layer at all: they run honkin' big Linux boxes, and they run your containers directly on them.

Isolation techniques and advantages are an interesting field of study.

OpenVZ had superior isolation due to the superior OpenVZ kernel patches up to version 6: version 7 is a total departure and I am unsure how different it is internally. OpenVZ also allowed for superior management of some resources.

LXC based containers were less mature, with somewhat less isolation, but based entirely upon the kernel upstream. LXC has continued to mature, but I am unsure just how well it now isolates functions that could affect the running kernel security. LXC had the additional advantage that it could be used to isolate and entire distro environment, or just ONE process, service, or software environment.

EITHER of them was superior to sandbox and chroot techniques at one time. I have not followed the changes for the last couple of years, and never did follow the LXD improvements from Ubuntu/Canonical.

Summary of my noise (?) : how effective the isolation is for your purpose depends upon your purpose, version, settings, and situation. What is the need, what (and what versions/configuration of) applications are involved, and what is the perceived threat? With those answers and some research one could be ready for some REAL WORLD TESTING: which is the only way you get a definitive answer.


Sounds like fun!

Hi simosx,

tnx for your answer.
O.K., that´s not my intention.
I was just looking for the principle differences of various isolation techniques.
The links you provided are also quite helpful.

Greetings.
Rosika

Hi sundialsvcs,

O.K., I have to admit I didn´t know that. Yet it explains a lot.
The overhead-argument is the one that got me interested in containers in the first place. Not having to power up a VM in the first place in order to quickly launch a certain process is a huge advantage.

Tnx a lot for further clarification.

Greetings.
Rosika

Hi wpeckham,

Tnx for poining that out. Sounds promising.
O.K., I see that there´s no really simple answer, at least not an universally applicable one.
As it all seems to be depending on the situation taking a good look at various container-technologies and doing some real-world testing seems to be advisable.

Now I know a lot more than I used to.
Thanks again to you and all the other helpers.

Rosika