LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Blogs > sag47
User Name
Password

Notices

Rate this Entry

sshd chroot jails based on hostname and SELinux

Posted 04-01-2013 at 10:46 AM by sag47
Updated 04-01-2013 at 10:51 AM by sag47

Why chroot jails based on host name?
If you don't care about the background info just jump to Setting up sshd chroot jail and SELinux.

So I have started configuring automated deployments through my organization using a continuous integration server called Jenkins. I'll post more on Jenkins regularly so that eventually you can have a series of posts which wrap up into a nice and secure howto for Jenkins. So here's why I want chroot jails based on host names.

I have a Jenkins master node running on RHEL 6 and several child nodes mixed with Windows and RHEL Linux for building a variety of cross platform applications. Different groups in my org use this tool so it's great to be so versatile. Jenkins is secured from the public a number of ways such as using LDAP, SSL, and firewall rules for IP blocks. However, Jenkins implementation of security gives you a false sense of security because anyone who actually has access to creating Jobs in Jenkins has the ability to do whatever they want, unhindered by the web interface. I'll explain.
  • On the master node, Jenkins is running as the jboss user which does not have root user access. The master node is what primarily runs the web interface.
  • On the Windows nodes (aka build agent), Jenkins is running as a domain account, call it domain-user-jenkins. In Windows specifically, domain-user-jenkins is required to have local Administrator rights in order to do the service bits it requires (according to the documentation). Eventually I'll investigate that further if it *really* needs admin rights but for now we'll take it at face value.

Why is this information important?
Because in Jenkins jobs you can run any action through a batch script, a shell script, a powershell script, a groovy script, even any executable if you set it up with svn. That means a user creating a Job in Jenkins is capable of doing anything the local build agent privileges has the ability to do. On the Windows nodes, where domain-user-jenkins has local administrator rights, that could be anything a local Administrator can do; which is anything. On the Master Node, where the user is restricted by the rights of the local jboss user it is possible for the user to modify web server configurations or even Jenkins core configurations and open up the security access.

How can we improve this model?

First and foremost, since the security model of Jenkins is very bad to begin with and gives you a false sense of security I recommend: be very careful who you give access to this system. While they may not be able to access many advanced features from the web interface a "normal" user who has "create jobs" permission on the master node can easily create a job that accesses and modifies any advanced feature of Jenkins on the command line using a simple bash script. So you put a lot of trust in users who get the ability to create build jobs. It's one of those security vs automation issues.

Jenkins is used for more than just building and compiling. It's used for deploying applications to both production and test servers. Therefore, it is important to think about how you approach deployments. This is important in case the Jenkins server itself is compromised and unparalleled access is given to an evil-doer across production if incorrectly configured.

Enough about Jenkins; the main focus of this is sshd chroot jails and SELinux.

Setting up sshd chroot jail and SELinux
Here's what I came up with. You should set up all of your deployments with a reversed access security model. You should give access to the Jenkins server as sftp-internal only with SELinux enabled inside of a ssh chroot jail. The user should only be able to read from the chroot jail and not write. This way Jenkins doesn't have access to servers at all and all of the servers have access to Jenkins in the scope of a read-only chroot jail.

Create ssh chroot
Create a chroot environment for ssh. Remember, the user on the system that will be locked in the chroot jail is called jboss.

Code:
#create a template jail which will be used by default if no system is specified
for x in /chroot/jail/home/jboss /chroot/jail/usr/libexec/openssh /chroot/jail/lib64;do
  mkdir -p $x
done
chmod -R 755 /chroot
chown jboss\: /chroot/jail/home/jboss
#use ldd to determine which libraries belong to
ldd /usr/libexec/openssh/sftp-server
#note the following libraries are used by sftp-server via ldd
for x in /usr/lib64/libcrypto.so.10 /lib64/libutil.so.1 /lib64/libz.so.1 /lib64/libnsl.so.1 /lib64/libcrypt.so.1 /lib64/libresolv.so.2 /lib64/libgssapi_krb5.so.2 /lib64/libkrb5.so.3 /lib64/libk5crypto.so.3 /lib64/libcom_err.so.2 /usr/lib64/libnss3.so /lib64/libc.so.6 /lib64/libdl.so.2 /lib64/libfreebl3.so /lib64/libkrb5support.so.0 /lib64/libkeyutils.so.1 /lib64/libpthread.so.0 /lib64/ld-linux-x86-64.so.2 /usr/lib64/libnssutil3.so /lib64/libplc4.so /lib64/libplds4.so /lib64/libnspr4.so /lib64/libselinux.so.1;do
  cp $x /chroot/jail/lib64/
done
cp /usr/libexec/openssh/sftp-server /chroot/jail/usr/libexec/openssh/
Set up some basic selinux rules for the chroot using the proper context.
Code:
#change selinux context
chcon -R -t chroot_user_t /chroot
#Do not allow ssh users to have rw access to chroot directory
setsebool -P ssh_chroot_rw_homedirs off
#disable selinux temporarily to test out the chroot; we'll handle that in a bit
setenforce 0
Modify /etc/ssh/sshd_config at the end so that the jboss user is chrooted by the jail automatically. I have a separate Match rule for the ChrootDirectory directive because once it is set it doesn't change. It must be done this way to allow different hosts to have different chroot directories.

Code:
Match User jboss
	PasswordAuthentication no
	AllowAgentForwarding no
	AllowTcpForwarding no
	GSSAPIAuthentication no
	PubkeyAuthentication yes
	PermitEmptyPasswords no
	RSAAuthentication no
	X11Forwarding no
	X11UseLocalHost no
#catchall
Match User jboss
	ChrootDirectory /chroot/jail
	ForceCommand internal-sftp
Restart sshd and test logging in as the jboss user using a public key from another system. I found the context of chroot_user_t by running the following command with SELinux set to permissive.

Code:
ps auZx | grep internal-sftp
SELinux and ssh chroot
Build a module so that local unconfined users have rw access to chroot directory. By default SELinux does not allow any user, not even root, to write or modify a chroot_user_t context type directory. This is why we have to build a module.

First you must create a policy violation alert with SELinux enforcing. Remember, we're trying to provoke a violation with all the different ways we might use the chroot jail for our deployments. That way we can write a module to allow that violation (so it will not longer be a violation).

Code:
mkdir /tmp/selinux_play
cd /tmp/selinux_play


#Set up your chroot jail for some violation tests
cd /chroot/jail/home/jboss
setenforce 0
touch env2.sh  env3.sh
mkdir -p ./dir/dir3
echo "test5" > env2.sh
echo "test6" > env3.sh
\rm env.sh
chown jboss\: *
setenforce 1

#now run tests with SELinux enforcing
#Log into Jenkins and try to run a job which attempts to write a file to the chroot directory.
#Now do some local commands on it
cd /chroot/jail/home/jboss
\echo "test" > env2.sh 
\touch env5.sh
\mv -f env3.sh env5.sh
\mv -f env3.sh /tmp/
\rm -f env3.sh 
ls
\mv -f dir dir2
\rmdir -f dir
\mkdir dir2
\rm -rf dir
\chmod 600 env2.sh
\chmod -f 600 env2.sh
\touch env3.sh

#do the same thing as the jboss user
su - jboss
cd /chroot/jail/home/jboss
\echo "test" > env2.sh 
\touch env5.sh
\mv -f env3.sh env5.sh
\mv -f env3.sh /tmp/
\rm -f env3.sh 
ls
\mv -f dir dir2
\rmdir -f dir
\mkdir dir2
\rm -rf dir
\chmod 600 env2.sh
\chmod -f 600 env2.sh
\touch env3.sh
Now let's create a module.
Code:
cd /tmp/selinux_play
audit2allow -M unconfined_user_rw_ssh_chroot -l -i /var/log/audit/audit.log
Here's the module that I came up with (along with a few manual edits).

source: unconfined_user_rw_ssh_chroot.te
Code:
#SELinux module by Sam Gleske
module unconfined_user_rw_ssh_chroot 1.1;

require {
	type unconfined_t;
	type fs_t;
	type locate_t;
	type chroot_user_t;
	type inetd_child_t;
	class dir { write rmdir rename getattr setattr remove_name create add_name };
	class file { rename create unlink setattr getattr };
	class filesystem associate;
}
#============= inetd_child_t ==============
allow inetd_child_t chroot_user_t:file setattr;

#============= locate_t ==============
allow locate_t chroot_user_t:dir getattr;

#============= chroot_user_t ==============
allow chroot_user_t fs_t:filesystem associate;

#============= unconfined_t ==============
allow unconfined_t chroot_user_t:dir { write rmdir setattr remove_name create add_name rename getattr };
allow unconfined_t chroot_user_t:file { create unlink setattr rename getattr };
If you need to edit and recompile the module here's how you can do it.
Code:
\rm -f unconfined_user_rw_ssh_chroot.mod unconfined_user_rw_ssh_chroot.pp
checkmodule -M -m -o unconfined_user_rw_ssh_chroot.mod unconfined_user_rw_ssh_chroot.te
semodule_package -o unconfined_user_rw_ssh_chroot.pp -m unconfined_user_rw_ssh_chroot.mod
Now install the module.
Code:
semodule -i unconfined_user_rw_ssh_chroot.pp
If you need to change and recompile the module you should remove it first.
Code:
semodule -l | grep chroot
semodule -r unconfined_user_rw_ssh_chroot
#recompile your changes and install again
Add a host to the chroot jail
Now that we've got the chroot jail and SELinux enabled and properly working. We want to start segregating hosts and chroot jails. That means your Jenkins job which is deploying should write any environment files or deployment files to the jboss home directory of the chroot jail for that host. Primarily because the jboss user coming from that host will only have access to the chroot jail with the sftp utility.

Let's set up the jail for myhost.com using our jail template.
Code:
cd /chroot
cp -r jail myhost.com
chcon -R -t chroot_user_t myhost.com
chown jboss\: myhost.com/home/jboss
Now edit /etc/ssh/sshd_config and add the host to use the new jail.
Code:
Match User jboss
	PasswordAuthentication no
	AllowAgentForwarding no
	AllowTcpForwarding no
	GSSAPIAuthentication no
	PubkeyAuthentication yes
	PermitEmptyPasswords no
	RSAAuthentication no
	X11Forwarding no
	X11UseLocalHost no
Match Host myhost.com
	ChrootDirectory /chroot/myhost.com
	ForceCommand internal-sftp
#catchall
Match User jboss
	ChrootDirectory /chroot/jail
	ForceCommand internal-sftp
The reverse call

Here's the last bit which I almost forgot to mention. I know you were probably wondering, "what is a reversed access security model?" Here's how I would define it since I pulled that phrase out of thin air:

Quote:
From the master node (Jenkins) set up your ssh keys. Copy your master node public key to the authorized_keys file of myhost.com. Modify the authorized keys file so that it executes a command rather than dropping into a shell when connected. That command will be a script which will sftp to the master node to gather all of the files from Jenkins.
So essentially the master node will only be able to execute a single command on the "deploy to" server and nothing else. And the "deploy to" server will only have access to the chroot directory on the master node. Here's a sample authorized_keys file.
Code:
command="/opt/deploy_something.sh" ssh-rsa KEY GIBERISH
Conclusion
Now whenever we ssh to the master node from the server myhost.com we get chrooted into a chroot directory. If we create more chroot directories for other hosts then we'll be able to keep a separate chroot jail for each host. This way hosts can't interfere with deployments to other hosts.

It helps to mitigate the amount of access a continuous integration server has to all of the production machines and visa versa. Hopefully I was concise enough for good understanding. For more information on the Match directive in sshd_config see the sshd_config man page.

Feel free to ask me any questions in the comments.
Views 3254 Comments 0
« Prev     Main     Next »
Total Comments 0

Comments

 

  



All times are GMT -5. The time now is 04:39 AM.

Main Menu
Advertisement

My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration