I had been using FreeBSD as my main web server OS since 2012 and I liked it so much that I even contributed money and code to it. However, since the FreeBSD guys (and gals) decided to install anti-tech feminism, I have been considering to move away from it for quite some time now.

As my growing needs require stronger hardware, it was finally time to rent a new server. I do not intend to run FreeBSD on it. Although the most obvious choice would be OpenBSD (I run it on another server and it works just fine), I plan to have a couple of databases running on the new machine, and database throughput has never been one of OpenBSD's strong points. This is my chance to give illumos another try. As neither WiFi nor desktop environments are relevant on a no-X11 server, the server-focused OmniOS seemed to fit my needs.

My current (to be phased out) setup on FreeBSD is:

• apache24 with SSL support, running five websites on six domains (both HTTP and HTTPS)
• a (somewhat large) Tiny Tiny RSS installation from git, updated via cronjob
• sbcl running a daily cronjob of my Web-to-RSS parser
• an FTP server where I share stuff with friends
• an IRC bouncer
• MariaDB and PostgreSQL for some of the hosted services

I would not consider anything of that too esoteric for a modern operating system. Since I was not really using anything mod_rewrite-related, I was perfectly ready to replace apache24 by nginx, remembering that the prepackaged apache24 on FreeBSD did not support HTTPS out of the box and I had ended up installing it from the ports. That is the only change in my setup which I am actively planning.

So here's what I noticed.

Preparations:

Before deciding on an operating system (I ruled out DragonFly BSD early - I could not install sbcl there, neither from pkg nor from DPorts), I spend quite some time reading about its motivations, heritage and development. I already feel thirty years older by just having read about general Solaris concepts. I was too young when Solaris blossomed, and two decades of (quite some) Linux and (quite some) BSD have spoiled my understanding of Unix. To say the least, I never had a real UNIX on any of my machines and no system except Windows which was that different from FreeBSD and/or Debian. Never settle, I guess.

The installation:

The most important reason to pick OmniOS over other illumos distributions was that OmniOS officially supports being installed on a system which is controlled over ssh. OpenIndiana seems to require "clicking" on stuff. I don't want that.

Most of OmniOS's "Kayak" installation reminds me a lot of OpenBSD's: I am asked a number of questions and I would probably get a working operating system by just accepting the defaults. Unlike OpenBSD, OmniOS wants numbers instead of "y" or "n" and sometimes asks for confirmation of what I have just entered, but that is not too bad since I don't think I will have to run the installer ever again. Updating OmniOS seems to be done without running this QA thing from the start. I consider this to be a significant advantage over OpenBSD, to be honest. I had to change the networking defaults on the post-installation screen though: As the default setting is "static" instead of "DHCP", I could not reach my server from the outside. Changing that fixed it. (So, yes, I needed to install OmniOS twice. On the plus side, I felt smarter on the second try.) I also like that the other options (adding users, enabling sshd etc.) are just a "Enabled"/"Disabled" switch - no giant blue fullscreen "dialogs" required.

OmniOS informs me that my new user account can be added as a primary admin and to sudo. So sudo is optional on Solaris - good to know. (According to the web, there is pfexec instead. I really prefer sudo for the time being.)

First impressions:

Hooray, a BSD boot loader! Finally an operating system without grub - I made my experiences with that and I don't want to repeat them too often.

It is weird that the installer won't accept "mydomain.org" as a hostname but sendmail complains that "mydomain" is not a valid hostname right from the start, OmniOS sent me into Maintenance Mode to fix that. A good start, right? So the first completely new thing I had to find out on my new shiny toy was how to change the hostname. There is no /etc/rc.conf in it and hostname mydomain.org was only valid for one login session. I found out that the hostname has to be changed in three different files under /etc on Solaris - the third one did not even exist for me. Changing the other two files seems to have solved this problem for me.

Does the network still work?

Uh... I think so, at least.

On to the packages!

Package woes:

OmniOS's pkg is quite different from FreeBSD's: It has a more verbose help screen and its packages have subpackages and something like flavours built-in from what I understand. A good number of core packages comes with OmniOS and I had updates (pkg update - no extra pkg upgrade required) to some of them even after a fresh installation.

The number of officially provided packages is not really overwhelming though: No Python 3, no sbcl, a lot of other things are missing as well. I could probably find some of them in third-party repositories - or I could go the extra mile and install Joyent's pkgsrc on OmniOS which combines the battle-tested pkgsrc package management with commercial-grade support by a company which actually needs it to be good enough for enterprises. Guess what I did?

The instructions from the website gave me a working pkgin, except that I added the PATH and MANPATH exports to my .profile file as well - I wanted them to be available automatically, of course. I managed to get all relevant applications - including nginx and php72-fpm - installed quickly.

Random findings:

~ I was wondering how many resources my (mostly idle) new web server was using - I always thought Solaris was rather fat, but it still felt fast to me.

Ah, right - we're in Unixland and we need to think outside of the box. This table was really helpful: although a number of things are different between OmniOS and SmartOS, I found out that the *stat tools do what top does. I could probably just install top from one of the package managers, but I failed to find a reason to do so. I had 99% idle CPU and RAM - that's all I wanted to know.

~ Trying to set up twtxt informed me that Python 3.6 (from pkgin) expects LANG and LC_ALL to be set. Weird - did FreeBSD do that for me? It's been a while ... at least that was easy to fix.

~ SMF - Solaris's version of init - confuses me. It has "levels" similar to Gentoo's OpenRC, but it mostly shuts up during the boot process. Stuff from pkgsrc, e.g. nginx, comes with a description how to set up the particular service, but I should probably read more about it. What if, one day, I install a package which is not made ready for OmniOS? I'll have to find out how to write SMF scripts. But that should not be my highest priority.

~ The OmniOS documentation talks a lot about "zones" which, if I understand that correctly, mostly equal FreeBSD's "jails". This could be my chance to try to respect a better separation between my various services - if my lazyness won't take over again. (It probably will.)

~ OmniOS's default shell - rather un-unixy - seems to be the bash. Update: I was informed about a mistake here: the default shell is ksh93, there are bogus .bashrc files lying around though.

~ Somewhere in between, my sshd had a hiccup or, at least, logging into it took longer than usual. If that happens again, I should investigate.

Conclusion:

By the time of me writing this, I have a basic web server with an awesome performance and a lot of applications ready to be configured only one click away. The more I play with it, the more I have the feeling that I have missed a lot while wasting my time with FreeBSD. For a system that is said to be "dying", OmniOS feels well-thought and, when equipped with a reasonable package management, comes with everything I need to reproduce my FreeBSD setup without losing functionality.

I'm looking forward to what will happen with it.

I gave up on FreeBSD a long time ago, for the same programs installed, OpenBSD was half that of a FreeBSD installation!

Sounds like you've found what works for you, & that is all that matters, thanks for the insight.

2 members found this post helpful.

... sorry didn't get to the end.
I've tried a couple of times to get into non-linux *nix, but the frustration of not knowing where anything is just got the better of me.

This is to remind myself to come back and read the entire initial post sometime. Thanks anyway for taking the time to let us know your thoughts.

2 members found this post helpful.

FreeBSD is incredibly user-friendly, really. But at a certain point, that does not matter anymore.

YesItsMe,

Thanks for the post. Been running Solaris since the early 90's and we have just decided to stop paying for support, which means no updates or patches. Was going to migrate everybody over to CentOS but I have been getting a major push back from my users.

Since our group has a very large code base of scientific applications this seems like an ideal solution. I'm going to start playing around with in a VM first, can't wait to get started.

Thanks again!

On desktops, OpenIndiana might be your first choice.

I'm glad that I could inspire you!

Thank you for the detailed post, quite inspiring.

Our company decided to do the same, more or less for same reasons.

It is interesting you mention sbcl, as we are into common lisp as well.

Some of our engineers have been running OpenIndiana on their laptops (ThinkPads) with success. Some of our workstations unfortunately do not seem to run well, or allow OpenIndiana to be installed (for example, X399 motherboards with AMD threadripper).

We are still testing OmniOS on servers.

The documentation, the "feel" of the OS, some of the technologies (like Zones) are leaps and bounds superior to FreeBSD, in same way FreeBSD was superior to others.

Please continue writing on illumos/OmniOS, it is inspiring and helpful for all like us wishing to use something better. Thanks again.

By the way, a (almost) recent article on installing X11 and a Desktop Environment on OmniOS

1 members found this post helpful.

I already like your company.

Sadly, nothing new to tell though.

1 members found this post helpful.

I was surprised to see this thread linked on /r/illumos:

This reply was quite inspiring, and also similar to the one at our company.

1 members found this post helpful.

Aw, fame!

Not quite un-interesting:
OmniOS forces me to understand how ZFS works by creating a new "/" container on each upgrade. That means: If you don't perform a

at least once, you're screwed. The data's not "gone" then - but you won't have fun restoring it.

FreeBSD does not do that. This seems rather dirty when compared to the clean approach.
So: Another + for OmniOS on this one.

1 members found this post helpful.

OmniOS r151026 has been released, it has an (optional) more colorful installer and upgrading it took three minutes and a reboot and everything still works.

Interesting, thank you.

Could you clarify a bit? You mean that one needs to make a dataset mountpoint explicit at creation time?

Is that somewhere in the wiki? If not, perhaps it should be added.

Thanks for the news.

I read there is a better AMD support. I am looking forward to test it on one of ours X399 motherboards with AMD threadripper.