Hi

What I have always been wondering is, How can I test my Linux system?

How is a distribution tested, and how can I verify all my installed packages is working correctly?

Can I only depend on the tests in each package (make check etc.) or is there some way to test my "brand new" LFS installation to see if something went wrong?

EDIT: For example, how can I test my locales is working as intended?

For starters you'd get better results asking in the Linux section rather than BSD.

As for your example: is the date in the right format? proper currency characters? accented characters showing up properly? time zone correct?

As LFS is a customized "distribution" only you know exactly how you organized it to work. In general you know when somethign misbehaves, and if not other users will let you know if you distribute your version. Then you do debugging and fix the problems.

I don't know the answer to that. Sorry. It would be neat if there was some kind of script or something that would test things and produce a report of some kind. Maybe there is. That goes on my Christmas list, I guess.

What I do is just start using it. Boot, log in, and start doing stuff. Establish and test your network connection (a simple ping of an Internet URL for now). Run tests with a bunch of the utilities from core-utils (cat, cp, df, echo, ls, md5sum, mkdir, rm, rmdir, sort, uname, and so on) and util-linux (dmesg, fdisk, mount, whereis, and so on). Test your vi editor. Make sure everything printed to console looks normal. Look at dmesg for errors that need work. Look at /var/log/boot.log for init stuff that needs tweaking.

I really think that's about it. There isn't much else a base LFS system can do. I guess the acid test is to push on to BLFS. Make and test a tarball backup. Then install GPM (mouse daemon) and Lynx (a text browser). Test navigating the Internet with Lynx, copying text with your mouse, and pasting it into another console. Try downloading a tarball with Lynx. Some people install wget now to download tarballs. Anyway, that is the routine many of us here use while building BLFS in a new LFS system.

If you can do all of that stuff, then it's good to go (IMO).

1 members found this post helpful.

Well I already have X and XFCE up and running, still are some tweaks to be done. But what I want is some kind of scripts that will test the LFS system before I continue on building BLFS.

Well now building LFS is as simple as prep+mount a partition to /mnt/lfs and run 'make world' from my source dir. But it would be nice to have some script to run in chroot after all packages has been built to verify that all packages is installed correct and working.

You mention using scripts to build LFS. Nothing wrong with that, but I do hope you build in some tests/escapes if something goes wrong along the way.

If you (automatically?) run all the make check/test commands in chapter 6 and they don't produce any unkown errors, I would trust the results. The newly build packages will be used once they are installed (have a look at the PATH setting), which "tests" them in a manner of speaking.

As already mentioned by stoat; Using your LFS system will also give you clues.

If you want to test specifics, say locale, you need to come up with it yourself, there's no out-of-the-box test. You could test if the ls command lists the content as set by LC_COLLATE: The output should be sorted differently.

1 members found this post helpful.

Yes, my scripts will stop on any error and they are running the tests (make check stuff) from the book.

When I'm using my LFS system and discover an error after I built Xorg and almost done with XFCE, I have to rebuild all of it. That is why I'm looking for some kind of test suite. Now I have extended my LFS core scripts so I have a FreeBSD ports like system for BLFS packages which I call Builds. And they includes meta packages (just a list of dependencies). So Rebuilding Xorg etc. is easy (cd /usr/builds/xorg && make install) just time consuming.

How is the "big guys" testing the existing distros, slackware, arch etc. Do they all rely on users reporting error when using the distro?

At this point you need to define what you mean by error. Any real errors would have been caught/shown during the build process.

Depending on the "error" you might not need to rebuild all of Xorg/Xfce/xyz. Fix the problem where it originates (not necessarily where it shows up) and restart from there. You do need scripts that can handle this scenario. But that's a (possible) scripting challenge.

Generally speaking: Most of the work is done by the developers/testers and most often a battery of (automated) tests are performed along the way. Testing in the professional world is serious business and goes far beyond testing a single package (WIKI - Software Testing).

Although the tests done are comprehensive, it cannot cover all the possible situations that exist in the wild. This is where users come in. They start working with the new (or RC) version and if they run into irregularities they report it.

My Builds is a Makefile, which includes helper Makefiles much like FreeBSD ports. And Makefiles stop executing when a command return code is > 0.

A simple Build would look like this:
to build mc I go to /usr/builds/apps/mc and run make install where the install rule is defined in builds.mk. This will download the source, if it doesn't exist in my source cache. Then it will call pkg_build rule from the Builds own Makefile (cd $(PKG_REPO) && make pkg_build) and if something goes wrong it will break on the return code.

And yes, I don't have to rebuild LFS core if something goes wrong in Xorg. Unless I track the error down and can see it's origin is in LFS core. And I do rebuild the entire Xorg if one package fails, to make sure my scripts works all the way.

Thanks for the link, I will read about it.

I mostly asked if there exists any kind of test suite because Linux+GNU is open source stuff then a test suite for the OS could exist.