LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Linux From Scratch
User Name
Password
Linux From Scratch This Forum is for the discussion of LFS.
LFS is a project that provides you with the steps necessary to build your own custom Linux system.

Notices

Reply
 
Search this Thread
Old 11-19-2012, 04:14 AM   #1
McZ
Member
 
Registered: Aug 2012
Posts: 42

Rep: Reputation: Disabled
Test suite for finished LFS?


Hi

What I have always been wondering is, How can I test my Linux system?

How is a distribution tested, and how can I verify all my installed packages is working correctly?

Can I only depend on the tests in each package (make check etc.) or is there some way to test my "brand new" LFS installation to see if something went wrong?

EDIT: For example, how can I test my locales is working as intended?
 
Old 11-19-2012, 01:21 PM   #2
NyteOwl
Member
 
Registered: Aug 2008
Location: Nova Scotia, Canada
Distribution: Slackware, OpenBSD, others periodically
Posts: 512

Rep: Reputation: 138Reputation: 138
For starters you'd get better results asking in the Linux section rather than BSD.

As for your example: is the date in the right format? proper currency characters? accented characters showing up properly? time zone correct?

As LFS is a customized "distribution" only you know exactly how you organized it to work. In general you know when somethign misbehaves, and if not other users will let you know if you distribute your version. Then you do debugging and fix the problems.
 
Old 11-20-2012, 04:47 PM   #3
stoat
Member
 
Registered: May 2007
Distribution: LFS
Posts: 464

Rep: Reputation: 133Reputation: 133
Quote:
Originally Posted by McZ

How is a distribution tested, and how can I verify all my installed packages is working correctly?
I don't know the answer to that. Sorry. It would be neat if there was some kind of script or something that would test things and produce a report of some kind. Maybe there is. That goes on my Christmas list, I guess.

What I do is just start using it. Boot, log in, and start doing stuff. Establish and test your network connection (a simple ping of an Internet URL for now). Run tests with a bunch of the utilities from core-utils (cat, cp, df, echo, ls, md5sum, mkdir, rm, rmdir, sort, uname, and so on) and util-linux (dmesg, fdisk, mount, whereis, and so on). Test your vi editor. Make sure everything printed to console looks normal. Look at dmesg for errors that need work. Look at /var/log/boot.log for init stuff that needs tweaking.

I really think that's about it. There isn't much else a base LFS system can do. I guess the acid test is to push on to BLFS. Make and test a tarball backup. Then install GPM (mouse daemon) and Lynx (a text browser). Test navigating the Internet with Lynx, copying text with your mouse, and pasting it into another console. Try downloading a tarball with Lynx. Some people install wget now to download tarballs. Anyway, that is the routine many of us here use while building BLFS in a new LFS system.

If you can do all of that stuff, then it's good to go (IMO).
 
1 members found this post helpful.
Old 11-22-2012, 12:59 AM   #4
McZ
Member
 
Registered: Aug 2012
Posts: 42

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by stoat View Post
I don't know the answer to that. Sorry. It would be neat if there was some kind of script or something that would test things and produce a report of some kind. Maybe there is. That goes on my Christmas list, I guess.

What I do is just start using it. Boot, log in, and start doing stuff. Establish and test your network connection (a simple ping of an Internet URL for now). Run tests with a bunch of the utilities from core-utils (cat, cp, df, echo, ls, md5sum, mkdir, rm, rmdir, sort, uname, and so on) and util-linux (dmesg, fdisk, mount, whereis, and so on). Test your vi editor. Make sure everything printed to console looks normal. Look at dmesg for errors that need work. Look at /var/log/boot.log for init stuff that needs tweaking.

I really think that's about it. There isn't much else a base LFS system can do. I guess the acid test is to push on to BLFS. Make and test a tarball backup. Then install GPM (mouse daemon) and Lynx (a text browser). Test navigating the Internet with Lynx, copying text with your mouse, and pasting it into another console. Try downloading a tarball with Lynx. Some people install wget now to download tarballs. Anyway, that is the routine many of us here use while building BLFS in a new LFS system.

If you can do all of that stuff, then it's good to go (IMO).
Well I already have X and XFCE up and running, still are some tweaks to be done. But what I want is some kind of scripts that will test the LFS system before I continue on building BLFS.

Well now building LFS is as simple as prep+mount a partition to /mnt/lfs and run 'make world' from my source dir. But it would be nice to have some script to run in chroot after all packages has been built to verify that all packages is installed correct and working.
 
Old 11-22-2012, 03:34 AM   #5
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
Quote:
Originally Posted by McZ View Post
Well now building LFS is as simple as prep+mount a partition to /mnt/lfs and run 'make world' from my source dir. But it would be nice to have some script to run in chroot after all packages has been built to verify that all packages is installed correct and working.
You mention using scripts to build LFS. Nothing wrong with that, but I do hope you build in some tests/escapes if something goes wrong along the way.

If you (automatically?) run all the make check/test commands in chapter 6 and they don't produce any unkown errors, I would trust the results. The newly build packages will be used once they are installed (have a look at the PATH setting), which "tests" them in a manner of speaking.

As already mentioned by stoat; Using your LFS system will also give you clues.

If you want to test specifics, say locale, you need to come up with it yourself, there's no out-of-the-box test. You could test if the ls command lists the content as set by LC_COLLATE:
Code:
# use en_US.utf8
LC_COLLATE=en_US.utf8 ls -la

# use POSIX
LC_COLLATE=POSIX ls -la
The output should be sorted differently.
 
1 members found this post helpful.
Old 11-23-2012, 01:01 AM   #6
McZ
Member
 
Registered: Aug 2012
Posts: 42

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by druuna View Post
You mention using scripts to build LFS. Nothing wrong with that, but I do hope you build in some tests/escapes if something goes wrong along the way.

If you (automatically?) run all the make check/test commands in chapter 6 and they don't produce any unkown errors, I would trust the results. The newly build packages will be used once they are installed (have a look at the PATH setting), which "tests" them in a manner of speaking.
Yes, my scripts will stop on any error and they are running the tests (make check stuff) from the book.

Quote:
Originally Posted by druuna View Post
As already mentioned by stoat; Using your LFS system will also give you clues.

If you want to test specifics, say locale, you need to come up with it yourself, there's no out-of-the-box test. You could test if the ls command lists the content as set by LC_COLLATE:
Code:
# use en_US.utf8
LC_COLLATE=en_US.utf8 ls -la

# use POSIX
LC_COLLATE=POSIX ls -la
The output should be sorted differently.
When I'm using my LFS system and discover an error after I built Xorg and almost done with XFCE, I have to rebuild all of it. That is why I'm looking for some kind of test suite. Now I have extended my LFS core scripts so I have a FreeBSD ports like system for BLFS packages which I call Builds. And they includes meta packages (just a list of dependencies). So Rebuilding Xorg etc. is easy (cd /usr/builds/xorg && make install) just time consuming.

How is the "big guys" testing the existing distros, slackware, arch etc. Do they all rely on users reporting error when using the distro?
 
Old 11-23-2012, 01:56 AM   #7
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
Quote:
Originally Posted by McZ View Post
When I'm using my LFS system and discover an error after I built Xorg and almost done with XFCE, I have to rebuild all of it.
At this point you need to define what you mean by error. Any real errors would have been caught/shown during the build process.

Depending on the "error" you might not need to rebuild all of Xorg/Xfce/xyz. Fix the problem where it originates (not necessarily where it shows up) and restart from there. You do need scripts that can handle this scenario. But that's a (possible) scripting challenge.

Quote:
How is the "big guys" testing the existing distros, slackware, arch etc. Do they all rely on users reporting error when using the distro?
Generally speaking: Most of the work is done by the developers/testers and most often a battery of (automated) tests are performed along the way. Testing in the professional world is serious business and goes far beyond testing a single package (WIKI - Software Testing).

Although the tests done are comprehensive, it cannot cover all the possible situations that exist in the wild. This is where users come in. They start working with the new (or RC) version and if they run into irregularities they report it.
 
Old 11-23-2012, 03:48 AM   #8
McZ
Member
 
Registered: Aug 2012
Posts: 42

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by druuna View Post
At this point you need to define what you mean by error. Any real errors would have been caught/shown during the build process.

Depending on the "error" you might not need to rebuild all of Xorg/Xfce/xyz. Fix the problem where it originates (not necessarily where it shows up) and restart from there. You do need scripts that can handle this scenario. But that's a (possible) scripting challenge.
My Builds is a Makefile, which includes helper Makefiles much like FreeBSD ports. And Makefiles stop executing when a command return code is > 0.

A simple Build would look like this:
Code:
PKG_NAME:=mc
PKG_VERSION:=4.8.3
PKG_GROUP:=apps
PKG_SRC:=$(PKG_NAME)-$(PKG_VERSION).tar.xz
PKG_URL:=http://www.midnight-commander.org/downloads

# PKG_DEP:=""

include ../../builds.mk

pkg_build:
   cd $(PKG_DIR) && \
   patch -Np1 -i $(PKG_REPO)/mc-4.8.3-missing-1.patch && \
   ./configure --prefix=/usr/local --sysconfdir=/usr/local/etc --with-screen=ncurses && \
   make

pkg_check:
   cd $(PKG_DIR) && \
   make check

pkg_install:
   cd $(PKG_DIR) && \
   make DESTDIR=$(DESTDIR) install
to build mc I go to /usr/builds/apps/mc and run make install where the install rule is defined in builds.mk. This will download the source, if it doesn't exist in my source cache. Then it will call pkg_build rule from the Builds own Makefile (cd $(PKG_REPO) && make pkg_build) and if something goes wrong it will break on the return code.

And yes, I don't have to rebuild LFS core if something goes wrong in Xorg. Unless I track the error down and can see it's origin is in LFS core. And I do rebuild the entire Xorg if one package fails, to make sure my scripts works all the way.

Quote:
Originally Posted by druuna View Post
Generally speaking: Most of the work is done by the developers/testers and most often a battery of (automated) tests are performed along the way. Testing in the professional world is serious business and goes far beyond testing a single package (WIKI - Software Testing).

Although the tests done are comprehensive, it cannot cover all the possible situations that exist in the wild. This is where users come in. They start working with the new (or RC) version and if they run into irregularities they report it.
Thanks for the link, I will read about it.

I mostly asked if there exists any kind of test suite because Linux+GNU is open source stuff then a test suite for the OS could exist.
 
  


Reply

Tags
lfs, test suite


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
finished lfs 6.3.... @d4M Linux From Scratch 9 02-10-2009 09:07 AM
Finished on LFS But how to continue in BLFS MajesticX Linux From Scratch 5 09-17-2008 07:12 AM
Errors when running Glibc-2.3.6 test suite (reference: LFS 6.2 chapter 6.9) PingFloyd Linux From Scratch 2 02-02-2007 09:43 PM
LFS 6-1: Glibc Test Suite Fails at end with 2 errors in Chapter 6 TGWDNGHN Linux From Scratch 1 07-26-2005 12:23 PM
Finished LFS - it's more than 1Gb ! Bert Linux From Scratch 4 12-06-2002 04:15 PM


All times are GMT -5. The time now is 10:22 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration