LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Linux Answers > Programming
User Name
Password

Notices

By ejspeiro at 2012-05-17 18:08
By. Eduardo J. Sanchez P. (ejspeiro)

Hi guys, since my research work has lead me to the necessity of using some important scientific libraries, I have decided to contribute to the tutorials section with this humble collection of summaries about how to install these libraries. An important purpose shall be to explicitly depict the hierarchical dependence among the libraries, while at the same time, as it's been already stated, to explain the basic mechanics of the installations.

I will assume (i) that a first semester Computer Science student has been asked to install these to perform basic operations with them. I believe that the arising knowledge from using them is highly more valuable that the annoying research required to figure this out for the novice user.

I will also assume that (ii) all the libraries are meant to be installed in the ~/Libraries directory. For example, in my computer:

Code:
$ echo ~
/home/ejspeiro/
The libraries under consideration in this part (Part 2 out of probably 3) are:

1. LAPACK (Linear Algebra PACKage) and LAPACKE [ http://www.netlib.org/lapack/ ]. Quoting from LAPACK's presentation section in their website: LAPACK is written in Fortran 90 and provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. The associated matrix factorizations (LU, Cholesky, QR, SVD, Schur, generalized Schur) are also provided, as are related computations such as reordering of the Schur factorizations and estimating condition numbers. Dense and banded matrices are handled, but not general sparse matrices. In all areas, similar functionality is provided for real and complex matrices, in both single and double precision.
2. ATLAS (Automatically Tuned Linear Algebra Software) [ http://math-atlas.sourceforge.net/ ]. Quoting: The ATLAS (Automatically Tuned Linear Algebra Software) project is an ongoing research effort focusing on applying empirical techniques in order to provide portable performance. At present, it provides C and Fortran77 interfaces to a portably efficient BLAS implementation, as well as a few routines from LAPACK.

Let us begin!

1. LAPACK: To install and use this library, we require to download and decompress its source code from here: http://www.netlib.org/lapack/lapack-3.4.1.tgz . Part 1 of these tutorials explains how to do this.

Once we have access to the downloaded folder, we can begin the building process. LAPACK requires cmake as it is stated in its README file. Part 1 of these tutorials explains how to install any required application easily.

Before building we need to configure the installation in the inclusion file for the Makefile. Specifically, we have to modify make.inc. One can easily obtain a very good approximation for that file by utilizing the provided example:

Code:
~/Libraries/lapack-3.4.1$ cp make.inc.example make.inc
In my case, this is what I configured in it:

Code:
SHELL = /bin/bash
OPTS     = -O3
BLASLIB      = /home/ejspeiro/Libraries/BLAS/libblas.a
Notice that LAPACK depends only on BLAS, which was studied on Part 1 of these tutorials.

We can then proceed to build instructing

Code:
~/Libraries/lapack-3.4.1$ make all
If the building process is successful, we will then read the summary reporting the tests performed by LAPACK:

Code:
			-->   LAPACK TESTING SUMMARY  <--
		Processing LAPACK Testing output found in the TESTING direcory
SUMMARY             	nb test run 	numerical error   	other error  
================   	===========	=================	================  
REAL             	1077227		0	(0.000%)	0	(0.000%)	
DOUBLE PRECISION	1078039		0	(0.000%)	0	(0.000%)	
COMPLEX          	521722		66	(0.013%)	0	(0.000%)	
COMPLEX16         	551318		66	(0.012%)	0	(0.000%)	

--> ALL PRECISIONS	3228306		132	(0.004%)	0	(0.000%)
If we change directory to lapacke, we can then do:

Code:
~/Libraries/lapack-3.4.1/lapacke$ make
to build its native wrapper library so that we can use LAPACK in C.

:)

Up to this point it is all going great; however, IF ATLAS is meant to be installed, which it should given its nature, we may have to redo all of this again, as we will see next.

2. ATLAS: To install this library we proceed as customary: we download and extract the source code, located at: http://sourceforge.net/projects/math-atlas/files/Stable/3.8.4/atlas3.8.4...

Notice that this specific file can be extracted as follows:
Code:
~/Libraries$ bunzip2 -c atlas3.8.4.tar.bz2 | tar xfm -
Once we have extracted the files, we may want to rename ATLAS' folder to a less generic name:
Code:
~/Libraries$ mv ATLAS/ ATLAS_3.8.4-CORE
Excellent!!! Let us proceed with installing ATLAS, which is not as straightforward as the other libraries!!! Mainly because we have to configure ATLAS in a way that it can study our architecture to provided for optimized BLAS and LAPACK routines, thus tailoring the construction of these to our architecture, thus boosting performance! Similarly, we want to test this attained performance boost!!!

STEP 1: Turn off CPU throttling (frequency scaling) when installing ATLAS.

This is necessary since, according to ATLAS' authors, this feature makes all the timing taken in order to study the architecture, well: to be junk! XD In my computer, I first inquired regarding this being on or off by doing the following:

Code:
# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies 
1733000 1333000 1067000 800000
since those are in Hertz, then my processor is capable of 1.73 GHz, 1.33 GHz and 1.06 GHz and 0.80 GHz. Now that we know that our CPU(s) can handle scaling, let us now inquire about the modes available:

Code:
# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
conservative ondemand userspace powersave performance
Nice! How do we control this? Well, to be honest I didn't want to devote too much time on figuring it out for KDE, so I basically did this:

Code:
# apt-get install gnome-applets
# dpkg-reconfigure gnome-applets
# cpufreq-selector -g performance
Thanks to [1] for helping me with this explanation!

STEP 2: Install a BOGUS ATLAS to learn about the preferred applications for your computer.

In order to minimize compiler/flag interoperability problems between ATLAS and both of your BLAS and LAPACK libraries, we want to build LAPACK with the same flags you use to compile ATLAS's FORTRAN API routines with. Clearly, this looks like a a chicken and egg problem, since we would like to install LAPACK BEFORE installing ATLAS. Therefore, we will only do a BOGUS CONFIGURATION step within the ATLAS' installation, in order to figure out the best flags. Notice that, I had been using -O3, for example, but ATLAS might inquire a different optimization flag that suits the best for my architecture! Therefore, I should go back AND RE INSTALL ALL OF THE OTHER LIBRARIES WITH THIS INFORMATION!!! Let us proceed with the bogus installation:
Code:
~/Libraries$ cd ATLAS_3.8.4-CORE/
~/Libraries/ATLAS_3.8.4-CORE$ mkdir BOGUS_ATLAS_INSTALL_DIR    # For bogus configuration.
~/Libraries/ATLAS_3.8.4-CORE$ mkdir ATLAS_3.8.4-BUILD-Citadel  # For real configuration.
~/Libraries/ATLAS_3.8.4-CORE$ cd BOGUS_ATLAS_INSTALL_DIR/
Notice how to instruct the configuration step:
Code:
~/Libraries/ATLAS_3.8.4-CORE/BOGUS_ATLAS_INSTALL_DIR$ ../configure -b 32 -D c -DPentiumCPS=1733 --prefix=/home/ejspeiro/Libraries/ATLAS_3.8.4-BUILD
(...)
DONE configure
Recall that you can inquire about the frequency of your CPU by means of

Code:
$ cat /proc/cpuinfo
Once we have performed our bogus configuration, let us analyze the created make.inc file, to learn about what does ATLAS prefer!

Code:
~/Libraries/ATLAS_3.8.4-CORE/BOGUS_ATLAS_INSTALL_DIR$ fgrep "F77 =" Make.inc 
   F77 = gfortran
~/Libraries/ATLAS_3.8.4-CORE/BOGUS_ATLAS_INSTALL_DIR$ fgrep "F77FLAGS" Make.inc 
   F77FLAGS = -O -m32
   FLINKFLAGS = $(F77FLAGS)
~/Libraries/ATLAS_3.8.4-CORE/BOGUS_ATLAS_INSTALL_DIR$ fgrep "ICC =" Make.inc 
   ICC = gcc
~/Libraries/ATLAS_3.8.4-CORE/BOGUS_ATLAS_INSTALL_DIR$ fgrep "ICCFLAGS" Make.inc 
   ICCFLAGS = $(CDEFS) -O -fomit-frame-pointer -m32
:O So ATLAS Prefers -O instead of -O3! Notice that the internal variable being called F77 implies nothing with respect to the fact that LAPACK is written in Fortran 90! The important thing is the information about the compiler that it contains! The same for F77FLAGS!

STEP 3: Erase the bogus installation directory AND reinstall BLAS AND LAPACK using the attained information from ATLAS!

This is just a matter of going back, cleaning and rebuilding both libraries!

This is how the new variable have changed for my LAPACK installation:

Code:
FORTRAN  = gfortran
OPTS     = -O -m32
DRVOPTS  = $(OPTS)
NOOPT    = -m32
LOADER   = gfortran
LOADOPTS = $(OPTS)
TIMER    = INT_ETIME    # Now we are sure that the compiler is indeed gfortran!
CC = gcc
CFLAGS = -O -fomit-frame-pointer -m32
STEP 4: Install ATLAS for real!

Again, it is just a matter of simply doing:

Code:
~/Libraries/ATLAS_3.8.4-CORE/ATLAS_3.8.4-BUILD-Citadel$ ../configure -b 32 -D c -DPentiumCPS=1733 --prefix=/home/ejspeiro/Libraries/ATLAS_3.8.4-BUILD --with-netlib-lapack=/home/ejspeiro/Libraries/lapack-3.4.1/liblapack.a
(..)
~/Libraries/ATLAS_3.8.4-CORE/ATLAS_3.8.4-BUILD-Citadel$ ls
ARCHS        bin         lib       Make.top  xarchinfo_linux  xctest    xf2cstr   xprobe_3dnow  xprobe_comp       xprobe_OS    xprobe_sse3
atlcomp.txt  include     Makefile  src       xarchinfo_x86    xf2cint   xf77test  xprobe_arch   xprobe_f2c        xprobe_sse1  xprobe_vec
atlconf.txt  interfaces  Make.inc  tune      xconfig          xf2cname  xflibchk  xprobe_asm    xprobe_gas_x8632  xprobe_sse2  xspew
Notice that we have now specified our working LAPACK library!

Before building verify that the produced Make.inc has correctly recognized both the shell and the ranlib application you are actually using!

Now do

Code:
~/Libraries/ATLAS_3.8.4-CORE/ATLAS_3.8.4-BUILD-Citadel$ make
You will see cool things as for example, how ATLAS computes the size of you L1 cache and so forth!!! Be aware that this step can take up to ONE HOUR depending on you platform. In my case it took me around .

STEP 5: Perform the sanity tests and check for a correct install! And finally, install!

Code:
~/Libraries/ATLAS_3.8.4-CORE/ATLAS_3.8.4-BUILD-Citadel$ make check
(...)
~/Libraries/ATLAS_3.8.4-CORE/ATLAS_3.8.4-BUILD-Citadel$ make time
(...)
~/Libraries/ATLAS_3.8.4-CORE/ATLAS_3.8.4-BUILD-Citadel$ make install
(...)
I would like to encourage carefully reading ATLAS User Guide, the one that comes with the installation! There you
will find valuable information regarding related topics to the installation!

Finally [2] provides an excellent example on how to use ATLAS' LAPACK!

:)


by mamunsultanpur on Thu, 2013-03-14 14:13
Really a nice post for all. Every one be helped 100%

by ejspeiro on Wed, 2013-03-20 09:40
Thanks mamunsultanpur!

\m/


  



All times are GMT -5. The time now is 11:21 PM.

Main Menu

My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration