LinuxQuestions.org - [Guide] HowTo: Multilib LFS

- Linux From Scratch (https://www.linuxquestions.org/questions/linux-from-scratch-13/)

- - [Guide] HowTo: Multilib LFS (https://www.linuxquestions.org/questions/linux-from-scratch-13/%5Bguide%5D-howto-multilib-lfs-4175649832/)

[Guide] HowTo: Multilib LFS

I'm successfully make multilib LFS with a little modifications from original LFS book. So i'm gonna share it with anyone wanna try it or its gonna be future reference. All this method i got from https://www.williamfeely.info/lfs-multilib/ which i'm simplified it and some google around. I've been using this method few times to build my LFS based distro, so far wine and steam work great :)

First you gonna need make multilib toolchain (chapter 5) then make final system multilib (chapter 6). Affected package need change build command is binutils, glibc, gcc and libstdc++. At the end all 32bit libraries is placed in /usr/lib32 directory.

On chapter 5,

1) binutils pass1, on configure cmd, replace:

Code:

--with-lib-path=/tools/lib

with

Code:

--with-lib-path=/tools/lib:/tools/lib32

then make lib32 directory in /tools directory

Code:

mkdir -p /tools/lib32

2) gcc pass1, replace:

Code:

case $(uname -m) in

  x86_64)

    sed -e '/m64=/s/lib64/lib/' \

        -i.orig gcc/config/i386/t-linux64

 ;;

esac

with

Code:

sed -i -e 's@/lib/ld-linux.so.2@/lib32/ld-linux.so.2@g' gcc/config/i386/linux64.h

sed -i -e '/MULTILIB_OSDIRNAMES/d' gcc/config/i386/t-linux64

echo "MULTILIB_OSDIRNAMES = m64=../lib m32=../lib32 mx32=../libx32" >> gcc/config/i386/t-linux64

on configure cmd, replace:

Code:

--disable-multilib

with

Code:

--with-multilib-list=m32,m64

3) glibc, you need build two times, first for 32bit lib, then 64bit. Make sure you build 32bit first.
for 32bit lib build using this cmd:

Code:

mkdir -v build32

cd      build32

        

echo slibdir=/tools/lib32 > configparms

../configure                            \

          --prefix=/tools                    \

          --host=i686-lfs-linux-gnu          \

          --build=$(../scripts/config.guess) \

          --libdir=/tools/lib32              \

          --enable-kernel=3.2                \

          --enable-add-ons                  \

          --with-headers=/tools/include      \

          libc_cv_forced_unwind=yes          \

          libc_cv_c_cleanup=yes              \

          CC="$LFS_TGT-gcc -m32"            \

          CXX="$LFS_TGT-g++ -m32"

make

make install

for 64bit, build normally like in the book. Then you can check if you can if compiling and linking working fine

Code:

echo 'int main(){}' > dummy.c

$LFS_TGT-gcc dummy.c

readelf -l a.out | grep ': /tools'

$LFS_TGT-gcc -m32 dummy.c

readelf -l a.out | grep ': /tools'

the result should be like this for 64 and 32 bit:

Code:

[Requesting program interpreter: /tools/lib64/ld-linux-x86-64.so.2]

[Requesting program interpreter: /tools/lib32/ld-linux.so.2]

4) libstdc++, you need to build two times just like glibc, build 32bit (make sure change 'GCC VERSION' below):

Code:

mkdir -v build32

cd      build32

        

../libstdc++-v3/configure          \

        --host=i686-lfs-linux-gnu      \

        --prefix=/tools                \

        --libdir=/tools/lib32          \

        --disable-multilib              \

        --disable-nls                  \

        --disable-libstdcxx-threads    \

        --disable-libstdcxx-pch        \

        --with-gxx-include-dir=/tools/$LFS_TGT/include/c++/<GCC VERSION> \

        CC="$LFS_TGT-gcc -m32"          \

        CXX="$LFS_TGT-g++ -m32"

make

make install

then build libstdc++ 64bit just like in the book.

5) binutils pass2, build like in the book then replace:

Code:

make -C ld LIB_PATH=/usr/lib:/lib

with

Code:

make -C ld LIB_PATH=/usr/lib:/lib:/usr/lib32:/lib32

6) gcc pass2, just like gcc pass1, replace:

Code:

case $(uname -m) in

  x86_64)

    sed -e '/m64=/s/lib64/lib/' \

        -i.orig gcc/config/i386/t-linux64

  ;;

esac

with

Code:

sed -i -e 's@/lib/ld-linux.so.2@/lib32/ld-linux.so.2@g' gcc/config/i386/linux64.h

sed -i -e '/MULTILIB_OSDIRNAMES/d' gcc/config/i386/t-linux64

echo "MULTILIB_OSDIRNAMES = m64=../lib m32=../lib32 mx32=../libx32" >> gcc/config/i386/t-linux64

and replace:

Code:

--disable-multilib

with

Code:

--with-multilib-list=m32,m64

build the rest of chapter 5 follow the book.

7) strip section, strip debugging symbols from 32bit libraries and remove libtool (*.la) files.

Code:

strip --strip-debug /tools/lib32/*

find /tools/lib32 -name \*.la -delete

On chapter 6, for main LFS system,

1) glibc, for 64bit build, add to configure cmd:

Code:

--enable-multi-arch --enable-obsolete-rpc

then build for 32bit libraries:

Code:

mkdir -v ../build32

cd ../build32

CC="gcc -m32" \

CXX="g++ -m32" \

../configure --prefix=/usr                  \

            --disable-werror                \

            --enable-kernel=3.2            \

            --enable-multi-arch            \

            --enable-obsolete-rpc          \

            --enable-stack-protector=strong \

            --libdir=/usr/lib32            \

            --libexecdir=/usr/lib32        \

            libc_cv_slibdir=/usr/lib32      \

            i686-pc-linux-gnu

make

make install_root=$PWD/DESTDIR install

install -vdm755 /usr/lib32

cp -Rv DESTDIR/usr/lib32/* /usr/lib32/

install -vm644 DESTDIR/usr/include/gnu/{lib-names,stubs}-32.h \

        /usr/include/gnu/

ln -sv ../usr/lib32/ld-linux.so.2 /lib/ld-linux.so.2

ln -sv ../usr/lib32/ld-linux.so.2 /lib/ld-lsb.so.3

ln -sv ../lib/locale /usr/lib32/locale

echo "/usr/lib32" > /etc/ld.so.conf.d/lib32.conf

2) binutils, add this to configure cmd:

Code:

--enable-multilib --with-lib-path=/usr/lib:/lib:/usr/lib32

3) gcc, add this to configure cmd:

Code:

--enable-multilib

4) Sanity test, its a little bit different from pure64 LFS.

for 64bit:

Quote:

+ echo 'int main(){}'
+ cc dummy.c -v -Wl,--verbose
+ readelf -l a.out
+ grep ': /lib'
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
+ grep -o '/usr/lib.*/crt[1in].*succeeded' dummy.log
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib/crt1.o succeeded
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib/crti.o succeeded
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib/crtn.o succeeded
+ grep -B4 '^ /usr/include' dummy.log
#include <...> search starts here:
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include
/usr/local/include
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include-fixed
/usr/include
+ grep 'SEARCH.*/usr/lib' dummy.log
+ sed 's|; |\n|g'
SEARCH_DIR("/usr/x86_64-pc-linux-gnu/lib64")
SEARCH_DIR("/usr/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib32")
SEARCH_DIR("/usr/x86_64-pc-linux-gnu/lib");
+ grep '/lib.*/libc.so.6 ' dummy.log
attempt to open /lib/libc.so.6 succeeded
+ grep found dummy.log
found ld-linux-x86-64.so.2 at /lib/ld-linux-x86-64.so.2

for 32bit (add '-m32' to cc):

Quote:

+ echo 'int main(){}'
+ cc -m32 dummy.c -v -Wl,--verbose
+ readelf -l a.out
+ grep ': /lib'
[Requesting program interpreter: /lib/ld-linux.so.2]
+ grep -o '/usr/lib.*/crt[1in].*succeeded' dummy.log
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib32/crt1.o succeeded
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib32/crti.o succeeded
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib32/crtn.o succeeded
+ grep -B4 '^ /usr/include' dummy.log
#include <...> search starts here:
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include
/usr/local/include
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include-fixed
/usr/include
+ grep 'SEARCH.*/usr/lib' dummy.log
+ sed 's|; |\n|g'
SEARCH_DIR("/usr/i386-pc-linux-gnu/lib32")
SEARCH_DIR("/usr/x86_64-pc-linux-gnu/lib32")
SEARCH_DIR("/usr/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib32")
SEARCH_DIR("/usr/i386-pc-linux-gnu/lib");
+ grep '/lib.*/libc.so.6 ' dummy.log
attempt to open /usr/lib32/libc.so.6 succeeded
+ grep found dummy.log
found ld-linux.so.2 at /usr/lib32/ld-linux.so.2

then build the rest follow the book.
Now you should have multilib LFS :D

for future build of any 32bit lib should build using:

Code:

CC="gcc -m32" \

CXX="g++ -m32" \

PKG_CONFIG_PATH="/usr/lib32/pkgconfig" \

./configure --prefix=/usr \

            --libdir=/usr/lib32

make

make DESTDIR=$PWD/DESTDIR install

cp -Rv DESTDIR/usr/lib32/* /usr/lib32

Extra

some package in BLFS need modification to make it work with its 32 version side by side

1) python 64bit (both v2.7 and v3.7), rename pyconfig.h to pyconfig-64.h

Code:

mv /usr/include/python2.7/pyconfig{,-64}.h

mv /usr/include/python3.7m/pyconfig{,-64}.h

then make a wrapper script called pyconfig.h to /usr/include/python2.7/ and /usr/include/python3.7m/ directory:

Code:

/* pyconfig.h stub */



#ifndef __STUB__PYCONFIG_H__

#define __STUB__PYCONFIG_H__



#if defined(__x86_64__) || \

    defined(__sparc64__) || \

    defined(__arch64__) || \

    defined(__powerpc64__) || \

    defined(__s390x__)

#include "pyconfig-64.h"

#else

#include "pyconfig-32.h"

#endif



#endif

for 32bit python, install to tmp location then copy all its 32bit libs to /usr/lib32/ and its pyconfig.h to /usr/include/python{2.7,3.7m}/pyconfig-32.h

2) llvm 64bit, rename llvm-config.h to llvm-config-64.h

Code:

mv /usr/include/llvm/Config/llvm-config{,-64}.h

then make a wrapper script called llvm-config.h to /usr/include/llvm/Config/ directory:

Code:

#include <bits/wordsize.h>



#if __WORDSIZE == 32

#include "llvm-config-32.h"

#elif __WORDSIZE == 64

#include "llvm-config-64.h"

#else

#error "Unknown word size"

#endif

for llvm 32bit, build using this:

Code:

mv ../cfe-$version.src tools/clang

mv ../compiler-rt-$version.src projects/compiler-rt



mkdir -v build

cd      build



CC="gcc -m32" \

CXX="g++ -m32" \

PKG_CONFIG_PATH="/usr/lib32/pkgconfig" \        

cmake .. -G Ninja \

      -DCMAKE_INSTALL_PREFIX=/usr          \

      -DLLVM_ENABLE_FFI=ON                  \

      -DCMAKE_BUILD_TYPE=Release            \

      -DLLVM_BUILD_LLVM_DYLIB=ON            \

      -DLLVM_LINK_LLVM_DYLIB=ON            \

      -DLLVM_TARGETS_TO_BUILD="X86;AMDGPU;BPF" \

      -DLLVM_LIBDIR_SUFFIX=32              \

      -DCMAKE_C_FLAGS:STRING=-m32          \

      -DCMAKE_CXX_FLAGS:STRING=-m32        \

      -DLLVM_TARGET_ARCH:STRING=i686        \

      -DLLVM_DEFAULT_TARGET_TRIPLE="i686-pc-linux-gnu" \

      -Wno-dev

ninja

DESTDIR=$PWD/DESTDIR ninja install



cp -Rv DESTDIR/usr/lib32/* /usr/lib32/



install -m 0755 -D DESTDIR/usr/bin/llvm-config /usr/bin/llvm-config-32

install -m 0644 -D DESTDIR/usr/include/llvm/Config/llvm-config.h /usr/include/llvm/Config/llvm-config-32.h

thats all i think for now.

for more 32bit package you can check here: https://github.com/emmett1/ports

I hope someone can make it work too using this method.
I already use this method many times but with my own script and package manager.

Anyway, if you faced any problem you can check my script to bootstrap my distro here: https://github.com/emmett1/venom
Or you can try out my LFS based distro which is used BSD-style init and port system for packages. Get the installable iso here: https://sourceforge.net/projects/venomlinux/

Edit: add sanity test result and strip symbols from 32bit libraries (/tools/lib32)

Looks like a good job I'll give this a go when I get the chance, well done.

Pity I've just built my new LFS 8.4! It would have been nice to do it this way, then I could have used my printer, which has 32-bit filters, in LFS.

Quote:

Originally Posted by Keith Hedger (Post 5972001)

Looks like a good job I'll give this a go when I get the chance, well done.

yeah :D

Quote:

Originally Posted by hazel (Post 5972061)

Pity I've just built my new LFS 8.4! It would have been nice to do it this way, then I could have used my printer, which has 32-bit filters, in LFS.

Well you can just make your current LFS multilib. Create the multilib toolchain then rebuild binutils, glibc and gcc with multilib enabled for your current LFS using multilib toolchain. Then you should have multilib LFS system running :D

Thank you! I was seriously considering a complete new build on one of my spare partitions.

btw I think this thread should be a sticky.

Quote:

Originally Posted by hazel (Post 5972309)

Thank you! I was seriously considering a complete new build on one of my spare partitions.

You're welcome. :D
I dont think you need complete rebuild, i already convert pure64 lfs to multilib before, so it should work.

Quote:

Originally Posted by hazel (Post 5972309)

btw I think this thread should be a sticky.

Yeah i think this too, so it could help anybody wanna build multilib lfs :)

OK, primary toolkit from chapter 5 completed. The sanity check in 5.7 is repeated in 5.10 with a slightly different syntax for the compilation step. You might want to include that in your instructions.

I plan to do the chapter 6 primary toolkit tomorrow. There are more detailed sanity tests there, which I assume must be carried out both with and without the -m32 option.

Is there anything new that I need to know about running the "official" test suites on these packages?

I've hit a snag, but I think I know what caused it. In the book (section 6.9) you are told to remove /usr/include/limits.h if it exists before building glibc. So I duly did. But the build then broke with this error:

Code:

In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include-fixed/syslimits.h:7,

                from /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include-fixed/limits.h:34,

                from /usr/include/sys/param.h:26,

                from rpc_main.c:45:

/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include-fixed/limits.h:194:61: error: no include path in which to search for limits.h

If you look at line 194 of this file, it specifies /usr/include/limits.h as a file to be searched. It seems your new configuration terms for the 64-bit build conflict with the book at this point. The directory in which the error occurs seems to be /sources/glibc-2.29/build/sunrpc, so perhaps it is the "obsolete rpc" thing that is causing the incompatibility here, but I can't be sure as I was using -j4 parallel build. I can run it again with -j1 to check that.

Think of me as the "naive user" who tests your instructions ;)

PS: Yes, that's the directory that gives the fault. Here is the print-out from a -j1 build

Code:

In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include-fixed/syslimits.h:7,

                from /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include-fixed/limits.h:34,

                from /usr/include/sys/param.h:26,

                from rpc_main.c:45:

/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include-fixed/limits.h:194:61: error: no include path in which to search for limits.h

 #include_next <limits.h>  /* recurse down to the real one */

                                                            ^

make[2]: *** [Makefile:174: /sources/glibc-2.29/build/sunrpc/cross-rpc_main.o] Error 1

make[2]: Leaving directory '/sources/glibc-2.29/sunrpc'

It's just occurred to me. Since I have 64-bit gcc and binutils already installed on this partition, wouldn't they be used in preference to the multilib ones in /tools? Perhaps I need to have /tools/bin at the head of my path rather than at the tail?

Maybe you dont need that limits.h file when rebuild. I also faced this problem before, then i'm remove rm -f /usr/include/limits.h from my build script, since then everything just fine. Maybe the book should remove rm -f /usr/include/limits.h line, its obsolete.

Yeah maybe its caused by --enable-obsolete-rpc, you just play around a bit with it. My script still has it and build just fine.

Ofcourse you need /tools/bin at the head of your path, not at the tail, so it will use toolchain in /tools/bin not other directory in your path.

Quote:

PATH=/tools/bin:$PATH

I've found out what was wrong. The chroot instructions in the book assume an almost empty system, but mine already has a functioning root account with .bashrc and .bash_profile. So when I chrooted with "bash --login" those scripts were run and it mucked up my environment. That's why I didn't have the path set correctly. I have to set the correct environmental variables by hand, then it works. I've built the 64-bit version successfully and tested it. 4 tests failed and that's the same result as I got with the standard 64-bit glibc.

I'll install it, then build and test the 32-bit version next.

You could also install subversion and do an SVN checkout of the book and apply this patch: https://io.ax.lt/LFS/lfs/patch-multilib-SVN.diff (apply it like any other patch: cd BOOK && patch -p1 -i ../That-Patch. Read carefully the two files in there about what you need to render the book to html. Now you have an "official" un-official version of LFS for multilib.

I did that for LFS-svn 20190301 (i wanted the gcc-8.3 compiler update). I haven't compiled all programs with the new compiler, but I've compiled many (including all of KDE) with no problems. My 32-bit brother printer driver now work with LFS - finally. Arch AUR has a PKGBUILD for my printer (hl2240d) but it didn't work right (margins off). Instead i just ran the scripts from the RPM after installing the drivers.

That version of the book will also include x32 support (subset of x86_64 architecture). I suppose you could omit that part - it's interesting but not really practical now. I went ahead and included it, so i have usr/lib, usr/lib32, and usr/libx32 for libraries.

You don't mention running any tests on the software apart from the LFS sanity tests. I ran the glibc package tests on both the 32-bit and 64-bit libraries; the results for 64-bit were exactly the same as when building from the book, but the 32-bit library failed a lot of tests. I'm attaching the relevant part of the log for interest.

Most of the failures (114 out of 142) are in nptl, the threads library, which is probably not relevant for running a small simple program like a printer driver. It doesn't look good though if you want full multilib capacity.

Postscript: Sanity checks yield two additional SEARCH_DIR entries compared with the Book: /lib32 and /usr/lib32. The latter I expected but I'm surprised to find /lib32, a directory that doesn't actually exist.

Toolchain complete. Both binutils and gcc passed all the expected tests in their respective test suites. But in carrying out the final sanity tests, I found one significant difference from the Book: in the search directories from both the 32-bit and 64-bit dummy compilations, the directories /lib64 and /usr/lib64 were missing. Now this makes a kind of sense because /lib64 contains no libraries, only linkers, and /usr/lib64 doesn't actually exist. All the same, it is a difference and I think you should include this in any final write-up of instructions.

The real test of course is if I can get my printer to work and I won't know that until I have installed cups.