LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Linux From Scratch
User Name
Password
Linux From Scratch This Forum is for the discussion of LFS.
LFS is a project that provides you with the steps necessary to build your own custom Linux system.

Notices


Reply
  Search this Thread
Old 06-03-2019, 11:38 AM   #1
GeekBoy
Member
 
Registered: Dec 2006
Distribution: Fedora v6
Posts: 56

Rep: Reputation: 0
v8.4 LFS systemd - chpt 6.9 Glibc: Error during "make" No module named '_posixsubprocess'


So as mentioned in title

Quote:
File "/tools/lib/python3.7/subprocess.py", line 152, in <module>
import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
Traceback (most recent call last):
File "../scripts/gen-as-const.py", line 28, in <module>
import glibcextract
File "/sources/glibc-2.29/scripts/glibcextract.py", line 22, in <module>
import subprocess
File "/tools/lib/python3.7/subprocess.py", line 152, in <module>
import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
make[2]: *** [../Makerules:271: /sources/glibc-2.29/build/tcb-offsets.h] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [../Makerules:271: /sources/glibc-2.29/build/ucontext_i.h] Error 1
Traceback (most recent call last):
File "../scripts/gen-as-const.py", line 28, in <module>
import glibcextract
File "/sources/glibc-2.29/scripts/glibcextract.py", line 22, in <module>
import subprocess
File "/tools/lib/python3.7/subprocess.py", line 152, in <module>
import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
Traceback (most recent call last):
File "../scripts/gen-as-const.py", line 28, in <module>
import glibcextract
File "/sources/glibc-2.29/scripts/glibcextract.py", line 22, in <module>
make[2]: *** [../Makerules:271: /sources/glibc-2.29/build/link-defines.h] Error 1
import subprocess
File "/tools/lib/python3.7/subprocess.py", line 152, in <module>
import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
make[2]: *** [../Makerules:271: /sources/glibc-2.29/build/tlsdesc.h] Error 1
make[2]: Leaving directory '/sources/glibc-2.29/csu'
make[1]: *** [Makefile:258: csu/subdir_lib] Error 2
make[1]: Leaving directory '/sources/glibc-2.29'
make: *** [Makefile:9: all] Error 2
To me it seems it is looking for a Python module names _posixsubprocess. Is that correct?

Shouldn't that of installed during section 5.30?

Thanks!
 
Old 06-03-2019, 01:30 PM   #2
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 11,646

Rep: Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360
That's a big page, compiling glibc. I have versions in /usr/lib64/python2.x/site-packages, and /usr/lib64/python3.x/lib-dynload/

Now You will have /usr/lib, not /usr/lib64 and probably only 1 python version in /tools. Check you have it in
/tools/usr/lib
My next suspect would be the clever stuff they did so you can see the libs in /tools because you will hardly compile without the stuff in /tools. That's what it's there for. See how you get on.
 
Old 06-03-2019, 04:39 PM   #3
GeekBoy
Member
 
Registered: Dec 2006
Distribution: Fedora v6
Posts: 56

Original Poster
Rep: Reputation: 0
There is no directory tools/usr/

I do see tools/lib/ and tools/lib64

In tools/lib there is /python3.7 directory with nothing with '_posixsubprocess' or 'posixsubprocess'

In tools/lib64 is also a /python3.7 which looks identical to tools/lib/python3.7 with no such posixsubprocess or _posixsubprocess.

Update:

I rebuilt both Python and Glibc, but this time with one difference. Instead of the usual "make -j4," this time I just did it with make. It was successful.

Any ideas on why they would be the case?

Last edited by GeekBoy; 06-03-2019 at 09:25 PM. Reason: Additional Information
 
Old 06-04-2019, 12:05 AM   #4
Luridis
Member
 
Registered: Mar 2014
Location: Texas
Distribution: LFS 9.0 Custom, Merged Usr, Linux 4.19.x
Posts: 616

Rep: Reputation: 167Reputation: 167
Tar up your /sources and save it in the build host. Wipe the disk, rebuild /tools (Chapter 5) the module paths hard coded into your python build are incorrect. Never try to rebuild a missing or corrupted chapter 5 package (./configure --prefix=/tools) during chapter 6 because the search paths are different.

CH5: PATH=/tools/bin:/bin:/usr/bin (In lfs user .bash_profile)
CH6: PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin (In chapter 6 chroot command.)

This is without bringing the linker changes into it, which complicates it even further. If you've been doing this a long time you can get away with it by building some simple tools like gawk with static linking. But, Python... That's a can of worms. Start over.

PS: Just saw your second message. Do you mean you rebuilt the chapter 5 libc (the one for /tools) during chapter 6? If that even worked I am surprised. It may cause you bizarre problems later...

Last edited by Luridis; 06-04-2019 at 12:13 AM.
 
Old 06-04-2019, 03:48 AM   #5
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 11,646

Rep: Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360
There's a big difference between 'tools/usr' which searches whatever directory you happen top be in, and '/tools/usr' which searches /.

'make' uses 1 make process; make -j4 uses 4, and is therefore faster.

I have misgivings also about rebuilding python (back in Chapter 5), but if glibc compiled, I'd probably keep going. If you didn't exit the chroot and set up for chapter 5 again to build python you're in a mess. I presume that whatever you did, chapter 6's glibc worked in the chroot? I'd take the advice in the previous post then.
 
Old 06-04-2019, 11:46 AM   #6
GeekBoy
Member
 
Registered: Dec 2006
Distribution: Fedora v6
Posts: 56

Original Poster
Rep: Reputation: 0
I did rebuild Python under chroot, but under the advice of another who mentioned to exclude the sed command because the path changes.

So it did compile. I did the "make check." While it is mentioned some of the tests will probably fail, what is not mentioned is what should happen if those tests fail, or you get a passing test.


Quote:
UNSUPPORTED: elf/tst-audit10
UNSUPPORTED: elf/tst-avx512
XPASS: elf/tst-protected1a
XPASS: elf/tst-protected1b
FAIL: inet/tst-idna_name_classify
FAIL: io/tst-copy_file_range
FAIL: io/tst-copy_file_range-compat
UNSUPPORTED: math/test-double-libmvec-alias-avx512
UNSUPPORTED: math/test-double-libmvec-alias-avx512-main
UNSUPPORTED: math/test-double-libmvec-sincos-avx512
UNSUPPORTED: math/test-float-libmvec-alias-avx512
UNSUPPORTED: math/test-float-libmvec-alias-avx512-main
UNSUPPORTED: math/test-float-libmvec-sincosf-avx512
UNSUPPORTED: misc/tst-pkey
FAIL: misc/tst-ttyname
UNSUPPORTED: nptl/test-cond-printers
UNSUPPORTED: nptl/test-condattr-printers
UNSUPPORTED: nptl/test-mutex-printers
UNSUPPORTED: nptl/test-mutexattr-printers
UNSUPPORTED: nptl/test-rwlock-printers
UNSUPPORTED: nptl/test-rwlockattr-printers
FAIL: nptl/tst-create-detached
FAIL: nss/tst-nss-files-hosts-getent
UNSUPPORTED: nss/tst-nss-test3
UNSUPPORTED: resolv/tst-resolv-ai_idn
UNSUPPORTED: resolv/tst-resolv-ai_idn-latin1
Summary of test results:
6 FAIL
5958 PASS
18 UNSUPPORTED
17 XFAIL
2 XPASS
make[1]: *** [Makefile:405: tests] Error 1
make[1]: Leaving directory '/sources/glibc-2.29'
make: *** [Makefile:9: check] Error 2
The book does not mentioned what to expect at the end. Should it fail like this just because there is an expected failed test, and this is normal, or do I have an issue because of the error reported at the end?

Thanks again!
 
Old 06-04-2019, 10:55 PM   #7
Luridis
Member
 
Registered: Mar 2014
Location: Texas
Distribution: LFS 9.0 Custom, Merged Usr, Linux 4.19.x
Posts: 616

Rep: Reputation: 167Reputation: 167
The book does mention what to expect and most of those are covered by the section immediately following the shown "make check" command in the Chapter 6 Glibc build instructions. The exceptions are these two:

Code:
FAIL: io/tst-copy_file_range
FAIL: io/tst-copy_file_range-compat
Which are likely due to timeouts during the tests.

As to Python: If you built Python in chapter 6 again before building (g)libc, you will need to build it again at the normal step 6.51. This is necessary because if you built it before installing libc, it will be linked to /tools/lib/{glibc.so,libstdc++.so} instead of /lib/{glibc.so,libstdc++.so}. Also note that libstdc++.so is part of the GCC package, so even if you were to rebuild Python right after glibc, you'd still be linking to things in /tools. Finally, since SSL was not installed when you built Python, the _ssl module wouldn't be built either. Lots of things use Python's _ssl module so, it is needed. So... be sure to build Python again at the normal step.

Last edited by Luridis; 06-04-2019 at 10:58 PM.
 
Old 06-05-2019, 01:38 PM   #8
GeekBoy
Member
 
Registered: Dec 2006
Distribution: Fedora v6
Posts: 56

Original Poster
Rep: Reputation: 0
I am using a flash drive as the partition location.
So they could be the cause of timeout.
But still does not really answer about what I asked. The book does not mention at all what the end result should be at all. While it does mention about some of the tests failing, and how critical the test is, it still does not address if you should be getting a fail as don't worry about it, or if fail, you have an issue.

Thanks for the response.
 
Old 06-06-2019, 06:07 AM   #9
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 11,646

Rep: Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360Reputation: 1360
Quote:
Originally Posted by GeekBoy
I am using a flash drive as the partition location.
Personally, I think that's crazy. Usb drives are best used as data disk, write and read slowly, and wear very fast. Even if you haven't a spare partition, you can use this dodge: choose a spot with 5G free, and do the following
Code:
dd if=/dev/zero of=/somewhere/LFS bs= 1G count=5
mkfs.ext4 /somewhere/LFS
mount -o loop /somewhere/LFS /lfs
 
1 members found this post helpful.
Old 06-06-2019, 11:03 AM   #10
Luridis
Member
 
Registered: Mar 2014
Location: Texas
Distribution: LFS 9.0 Custom, Merged Usr, Linux 4.19.x
Posts: 616

Rep: Reputation: 167Reputation: 167
Quote:
Originally Posted by GeekBoy View Post
But still does not really answer about what I asked. The book does not mention at all what the end result should be at all. While it does mention about some of the tests failing, and how critical the test is, it still does not address if you should be getting a fail as don't worry about it, or if fail, you have an issue.
They do address it as best they can. Different tests will fail on different machines. There are many things related to the hardware and/or build host that can cause a test to fail: kernel configuration, shell environment, chroot settings, installed software, lack of microcode updates, host cpu available features, and even BIOS bugs can cause tests to fail. The problem you're having is that you expect certainty, the exact output you can expect from the test suites but, that is impossible in the open source ecosystem.

Generally, if the majority of the tests pass the system will be fine. There really is only reason for concern if hundreds of tests fail. Test suites on these core libraries look for things that most users will not encounter through the normal use of the machine. A lot of the tests look for things the developers call "corner cases". These are mostly obscure problems that only show up in very specific circumstances that will not occur for most uses of the library or tool. I would not be concerned about the failures you encountered... You can move on to the next step.
 
1 members found this post helpful.
Old 06-07-2019, 07:00 PM   #11
GeekBoy
Member
 
Registered: Dec 2006
Distribution: Fedora v6
Posts: 56

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by business_kid View Post
Personally, I think that's crazy. Usb drives are best used as data disk, write and read slowly, and wear very fast. Even if you haven't a spare partition, you can use this dodge: choose a spot with 5G free, and do the following
Code:
dd if=/dev/zero of=/somewhere/LFS bs= 1G count=5
mkfs.ext4 /somewhere/LFS
mount -o loop /somewhere/LFS /lfs

I have some spare SATA3 SSD drives laying around I could use with a USB 3.0 adapter instead.
 
Old 06-07-2019, 07:03 PM   #12
GeekBoy
Member
 
Registered: Dec 2006
Distribution: Fedora v6
Posts: 56

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Luridis View Post
They do address it as best they can. Different tests will fail on different machines. There are many things related to the hardware and/or build host that can cause a test to fail: kernel configuration, shell environment, chroot settings, installed software, lack of microcode updates, host cpu available features, and even BIOS bugs can cause tests to fail. The problem you're having is that you expect certainty, the exact output you can expect from the test suites but, that is impossible in the open source ecosystem.

Generally, if the majority of the tests pass the system will be fine. There really is only reason for concern if hundreds of tests fail. Test suites on these core libraries look for things that most users will not encounter through the normal use of the machine. A lot of the tests look for things the developers call "corner cases". These are mostly obscure problems that only show up in very specific circumstances that will not occur for most uses of the library or tool. I would not be concerned about the failures you encountered... You can move on to the next step.
Yes, I understand that, but saying it is critical to run the test but then having the "make" fail is the part which could be explained a bit better. Maybe it could be mentioned a little better why it is so critical and possibly explain when you have a problem when make check fails
 
Old 06-15-2019, 10:03 PM   #13
Luridis
Member
 
Registered: Mar 2014
Location: Texas
Distribution: LFS 9.0 Custom, Merged Usr, Linux 4.19.x
Posts: 616

Rep: Reputation: 167Reputation: 167
I'm confused now. You showed the results of "make check" and then mention afterward that the "make" step failed and didn't post the output of that.
 
Old 06-15-2019, 10:37 PM   #14
GeekBoy
Member
 
Registered: Dec 2006
Distribution: Fedora v6
Posts: 56

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Luridis View Post
I'm confused now. You showed the results of "make check" and then mention afterward that the "make" step failed and didn't post the output of that.
It's the same thing.

Anyway it is moot now, and I have started over now.
Thanks anyway for taking the time for responding.
 
Old 06-15-2019, 11:15 PM   #15
Luridis
Member
 
Registered: Mar 2014
Location: Texas
Distribution: LFS 9.0 Custom, Merged Usr, Linux 4.19.x
Posts: 616

Rep: Reputation: 167Reputation: 167
LOL, a fail on make is different. The "make" step builds the package, make check builds and runs the tests. If the build fails, that needs to be fixed before running tests.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] v8.4 LFS systemd - chpt 6.9 configure error: python not found GeekBoy Linux From Scratch 2 06-03-2019 11:25 AM
[SOLVED] LFS 8.3 Chpt. 5.2 toolchain issue mag911 Linux From Scratch 8 12-08-2018 05:18 PM
file /var/lib/named/var/named/reverse/named.zero failed: file not found Toadman Linux - Software 15 03-18-2009 07:01 PM
chown -R named:named /var/named crash the system? joangopan Fedora 2 09-09-2007 02:46 AM
Chpt 6.14 GCC no such file creznedmick Linux From Scratch 3 01-25-2007 08:16 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Linux From Scratch

All times are GMT -5. The time now is 12:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration