LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (https://www.linuxquestions.org/questions/slackware-14/)
-   -   64-bit kernel, 32-bit userspace? (https://www.linuxquestions.org/questions/slackware-14/64-bit-kernel-32-bit-userspace-705956/)

piete 02-19-2009 12:04 PM

64-bit kernel, 32-bit userspace?
 
Originally started here: http://www.linuxquestions.org/questi...73#post3444973

The gist of it being "What happens if I throw in a 64-bit kernel with slackware stock 32 bit userspace applications?"

I've started a new thread here because this is not related to the original post, really.

Quote:

Originally Posted by Shingoshsi
The point would be to find out how little of Slamd64 would you need to install in Slackware, to keep full functionality of the original Slackware 32-bit package set. ... It would be much better, if we could simply compile glibc to support both 32-bit and 64-bit in one single package, instead of having a compat to get around this.

Since slamd64 intends to replicate the slackware experience, surely the amount of slamd64 you install is arbitrary (dependencies notwithstanding) to retain that functionality?

The reason for compatibility libraries (which are no more than stock slack libraries) is one of separation*. Not everyone needs them, so it makes sense to separate them out so those who do use them can install them, and those who don't ... don't.

However, if you want a LotR (one glibc to rule them all ... ) approach to toolchains in your slack, then you probably need to investigate biarch glibc:

http://devpit.org/wiki/Gnu_Toolchain...Building_GLIBC
http://ozlabs.org/pipermail/linuxppc...ry/000852.html

The general theory being a biarch compiler backed up by a biarch glibc (and binutils) can produce two binaries at the flick of a switch.

I question how necessary having a biarch toolchain is, when considering the amount of effort to generate one vs using two separate (and already prebuilt) toolchains; it's not like you'd have cause to suddenly swap architecture mid-build. Is it? My head is hurting just contemplating the mess that would make!

Quote:

Originally Posted by Shingoshi
But when I want to build something, let's say like kvm which builds a kernel module as well as binaries, the modules need to be 64-bit, while the binaries need to be 32-bit.

Does anyone have any ideas as to how to resolve this?

Two toolchains, one for each architecture, or one toolchain that supports both arches. They amount to the same thing in this case: you need to compile the kernel module with the kernel arch toolchain (64-bit) and the application layer with the application arch toolchain (32-bit).

This may mean dismantling the makefiles and ensuring there is a clean separation between module and application. It could be as easy as `ARCH=x86_64 make module && ARCH=i386 make application` but I doubt it.

You've said that a 64 bit kernel is better than PAE (http://kerneltrap.org/node/2450) for memory addressing, but does your 32 bit userspace not throttle (at least in part) you because your apps don't understand a bigger memory space? I'm thinking of place and route operations or mathematical simulation. If you have a lot of smaller apps, there may be some benefit to be had.

The one final question I have: in this mix of 32 bit and 64 bit stuff, what should be reported by uname?

- Piete.

* I am not a slamd64 contributor or Fred. In my mind, c/ is used to separate things sensibly. I have in the past gotten things wrong, so if there is an official stance on the whys and wherefores of c/ ... it's not being made by me.

weibullguy 02-19-2009 12:52 PM

Quote:

Originally Posted by piete (Post 3449909)
The reason for compatibility libraries (which are no more than stock slack libraries) is one of separation*. Not everyone needs them, so it makes sense to separate them out so those who do use them can install them, and those who don't ... don't.

They're separated because the 32-bit and 64-bit libraries have the same name. You couldn't put them in the same directory (/usr/lib for example) or they would overwrite each other and nuthin' would work.

Quote:

Originally Posted by piete (Post 3449909)
However, if you want a LotR (one glibc to rule them all ... ) approach to toolchains in your slack, then you probably need to investigate biarch glibc:

The general theory being a biarch compiler backed up by a biarch glibc (and binutils) can produce two binaries at the flick of a switch.

A multilib toolchain requires a 32-bit glibc and a 64-bit glibc. Glibc provides ABI-specific libraries, there's no way around this. Both binutils and gcc will build with mulitlib support unless explicitly told not to with the --disable-multilib switch. Then, yes, it's a matter of calling gcc with the -m32 or -m64 switch when building packages.

Quote:

Originally Posted by piete (Post 3449909)
I question how necessary having a biarch toolchain is, when considering the amount of effort to generate one vs using two separate (and already prebuilt) toolchains; it's not like you'd have cause to suddenly swap architecture mid-build. Is it?

I'd say it's more effort to build two separate toolchains than one multilib toolchain. The only thing that needs to be built in duplicate is glibc in a mulitlib toolchain. Two separate toolchains will require you to build two binutils and two gcc also.

Anyway, my :twocents:.

piete 02-19-2009 07:14 PM

You're perfectly correct of course, you're still going to end up with two sets of libraries. Apologies for the confusion.

Shingoshi 02-19-2009 07:25 PM

piete: Thank you so much!!
 
I really appreciate seeing that others have more than a passing interest in this subject. I'm glad that you went back and posted the link to this thread in my original comments. Thank you friend!

I just signed up for membership on the first link. I will now take time to look at the rest of this.

Thanks friend!
Xavian-Anderson Macpherson
Shingoshi

Ok. Additionally, I've thought it best to simply build everything from this point forward as 64-bit. There's a good reason for this, in that Slackware already has most if not all that it needs. Slamd64 on the other hand doesn't. So encouraging users to build 64-bit applications should be the priority. As I've said elsewhere, I think only those applications that won't for any reason run as 64-bit, should be built as 32-bit. This will however mean that I install the rest of Slamd64. That's the point of concern that I've had with this. If I can do that without any compromise to my Slackware system, then there's no problem.

Shingoshi 02-20-2009 12:04 AM

Ok. Maybe this sums up the dilemma...
 
With having nothing but 32-bit packages installed, you also only have 32-bit libraries also. So when you go to build new applications (from your Slackware system), they're looking for libraries in /usr/lib64 or /lib64 (because that's where uname tells the build to look), except you don't have anything there. So the builds fail, even though you have the libraries installed. They're just not the right versions in the right place. So you need a means to build the packages against the 32-bit libraries (and maybe create 64-bit packages from those, if that's you highest priority). Because once you have as large a database of (3600+) packages installed as I do, the whole point is to use those packages, even for building others, not just to run the binaries. So there's the problem that hasn't yet been answered. I realized that I needed to write this after just having another package fail to build. In the report, I then saw it was looking in /usr/lib64.

So, now that hopefully my question is clearer, please explain how to resolve this.

Thank you,
Xavian-Anderson Macpherson.

ps. With Gentoo, this process would all be handled automatically, by doing an "emerge system" (for the toolchain), and then an "emerge world" (for the packages in their proper order).

Alien Bob 02-20-2009 03:08 AM

Quote:

Originally Posted by Shingoshi (Post 3450578)
ps. With Gentoo, this process would all be handled automatically, by doing an emerge system (for the toolchain), and then an emerge world (for the packages in their proper order).

If there was a 64bit Slackware, all you would have to do is pointing slackpkg at a 64bit mirror, and running "slackpkg update ; slackpkg install-new; slackpkg upgrade-all" to end up with a fully 64bit capable system.

But since Slackware is not available in 64bit, and Gentoo supports it, comparing the two is pointless.

Eric

Shingoshi 02-20-2009 04:24 AM

The reality about moving from one to another...
 
Quote:

Originally Posted by Alien Bob (Post 3450752)
If there was a 64bit Slackware, all you would have to do is pointing slackpkg at a 64bit mirror, and running "slackpkg update ; slackpkg install-new; slackpkg upgrade-all" to end up with a fully 64bit capable system.

But since Slackware is not available in 64bit, and Gentoo supports it, comparing the two is pointless.

Eric

Switching repositories was all I did to move from Slackware to Slamd64. Mind you it was not advised, but I did it anyway. And for the most part, it worked, with minor configuration changes that had to be done. But then again, I used slapt-get instead of slackpkg. One provides some degree of dependency checking, and the other doesn't. Without dependency verification, you simply can't do it with slackpkg. Besides, that always crashed for me.

Just try it with a test installation of Slackware. If you have no or very few packages on your system except what's from Slackware, Slamd64 will install just fine. In fact, it will install just fine anyway. It's just that's you'll have to recompile all of your old packages which Slamd64 doesn't have. That's my case now with having so many packages from Slacky.eu, which has a phenomenally larger database than Slackware proper. As mentioned above, I have 3600 packages installed. So I'm facing a big undertaking in switching this time around.

Xavian-Anderson Macpherson
Shingoshi

piete 02-20-2009 05:09 AM

Quote:

Originally Posted by Shingoshi (Post 3450578)
Because once you have as large a database of (3600+) packages installed as I do, the whole point is to use those packages, even for building others, not just to run the binaries. So there's the problem that hasn't yet been answered. I realized that I needed to write this after just having another package fail to build. In the report, I then saw it was looking in /usr/lib64.

So, now that hopefully my question is clearer, please explain how to resolve this.

Off the top of my head there are two methods you could adopt to work around the fact that uname returns x86_64:

1. Treat your system as a pure 64-bit system, and treat every build you do as a cross-compile.
2. Hack uname to return x86 instead of x86_64.

I've seen 2. used in reverse in a 32-bit chroot environment on x86_64. Basically you build a shell script, chmod it and install it over the uname binary, to return the values you want it to and fool any programs that call it to use the result to determine the architecture.

For 1. this means having one of:
a) x86_64/x86 biarch toolchain
ii) x86_64->x86 cross toolchain
3) x86 native toolchain

I confess to not knowing the current state of the slamd64 toolchain, however for this explanation, let's assume it's native x86_64, so the first two options aren't available.

Since slackware's toolchain is native x86, in my mind it makes the most sense to use that one to build your userspace apps. It should just be enough to ensure that your x86 toolchain binaries are ahead in the path of the x86_64 binaries, and then that's it, job done. Pass the necessary ARCH options to the new sources so they know (or are fooled enough not to care) about the arch and voila.

I could go on and on, but back to you :)

- Piete

weibullguy 02-20-2009 08:50 AM

Quote:

Originally Posted by Shingoshi (Post 3450578)
So the builds fail, even though you have the libraries installed. They're just not the right versions in the right place.

Sure they are. They're 32-bit libraries installed in the /lib directories right where they're supposed to be installed.
Quote:

Originally Posted by Shingoshi (Post 3450578)
So you need a means to build the packages against the 32-bit libraries (and maybe create 64-bit packages from those, if that's you highest priority).

Have you tried using the PKG_CONFIG_PATH environment variable? As long as you don't use something like /usr/share/pkgconfig for your metadata files (and you probably shouldn't on a mulitlib machine), the metadata files for the 32-bit stuff will be in /usr/lib/pkgconfig and the metadata files for the 64-bit stuff will be in /usr/lib64/pkgconfig. This only leaves a fairly small number, albeit common, apps that use -config files in /usr/bin to provide build information to other packages.

Anyway, my :twocents:.

Quote:

Originally Posted by Shingoshi (Post 3450578)
With Gentoo, this process would all be handled automatically, by doing an emerge system (for the toolchain), and then an emerge world (for the packages in their proper order).

Actually, Gentoo doesn't build the 32-bit apps on multilib machines. IIRC, they install precompiled 32-bit libs and apps so it won't be looking for 32-bit libraries to build against.

i92guboj 02-20-2009 09:04 AM

Quote:

Originally Posted by weibullguy (Post 3451096)

Anyway, my :twocents:.

Actually, Gentoo doesn't build the 32-bit apps on multilib machines. IIRC, they install precompiled 32-bit libs and apps so it won't be looking for 32-bit libraries to build against.

To tell the truth, it uses a mixed approach. Wine and grub are two tipical examples of apps that are built in 32 bits mode (they can't be compiled as 64 bits apps anyway), and not installed from prebuilt binaries.

Shingoshi 02-20-2009 02:13 PM

I was speaking in the context of compile process...
 
Quote:

Originally Posted by weibullguy (Post 3451096)
Sure they are. They're 32-bit libraries installed in the /lib directories right where they're supposed to be installed.Have you tried using the PKG_CONFIG_PATH environment variable? As long as you don't use something like /usr/share/pkgconfig for your metadata files (and you probably shouldn't on a mulitlib machine), the metadata files for the 32-bit stuff will be in /usr/lib/pkgconfig and the metadata files for the 64-bit stuff will be in /usr/lib64/pkgconfig. This only leaves a fairly small number, albeit common, apps that use -config files in /usr/bin to provide build information to other packages.

Anyway, my :twocents:.

Actually, Gentoo doesn't build the 32-bit apps on multilib machines. IIRC, they install precompiled 32-bit libs and apps so it won't be looking for 32-bit libraries to build against.

I thought this should have been clear. I definitely stated that the packages are in fact installed. That alone, means they are installed in their proper locations. The context that matters here though, is the compile of the source. And that compile process is looking in locations it expects to find libraries for 64-bit systems. Of course, this isn't a 64-bit system, but 32-bit instead. So, only in this context, the processes involved thinks things are not installed, when in fact, they are.

I'm waiting for the author of my build tool (src2pkg) to update it to handle this situation. This would be good, because it will clear the way for others to pursue the same path, and have cross-compilation capacity (in various environments) immediately available.

And about Gentoo. You missed my point. The point was that Gentoo has an automated build process.
Quote:

ps. With Gentoo, this process would all be handled automatically, by doing an "emerge system" (for the toolchain), and then an "emerge world" (for the packages in their proper order).
I said nothing about what it would build. What I meant was that all of the packages on the system, from the toolchain (first) to the applications (second), would each and all be built in their proper order, to ensure that all dependencies are rebuilt as well. So you start out with a new toolchain (which would now be 64-bit), against which everything thereafter would be built. The result is a clean 64-bit system.

Hopefully, this clears things up
Xavian-Anderson Macpherson
Shingoshi

fred87 02-20-2009 02:42 PM

Quote:

Originally Posted by piete (Post 3450833)
2. Hack uname to return x86 instead of x86_64.

Better:

Use the linux32 utility. This changes the kernel personality so that for all subprocesses, the kernel reports itself as i386 - including an unpatched uname, and binaries that use syscalls/other methods to work out the current architecture.

fred87 02-20-2009 03:07 PM

Quote:

Originally Posted by piete (Post 3450833)
a) x86_64/x86 biarch toolchain
<snip>
I confess to not knowing the current state of the slamd64 toolchain

It's 'a' :)

Shingoshi 02-20-2009 06:26 PM

If we set aside all the issues related to cross-compiling...
 
What packages from Slamd64 are needed to be added to Slackware along with the 64-bit kernel, that will allow a user to run 64-bit applications? We only need those packages listed that are absolutely necessary for 64-bit operations.

Whatever that set of packages are, that's the set that should comprise the meta-slamd64 package for Slackware users to install, to upgrade to Slamd64. This would be the same as an absolute minimal install for Slamd64.

So, taking a guess:
aaa_base
aaa_elflibs
aaa_terminfo (?)
glibc (the whole set, with compat as well)

Xavian-Anderson Macpherson
Shingoshi

Shingoshi 03-05-2009 10:18 PM

Here's an interesting thing to be aware of...
 
1 Attachment(s)
When running a 64-kernel on a 32-bit system, you may find that Firefox complains about the type of FlashPlayer you have installed. I've had it work as often as not. But a simple solution may simply be to install this package, and run "linux32 firefox" when you start Firefox. The linux32 supplied by Slamd64 is a 64-bit application, and won't work on a 32-bit system.

Change the extension from .txt to .tlz, to install with pkgtools-tukaani.

Xavian-Anderson Macpherson
Shingoshi


All times are GMT -5. The time now is 01:18 AM.