Slackware package managing system

moses · 01-05-2003, 06:20 PM

This is a continuation of an off-topic thread from another post. It
concerns creating a decent package managment tool for
Slackware, probably based off of the current pkgtools. This may
end up being an extension of those tools to provide download/
compilation support (that's what the user will see).

Quote:

NSKL:
I was thinking about what you said, and creating such a database, or using an existing one (such as the debian one) is not an option i like. But i think a good idea would be to create a tool, something like installwatch, but specific for slack, that will create a slackware package from source. So people download a source of a program, tar zxvf'it, cd to it's base dir, and then the user can execute this new tool/script that will ask for any configure option you wish to put (and perhaps display ./configure --help to show the available options) then go on to compile and make the package, and make it using makepkg, and finally install it with installpkg, leaving a packagename.tgz in the source base directory. Later this package can be uploaded to linuxpackages.net. I dunno if this script would be useful at all, but as i learn more bash scripting i will attempt to create something like this, perhaps some people might find it useful, and if i get more people to use it, we will see more .tgz packages around, beacuse it will be easier to create them. Also, this way it will be easier to uninstall software from a slack installation (with removepkg). This will increase the use of pkgtool since all packages would be compiled from source, but installed as slackpacks, and will appear in pkgtool's package database on every system.

Does anybody have any other ideas on improving slack package management? I'm pretty happy compiling from source, but if make uninstall script is missing, it can be a pain to remove packages from slack. Also new slack users don't like source. So they can download pre-compiled binaries from linuxpacgaes.net. This always leaves the dependency issues, but this is easily solved with ldd and downloading other needed packages.

Quote:

Mephisto
I toyed with the idea for a Slack package manager. I am actually still working on it. It will probably never get finished, and even if it does it will be too deviant from the Slack ethic for most people.

It is a multi tier p2p structure with different layers providing different services necessary to assemble the components and verify that they are valid. One tier to keep/submit md5 checksums and descriptions. Another layer to retrieve the files. And a third (optional) for dependency checking and compilation. It is actually fascinating from an intellectual standpoint, and even an achievable goal since 8.1 (with the inclusion of version numbers on the packages).

The hard part is dependency and GCC version checking. Since by it's nature Slackware does not, and never intends to, include dependency checking it would need to be added on externally. Then comes the question of how do you establish dependency? based on the libraries, or upon the package? And to what extent do you need to concern yourself with versions and under what version of GCC the library was compiled.... I think you see my conundrum. The p2p network is easy, the semi-securable MD5 network (based on signatures from known peers) is achievable, dependency checking on the other hand is a little more tricky, for me a Nth difficulty problem.

Oh yeah, a more automated way of creating the tgz packages would be nice as well. A chroot jail is the easiest way I have found but it is not necessarily straight forward.

Can we start by getting a list of all the available slackware-like
package managers available? We don't want to reinvent the
wheel where we don't have to. I'm aware of an old one called
autoslack that allowed updating of Slackware (it's relevant), and
autopkg, which I'm not too familiar with. autoslack is no longer
in development (as far as I'm aware). Don't forget the FreeBSD
solution, which seems to be elegant, though I've never used it.

After that, the first thing that's needed is to make a decision
about how we will serve the necessary information. Will we have
a server like http://linuxpackages.net that provides a central place
for downloading (at least the relevant information about where
to get the package)? We can't store everything on the user's
machine.

Mephisto · 01-05-2003, 07:21 PM

The first thing that leaps to mind is what are you/we trying to accomplish? My personal project was simply a way for me to muck around with JXTA, so let's disregard that for the moment. (it is really cool to play with though

)

Package Managers:
Slackware's default tools. Needed by all other "package managers" I know of for Slack. For their function they work and they are stable.

AutoPkg - a shell script to allow you to download and install packages from designated sites (configured by a text file) very much like apt-get. My 2 main issues (last time I used it) were that it only could retrieve things from the base tree of a Slack snapshot. i.e. you would not get updates from the patch folder. Useful for Slack-current, less so for a stable release. Related to this issue was there was no way to retrieve a package that is not part of the Slackware distribution.

AutoSlack - never used it.

FreeBSD solution?

LinuxPackages.net - only slightly more organized than IBM's partner zone (which I have heard referred to as "Website design by Picasso"). Has potential but does not deliver. Maybe if there was a CLI way of connecting to the DB.

Features that would add value to Slackware:
Package creator An easy way to create a tgz package from a source tarball. An idea I just had would be to tripwire a system, install the package, then get the difference in the tripwire. Then assemble your tgz based on these differences. Think it would work?

source install manager Maybe I am missing something here but to me installing from source is easy, it is removing it or upgrading later that is a problem. It is very difficult for me to see how a source tgz would work since compile options would make a difference in the package contents. The flip side is that if a package creator can be made sufficiently robust it also solves this issue in that you install from source and a variant of the package creator creates a "phantom package" for uninstalling based on it's results. Regardless I think source based packages have less value to your average user than a way to retrieve binary packages which leads us to:

Distributed Content Manager an apt-get style DB to retrieve packages from, or in my eyes somewhere between apt-get and linuxpackages.net. I gave this a lot of thought and my conclusion that a central repository of packages was unrealistic. What is needed is an automated way of discovering a package location based on criteria (probably configured by ini file) The problem with a centralized DB approach is maintenance and scalability, both can be achieved but not without a community (for maintenance) and funding (for scalability, specifically bandwidth costs) a p2p network is also an option and doable, the issue there is ensuring the quality of content and grazing/blight topology issues.

all of this does not also address the other "issue" which is
Dependency Checking I have no idea there, but I also never had a serious problem with handling dependencies by hand.

Whew that's it for now. Hopefully that was at least marginally coherent. There is a reason I keep my posts short.

moses · 01-05-2003, 08:00 PM

Quote:

Originally posted by Mephisto
The first thing that leaps to mind is what are you/we trying to accomplish? My personal project was simply a way for me to muck around with JXTA, so let's disregard that for the moment. (it is really cool to play with though :D )

My first goal is to set up a system that will allow an "easy" way
to download, compile, install packages for (at first) the Slackware
linux distribution. Now, the real trick is that I want it set up in
such a way as to do something like:

Code:

% pkgget gnome

and it goes and gets gnome, compiles it (if that's what is desired),
installs it, logs the installation (as well as where EVERY little
bit was installed), and then cleans up after itself. What I don't
want to do is store every package on a central server, though
storing a bit of information about where to get the package,
how to build the package and its dependencies may work.

Quote:

Package Managers:
Slackware's default tools. Needed by all other "package managers" I know of for Slack. For their function they work and they are stable.

Absolutely necessary.

Quote:

AutoPkg - a shell script to allow you to download and install packages from designated sites (configured by a text file) very much like apt-get. My 2 main issues (last time I used it) were that it only could retrieve things from the base tree of a Slack snapshot. i.e. you would not get updates from the patch folder. Useful for Slack-current, less so for a stable release. Related to this issue was there was no way to retrieve a package that is not part of the Slackware distribution.

I don't want to only be getting Slackware distributed packages. I
want to be able to get any package.

Quote:

AutoSlack - never used it.

It's old, and was a way to keep up with slack-current. It used
Perl and would go to ftp.slackware.com to download the
necessary files. It would check md5 files, check for what could
be upgraded, or what was new, etc.

Quote:

FreeBSD solution?

FreeBSD has a central repository of all the packages that are
available for FreeBSD. They have something like apt-get, but
you do a: make gnome install, and it'll go out and get the gnome
package from FreeBSD and install it (compilation occurs if
necessary). I'm not *really* familiar with it, but its code could
help. . .

Quote:

LinuxPackages.net - only slightly more organized than IBM's partner zone (which I have heard referred to as "Website design by Picasso"). Has potential but does not deliver. Maybe if there was a CLI way of connecting to the DB.

There absolutely needs to be a CLI/non-interactive way of doing
this.

Quote:

Features that would add value to Slackware:
Package creator An easy way to create a tgz package from a source tarball. An idea I just had would be to tripwire a system, install the package, then get the difference in the tripwire. Then assemble your tgz based on these differences. Think it would work?

That seems like it would be overkill. Why not modify pkgtool a
bit so that the user can pass parameters to "./configure", "make",
"make install" if they need to have the package installed in a
particular place. If something like install watch is included, one
can keep track of where things went, so if we need to remove
that package, we can. You've already touched on some of
this below.

Quote:

source install manager Maybe I am missing something here but to me installing from source is easy, it is removing it or upgrading later that is a problem. It is very difficult for me to see how a source tgz would work since compile options would make a difference in the package contents. The flip side is that if a package creator can be made sufficiently robust it also solves this issue in that you install from source and a variant of the package creator creates a "phantom package" for uninstalling based on it's results. Regardless I think source based packages have less value to your average user than a way to retrieve binary packages which leads us to:

Distributed Content Manager an apt-get style DB to retrieve packages from, or in my eyes somewhere between apt-get and linuxpackages.net. I gave this a lot of thought and my conclusion that a central repository of packages was unrealistic. What is needed is an automated way of discovering a package location based on criteria (probably configured by ini file) The problem with a centralized DB approach is maintenance and scalability, both can be achieved but not without a community (for maintenance) and funding (for scalability, specifically bandwidth costs) a p2p network is also an option and doable, the issue there is ensuring the quality of content and grazing/blight topology issues.

I agree that a central repository of packages is unrealistic. What
may not be unrealistic is a central repository of "package
descriptions". Something that includes where to download the
latest version, its dependencies, etc. If we get it started and it's
useful, it should be able to be updatable by the package creator,
or by the public.

Quote:

all of this does not also address the other "issue" which is
Dependency Checking I have no idea there, but I also never had a serious problem with handling dependencies by hand.

I've never had a problem, and now it is quite common for groups
to use "name-config" scripts. For example, one can call gnome's
config tool to find out where it was installed, or which version is
installed:

Code:

root@valhalla:~# gnome-config  --prefix
/usr

root@valhalla:~# gnome-config  --version
gnome-libs 1.4.1.7

While this isn't used everywhere, it's getting more popular, and
many auto configure scripts already use it.

Anyway, thanks for responding. . .

Mephisto · 01-05-2003, 10:00 PM

With the concept of the Package Creator I am not talking about a tool for the end user but as a tool for someone who wants to create packages for other people.

This kind of tool would lead to the ability to keep track of arbitrary file sets as "packages" in the Slackware sense in that you could upgrade/remove these sets of files from a Slack system though that was not my intent.

Quote:

My first goal is to set up a system that will allow an "easy" way to download, compile, install packages for (at first) the Slackware linux distribution.

If this is what you want why not just use one of the source distributions? Even the command you use implies a normal Slack package versus source. OTOH I can see having a

pkgget gnome

which retrieves all packages necessary to install gnome. a "package set" if you will. iOW on some data map have an entry that says "if they request gnome they are requesting these packages, install any missing upgrade those that are out of date." I am making in essence a variation on the argument in the original thread. What is the value gained in making Slackware a source based distro? Would it not be better to keep Slack in it's original packaging format (tgz)? Instead of changing it's packaging context, supplement it to help on dependency checking and installation of package sets?

The other, or possibly another, possibility that has potential is the extending of the pkgtool you mention to include more functionality on handling the install and removal of packages that are compiled at install time. IOW wrap a source for a program in a tgz with directions for how to install/remove itself. It should already be possible in a tgz, though creating such a package would not necessarily be straight forward.

In a nutshell... I was trying to come up with a summary but can't think right now. Time for bed.

Ja ne,
Meph

moses · 01-05-2003, 11:27 PM

What I'm talking about is not making Slackware a source based
distro. I'm talking about

Quote:

supplement it to help on dependency checking and installation of package sets

but make it useful for more than just Slackware. I have yet to see
a decent tool that allows this. Debian's apt-get is one that comes
close, but I don't like the Debian package manager, and it relies
on a central server. FreeBSD has something similar to this, but
it's not quite what I want, and it relies on a central server.
I have yet to see something that is portable to any linux distro.
RPM sucks because it can't see what's on your computer, just
what it's installed. There's no reason to require the user to only
use such a package manager. I've tried a couple other distros
that proported to offer such a thing (sorcery, grimoires, etc.), but
they all fell short of what I want, and Slackware is, by far, the
best distribution in my opinion.
A package creator is something that would be a big part of this,
since there are so few slackware packages out there, even
though it's currently pretty easy to make a slackpack. We would
want to make it easier for maintiners to create slackpacks.

NSKL · 01-06-2003, 03:14 AM

Theres also checkinstall (http://asic-linux.com.mx/~izto/checkinstall/index.php) that claims it wills create a slack, or a debian package from source and install it using make install. If we make a script similar to this, which can take the source, copy it to /tmp, compile it, then make a slackpack using makepkg, this will make more slackpacks available around since slackpack creation would be made easy.

Then, are we allowed to use any of the existing databases (Gentoo's or debian's) for this project? Because if we can, then we can use gentoo's source database, and write a program that will fetch source from it, compile the package and check for dependencies, make it into a slackpack and install. Basically we could rewrite Gentoo's package manager (i never used it but heard it fetches source from the database, along with any dependencies it finds, compiles it, and installs it) but rewrite so it creates and installs a slackpack. I dont know if this is a good idea, im just giving suggestions to you guys, you discard them, or decide if they are plausible for this project.

I'm not a big help, but if i can help in some way let me know, i would like to...
-NSKL

moses · 01-06-2003, 11:21 PM

The use of existing databases would be perfectly reasonable, but
the requirement that any central package server exist isn't. There
are internet tools available that can go to the packages' sites and
download the data. Another tool that may be a useful model is
the Loki demo and update tool. I haven't delved too deeply into
the source yet, but it looks like it doesn't require a central server,
just a URL for a package.
In the end, I don't care if the package is source or binary, what I
think would be nice is to be able to easily get the necessary
packages and their dependencies without having to spend
hours looking for them (gnome is the case in point, and yeah, I
know about dropline gnome). I don't have a problem finding out
what the dependencies are, but I do sometimes have trouble
getting those dependencies, and would like to make it easier.
I don't know if this will ever take off, but at least it can be
discussed. . .

Astro · 01-07-2003, 12:35 AM

Whats the deal with this pkgtool2? I didn't see that mentioned in anyones posts..

From what I can find it this...
==============================================
This is pkgtool2 v2.0.9. It's a supplement to pkgtool with six options.

This taken from the http://gd.tuwien.ac.at/opsys/linux/l...kgtool2.readme site

A friend of mine on #slackware on irc.undernet.org was talking about it, I had never heard of it before...

NSKL · 01-07-2003, 07:59 AM

Have you checked userlocal.com today? (Jan 07 )
They have a column about a program i have never heard of before, tgz-get:
"Description

TGZ-GET help to install and upgrade tgz packages on Slackware or Current from FTP with standard version (tgz-get) or CDROM and directory with menu version tgz-get-menu).

This program use wget, pkgtool 8.1 and dialog (only menu version).
wget: http://www.gnu.org/software/wget/
pkgtool:http://www.slackware.com/
dialog: http://www.hightek.org/dialog/
All these programs are in Slackware 8.1 and 9.0.

How to use?

TGZ-GET is very easy to use.You must install the TGZ-GET package with "installpkg" or "pkgtool". Then, you can use the program with the command "tgz-get" in root. Enter "tgz-get --help" for tgz-get help or "tgz-get-menu" for menu version of tgz-get."

It seems there are alredy quite a few projects aiming at improving slackware's package management. If Patrick could only talk with the developers and integrate at least one of these into slackware, it would become pretty much the main one, and if all slackware users have to use it, then more packages and will be created and eventually we will get something like a ftp.packages.slackware.com to fetch packages from via autopkg, TGZ-get or autoslack.
-NSKL

moses · 01-07-2003, 09:49 AM

The deal with tgz-get is that it's Slackware packages specific, and
it's got all the disk names hard coded into it. It's basically just a
way to go out and update your Slackware install. I'm thinking of
grander things like adding packages that aren't a part of the
Slackware distribution.

moses · 01-07-2003, 09:55 AM

Astro:
That's a good tool, hadn't heard of it before. Thanks! It's not
everything I envision, but it's a good start for setting up configure
rules and the like, and for using installwatch to make a tarball
package for distribution later. . .

Astro · 01-08-2003, 01:55 AM

Yeah, thanks to Rudeboy in #slackware on irc.undernet.org for that... He said he's been using it for a while and absolutely loves it, so I thought why not mention it here, I didn't know if anyone else heard of it or not. At least that gives you a little more of an idea of what it has/does and what else there is out there.