Because this
thread has gotten a bit derailed, I thought it prudent to start again when I had some more thoughts on this topic.

This is sort of carried on from earlier discussions involving src2pkg, but this isn't strictly limited to src2pkg.

What I find a particularly nice feature of pkgtools is that no matter where you put files on install (/var/log/buildscripts, /usr/doc, /usr/src, et al), pkgtools will register the location. So to find a build script included with your package, grep /var/log/packages/$progname for *.SlackBuild or *.src2pkg and shazzam, you've got your answer.

On the flip side of this, I draw your attention to a comment that didn't really get talked about much by jong:

(Emphasis mine)

The whole point of a buildscript is to ensure repeatability, and by extension, gives you a log of how you got it to work. When I'd first heard about src2pkg, it gave me something that ./configure && make && checkinstall didn't, and that was the ./configure line saved as a file that I could use later on.

So on to the meat, then. Should buildscripts be a part of a binary package?

Actually, dispite my previous enthusiasm, I'm now starting to think: no, they shouldn't. Binaries should be distributed with that knowledge implicit. When you download one it should be clear what it's been built on top of, and therefore what it's expecting on a system to run. If you don't know the answer to those questions, don't use that binary.

The inclusion of a build script into a binary package is on the surface useful to some, but it would be better off placed in a build script repo to allow those people to build what they need rather than adding it to the filesystem to take up space, when they've already got a pre-built package installed ...

Finally, if the binary distributor (not necessarily the original developer) doesn't provide that information, should the developer? The gcc guys do:

In short: if an official slackware package doesn't do it, why should we?

If you want to distribute your buildscript with your package:

* tar them both up together but keep the buildscript out of the package
* provide a *-dev style package that includes the buildscript so people know they are getting a non-standard slackware tarball
* make the buildscript downloadable next to the package

Your thoughts then!

1. How do others (package managers) deal with buildscripts (I know some, but I've written too much already ... )
2. Is it a good idea to put your buildscripts in a binary package, or conversely, as a package users, do you care about the script that was used to build it once it's installed?
3. How do you like to interact with buildscripts? Downloads from trusted sources (AlienBob, SB.org, rworkman, etc) by hand, automated scripts to pull them (SBo, emerge etc), writing your own, or just don't care about the script?

I'm done, your turn
- Piete.

PS: Reading this back, some statements sound like opinion but are presented as fact; add "imho" to anything that looks like that.

Piete,

I pretty much only build from source either using slackbuild scripts (mostly from Slackbuilds.org) or src2pkg -- not counting the official Slackware packages I download to keep my system up-to-date. However, the couple of times that I used a different approach, I downloaded from repositories maintained by Robby Workman, Eric (Alien Bob) Hameleers, or Slacky.eu. In all three cases, the slackbuild scripts and sometimes the source code are available for download with the binary -- in the same directory tree, but separate from the binary package.

This is the perfect way to distribute Slackware packages. Doing so gives users the choice of simply installing the binary or building it themselves. Even if users choose to simply install the binary, they can read the slackbuild script to see how the program was built. I prefer that the binary only contain the program data and documentation since I manage the build scripts and source code myself.

Regards,

-Drew

P.S. Could you reformat your code example above? This page is really really wide.

No, it shouldn't be included, but people must be able to get buildscript if they want. If they don't want it, they don't need it. People who download compiled package, don't want buildscript.

Normally I create packages manually (tar -xzf , ./configure, make, make DESTDIR=1234 install, makepkg) and only use buildscripts when it's something complicated (doesn't support DESTDIR, uses hand-written makefile, unusual buildsystem, etc) and checkinstall can't handle it.
This is because I don't like the way normal slackbuilds work. They create build directory in /tmp and store finished package there (and also don't clean directories/compilation junk when done), so you'll have to move it from /tmp to place you need. Also scripts are sometimes messy and more complicated than they should/could be - 7 lines is normally enough to make package, but normal slackbuild normally contains several dozens lines. Also I don't like the fact that slackbuild should be run as root. There is "su" command and buildscript could just ask password for "privileged" operations (makepkg, anything else don't need it).

So, I'll chip in here since I'm the one who was de-railing the original thread. I've just worked the code for including build scripts into src2pkg today. But, the default will be to not include them. If the user chooses to include them, the default location will be in /usr/doc/$NAME-$VERSION. But the location can be configured and a link can be created in /usr/doc/$NAME-$VERSION to the real location if wanted. I think that for src2pkg, that covers the various otptions and attitudes on the matter.
As far as not including them at all -the binary packages I have on my site always have the build script there.
Side notes to ErV:
1. I believe src2pkg answers your complaints about using SlackBuilds. src2pkg scripts can be as short as 2-3 lines. Of course they are usually longer, but the format is really clean, short and orderly which makes it easy to see if any special changes have been made or not.

2. src2pkg also lets you configure where the sources get unpacked and built, where the packages are assembled and where the final packages are put. You can configure each of these separately and do it globally in the src2pkg.conf file or from the command-line. I prefer to have all temporary files and dirs in the same dir where the sources are as it makes it easier to examine them. But this would be an unacceptable default for sticklers, so the default is /tmp just like SlackBuilds.

3. src2pkg makes it possible to build nearly any package as a normal user. I experimented with using fakeroot and other methods for doing this. But, I found a better way that reduces the risks to a minimum. When the final apckage is created, a feature of tar is used which resets the ownership of all files to 'root' so that conformant packages are created. The only times this won't work properly is when the package contains directories which should belong to a special group(like daemons which run as user 'nobody'). even these can still be built if you include code in the doinst.sh which sets the proper UID/GID when the package is installed. And there are a very few packages which are almost impossibble to create as a non-root user because the Makefiles are so screwed up that none of src2pkg's install methods can properly track the file locations.

4. src2pkg provides superior performance compared to checkinstall and is able to more accurately log the creation of files and directories and any chown/chmod operation which are done to them. checkinstall drops the ball for some operations ever since Slackware-12.0.

5. src2pkg can accurately handle many builds which don't support DESTDIR -even some Makefiles which don't include an 'install' rule.

6. src2pkg can automatically clean up temporary files if you choose that option. The default is to *not* do this. This encourages the user to examine the uncompressed package content and preserves both source and package content in case something goes wrong during the build.

Any SlackBuild worth its' salt has to be much longer than 7 lines in order to assure a sane package. src2pkg relies on over 4,000 lines to do it even more thoroughly, but only the most interesting and important info is written into the script.

Now back to this thread...
I agree that most people who download third-party binary packages are not usually interested in the details of the build script. But they may sometimes be interested in the configure details as a way of knowing what to expect from the program -in cases where the program itself doesn't provide this info, and most programs do not.
I also agree that one should be careful about where they acquire third-party packages. Sites like linuxpackages.net where nearly anyone can upload packages should mostly be avoided. But even there you can get packages from a couple of fellows who are probably trustworthy. Knowing which fellows is the problem. And exactly because they don't always include the build script or even link to it, you have no way of trying to build your own package using their supposed options and method.

"if an official slackware package doesn't do it, why should we?" sounds reasonable at first. But really, exactly because we third-party packagers are not the trusted PatV, we should take extra steps to assure and/or assist the user who downloads our packages. If the build scripts were included, a paranoid user could still download the package, use explodepkg on it, and rebuild the package on their own machine.
I hate to point this out, but sometimes even the official SlackBuilds will not work. Usually this happens because a package was built on an earlier version of Slackware and is included in the current or stable version without having been rebuilt on that platform. The point is that having a build script is not all of the story unless your system exactly matches the system on which the original script was used. And even if it does -what about build order? Sometimes packages have to built in a certain order. Also, some SlackBuild scripts are a kind of afterthought. What I mean is that the create packages of stuff that was compiled using some other method or script -cases in point: kernel, kernel-modules, kernel-headers.

This is exactly why I stated in the other thread why I think SlackBuild scripts should always be included by default in third-party packages, but that's just my opinion. Having said that, I have not even done that myself for sbopkg so, of course, I fail to live up to my own standard. :-)

No offense, but when someone says a lot of good things about some kind of software I always get paranoid and extremely suspicious about real software abilities/quality and start thinking that review/opinion is completely biased. Suspiciously good "reviews" also provides predetermined negative opinion about software. Maybe it's automatic defensive reaction. Anyway, I've been thinking about making my own script for compiling packages on my machine. Just simple-like-a-brick thing (bash or python) that'll guess current package installation scheme, and then perform operations I normally do manually. The script should take only one argument - the name of source code.

Anyway, about src2pkg (here is some criticism, everything is my own opinion):
1) it's very strange that it has 5MB download size.
2) folder in /usr/doc/src2pkg-1.9.6 has 644 access right, which means it's unreadable for everyone except root. Maybe it's fixed by doinst.sh script, or by installpkg, but it looks like it isn't.
3) Installation of src2pkg doesn't seem to comply with default behavior of slackware package:
3.1) Looking at the scripts it seems that src2pkg is required to compile tools for itself. This is WRONG. Normal slackware package is ready to use right after installation, and doesn't require specific manipulation. If program is required to be compiled, don't provide it in *.tgz form, and make some kind of other compilation mechanism, (similar to commercial installers) when main program will be provided as executable bin file which will extract itself and then compile slackware packages for this particular machine. Nvidia drivers make machine-dependant in very good way, just don't install stuff after compilation.
3.2) If package is marked "noarch" this normally means that it is python, java, perl, bash-based software that can be run right after installation - i.e. it's (semi-) interpreted language, independant to platform. Compiling libraries for itself is incorrect. Better idea would be to provide "normal" *.tgz package for current kernel version and source for everyone else.
3.3) Placing archive within archive is also incorrect. If utilities are required by main package, they should be either provided separately or included into package which won't be "noarch" anymore. Otherwise we get a ridiculous picture when seemingly bash-based software takes 5 megabytes of space. This archive also isn't required for normal program operation, it shouldn't exist after installation step. If it is removed after src2pkg setup, this is also incorrect behaviour, because installed program should never modify it's own installation, only user data.

So far it seems too much for making packages (doesn't follow "keep it simple" principle). Most installations can be handled by much easier methods.

There are other build mechanisms.
1)What about making python modules into package? These normally requires running "setup.py".
2) What about scons-based packages (blender)? Those normally have NO install rules, NO makefile and often can't be installed system-wide without using tricks.
3) What about compiling linux kernel? compiling kernel modules? There is a makefile, but installation step is usually manual.
4) What about qmake - based software? This also doesn't have makefile. There also several versions of qmake - qmake for Qt 3 and qmake for Qt 4. Also, qmake for Qt 4 might be not included into path, might be located in another place, and might be named differently.
5) what about dwd+rw tools? (cdrecord & co) They use their own "schilly make" build system.
6) what about cmake?

For 95% of packages following produces "sane" package (it's larger than 7 lines, but that's not 4000):
13 lines.
It will fail on anything that doesn't support makefiles and destdir, of course.

Providing digital signatures for packages should be good enough. And 3rd-party-created packages should comply with default behavior for packages.


doesn't anyone use mc to read tgz contents? using explodepkg isn't necessary.

This is sound reasoning and I whole-heartedly agree. Binary packages should not be provided without some sort of extra layer. The point I was getting at is that while we have a responsibility to take those extra steps (ie: providing the buildscript), it's not a package issue, per se, but a distributor issue. Should the buildscript be provided but not as part of the installable item?

Great discussion, keep it rolling.

Normally, build script should be provided somewhere separately, and package should be GPG-signed by trusted party. This is how it's done in original slackware, and that's should be enough for anyone paranoid.

Of course, "trusted party" being the operative phrase there. Downloading binary packages that have been GPG-signed by PV from ftp.slackware.com is one thing, but downloading binary packages from a random site that have been GPG-signed by a random person is another. Just because a binary package has been GPG-signed doesn't provide any assurance that the package was built properly or in a secure manner.

I look at this as a cost-benefit analysis. The 'cost' of including a SlackBuild script is a minimal amount of disk space; the 'benefit' is twofold -- peace of mind to address the security issues and tweakability/learning for those who want to modify the build or see what configure options were chosen. The benefits outweigh the costs, IMHO.

I stand by my original point: SlackBuild scripts should always be included in third-party binary packages and they should go in /usr/doc/$PRGNAM-$VERSION.

It is simple.
If you provide a binary package of GPL-ed source code, you are obliged to make the original sources available at the same location where you offer the binaries for download.
This includes the scripts used for building the binaries. Such as the SlackBuild script used to create many of the 3rd party Slackware packages.

I use this approach: next to the "pkg" directory with the actual Slackware packages, I keep a "build" directory which contains everything you need to re-create this package. There are cases of patent-encumbered software like LAME where I omit the source tarball because it will not be allowed on the Slackware server, but my build scripts will automatically download any missing source anyway.

I also add the SlackBuild used to create the package into the /usr/doc/$PRGNAM-$VERSION/ directory of the package. This serves an important purpose. The SlackBuild script for a certain piece of software will evolve over time, and as many surely have found out it is not always simple case of updating the VERSION value to build a new package when upstream releases a new version of their software.

For this reason, I add the actual SlackBuild script used for the package to that package, so that you can download an earlier version of a certain package and will still be able to determine how the package was built. The fact that this script will be contained in a binary package which is GPG-signed by me will ensure that this is the original script that I ran and not some hacked-up version someone else added at a later date (think explodepkg/makepkg).

The SlackBuild scripts you will find in my (and other) repository at any given time, will be the versions used to build the latest release of a package. Without the existence of the SlackBuild script inside the doc-directory of the package, it would be impossible for visitors to determine how older versions of a package were built. Unlike past Slackware releases, I do not keep older versions of packages plus source available for download. I do use a revision control system for my scripts but that is of no use to other people.

Eric

Thanks Eric, for weighing in on this subject. I'm convinced that most Slackware users who have been around for awhile trust your judgement and consider you as a trusted part of the Slackware team.
You brought up a couple of points which I didn't mention in order to save space. I think your reasoning is perfectly sound. On my site I always have the complete set of build-materials for each package available in the same directory. Since I *do* keep older versions around with the complete build-set, I haven't been including the build scripts in the binary package. In fact, I don't even use gpg or md5sum. My site is not really pointed at current Slackware users -I mean I don't push myself as a supplier of third-party packages. I encourage people to use my build scripts, patches and other materials to build their own package.

Most of my src2pkg build scripts do not include the necessary code to download all the needed material -this feature wasn't available in src2pkg until less than a year ago. It is now possible, though, and I have a few example packages which show how a user can create and use such scripts. 'src2pkg URL' can download the build script itself which then downloads the sources and extra materials, when the src2pkg script has been setup for that. Makes it pretty easy to create your own repository if you wanted, or do builds of complete sets of interdependent programs.

My site is full of mostly non-mainstream stuff and don't expect or hope that many people download the binary packages. The main thrust of my site is as 'heaven' for GTK1 software and a place to keep exampls src2pkg scripts for hard-to-package or hard-to-find software. Of course, the materials on my site are closely-coupled to src2pkg itself, but even people who don't use src2pkg can use all the other materials which are there and the src2pkg scripts are one of the most concise ways to document any regular configuration information and/or non-standard aspects of the build.

Regarding the topic of the thread, I have always tried to have src2pkg do the sanest and least intrusive thing by default, while still allowing for lots of flexibility so the user can have it do things the way they want.

ErV, thanks for telling me about the perms on /usr/doc/src2pkg-1.9.6. Funny though, that 'tar -tzvf src2pkg-1.9.6-noarch-3.tgz shows this:
drwxr-xr-x root/root 0 2008-05-05 03:56:32 usr/doc/src2pkg-1.9.6/
I'm sorry, but it shows that you haven't looked into src2pkg much yet. In the included HOWTO.txt:
Supported formats:

a. autoconf sources -most commonly available sources use the
'autoconf/automake' system for configuration. src2pkg handles
these automatically -even when the sources are incomplete or
outdated. Incomplete sources are sources from CVS or SVN which
must have their configuration files generated first before
running 'configure'. Many sources also use outdated configuration
files which are incompatible with the versions of autoconf and
automake which are installed on your system. src2pkg recognizes
these and runs extra commands to update them first. It also
properly handles sources which use GNUmakefiles instead of the
more common Makefiles.

b. src2pkg automatically handles sources which must be configured
using cmake

c. It also handles sources which use the 'scons' SConstruct system.

d. It also handles sources which are configured to use the 'jam' system.

e. Sources which use no configuration and contain pre-made Makefiles
are also handled automatically. src2pkg can even correct the hard-coded
installation paths when these are different than your default choice.

e. src2pkg also handles sources whch use the Imake system for configuration.

f. It can also automatically handle creating packages from python
modules which use a setup.py script

g. And it can also automatically handle sources which are normally
installed by running an install.sh script.

h. binary packages -src2pkg automatically handles the conversion of
binary archives which use the .rpm or .deb suffix. When these are
unpacked, src2pkg recognizes them and converts them to the Slackware-
conformant format regarding placement of files and ownership and
permissions of files and directories.

i. generic binary content - Some programs sucg as the Opera browser
are delivered as pre-compiled generic content. src2pkg can recognize
these and create a package from such content.

j. source code with no installation routine- Some sources don't come
with any Makefile rule or script to do the installation for you. Many
times src2pkg is able to create a package from them, finding the binaries,
included documents and man-pages and packaging them for you.

k. conversion/verification of Slackware-type packages- src2pkg can be
used to repackage or verify the content of packages which already are
in the Slackware format. This may sound strange at first, but it can be
used to verify packages made by others which you have downloaded. A couple
of users have reported using src2pkg to verify packages that they have
made using other methods -src2pkg is able to scan the content for comnmon
mistakes such as misplaced documents/manpages and also correct wrong
file or directory permissions and ownership

l. perl/CPAN modules- src2pkg automatically handles perl or CPAN modules
which are built using a Makefile.pl file.

Sorry, I don't ever install KDE or use KDE/QT stuff excpet static-opera, so I have never gotten around to even see how to use qmake, although I think that this already works. src2pkg lets you specify the command and/or arguments for each of the normal 'configure', 'make' and 'make install' commands. Something like this:
'src2pkg -c=skip -m='make linux' dvd+rw-tools-x.x.xtar.bz2

The next release of src2pkg, which will be out any day now, includes example src2pkg scripts which make kernel and/or kernel-modules building and packaging extremely easy without having to do anything as root.

Regarding the way that src2pkg is itself packaged. It is so large because it contains the sources for some libraries and bins which are needed for it to work properly. I do provide binary packages of the libs and bins(src2pkg-helpers) for various versions of Slackware. But I encourage users to just download the main sr2pkg package. It is a noarch package since it comntains no arch-specific compiled items. Installing the package allows you to bootstrap src2pkg for use on your system no matter which architecture, kernel,glibc or OS version you use. Some people use it on amd64, ppc and other platforms. bootstrapping it simply creates and installs the separate sr2pkg-helpers package natively on your system. The original content of the src2pkg package remains noarch and, yes, partly source.
It was not easy to get this to work and have it be palatable and semantically correct. But remember, Slackware also has source packages.

I am still not quite satisfied with the packaging of src2pkg, myself. I have thought about having an option to remove the src2pkg-helpers sources once the main package is installed. I do not like the idea of using an install.sh method which complicates the distribution. Basically, if you already have pkgtools installed and have a working development environment, I want you to be able to download and install one package and go from there. For a couple of releases, the bootstrapping was done as part of the installation (from doinst.sh), but that was too 'far out' so I took that out and made 'srcpkg --setup' a purely post-install operation. Actually you don't even have to do it separately -if you are running as root the first time you try to use src2pkg, it will bootstrap itself before building the package you asked for. If you try at first as a non-root user, you are prompted that you must login as root and run 'src2pkg --setup'.

I know that src2pkg is an 'unusual' package, but I think it is conformant enough for those who choose to use it. Your 13 line script will for around 40-60% of sources, depending on your software tastes. Anyone who has written 'real' SlackBuilds will right away spot the weaknesses in those 13 lines. Much of the 4,000+ lines of code in src2pkg deals with the other exceptional cases which can't be correctly handled with such a simple build.

Sorry again for sidetracking the line of this thread somewhat, but since my software is meant to correct misconformancies, I remain ineterested in what anyone has to say about the topic.

Qt-based programs normally have *.pro file (project file). To compile program, you'll need to run "qmake programname.pro", which will generate Makefile. Problem is that there are two version of qmake - one for qt 3 and one for qt 4. Both are named "qmake". Another problem is that qt 4 doesn't install its qmake into path, so it normally resides in /usr/local/Trolltech/Qt-4.x.x/bin. Simple qt 4 program don't use any external files, so it simply can be copied into any bin directory.

Lol!!

By the way, my osn post was purely to explain how I do things, it is not meant to be regarded as a new requirement for SlackBuilds.
Basically, as long as a packager makes sure his package sources are available to the users of his packages, it's OK by me. To everyone his own.

Eric

Hi all,

This has been a great discussion so far.

Gilbert: I'm looking forward to the next src2pkg!

Chess: I'm actually reviewing sbopkg for Linux.com right now. It's good software. I've got some questions and a feature request that I'll post on the sbopkg group before I submit my article.

ErV: Your comments are particularly insightful. They present a different perspective/experience than my own. In the past couple of years I've learned just how easy it is to compile software and how hard it can be to do so correctly.

I was pretty cocky about compiling software until recently when I built QGIS. For the first time, I had to compile more than one or two extra libraries -- QGIS required more than a dozen dependencies, with a couple of those also having dependencies I needed to compile. I used slackbuilds (mostly from Slackbuilds.org) and src2pkg to build nearly everything I needed. Without those two resources I would not have been able to build QGIS in a reasonable amount of time.

As a Slackware user, I expect to have to do a lot of work on my own, and I wouldn't have it any other way. However, I also appreciate that, for example, Aleksandar Samardzic has figured out in his slackbuild how to get QT4 to install and work correctly in a default Slackware 12.1 installation without conflicting with QT3.

QGIS itself was a bit of a pain in part because it uses Cmake. Cmake's documentation is okay, but QGIS's build instructions for Linux are not very helpful. Again, a combination of a slackbuild and some help from Gilbert on src2pkg helped me build the package.

If I read your comments correctly, you are asserting that user aids like slackbuilds and src2pkg shouldn't really be necessary. I disagree. I have found them to be exceptionally helpful.

Back to the original topic: I don't expect the slackbuild or other build information to be packaged in the binary, but I don't object to it either. I like Eric's approach where he puts the build information in the program's documentation directory.

Since I generally do not simply download and install binary packages, I already usually have the source code and build scripts (or the build script I generate with scr2pkg), so putting that info in the binary package is redundant.

Regards,

-Drew

Assuming the environment is set so qmake is in your path:
src2pkg -c='skip'-m='qmake programname.pro' qt-based-programname.tar.bz2
might work for compiling such stuff. That's if the configuration step is not needed. If the qmake system is like cmake which only uses cmake to configure and make to compile then:
src2pkg -c='qmake programname.pro' qt-based-programname.tar.bz2
might do the trick.

Eric >> LOL