Package building framework

polch · 03-08-2011, 03:45 PM

Hi.

Why isn't there any package building framework ?

It could be a set of functions (downloading from source, from git/svn/..., extract, patch, etc...), it could be a set of variables plus some external scripts, etc...

I have begun to read Arch Linux doc, and they seems to have such "framework".

Could you give me the opinion of maintainers on this subject please ?

bgeddy · 03-08-2011, 04:26 PM

I don't know whether you'd qualify any of my suggestions as "package building frameworks" however have a look at solutions such as Gilbert Ashley's src2pkg, Chess Griffin's (and other's) sbopkg and also Dave Woolfall's mkslack. All very useful tools - maybe "frameworks". Apologies if I've missed anyone's package building solutions out.

Edit: Perhaps I should have accredited sbopkg with it's other authors being Mauro Giachero and slakmagik. Apologies to both - no offence intended.

sycamorex · 03-08-2011, 04:39 PM

Quote:

Originally Posted by polch

It could be a set of functions (downloading from source, from git/svn/..., extract, patch, etc...), it could be a set of variables plus some external scripts, etc...

What about slackbuilds?
http://slackbuilds.org/howto/

It's a set of instructions on building a package.

Don't know if it qualifies as a framework, though.

polch · 03-09-2011, 01:42 AM

I have read the Writing_A_SlackBuild_Script from the slackbuild site, and the http://www.slackbook.org/html/book.h...AGE-MANAGEMENT section of the slackbook. But it seems that every one is free to write the script anyway he wants, but that he can't reuse kind of "macro" for repetitives task we can find in every scripts. Every one who write a generator for slackbuilds could generate different structures... A slackbuild script seems to has only one requirement : generate a package. No naming convention, no predefined variables, no restriction on external programs, etc...

So the tools you suggest are not really the framework i would imagine....

gnashley · 03-09-2011, 02:19 AM

src2pkg definetly fits your definition of 'framework'.

Ramurd · 03-09-2011, 03:07 AM

If every program would compile the exact same way, such a thing would be practical

./configure is seen most of the time, but the options to it differ.
make install DESTDIR=/foo/bar same, but not all programs follow this defacto standard; sometimes it's ISNTALLDIR=/foo/bar or make prefix=/foo/bar install

I can continue for a while, so the various projects all use different standards.
pkgtools "recognize" only one standard naming convention, hence packagers will follow this standard as the naming convention. As with many more things, Slackware doesn't hold your hand here.

There are frameworks many people use and find useful. src2pkg is one that would fit your description. I prefer to do things a little bit different, and developed my own slackbuild scripts which are function based. Most of the times I only have to set the application name, version and the parameters to configure. But often enough I have found that a standard does not work.

Also, people will wan their own options to applications; for example php: an src2pkg or any framework would not do (for me), as I want to compile things in, that others don't want. Also the default package doesn't provide what I want, so I have to build it for myself. I never wanted a one-flavor-for-all approach.

ruario · 03-09-2011, 04:26 AM

Quote:

Originally Posted by polch

I have begun to read Arch Linux doc, and they seems to have such "framework".

Since you used Arch you might like slkbuild (not to be confused with SlackBuilds). slkbuild is "An Arch-like tool for easy Slackware packaging" and is used extensively in the Slackware based distro SalixOS. The format of the SLKBUILD build/packaging meta-files is very to that of Arch PKGBUILDs but will generate Slackware packages.

ruario · 03-09-2011, 04:45 AM

Correction: It actually generates a build script and that build script generates a Slackware package. Which is handy as you can share the build script with someone who does not have slkbuild installed.

See also: http://www.salixos.org/wiki/index.ph..._with_slkbuild

GazL · 03-09-2011, 04:49 AM

What I like about the slackbuild scripts is that as everything is in-line, nothing is hidden from you and is much easier to customise. If you know shell scripting, you're set. No need to learn any extra functions or macros, and the templates that are available give you a good starting point.

Also, if using an external framework, you also need to worry about what version of it is available. Slackbuild templates are self contained thus avoiding this consideration.

ruario · 03-09-2011, 05:10 AM

Edit: Others have already said the following. This is just my own spin

Alternatively, if you don't want to go down the slkbuild route because you don't like relying on a third party tool, then you may find these useful for ensuring you make high quality build scripts.

http://slackbuilds.org/templates/
http://www.slackwiki.org/Writing_A_SlackBuild_Script

The templates are particularly handy. All the boring stuff like untarring, setting man/doc install directories, setting permissions, stripping binaries, compressing man and info pages, removing unneeded files, etc. will probably just work as is. Generally you only need to change a few variables and do a few other minor tweaks.

I would argue that the Wiki and templates are as close to an 'official' framework as you are likely to get, given the involvement of Slackware team members (like Robby Workman) in their creation and maintenance.

dugan · 03-09-2011, 08:28 AM

There's also slacktrack.

psionl0 · 03-09-2011, 11:19 PM

To create a package creator that conforms with slackbuilds.org guidelines, you need to create several files and before running the slackbuild script, you need to manually download the source tarball.

If you prefer a slackbuild that does all of the work in a single file, you might like to have a look at https://slack.sarava.org/slackbuilds/. The scripts there would make a useful template for one of your own scripts.

polch · 03-10-2011, 01:48 AM

Thank you for your replies.

Quote:

What I like about the slackbuild scripts is that as everything is in-line, nothing is hidden from you and is much easier to customise

One one hand i share this point of view.

On the other hand, i maintain my own embedded system builder, a kind of very basic openembedded or buildroot (if you are curious, here is the source http://paul.chavent.free.fr/sdk/sdk.tar.bz2). And as i have scripts for building my packages, i wondered if i could stick to an already existing convention. As i use slackware as OS, i asked you this question.

For an example of what i currently do :

Code:

#!/bin/bash

################################################################################
# Setup framework                                                              #
################################################################################

source env_pkg.sh

################################################################################
# Set configuration variables                                                  #
################################################################################

OPENCV_VERSION=${VERSION:-2.2.0}

OPENCV_LOCATION=http://freefr.dl.sourceforge.net/project/opencvlibrary/opencv-unix/${OPENCV_VERSION%.*}/

################################################################################
# build                                                                        #
################################################################################
opencv_build()
{
    # download
    pkg_download ${OPENCV_LOCATION}/OpenCV-${OPENCV_VERSION}.tar.bz2

    # extract
    pkg_extract OpenCV-${OPENCV_VERSION}.tar.bz2

    # build for target
    pkg_cd_before_build OpenCV-${OPENCV_VERSION}

    cmake ${PKG_SOURCES}/OpenCV-${OPENCV_VERSION} \
	-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE} \
	-DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_BUILD_TYPE=MinSizeRel \
        -DBUILD_EXAMPLES=OFF \
	-DBUILD_LATEX_DOCS=OFF \
	-DBUILD_NEW_PYTHON_SUPPORT=OFF \
	-DBUILD_TEST=OFF \
	-DENABLE_OPENMP=ON \
	-DOPENCV_WHOLE_PROGRAM_OPTIMIZATION=ON \
	-DWITH_1394=OFF \
	-DWITH_FFMPEG=OFF \
	-DWITH_GSTREAMER=OFF \
	-DWITH_GTK=OFF \
	-DWITH_JASPER=OFF \
	-DWITH_JPEG=OFF \
	-DWITH_PNG=OFF \
	-DWITH_TIFF=OFF \
	-DWITH_UNICAP=OFF \
	-DWITH_V4L=OFF \
	-DWITH_XINE=OFF

    $MAKE
    $MAKE install DESTDIR=${PKG_TOOLCHAIN_SYSROOT}

    pkg_cd_after_build OpenCV-${OPENCV_VERSION}
}

################################################################################
# pack                                                                         #
################################################################################
opencv_pack()
{
    local system_root=${PKG_TMP}/OpenCV-${OPENCV_VERSION}

    rm -fr ${system_root}
    mkdir  ${system_root}

    mkdir -p ${system_root}/usr/lib${LIBDIRSUFFIX}
    cp -d ${PKG_TOOLCHAIN_SYSROOT}/usr/lib${LIBDIRSUFFIX}/libopencv_core*so* ${system_root}/usr/lib${LIBDIRSUFFIX}/
    cp -d ${PKG_TOOLCHAIN_SYSROOT}/usr/lib${LIBDIRSUFFIX}/libopencv_ml*so*   ${system_root}/usr/lib${LIBDIRSUFFIX}/

    pkg_pack ${system_root}
}



################################################################################
#                                                                              #
################################################################################

opencv_build
opencv_pack

It's my "slackbuild" script for opencv.

and an other one different for the libdc1394 :

Code:

#!/bin/bash

################################################################################
# Setup framework                                                              #
################################################################################

source env_pkg.sh

################################################################################
# Set configuration variables                                                  #
################################################################################

LIBDC1394_VERSION=${VERSION:-2.1.3}

LIBDC1394_LOCATION=http://freefr.dl.sourceforge.net/project/libdc1394/libdc1394-2

################################################################################
# build                                                                        #
################################################################################
libdc1394_build()
{
    # download
    pkg_download ${LIBDC1394_LOCATION}/libdc1394-${LIBDC1394_VERSION}.tar.gz

    # extract
    pkg_extract libdc1394-${LIBDC1394_VERSION}.tar.gz

    # patch
    pkg_patch libdc1394-${LIBDC1394_VERSION} "build_juju_out_of_src_tree "

    # regenerate autotools files
    cd ${PKG_SOURCES}/libdc1394-${LIBDC1394_VERSION}
    aclocal
    libtoolize
    autoconf
    automake
    cd -

    # build for target
    pkg_cd_before_build libdc1394-${LIBDC1394_VERSION}

    ${PKG_SOURCES}/libdc1394-${LIBDC1394_VERSION}/configure \
	--prefix=/usr \
	--libdir=/usr/lib${LIBDIRSUFFIX} \
	--build=${HOST} \
	--host=${TARGET} \
	--disable-examples \
	--disable-doxygen-doc \
	-v \
	ASFLAGS="${ASFLAGS} ${TARGET_ASFLAGS}" \
	CFLAGS="${CFLAGS} ${TARGET_CFLAGS}"

    $MAKE
    $MAKE install DESTDIR=${PKG_TOOLCHAIN_SYSROOT}

    pkg_cd_after_build libdc1394-${LIBDC1394_VERSION}
}

################################################################################
# pack                                                                         #
################################################################################
libdc1394_pack()
{
    local system_root=${PKG_TMP}/libdc1394-${LIBDC1394_VERSION}

    rm -fr ${system_root}
    mkdir  ${system_root}

    mkdir -p ${system_root}/usr/lib${LIBDIRSUFFIX}
    cp -d ${PKG_TOOLCHAIN_SYSROOT}/usr/lib${LIBDIRSUFFIX}/libdc1394*so* ${system_root}/usr/lib${LIBDIRSUFFIX}/

    pkg_pack ${system_root}
}


################################################################################
#                                                                              #
################################################################################

libdc1394_build
libdc1394_pack

GazL · 03-10-2011, 07:48 AM

Quote:

Code:

# download
    pkg_download ${OPENCV_LOCATION}/OpenCV-${OPENCV_VERSION}.tar.bz2

    # extract
    pkg_extract OpenCV-${OPENCV_VERSION}.tar.bz2

    # build for target
    pkg_cd_before_build OpenCV-${OPENCV_VERSION}

That's exactly the sort of thing I don't like to see. Download to where? Extract to where? Without digging into your function library I simply don't know and if I ever needed to change the location
then that gets ugly

I'd have less of a problem with a framework that worked something like

Code:

pkg_extract --from "$tarfile" --to "$tmpdir"

as everything important is still visible to the reader. My philosophy on functions is that you should use them to hide 'how', but never 'what'.

If you plan to submit to slackbuilds.org, then they expect you to do it their way and use their templates to keep things consistent.
If you're going to make them available on your own site, or just for personal use, then use whatever you're comfortable with.

This type of stuff is all personal preference.

allend · 03-10-2011, 08:56 AM

From post #13, opencvlibrary has a BSD licence and libdc1394 has a GPL license, yet your framework does not appear to meet the requirements for redistribution.
Just one of those little annoyances that the Slackbuild maintainers might pick up.