LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux From Scratch (https://www.linuxquestions.org/questions/linux-from-scratch-13/)
-   -   Custom pkgtools for the toolchain (https://www.linuxquestions.org/questions/linux-from-scratch-13/custom-pkgtools-for-the-toolchain-913642/)

jimmy_page_89 11-15-2011 07:09 AM

Custom pkgtools for the toolchain
 
Hi guys,
i have modified the pkgtool scripts to have t{g,z}x packaging support on LFS.
This is ONLY for the toolchain system (for now..)

https://github.com/jpage89/LFS_pkgtools

Changelog:
Code:

upgradepkg:
 - Removed /sbin when installpkg and removepkg are called (it calls installpkg and removepkg from the first dir of PATH, ergo /tools/bin)
 - Changed input of root dir (upgradepkg --root foo instead of ROOT=dir upgradepkg foo)
installpkg:
 - Removed warn, infobox, terse, menu, ask, priority, tagfile, md5sum options (i think we don't need this, i keep just the --root)
 - Adjusted output (i removed the slack-desc support)
 - Removed tar control (tar-1.13 or tar-1.blahblah, is it really important?)
removepkg:
 - Changed input of root dir (removepkg --root foo instead of ROOT=dir removepkg foo)
makepkg:
 - Added auto-fixing non-gzipped pages (to have a slackware-style gzipping man and info)
 - Removed description (same as installpkg)
 - Added strip option (auto-stripping binaries with --strip y or -s y)

I know that there are less flexible than original, but i think they are more useful than.

Usage (if you don't know it): when you have to do "make install", just create a temporary dir (such as /tmp/foodir) and launch
Code:

make install DESTDIR=/tmp/foo
make the package with
Code:

cd /tmp/foo
makepkg -l y -c n -s y /root/foo-versionnumber-arch-build.txz

and install it
Code:

installpkg --root $LFS /root/foo-versionnumber-arch-build.txz
I don't think that pkgtool and explodepkg are needed (we're just compiling, DAMN!)
Just copy the scripts in /tools/bin and launch a chmod +x on it.

PS: In a second time, i'm going to modify pkgtool for using into the LFS main system to make it more comfortable ;)

Every bug report and patch is well-accepted! :D

magiknight 11-15-2011 10:03 AM

Sorry if this sounds ignorant, but what are the benefits of using this over Tukaani pkgtools?
It's great work you have done, but why? Why change the pkgtools Slack has already? It's not like they impose Slackware limitations on your system. Same with Tukaaki's pkgtools.

I am developing an Linux flavor based off LFS that will eventually be it's own monster; thats why I ask these questions.

jimmy_page_89 11-15-2011 10:31 AM

Hi,
i don't know tukaani pkgtools, but i have read now the page, and they have too much things.
My goal is to write an essential pkgtool suite to give a way to install-remove packages from a LFS, from the toolchain to the main system.
I don't want to add things.
Slackware's pkgtools also have too much things, that are useless inside an LFS (like slack-desc)
And they use some commands (like rev) that they aren't into the toolchain, so i have to write some workarounds.
Original pkgtools are unusable on a raw LFS.

jimmy_page_89 11-15-2011 11:50 AM

Ok, guys, now i have the first problem
when i try removepkg, all works, but at the end i have this:
Code:

WARNING: Unique directory /usr/ contains new files
WARNING: Unique directory /./ contains new files

On the head, i have this
Code:

find: warning: you have specified the -maxdepth option after a non-option argument -type, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it).  Please specify options before other arguments.
Any ideas?

jimmy_page_89 11-16-2011 01:32 AM

Ok, now i've solved some issues:
I have to create a package that recreate the tree in / (the manual creation was the cause of the warnings)
The /bin/sh must be modified in /tools/bin/sh ('cause i don't create the links manually, and i don't need it)
Now there is only one "problem": removepkg works fine, but i have this output:
Code:

WARNING: Unique directory /./ contains new files
it doesn't make anything, but is so UGLY

If you want to help me, all the files are on the github

ruario 12-22-2011 09:13 AM

The problem is that makepkg is expecting to use tar 1.13 and this version of tar stored sub directories of '.' differently. Here is a comparison so you can see the difference.

GNU tar 1.13:
Code:

$ tar -cvf ../test.tar .
./
usr/
usr/bin/
usr/bin/example
install/
install/doinst.sh

GNU tar 1.26:
Code:

$ tar -cvf ../test.tar .   
./
./usr/
./usr/bin/
./usr/bin/example
./install/
./install/doinst.sh

You removed tar from your custom version but did not account for this change. This means all the packages you have created thus far are invalid. :(

On the plus side it is relatively trivial to work around this problem using tar's transform option and a regex to make modern tar save the directories in tar 1.13 format, e.g.:

GNU tar 1.26:
Code:

$ tar --transform="s,^\./\(.\),\1," --show-stored-names -cvf ../test.tar .
./
usr/
usr/bin/
usr/bin/example
install/
install/doinst.sh


knudfl 12-22-2011 09:23 AM

The Slackware 'pkgtools' can be used on any Linux OS.

Just make sure, you have an executable copy of the binary 'tar-1.13' in /bin/.

ruario 12-22-2011 09:27 AM

By the way, you could use spkg (Edit: the source package offered by the main site seems to be broken right now. You can fetch a non-broken version here: spkg-1.0.tar.gz) instead of your patched Pkgtools. It is a reimplementation of Slackware pkgtools optimised for speed (it is very fast) and is actually used by some Slackware derivatives, like SalixOS. Because it is an actual program and not a shell script it has a few extra dependencies but they are not outrageous. Additionally it has an option to compile statically. That way you can make a binary on a machine that has the required dependencies and then move this to your more minimal LFS setup and it will continue to work whether those libs are present in your LFS install or not, meaning you can use it right from the beginning. ;)

The only thing that spkg is missing is a makepkg equivalent but it is possible to do without this. The two major things that makepkg does for you are converting symlinks to shell script code (in the format that pkgtools/spkg expect) and making sure you output tar packages that use tar 1.13 directory formatting.

Since you already know how to do the latter, here is one way to do the former:

Code:

find . -type l -printf "%f@-&%h@-&" -exec readlink {} \; | sed "s,^\(.*\)@-&\(.*\)@-&\(.*\)$,( cd \2 ; rm -rf \1 )\n( cd \2 ; ln -sf \3 \1 ),;s,\(( cd \)\./,\1,g"
Just save the result in your doinst.sh and then delete all symlinks before you tar up into a package. An easy way to delete all the symlinks would be:

Code:

find . -type l -delete
Here is a minimal makepkg that I quickly knocked up that implements these two ideas.

Have fun! :D

ruario 12-22-2011 09:38 AM

Quote:

Originally Posted by knudfl (Post 4556210)
The Slackware 'pkgtools' can be used on any Linux OS.

Just make sure, you have an executable copy of the binary 'tar-1.13' in /bin/.

You don't need it. You just need to use a transform in makepkg to force tar-1.13 directory format and to alter 'installpkg' to not use '-l' '-U' on extraction.

There is a whole thread about why tar 1.13 is used. It is long but an interesting read.

The key bits if you can't be bothered to read it all are that you can use a modern tar if you make those changes. Patrick doesn't do this as it is less maintenance to stick with one version of tar that reacts exactly as his tools expect, rather than having to update them again every time the GNU tar authors decide to change (or break) something.

ruario 12-22-2011 09:41 AM

He he, just re-read your original post and noticed this:

Quote:

Originally Posted by jimmy_page_89 (Post 4524552)
Hi guys,
- Removed tar control (tar-1.13 or tar-1.blahblah, is it really important?)

Yes, as you discovered. You shouldn't remove stuff like that before you understand why it was done in the first place. ;)

ruario 12-22-2011 03:07 PM

Quote:

Originally Posted by jimmy_page_89 (Post 4524715)
Slackware's pkgtools also have too much things, that are useless inside an LFS (like slack-desc)

I see little point in removing this. slack-desc is optional anyway. Just don't write them if you don't need them. However, if for some package of other you decide you do want to record notes (even if it is just a link to the website) you can use the slack-desc.

Quote:

Originally Posted by jimmy_page_89 (Post 4524777)
On the head, i have this
Code:

find: warning: you have specified the -maxdepth option after a non-option argument -type, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it).  Please specify options before other arguments.
Any ideas?

Swap the order of the '-type f' and '-maxdepth 1' find options in removepkg. I had to do the same thing when I customised pkgtools myself. It will get rid of the warning.

ruario 12-22-2011 03:24 PM

Quote:

Originally Posted by jimmy_page_89 (Post 4525244)
Now there is only one "problem": removepkg works fine, but i have this output:
Code:

WARNING: Unique directory /./ contains new files
it doesn't make anything, but is so UGLY

If you want to help me, all the files are on the github

Quote:

Originally Posted by ruario (Post 4556202)
You removed tar from your custom version but did not account for this change. This means all the packages you have created thus far are invalid. :(

Sorry for the multitude of replies but a workaround for this suddenly occurred to me. You could fix your log files using the same regex I suggested in the tar transform. Previously installed packages would then uninstall cleanly, without a warning.

I haven't tested the following but I am 99% sure it will work, nonetheless backup the files in /var/log/packages/ first, then issue the following as root:

Code:

sed -i "s,^\./\(.\),\1," /var/log/packages/*
Now the logs will look like they were installed with packages made by tar 1.13 and hence will remove cleanly without warnings.

jimmy_page_89 12-28-2011 01:45 PM

Hi guys,
my goal is to use pkgtools with tar-1.26 without add anything
spkg wants Judy, so i exclude it
@ruario your solution with --transform is so clean, thank you. I will adopt it.
For the find problem, i'm arrived to your same solution, so the problem is fixed.

Just in case, if you have seen oh github, i've added:
- auto-stripping option (-s y|n)
- "--root" standard parameter (now i don't remember which ones, but one of them wants --root, another one wants ROOT=/mnt/blahblah foopkg, i think it was soooo ugly)
- auto-fixing for man-pages, info-pages and doc-pages (i put it respectively in /usr/man, /usr/info and /usr/doc, and i've put an auto-gzip for man and info pages, in fully Slackware-style)

Now i want to edit the "pkgtool" script to remove all unnecessary things and have a comfortable tool to select the package you want to remove. It will works on a working LFS (not toolchain).
I'll keep you updated. Stay tuned!


All times are GMT -5. The time now is 10:53 AM.