Oh, that is so good, thank you slalik

Glad I could provide something back

That's a bummer. Luckily, I don't think that will bother me too much. I've been building everything as root (no fakeroot set up), so if the permissions of root's home directory get chowned, even to another user, I don't think it will cause me much grief. But that's good that you're aware of it and looking for a fix.

Would there be a possibility to just keep the ccache within the chroot, but then copy it outside of the chroot to the home directory before the chroot is destroyed? Actually, I guess you wouldn't even need to do it before since all the changes would still exist in the tmp folder (maybe... I can't remember how/when/if you clear those directories used for the overlay directory). Could it be so simple to just copy the contents of $MYTMPDIR/changes/$HOME/.ccache/. to $HOME/.ccache/? That should still keep everything protected but provide you with the benefits of ccache.

Well, if you change your mind, I'm willing to throw some money your way. I do it to other people that have benefited me, so I would certainly have no problems sending some to you (between this and your time you spent helping me grasp git, you deserve more than I could send).

2 members found this post helpful.

Hi,

Here's my wishlist ;-)

• handling of build-time and run-time only dependencies. Some packages depend on another package only for e.g. building the docs and don't have to be installed on the target system. This option would probably mean not placing the dependency in *.dep file. Then, run-time dependency would mean that the package does not need to be rebuilt if the run-time dependency is updated.
• handling of circular dependencies. Some way of describing when/how many times the dependencies should be build/installed.
• possibility of having source code tarballs in single/flat location,
• solution not to rebuild the package after git commit, i.e. I modify SlackBuild (git is dirty), I test the SlackBuild and if it's fine, I commit. But after the commit, even if the committed SlackBuild is the same as the dirty one, then slackrepo will issue rebuild of it.

For replacing/removing the REQUIRES dependencies I support the idea of hintfile allowing to remove the require dependency.
I suggest shorter name, i.e. DELREQUIRES/REMREQUIRES.
Then you do:
and you're back home :-)

P.S. @David, should you implement or not what we request, you already have saved us tons of time.

--
Best regards,
Andrzej Telszewski

1 members found this post helpful.

Inside my own head I have always thought, "never -- this is too complicated". But actually, what you describe is reasonable and helpful and very easy Maybe two hints, BUILDTIME="dep1 dep2" and RUNTIME="dep3 dep4" would solve the problem. As you say, BUILDTIME would simply delete the deps from the .dep file after building. RUNTIME would be exactly the same as DELREQUIRES (as below) but would add the deps to the .dep file after building. YaY !!

I want to do this, and I had some ideas for how to do this automatically, but I never got further than some buggy code to detect circles. If the user could specify exactly where to break the circle, that would be a big help. One possible solution is that I want to make the command line more flexible, mixing verbs and variable assignments and package names left-to-right, so you could say

slackrepo build ffmpeg transcode vid.stab rebuild ADDREQUIRES="vid.stab" ffmpeg

... but now that I have actually written that down, it looks bloody awful

The file name is not always the same as the last part of the download URL. There are three cases:

(1) urls with ?name=value&name2=value2 on the end
(2) % encoded urls
(3) downloads where the server pushes a different name (content disposition) -- github, bitbucket and others

I can use sed to deal with (1) and (2), but (3) is difficult. If you *don't* use content disposition, and if you keep all the downloads in a flat location, then it doesn't work properly when there are two different packages that have tarballs named 'v0.1.tar.gz'

So the only certain way to find the right file in a directory is to look at the md5sum of all the files and see if one matches This is why I'm thinking of abandoning my own code and using Urchlay's sbosrcarch for everything, because sbosrcarch stores source tarballs both 'by_name' and 'by_md5sum'. It saves bandwidth and storage space when a download is shared by multiple packages (e.g. nvidia-*), and will allow us to keep multiple source versions if we are building for 14.1 and 14.2 and -current. But unfortunately, sbosrcarch is the complete opposite of what you're suggesting, it *loves* subdirectories.

You're right, this is a pain, this happens to me ten times a night when I'm reviewing submissions. The same thing happens with a rebase, or when your submission is approved. There's a solution for this in my head, it might work

*perfect* -- this is the right one -- thank you.

1 members found this post helpful.

I also have some wishes in addition to above ones . They are:

1. a hook to run a script after slackrepo run
2. a possibility to ask slackrepo not to complain to one or two installed SBo packages. For example, I like on a https server to have 'dehydrated' (for letsencrypt ssl certificates), it will not harm slackrepo.
3. when slackrepo builds a package that already installed, it upgrades it to the newly built version. This is not always good, e.g., I build a new version of freetype, but I don't want to install this new version to the build server, but only to some other machines. Would be nice to have an option to configure this behavior. For the moment I commented out the following fragment at the end of the install_packages function in installfunctions.sh:
   
   Code:

           # istat=1 (already installed, different version/arch/build/tag)
           # or istat=3 (broken /var/log/packages) or istat=whatever
           [ "$istat" = 1 ] && log_normal -a "Upgrading $R_INSTALLED ..."
           [ "$istat" = 3 ] && log_warning -n "Attempting to upgrade or reinstall $R_INSTALLED ..."
           if [ "$OPT_VERBOSE" = 'y' ]; then
             set -o pipefail
             ROOT=${CHROOTDIR:-/} ${SUDO}upgradepkg --reinstall "$pkgpath" 2>&1 | tee -a "$ITEMLOG"
             pstat=$?
             set +o pipefail
           else
             ROOT=${CHROOTDIR:-/} ${SUDO}upgradepkg --reinstall "$pkgpath" >> "$ITEMLOG" 2>&1
             pstat=$?
           fi
           [ "$pstat" = 0 ] || { log_error -a "${itemid}: upgradepkg $pkgbase failed (status $pstat)"; return 1; }

Thank you very very much for the slackrepo!

1 members found this post helpful.

You're the second person to suggest this, so it must be a good idea

It's very helpful to hear these things that you find annoying -- this annoyed me too, but I've seen it so many times that I'm desensitised now, and it's always bad to be desensitised by too many warnings. So yes, thanks, it needs to be fixed.

That's really interesting feedback, thank you for making me think

This is almost the same problem as your second suggestion, and almost the same problem as another idea someone suggested: that we should be able to "pin" a particular version of a package. (I wanted to do that with keepassx recently.) Also, it's going to be needed for slackrepo to build core Slackware packages (which is a long term goal). I wonder if there's some good way to solve all these problems ... or maybe I should just do the option that you suggest; that sounds much easier

1 members found this post helpful.

David, I'm not sure if you missed my suggestion for a possible fix to your ccache issue. I have no idea on the feasibility of using this as I tried to find in the code where this was happening and came up empty.

Another thought I just had, maybe you could only allow ccache to be saved by using a new entry in the hintfiles. I know ccache for building Android ROMs can take up a substantial amount of space, and I'd imagine for some of the bigger projects here, it could as well. For those who have ample space, the extra ccache space wouldn't be a big deal, but if you're fairly limited (I started with a 30GB container and I'm going to have to grow it soon as I think I might be bumping into space issues with compiling qt5), that ccache could take up valuable space.

One other thing that I asked about before but didn't get an answer... is there any way to use a pre-compiled package as a dependency for another package? I am trying to build mkvtoolnix, but I wanted the qt5 interface. Unfortunately, my htpc is pretty slow and I'm running a VM on there for slackrepo, so RAM is a premium as well. I've tried several times to build qt5, but each attempt has been unsuccessful, but sometimes it takes a long time for it to error out (the last one ran into "file not found" issues after 12ish hours -- I'm hoping it was just a space issue, and I cleared the source directories (except for jdk and qt5), old packages, and the /tmp/ directory to try and get more space) so I'm struggling to get a working package. In the past, I've always just used Eric's qt5 packages, and I'm wondering if there's some way I can do that here (other than installing it manually into the base install then removing qt5 from the ADDREQUIRES line in the mkvtoolnix hint file but keeping the QTGUI=yes in the OPTIONS line).

Oh, and @atelszewski, DELREQUIRES or REMREQUIRES are both much better than my suggestion. I was trying to think of something shorter, but by my brain wasn't cooperating, so thanks!

1 members found this post helpful.

Hi,

I think this is the only solution for now.
You could try edit slackrepo sqlite database, but that would be counterproductive, I guess.

I started with slackrepo a few days earlier than you ;-)

--
Best regards,
Andrzej Telszewski

Sorry to have not replied explicitly -- about ccache, I was thinking about bind-mounting that directory into the chroot instead of copying (which could be horribly slow, I was using a 5Gb cache when I was fixing SBo for 14.2). It's more interesting that you do actually care about ccache, it encourages me to find an answer, because when I think it's only me that cares I don't try so hard

About qt5, I agree with Andrzej that you've already thought of the least worst answer currently available ... maybe the hacks left over in the code to cope with packages moving from SBo to -current before 14.2 came out might be useful. It would be somewhere around line 25 of parsefunctions.sh .. sorry, let me sleep on it.

1 members found this post helpful.

No worries. Sometimes it's nice to throw your ideas out there and let other people provide suggestions. Hopefully we've helped you along, and it's not a big deal if you don't add something to handle using alternate packages like Eric's qt5. I can handle that Just figured I'd ask in case there was something already in place (or in the works).

And yeah, copying the ccache would add a substantial amount of time for compiles, so I suppose that probably isn't the best idea. Using bind mount seems like a much better idea. And no rush trying to implement it. While adding proper support would be nice, it isn't a high priority for me (although, with how often you likely rebuild everything in your testing, I could see it'd be highly beneficial for you and people like Panagiotis Nikolaou, the maintainer of slackonly).

For posterity's sake (and for people who have corrections or better options), here's what I needed to do when I couldn't use a SlackBuild as is.

I tried for about a week to compile qt5 on my computer. It kept errorring out, I think during the linking process (but there were errors of missing files as well, so that could be related too), but I don't know enough about qt to figure it out, and it usually would fail around the 24 hour mark (it is a basic htpc computer). Anyway, rather than just installing a pre-compiled qt package, which would "dirty the system" and would require a lot of manual intervention if I didn't want to keep it permanently installed, plus it would cause issues with the packages' dep files not listing the qt5 package.

Anyway, so onto what I did. This could also be used to add additional packages that aren't part of SBo, you'll just get a warning that the git is dirty. (I'm not positive on how this will affect updates, so I'll update that when I find out.)

First, cd to the correct directory.

Now, we'll create a git branch for your new project. (This will hopefully allow updates of the repo without issue, but we'll see once the next update is pushed out.)

Now, we can create a new folder for the package, I called mine qt5-repack (because I will just be repacking a pre-compiled binary), and then create the needed files.

I ended up using Eric's package because his was a higher version than slackonly's (5.7.0 vs 5.6.1), which was built from SBo, so I grabbed the urls for both the 32bit and 64bit versions along with the md5s for each.

qt5-repack.info
qt5-repack.SlackBuild
Then I just had to change the hint file for mkvtoolnix (the program I was attempting to build qt5 for) to have the ADDREQUIRES list qt5-repack instead of regular qt5. For any other programs relying on qt5, whether option or required, you would need to replace those manually (within the branch to prevent screwing up potential updates).

If anyone is aware of a better alternative, I'd be open to it.

I also wonder if a future iteration of slackrepo would be able to maybe have another folder under /etc/slackrepo/SBo/ that can override the .SlackBuild and/or .info files (maybe if the folder exists, it doesn't look at the folder in the git repo). This could be used much like the default_hintfiles being overridden by hintfiles in the regular directory. So for my qt5 things, I could just do something like the following and have it override whatever's in the repo without the warning of a dirty git.

/etc/slackrepo/SBo/slackbuilds/qt5/qt5.{SlackBuild,info}

Just a thought

2 members found this post helpful.

Hi,

Sorry for no proper quoting, I'm too tired ;-)

Could your problem with compiling qt5 come from having too many cores and too little memory?
I.e. this might happen if you have, say, 4 cores (or 2 cores, but 4 threads [Hyper Threading]) and 1GB of memory.
Just a blind guess.

The problem with dirty git is that, your packages will get rebuilt again after you commit your changes.
And it doesn't matter if the SlackBuild has changed, going from dirty => clean will trigger the rebuild.
We discussed this problem with David some posts earlier.

When it comes to local changes, this is what I do (dunno if it's the best approach):
1. I have local branch, as you (I keep it "clean", don't know if it matters).
2. When public update comes, I do:
If I understand correctly, after rebase, your local branch will be the same as 14.2, with your local changes re-applied on top of it. This might mean that the package you maintain locally does not receive the updates from 14.2, but I am not sure how it works exactly ;-)

Once you're on your local branch, you can compare it against 14.2:
And if you have shitty looking output from git commands, try adding -R to less env variable:
--
Best regards,
Andrzej Telszewski

2 members found this post helpful.

I thought this as well, so the last few attempts had NUMJOBS set to -j1, and I had 3GB of RAM plus 2GB of swap for this, and it is running in headless mode, so on a fresh boot, it's only using around 50MB of that 3GB and no swap. I suppose it could need more RAM (I tried to find a suggested amount, but came up empty), but I got tired of trying to build it. The one that went the furthest was using -j1 and I think it failed around the 30 hour mark. After dealing with this for a week, I grew tired of trying to compile it and just decided to bypass it, which is what led to this post. I probably tried it a dozen times, with quite a few of them surpassing at least 12 hours before failure. I had 16GB of free space on the drive.

I'm not too familiar with git (David gave me some quirk and dirty lessons so I now know enough to cause trouble), but if I don't commit and just keep them as a separate branch, can't I update the initial branch without issue? Then, before I go to actually build, I just recheckout the custom branch so it doesn't try to rebuild qt5.

I'll have to dig through these commands online so I can figure out what they're all doing. I only started with git earlier this year, and I only use it for a few minor projects. I've not really touched branching and definitely haven't touched rebasing, so this is relatively new territory for me.

I did run into this in the past and ended up setting git's pager default rather than system-wide. I didn't know what changes that might make to normal operations (if any), so I elected to go the safer route

But I do appreciate your suggestions. I'll have to look into them and see how I can best implement this for me.

1 members found this post helpful.

I thought I was going crazy... I was digging through that file and couldn't find any reference to dropbox. I just assumed the latest package mirrored what was on github. Now I realize that he has made some changes since his 2.0rc1 release, so I'll clone the repo and start using that instead. (With that nifty change added in.)

@55020, one other suggestion, and this could be used in conjunction with a few others in here, could a future version add an IGNOREUPDATES option? I'm thinking about things like qt5, libreoffice, etc that are monstrous builds. If the developer and maintainer are quick to push out new versions, this could lead to a lot of long compiles, and when you have many programs that have qt5 as a dependency that would need to be rebuilt if qt5 gets updated, it might be nice to manually choose to not update a program. This could be done by using something like an IGNOREUPDATES="yes" option that is added to the hint file after the initial build is done.

If there's a worthwhile change that the user feels is worth a recompile, they could just remove or comment out that line and then run slackrepo update to build the new version and any downstream updates that required that dependency. Once the compile is finished, the user could add that line back to the hint file.

1 members found this post helpful.

Yes, that's what the inside of my head calls "version pinning" or "freezing". I must say, thank you for the feedback. My own use is primarily for test building, and I get the SBo packages I use every day as an accidental reward but what you're using it for is primarily to make packages, which is a bit more interesting and important

I might have the chance to restart work on slackrepo this weekend

It's still a bit puzzling and worrying why qt5 hates you; congrats on the workround.

1 members found this post helpful.