LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (http://www.linuxquestions.org/questions/slackware-14/)
-   -   Boost "slackpkg upgrade-all" performance (http://www.linuxquestions.org/questions/slackware-14/boost-slackpkg-upgrade-all-performance-749412/)

grissiom 08-21-2009 08:09 PM

Boost "slackpkg upgrade-all" performance
 
Talk is cheap, patch here: (against slackpkg-2.80.2-noarch-3)
Code:

--- /ext4/slackware_rsync/slackware64-current/slackware64/ap/slackpkg-2.80.2-noarch-3/usr/libexec/slackpkg/core-functions.sh  2009-07-27 09:33:49.000000000 +0800
+++ /usr/libexec/slackpkg/core-functions.sh    2009-08-22 07:41:30.106445523 +0800
@@ -567,7 +567,11 @@
                        done
                ;;
                upgrade-all)
-                      for i in $(cut -f2 -d\  ${TMPDIR}/tmplist); do
+                      cut -f2-6 -d\  ${TMPDIR}/tmplist > ${TMPDIR}/lpkg
+                      cut -f2-6 -d\  ${TMPDIR}/pkglist > ${TMPDIR}/spkg
+                      potential=$(cat ${TMPDIR}/{s,l}pkg | sort | uniq -u | cut -f1 -d\  - | uniq -d )
+                      #for i in $(cut -f2 -d\  ${TMPDIR}/tmplist); do
+                      for i in $potential; do

                                givepriority ${i}
                                [ ! "$FULLNAME" ] && continue

Before applying this patch, slackpkg will loop 800+ times which will take minutes for one "upgrade-all" on my box. After this patch, it only loop 5 times... There is even no time for spinning bar to appear ;D

Although I haven't tested it so much (I reverted two packages and "new slackpkg" can find them) but I think it would have no regressions. Any feedback is strongly welcome.

dive 08-21-2009 08:16 PM

You might want to send this to Piter Punk.

Bruce Hill 08-21-2009 09:11 PM

grissiom,

Are you updating from a local mirror, such as I do from my Slackware64 -current
mirror; or are you updating via the internut (Great Chinese Firewall version)?

I applied the patch and it took maybe 2 seconds for the line:
Code:

Looking for packages to upgrade. Please wait... DONE
to pop up in my terminal. Then the spinner appeared and BAM! the window with
the packages listed was there. And, yes, they are exactly the ones from the
ChangeLog that I had not updated, minus my blacklist such as kernel.

Thanks, and I too think you should send it to PiterPUNK. It's still his choice
what to do, and probably won't happen until Slackware-13.0 comes out.

grissiom 08-21-2009 11:13 PM

Quote:

Originally Posted by dive (Post 3653064)
You might want to send this to Piter Punk.

Yes, I have sent this to Piter. But I put it here to get more testing/feedback. Somebody wants to try something new may like it as well ;)

grissiom 08-21-2009 11:22 PM

Quote:

Originally Posted by Bruce Hill (Post 3653095)
grissiom,

Are you updating from a local mirror, such as I do from my Slackware64 -current
mirror; or are you updating via the internut (Great Chinese Firewall version)?

Local mirror. But, anyway, rsync will have to go through the Great Firewall right? ;)

Quote:

Originally Posted by Bruce Hill (Post 3653095)
I applied the patch and it took maybe 2 seconds for the line:
Code:

Looking for packages to upgrade. Please wait... DONE
to pop up in my terminal. Then the spinner appeared and BAM! the window with
the packages listed was there. And, yes, they are exactly the ones from the
ChangeLog that I had not updated, minus my blacklist such as kernel.

Glad to here that ;) Now I know the patch works on your box at least ;)

Quote:

Originally Posted by Bruce Hill (Post 3653095)
Thanks, and I too think you should send it to PiterPUNK. It's still his choice
what to do, and probably won't happen until Slackware-13.0 comes out.

Yes, I mailed Piter. But it seems Piter is out for a vacation or something like that? I sent "search for .new file" patch to him weeks ago but don't get his reply yet...

niels.horn 08-22-2009 12:41 AM

Worked very well here!
Slackpkg was slow on my virtual machines especially, but after this patch it's a lot faster.

Thanks!

Bruce Hill 08-22-2009 01:03 AM

rsync doesn't have to go through the firewall, if you just use it local -- which I do
for everything where the files are important. Maybe you have better bandwidth in
Beijing that I do in a village here in Dali. Heck, the phone wires coming in to my
house to use the ADSL modem look like chicken intestines, so I doubt the copper
quality is very good. And they're hung all over roofs, trees, etc.

PiterPUNK most probably wouldn't submit any changes until after Slackware-13.0 anyway.

Pat surely has a freeze on anything that's not broken, and important.

Now you have niels.horn (also in PiterPUNK's area) using it with good results.

PiterPUNK really does appreciate us using and submitting suggestions for slackpkg.

Chuck56 08-22-2009 08:52 AM

Quote:

Originally Posted by grissiom (Post 3653058)
Although I haven't tested it so much (I reverted two packages and "new slackpkg" can find them) but I think it would have no regressions. Any feedback is strongly welcome.

Feedback:

I am updating a desktop machine expecting to see the 8/20 thunderbird-2.0.0.23 patch from an LAN mirror via NFS. I replaced the 1 line (-) with the 5 lines (+) in /usr/libexec/slackpkg/core-functions.sh and got the following response.

Code:

root@slacker:~# slackpkg upgrade-all

Looking for packages to upgrade. Please wait... cut: /tmp/slackpkg.BK9mE8/pkglist: No such file or directory
DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall


root@slacker:~#

When I backed out the changes it worked as expected. Did I do something wrong?

Bruce Hill 08-22-2009 09:00 AM

Please post the output of:
Code:

diff -u /usr/libexec/slackpkg/core-functions.sh /usr/libexec/slackpkg/core-functions.sh~
where /usr/libexec/slackpkg/core-functions.sh is the new file and /usr/libexec/slackpkg/core-functions.sh~ the old one.

Chuck56 08-22-2009 09:10 AM

Quote:

Originally Posted by Chuck56 (Post 3653468)
Did I do something wrong?

Self-inflicted wound...

I'm running slackpkg-2.70.5-noarch-1, not slackpkg-2.80.2-noarch-3. My modified feedback is that it doesn't appear to work in the stock 12.2 slackpkg.

grissiom 08-22-2009 11:35 AM

Quote:

Originally Posted by Bruce Hill (Post 3653216)
PiterPUNK most probably wouldn't submit any changes until after Slackware-13.0 anyway.

Pat surely has a freeze on anything that's not broken, and important.

Yes, it's reasonable.

Quote:

Now you have niels.horn (also in PiterPUNK's area) using it with good results.

PiterPUNK really does appreciate us using and submitting suggestions for slackpkg.
Yes, I think every developer's dream is getting people's suggestions and patches. Maybe he is busy this time. But I don't think he will miss it. Never mind~;-)

grissiom 08-22-2009 11:42 AM

Quote:

Originally Posted by Chuck56 (Post 3653468)
Code:

root@slacker:~# slackpkg upgrade-all

Looking for packages to upgrade. Please wait... cut: /tmp/slackpkg.BK9mE8/pkglist: No such file or directory
DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall


root@slacker:~#

When I backed out the changes it worked as expected. Did I do something wrong?

Thanks~ So I know the patch won't work with slackpkg-2.70.5-noarch-1. Make it to generate pkglist and apply my patch is just painful...

gegechris99 08-24-2009 04:38 PM

Hello Grissiom,

I've used your patch and I'd like to report that "slackpkg upgrade-all" detected correctly the new upgraded packages for my -current machine.

And also, it was way much faster :)

Thank you.

BrZ 08-30-2009 09:17 AM

Thanks grissiom, worked fine on current 32 and 64.

gegechris99 09-14-2009 03:12 PM

@ grissiom,

I'm sorry but it seems that your patch does not work for security updates to a stable release.

In my case, I used mirror: http://slackware.osuosl.org/slackware-13.0/.

Your patch failed to detect the 3 patches since release of 13.0 (namely, mozilla-firefox, seamonkey and seamonkey-solibs).

Could the reason be that file PACKAGES.TXT is not updated for a stable release (it's rather patches/PACKAGES.TXT) contrary to -current?


All times are GMT -5. The time now is 10:19 AM.