LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices

Reply
 
Search this Thread
Old 08-21-2009, 07:09 PM   #1
grissiom
Member
 
Registered: Apr 2008
Location: China, Beijing
Distribution: Slackware
Posts: 421

Rep: Reputation: 45
Boost "slackpkg upgrade-all" performance


Talk is cheap, patch here: (against slackpkg-2.80.2-noarch-3)
Code:
--- /ext4/slackware_rsync/slackware64-current/slackware64/ap/slackpkg-2.80.2-noarch-3/usr/libexec/slackpkg/core-functions.sh  2009-07-27 09:33:49.000000000 +0800
+++ /usr/libexec/slackpkg/core-functions.sh     2009-08-22 07:41:30.106445523 +0800
@@ -567,7 +567,11 @@
                        done
                ;;
                upgrade-all)
-                       for i in $(cut -f2 -d\  ${TMPDIR}/tmplist); do
+                       cut -f2-6 -d\  ${TMPDIR}/tmplist > ${TMPDIR}/lpkg
+                       cut -f2-6 -d\  ${TMPDIR}/pkglist > ${TMPDIR}/spkg
+                       potential=$(cat ${TMPDIR}/{s,l}pkg | sort | uniq -u | cut -f1 -d\  - | uniq -d )
+                       #for i in $(cut -f2 -d\  ${TMPDIR}/tmplist); do
+                       for i in $potential; do

                                givepriority ${i}
                                [ ! "$FULLNAME" ] && continue
Before applying this patch, slackpkg will loop 800+ times which will take minutes for one "upgrade-all" on my box. After this patch, it only loop 5 times... There is even no time for spinning bar to appear ;D

Although I haven't tested it so much (I reverted two packages and "new slackpkg" can find them) but I think it would have no regressions. Any feedback is strongly welcome.
 
Old 08-21-2009, 07:16 PM   #2
dive
Senior Member
 
Registered: Aug 2003
Location: UK
Distribution: Slackware
Posts: 3,211

Rep: Reputation: 292Reputation: 292Reputation: 292
You might want to send this to Piter Punk.
 
Old 08-21-2009, 08:11 PM   #3
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: Tupelo, MS
Distribution: Gentoo
Posts: 6,926

Rep: Reputation: 124Reputation: 124
grissiom,

Are you updating from a local mirror, such as I do from my Slackware64 -current
mirror; or are you updating via the internut (Great Chinese Firewall version)?

I applied the patch and it took maybe 2 seconds for the line:
Code:
Looking for packages to upgrade. Please wait... DONE
to pop up in my terminal. Then the spinner appeared and BAM! the window with
the packages listed was there. And, yes, they are exactly the ones from the
ChangeLog that I had not updated, minus my blacklist such as kernel.

Thanks, and I too think you should send it to PiterPUNK. It's still his choice
what to do, and probably won't happen until Slackware-13.0 comes out.
 
Old 08-21-2009, 10:13 PM   #4
grissiom
Member
 
Registered: Apr 2008
Location: China, Beijing
Distribution: Slackware
Posts: 421

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by dive View Post
You might want to send this to Piter Punk.
Yes, I have sent this to Piter. But I put it here to get more testing/feedback. Somebody wants to try something new may like it as well
 
Old 08-21-2009, 10:22 PM   #5
grissiom
Member
 
Registered: Apr 2008
Location: China, Beijing
Distribution: Slackware
Posts: 421

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by Bruce Hill View Post
grissiom,

Are you updating from a local mirror, such as I do from my Slackware64 -current
mirror; or are you updating via the internut (Great Chinese Firewall version)?
Local mirror. But, anyway, rsync will have to go through the Great Firewall right?

Quote:
Originally Posted by Bruce Hill View Post
I applied the patch and it took maybe 2 seconds for the line:
Code:
Looking for packages to upgrade. Please wait... DONE
to pop up in my terminal. Then the spinner appeared and BAM! the window with
the packages listed was there. And, yes, they are exactly the ones from the
ChangeLog that I had not updated, minus my blacklist such as kernel.
Glad to here that Now I know the patch works on your box at least

Quote:
Originally Posted by Bruce Hill View Post
Thanks, and I too think you should send it to PiterPUNK. It's still his choice
what to do, and probably won't happen until Slackware-13.0 comes out.
Yes, I mailed Piter. But it seems Piter is out for a vacation or something like that? I sent "search for .new file" patch to him weeks ago but don't get his reply yet...
 
Old 08-21-2009, 11:41 PM   #6
niels.horn
Senior Member
 
Registered: Mar 2007
Location: Rio de Janeiro - Brazil
Distribution: Slackware64-current
Posts: 1,004

Rep: Reputation: 89
Worked very well here!
Slackpkg was slow on my virtual machines especially, but after this patch it's a lot faster.

Thanks!
 
Old 08-22-2009, 12:03 AM   #7
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: Tupelo, MS
Distribution: Gentoo
Posts: 6,926

Rep: Reputation: 124Reputation: 124
rsync doesn't have to go through the firewall, if you just use it local -- which I do
for everything where the files are important. Maybe you have better bandwidth in
Beijing that I do in a village here in Dali. Heck, the phone wires coming in to my
house to use the ADSL modem look like chicken intestines, so I doubt the copper
quality is very good. And they're hung all over roofs, trees, etc.

PiterPUNK most probably wouldn't submit any changes until after Slackware-13.0 anyway.

Pat surely has a freeze on anything that's not broken, and important.

Now you have niels.horn (also in PiterPUNK's area) using it with good results.

PiterPUNK really does appreciate us using and submitting suggestions for slackpkg.
 
Old 08-22-2009, 07:52 AM   #8
Chuck56
Member
 
Registered: Dec 2006
Location: Colorado
Distribution: Slackware
Posts: 421

Rep: Reputation: 58
Quote:
Originally Posted by grissiom View Post
Although I haven't tested it so much (I reverted two packages and "new slackpkg" can find them) but I think it would have no regressions. Any feedback is strongly welcome.
Feedback:

I am updating a desktop machine expecting to see the 8/20 thunderbird-2.0.0.23 patch from an LAN mirror via NFS. I replaced the 1 line (-) with the 5 lines (+) in /usr/libexec/slackpkg/core-functions.sh and got the following response.

Code:
root@slacker:~# slackpkg upgrade-all

Looking for packages to upgrade. Please wait... cut: /tmp/slackpkg.BK9mE8/pkglist: No such file or directory
DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall


root@slacker:~#
When I backed out the changes it worked as expected. Did I do something wrong?
 
Old 08-22-2009, 08:00 AM   #9
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: Tupelo, MS
Distribution: Gentoo
Posts: 6,926

Rep: Reputation: 124Reputation: 124
Please post the output of:
Code:
diff -u /usr/libexec/slackpkg/core-functions.sh /usr/libexec/slackpkg/core-functions.sh~
where /usr/libexec/slackpkg/core-functions.sh is the new file and /usr/libexec/slackpkg/core-functions.sh~ the old one.
 
Old 08-22-2009, 08:10 AM   #10
Chuck56
Member
 
Registered: Dec 2006
Location: Colorado
Distribution: Slackware
Posts: 421

Rep: Reputation: 58
Quote:
Originally Posted by Chuck56 View Post
Did I do something wrong?
Self-inflicted wound...

I'm running slackpkg-2.70.5-noarch-1, not slackpkg-2.80.2-noarch-3. My modified feedback is that it doesn't appear to work in the stock 12.2 slackpkg.
 
Old 08-22-2009, 10:35 AM   #11
grissiom
Member
 
Registered: Apr 2008
Location: China, Beijing
Distribution: Slackware
Posts: 421

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by Bruce Hill View Post
PiterPUNK most probably wouldn't submit any changes until after Slackware-13.0 anyway.

Pat surely has a freeze on anything that's not broken, and important.
Yes, it's reasonable.

Quote:
Now you have niels.horn (also in PiterPUNK's area) using it with good results.

PiterPUNK really does appreciate us using and submitting suggestions for slackpkg.
Yes, I think every developer's dream is getting people's suggestions and patches. Maybe he is busy this time. But I don't think he will miss it. Never mind~;-)
 
Old 08-22-2009, 10:42 AM   #12
grissiom
Member
 
Registered: Apr 2008
Location: China, Beijing
Distribution: Slackware
Posts: 421

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by Chuck56 View Post
Code:
root@slacker:~# slackpkg upgrade-all

Looking for packages to upgrade. Please wait... cut: /tmp/slackpkg.BK9mE8/pkglist: No such file or directory
DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall


root@slacker:~#
When I backed out the changes it worked as expected. Did I do something wrong?
Thanks~ So I know the patch won't work with slackpkg-2.70.5-noarch-1. Make it to generate pkglist and apply my patch is just painful...
 
Old 08-24-2009, 03:38 PM   #13
gegechris99
Member
 
Registered: Oct 2005
Location: France
Distribution: Slackware 14.1 32bit
Posts: 720
Blog Entries: 3

Rep: Reputation: 76
Hello Grissiom,

I've used your patch and I'd like to report that "slackpkg upgrade-all" detected correctly the new upgraded packages for my -current machine.

And also, it was way much faster

Thank you.
 
Old 08-30-2009, 08:17 AM   #14
BrZ
Member
 
Registered: Apr 2009
Distribution: Slackware
Posts: 492

Rep: Reputation: 80
Thanks grissiom, worked fine on current 32 and 64.

Last edited by BrZ; 08-30-2009 at 08:18 AM.
 
Old 09-14-2009, 02:12 PM   #15
gegechris99
Member
 
Registered: Oct 2005
Location: France
Distribution: Slackware 14.1 32bit
Posts: 720
Blog Entries: 3

Rep: Reputation: 76
@ grissiom,

I'm sorry but it seems that your patch does not work for security updates to a stable release.

In my case, I used mirror: http://slackware.osuosl.org/slackware-13.0/.

Your patch failed to detect the 3 patches since release of 13.0 (namely, mozilla-firefox, seamonkey and seamonkey-solibs).

Could the reason be that file PACKAGES.TXT is not updated for a stable release (it's rather patches/PACKAGES.TXT) contrary to -current?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] "slackpkg upgrade-all" finishes with WARNING Bruce Hill Slackware 4 08-01-2009 09:06 PM
upgrade to kernel 2.6.16.1 : "make bzImage" print "parse error" math_physics Red Hat 2 06-29-2007 11:04 PM
Blacklisting packages with "slackpkg"??? Southpaw76 Slackware 2 05-19-2007 03:50 PM
"Shutdown" and "Restart" option missing after upgrade hansalfredche Mandriva 8 11-02-2006 07:23 AM
How to use synaptic "smart upgrade or "apt-get dist-upgrade" PenguinMolester Ubuntu 2 04-04-2005 02:11 PM


All times are GMT -5. The time now is 06:13 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration