LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 01-01-2013, 12:52 PM   #1
muggabug
LQ Newbie
 
Registered: Aug 2011
Posts: 18

Rep: Reputation: Disabled
Is Scheme shell slow or is it me ?


Hi,
I have some scripts for which bash is too slow. I thought it would be a nice idea to try scsh for it. I just picked some lines from their manual, so I might be missing out on something, but to me it seems extremely slow.

I have tried to copy half a million empty files from one directory to the other.The first 20000 or so take 20 secs on my old PC to complete with bash.
With scsh it takes four times as long.

Using directory-strings and for-each, I just get the first 100,000 names, and the loop crashes at 20,000 or so, after 1 min 20 secs. An earlier test took ages, but at least got to 75000 files.

So I tried a straight loop like this:

(define (cpy f) (run (cp ,f "copies")))

(do ((it 0 (+ it 1))) ((> it 20000)) (cpy (string-append "file" (number->string it))))

This crashes at 20.000 as well.
Using
(define (loop it itend)
(cond ((< it itend) (begin
(cpy (string-append "files" (number->string it)))
(loop (+ it 1) itend )
))))
crashes after 500 files, even.
What am I doing wrong here ??

cheers
 
Old 01-02-2013, 09:00 AM   #2
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,426

Rep: Reputation: 821Reputation: 821Reputation: 821Reputation: 821Reputation: 821Reputation: 821Reputation: 821
Quote:
I have tried to copy half a million empty files from one directory to the other.
I wouldn't expect to see any performance difference with different shells for a task like this, since it seems likely that disk IO would be the bottleneck. However, since you are seeing a difference (and also crashing), my guess is you've hit a bug in scsh. Note that scsh hasn't seen a release since 2006, and I get the impression that it was always more of prototype/research project.
 
Old 01-02-2013, 03:01 PM   #3
muggabug
LQ Newbie
 
Registered: Aug 2011
Posts: 18

Original Poster
Rep: Reputation: Disabled
Quote:
it seems likely that disk IO would be the bottleneck.
I hoped it would loop faster than the regular shell. I do not know anything about Python (that's why I tried scsh first), but I just picked some commands from their website to test it, instead of scsh. Comparing the following two loops, I think different scripting languages already loop faster than ksh/bash.
It must be that scheme shell is buggy. Pity, I had the impression it was something I would like.
Code:
####################
##ksh
####################
((index=1))
while ((index < 100001)) ; do
 touch file$index
((index++))
done

mkdir copies

(cat << EOT
#!/usr/bin/python
import shutil
import glob
files=glob.glob('file*')
for f in files:
shutil.copy(f, 'copies')
print 'done'
EOT) > cpy.py

chmod u+x cpy.py
time ./cpy.py

#real 0m17.59s
#user 0m12.35s
#sys 0m4.86s

rm -rf copies
mkdir copies
ksh

time for file in file* ; do
cp $file copies
done
echo 'done'

#real 1m36.55s
#user 0m43.39s
#sys 0m49.90s
cheers
 
Old 01-03-2013, 08:36 AM   #4
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,426

Rep: Reputation: 821Reputation: 821Reputation: 821Reputation: 821Reputation: 821Reputation: 821Reputation: 821
Quote:
Originally Posted by muggabug View Post
Comparing the following two loops, I think different scripting languages already loop faster than ksh/bash.
Yes, shells are usually slow, plus you are invoking the cp program every iteration of the loop (the scsh loop has the same problem). Somehow I thought the overhead wouldn't be that significant, but upon reflection empty files could be cached pretty easily so the disk IO wouldn't be that bad. You could also accomplish this task without a loop:
Code:
find . -maxdepth 1 -name 'file*' -print0 | xargs -0 cp -t copies
Quote:
It must be that scheme shell is buggy. Pity, I had the impression it was something I would like.
Have you considered just using some scheme implementation with a POSIX library? A quick web search turns up a reimplementation of scsh's run notation in chicken.
 
  


Reply

Tags
shell


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] REALLY Slow Shell Script Crowey Programming 9 03-02-2010 07:02 PM
LXer: Scripting with Scheme Shell LXer Syndicated Linux News 0 05-23-2008 03:50 PM
interactive shell very slow infinity005 Linux - Software 11 12-05-2007 06:13 PM
Password generation failed for scheme {CRYPT}: scheme not recognized olva Linux - General 0 11-05-2006 11:21 AM
Changed IP scheme on network, Samba browsing really slow z3r0star21 Linux - Networking 5 03-05-2006 01:06 PM


All times are GMT -5. The time now is 12:10 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration