LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   Is Scheme shell slow or is it me ? (http://www.linuxquestions.org/questions/programming-9/is-scheme-shell-slow-or-is-it-me-4175443701/)

muggabug 01-01-2013 12:52 PM

Is Scheme shell slow or is it me ?
 
Hi,
I have some scripts for which bash is too slow. I thought it would be a nice idea to try scsh for it. I just picked some lines from their manual, so I might be missing out on something, but to me it seems extremely slow.

I have tried to copy half a million empty files from one directory to the other.The first 20000 or so take 20 secs on my old PC to complete with bash.
With scsh it takes four times as long.

Using directory-strings and for-each, I just get the first 100,000 names, and the loop crashes at 20,000 or so, after 1 min 20 secs. An earlier test took ages, but at least got to 75000 files.

So I tried a straight loop like this:

(define (cpy f) (run (cp ,f "copies")))

(do ((it 0 (+ it 1))) ((> it 20000)) (cpy (string-append "file" (number->string it))))

This crashes at 20.000 as well.
Using
(define (loop it itend)
(cond ((< it itend) (begin
(cpy (string-append "files" (number->string it)))
(loop (+ it 1) itend )
))))
crashes after 500 files, even.
What am I doing wrong here ??

cheers

ntubski 01-02-2013 09:00 AM

Quote:

I have tried to copy half a million empty files from one directory to the other.
I wouldn't expect to see any performance difference with different shells for a task like this, since it seems likely that disk IO would be the bottleneck. However, since you are seeing a difference (and also crashing), my guess is you've hit a bug in scsh. Note that scsh hasn't seen a release since 2006, and I get the impression that it was always more of prototype/research project.

muggabug 01-02-2013 03:01 PM

Quote:

it seems likely that disk IO would be the bottleneck.
I hoped it would loop faster than the regular shell. I do not know anything about Python (that's why I tried scsh first), but I just picked some commands from their website to test it, instead of scsh. Comparing the following two loops, I think different scripting languages already loop faster than ksh/bash.
It must be that scheme shell is buggy. Pity, I had the impression it was something I would like.
Code:

####################
##ksh
####################
((index=1))
while ((index < 100001)) ; do
 touch file$index
((index++))
done

mkdir copies

(cat << EOT
#!/usr/bin/python
import shutil
import glob
files=glob.glob('file*')
for f in files:
shutil.copy(f, 'copies')
print 'done'
EOT) > cpy.py

chmod u+x cpy.py
time ./cpy.py

#real 0m17.59s
#user 0m12.35s
#sys 0m4.86s

rm -rf copies
mkdir copies
ksh

time for file in file* ; do
cp $file copies
done
echo 'done'

#real 1m36.55s
#user 0m43.39s
#sys 0m49.90s

cheers

ntubski 01-03-2013 08:36 AM

Quote:

Originally Posted by muggabug (Post 4861548)
Comparing the following two loops, I think different scripting languages already loop faster than ksh/bash.

Yes, shells are usually slow, plus you are invoking the cp program every iteration of the loop (the scsh loop has the same problem). Somehow I thought the overhead wouldn't be that significant, but upon reflection empty files could be cached pretty easily so the disk IO wouldn't be that bad. You could also accomplish this task without a loop:
Code:

find . -maxdepth 1 -name 'file*' -print0 | xargs -0 cp -t copies
Quote:

It must be that scheme shell is buggy. Pity, I had the impression it was something I would like.
Have you considered just using some scheme implementation with a POSIX library? A quick web search turns up a reimplementation of scsh's run notation in chicken.


All times are GMT -5. The time now is 08:01 PM.