LinuxQuestions.org - Is Scheme shell slow or is it me ?

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Is Scheme shell slow or is it me ? (https://www.linuxquestions.org/questions/programming-9/is-scheme-shell-slow-or-is-it-me-4175443701/)

muggabug

01-01-2013 12:52 PM

Is Scheme shell slow or is it me ?

Hi,
I have some scripts for which bash is too slow. I thought it would be a nice idea to try scsh for it. I just picked some lines from their manual, so I might be missing out on something, but to me it seems extremely slow.

I have tried to copy half a million empty files from one directory to the other.The first 20000 or so take 20 secs on my old PC to complete with bash.
With scsh it takes four times as long.

Using directory-strings and for-each, I just get the first 100,000 names, and the loop crashes at 20,000 or so, after 1 min 20 secs. An earlier test took ages, but at least got to 75000 files.

So I tried a straight loop like this:

(define (cpy f) (run (cp ,f "copies")))

(do ((it 0 (+ it 1))) ((> it 20000)) (cpy (string-append "file" (number->string it))))

This crashes at 20.000 as well.
Using
(define (loop it itend)
(cond ((< it itend) (begin
(cpy (string-append "files" (number->string it)))
(loop (+ it 1) itend )
))))
crashes after 500 files, even.
What am I doing wrong here ??

cheers

ntubski

01-02-2013 09:00 AM

Quote:

I have tried to copy half a million empty files from one directory to the other.

I wouldn't expect to see any performance difference with different shells for a task like this, since it seems likely that disk IO would be the bottleneck. However, since you are seeing a difference (and also crashing), my guess is you've hit a bug in scsh. Note that scsh hasn't seen a release since 2006, and I get the impression that it was always more of prototype/research project.

muggabug

01-02-2013 03:01 PM

Quote:

it seems likely that disk IO would be the bottleneck.

I hoped it would loop faster than the regular shell. I do not know anything about Python (that's why I tried scsh first), but I just picked some commands from their website to test it, instead of scsh. Comparing the following two loops, I think different scripting languages already loop faster than ksh/bash.
It must be that scheme shell is buggy. Pity, I had the impression it was something I would like.

Code:

####################

##ksh

####################

((index=1))

while ((index < 100001)) ; do

 touch file$index

((index++))

done



mkdir copies



(cat << EOT

#!/usr/bin/python

import shutil

import glob

files=glob.glob('file*')

for f in files:

shutil.copy(f, 'copies')

print 'done'

EOT) > cpy.py



chmod u+x cpy.py

time ./cpy.py



#real 0m17.59s

#user 0m12.35s

#sys 0m4.86s



rm -rf copies

mkdir copies

ksh



time for file in file* ; do

cp $file copies

done

echo 'done'



#real 1m36.55s

#user 0m43.39s

#sys 0m49.90s

cheers

ntubski

01-03-2013 08:36 AM

Quote:

Originally Posted by muggabug (Post 4861548)

Comparing the following two loops, I think different scripting languages already loop faster than ksh/bash.

Yes, shells are usually slow, plus you are invoking the cp program every iteration of the loop (the scsh loop has the same problem). Somehow I thought the overhead wouldn't be that significant, but upon reflection empty files could be cached pretty easily so the disk IO wouldn't be that bad. You could also accomplish this task without a loop:

Code:

find . -maxdepth 1 -name 'file*' -print0 | xargs -0 cp -t copies

Quote:

It must be that scheme shell is buggy. Pity, I had the impression it was something I would like.

Have you considered just using some scheme implementation with a POSIX library? A quick web search turns up a reimplementation of scsh's run notation in chicken.

All times are GMT -5. The time now is 03:38 AM.