LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 09-21-2011, 12:51 AM   #1
saad.imran
LQ Newbie
 
Registered: Sep 2011
Posts: 2

Rep: Reputation: Disabled
How can i do Multithreading in shell scripting.


Hi,

I have shell script "A" which is calling a oralce pl/sql procedure. , i want to initiate its multiple threads at a time.

Problem statement is : user will give any value and value will be any numeric value let user give 25 then there should be 25 instance of script A per second and it will keep on continue [26 threads per second] no matter how a much time a thread take to complete it should create 25 threads after every second.

Any body can help !!
 
Old 09-21-2011, 01:13 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405
Hi,

This should work:
Code:
#!/bin/bash
# Run script: ./scriptname value
UserValue=$1

# loop forever
while true
do
  # start threads
  for i in $(seq 1 $UserValue)
  do
    echo "started instance no: $i"
    # /path/to/script/A &
  done
  # sleep 1 second
  sleep 1
done
A few comments on the above:
- The sleep 1 sleeps one second, but does this after all the oracle procedure's are started.
- The commented out part is were your procedure goes. Do add an & at the end. This makes sure the process is pushed to the background and bash can continue doing its thing.
- No error checking is implemented (example: did the user give a numerical value).
- Script can be stopped by hitting ctrl-c

Do test the above as is first to see if you want this (./script 10 for example).

Hope this helps.
 
1 members found this post helpful.
Old 09-21-2011, 01:17 AM   #3
corp769
LQ Guru
 
Registered: Apr 2005
Location: /dev/null
Posts: 5,818

Rep: Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007
Hello,

Within bash, you technically can not run via multithreading, but you can run in parallel. A very good example can be seen here - http://blog.dbotelho.com/2009/01/mul...h-bash-script/
I hope this helps in your understanding of what you will need to do.

Cheers,

Josh

Edit - And once again, I got beat.
 
1 members found this post helpful.
Old 09-21-2011, 01:23 AM   #4
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405
Hi,
Quote:
Originally Posted by corp769 View Post
Within bash, you technically can not run via multithreading, but you can run in parallel.
True!

But looking at the OP's problem description I assumed that real multi-threading wasn't what s/he was after.

Quote:
Edit - And once again, I got beat.
I just got out of bed. Fresh coffee and cigarettes handy, you on the other hand are doing the last stages of a long shift (if I'm not mistaken)
 
Old 09-21-2011, 01:28 AM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,011

Rep: Reputation: 3194Reputation: 3194Reputation: 3194Reputation: 3194Reputation: 3194Reputation: 3194Reputation: 3194Reputation: 3194Reputation: 3194Reputation: 3194Reputation: 3194
There is also a command called parallel which may be of interest.
 
1 members found this post helpful.
Old 09-21-2011, 01:30 AM   #6
corp769
LQ Guru
 
Registered: Apr 2005
Location: /dev/null
Posts: 5,818

Rep: Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007
Quote:
Originally Posted by druuna View Post
I just got out of bed. Fresh coffee and cigarettes handy, you on the other hand are doing the last stages of a long shift (if I'm not mistaken)
Dude, I still have three and a half hours to go! Eight and a half hours down so far.... Come on 0600!

But yes, I do see where you are coming from. I've written my share of multi-threaded applications, and each time I see the term, I think of straight-forward multi-threading. As far as your script, if I were to use that, to think about a different approach on running the script, I am thinking more towards using nohup just to be safe.
 
Old 09-21-2011, 01:31 AM   #7
corp769
LQ Guru
 
Registered: Apr 2005
Location: /dev/null
Posts: 5,818

Rep: Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007
Quote:
Originally Posted by grail View Post
There is also a command called parallel which may be of interest.
That would also definitely work. I was just thinking in the favor of the OP, as far as using a straight up script.
 
Old 09-21-2011, 01:40 AM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405
Hi
Quote:
Originally Posted by grail View Post
There is also a command called parallel which may be of interest.
That would be a nicer solution for the inner loop! It might even be faster, which would produce a better sleep delta.
 
Old 09-21-2011, 01:49 AM   #9
corp769
LQ Guru
 
Registered: Apr 2005
Location: /dev/null
Posts: 5,818

Rep: Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007Reputation: 1007
Quote:
Originally Posted by druuna View Post
Hi
That would be a nicer solution for the inner loop! It might even be faster, which would produce a better sleep delta.
I was thinking the same thing. Have that within the script will also cut out some code and condense it a bit, making it more user-effecient too.
 
Old 09-21-2011, 06:07 AM   #10
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,161

Rep: Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125
Sounds awfully like homework to me.
 
Old 09-26-2011, 02:05 AM   #11
saad.imran
LQ Newbie
 
Registered: Sep 2011
Posts: 2

Original Poster
Rep: Reputation: Disabled
Dear Druuna,
Thanks for reply.

As am executing : "# /path/to/script/A & " script "A" in below mentioned script and Script "A" is executing a pl/sql procedure. it runs fine if i use : /path/to/script/A . AND when i use : " /path/to/script/A & " if given value in loop is equal to 1 then this code is working fine but if given value in loop is more than 1 then it is throwing error [Error is given below ] Means if i execute more than one backend instance with " /path/to/script/A & " below given errors are there .. :-(


code of script calling script "A"

#!/bin/bash
# Run script: ./scriptname value
UserValue=$1

# loop forever
while true
do
# start threads
for i in $(seq 1 $UserValue)
do
echo "started instance no: $i"
# /path/to/script/A &
done
# sleep 1 second
sleep 1
done

----
code of script "A" executing pl/sql procedure

#!/bin/sh
RACLE_SID=GNVDEV


#ORACLE_HOME=/u21/app/oracle/product/10.2.0/db_1/
ORACLE_HOME=/oracle/home/product/10.2.0/db_1


PATH=/usr/local/bin:$ORACLE_HOME/bin:$PATH
export PATH

#sqlplus -s geneva_admin/gen89eva@gnvdev<<THEEND
sqlplus -s sip/sipdev@sipdev<<THEEND

SET PAGESIZE 0 FEEDBACK OFF TRIMSPOOL ON TRIMOUT ON VERIFY OFF
SET SERVEROUTPUT ON SIZE 1000000

DECLARE
accountId varchar2(100);
result varchar2(100);
error varchar2(100);
begin
accountId:=1225;

TEST_DISPLAY2(accountId,result,error);

commit;
end;
/
QUIT;
THEEND





---
Erors while using /path/to/script/A & in above script



ERROR:
ORA-12537: TNS:connection closed


ERROR:
ORA-12537: TNS:connection closed


ERROR:
ORA-01017: invalid username/password; logon denied


SP2-0306: Invalid option.
Usage: CONN[ECT] [logon] [AS {SYSDBA|SYSOPER}]
where <logon> ::= <username>[/<password>][@<connect_identifier>] | /
SP2-0157: unable to CONNECT to ORACLE after 3 attempts, exiting SQL*Plus
ERROR:
ORA-01017: invalid username/password; logon denied


SP2-0306: Invalid option.
Usage: CONN[ECT] [logon] [AS {SYSDBA|SYSOPER}]
where <logon> ::= <username>[/<password>][@<connect_identifier>] | /
SP2-0157: unable to CONNECT to ORACLE after 3 attempts, exiting SQL*Plus
 
Old 09-26-2011, 03:24 AM   #12
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405
Hi,

Does the above pl/sql script work if you execute it from the command line?

I'm asking because of these things I noticed:

- RACLE_SID=GNVDEV
shouldn't that be: export ORACLE_SID=GNVDEV (mind the O. RACLE_SID vs ORACLE_SID !!)

- ORACLE_HOME=/oracle/home/product/10.2.0/db_1
shouldn't that be: export ORACLE_HOME=/oracle/home/product/10.2.0/db_1

- You seem to be using two different oracle SID's: ORACLE_SID=GNVDEV and sqlplus -s sip/sipdev@sipdev

- All the errors shown seem to be Oracle specific.

I'm by no means an Oracle expert, just trying to make sure that the pl/sql script works the way it should.

Hope this helps.

Last edited by druuna; 09-26-2011 at 08:05 AM. Reason: spelling
 
1 members found this post helpful.
Old 09-26-2011, 10:46 AM   #13
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,706
Blog Entries: 4

Rep: Reputation: 3949Reputation: 3949Reputation: 3949Reputation: 3949Reputation: 3949Reputation: 3949Reputation: 3949Reputation: 3949Reputation: 3949Reputation: 3949Reputation: 3949
Write your code so that you can spawn any number of instances of it ... using the good ol' "&" capability of the shell ... and so that each one of those instances will do their appointed job and survive until expressly terminated (e.g. by a kill command).

Design them ... using bash-scripting or (probably) not ... so that they listen for requests from some queue and, when they receive a request, carry it out then deliver the results then listen on the queue once again.

What you've created is exactly the sort of "server" that you find in the back room or behind the counter in any fast-food restaurant. The people who work there survive the shift. They might be busy-as-hell or they might be twiddling their thumbs (and five tour-buses full of hungry passengers might show up at any moment). But no one's sitting in the back room with a box full of "Instant Employee - Just Add Water," and no one's keeling over after serving you and nobody's shooting them dead.

Each instance of the service process concentrates only upon its assigned task and is neither aware of nor in direct communication with any of the others. Meanwhile, there's a manager process (maybe it's just you and the shell) overseeing their activities while (of course... ) not actually doing any of the work.

Also... remember that "spawning multiple threads" does not multiply the capabilities of the underlying system: it divides it. If you have a lot of work to do and think that "throwing multiple processes at it" will help, you might well find that the opposite is true. "Don't guess... Measure."

Last edited by sundialsvcs; 09-26-2011 at 10:50 AM.
 
1 members found this post helpful.
Old 05-19-2012, 02:23 AM   #14
yo8rxp
Member
 
Registered: Jul 2009
Location: Romania
Distribution: Ubuntu 10.04 Gnome 2
Posts: 102

Rep: Reputation: 31
Runnig bash parallel

True , bash aint running parallel.
still, i did wrote a piece of software script.cfg for some test.csv word processing.
bash is counting lines in that file , counting how many CPU u got, split lines/cores +1 ,clone script.cfg as many times as cores u got, "sed" some variables inside clones , make them executable and run them simultaneous. make a messaging mechanism to notify when all are done, and after that recombine partial result into a single result.
In this way , some test file having 30k lines is split in 4 parts( amd quad core) ,and each clone is processing just 1/4 of test file, thus CPU is running 100% on all cores. When finish , it combines all 4 parts into a single file.
mechanism to check out when all cores are done can be made using " wait", but for the sake of controlling ,i did it in my way.

#!/bin/bash
cores=$(ls /proc/acpi/processor/ | wc -l)
touch images.list
cat TEST.csv > fisier.tmp
sed -i '1d' fisier.tmp
file_lines=$(cat fisier.tmp | wc -l)

let "split_lines=$file_lines/$cores"
let "split_lines=$split_lines+1"
split -d --lines=$split_lines fisier.tmp CPU

( for script in `seq 1 $cores`;
do
let sufix=script-1
cat script.cfg > "CPU0$sufix.script"
chmod +x CPU0$sufix.script
sed -i 's/filenumber/CPU0'$sufix'/g' CPU0$sufix.script

./CPU0$sufix.script &
done
)

( while [ 1 ]
do
ok=$(ls ok.* | wc -l)
if [ "$ok" = "$cores" ] ; then ./script_reintregire.sh ; exit
fi
sleep 1
linii_realizate=$(cat final.CPU* | wc -l)

let procentaj=(linii_realizate*100)/file_lines
echo $procentaj
done )| zenity --width=400 --height=100 --progress --title="Procesez $file_lines articole" --text="Please wait ...parsing in progress" --percentage=0 --auto-close ;


Sorry about romanian words, aint got time to translate !

for info
http://www.linux-romania.com/product.php?id_product=76
gabriel@linux-romania.com

Last edited by yo8rxp; 05-19-2012 at 02:31 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Terminal functions for shell scripting with Shell Curses LXer Syndicated Linux News 0 03-26-2008 11:50 PM
SHELL scripting/ shell functions mayaabboud Linux - Newbie 6 12-26-2007 08:18 AM
Shell Scripting: Getting a pid and killing it via a shell script topcat Programming 15 10-28-2007 02:14 AM
teaching shell scripting: cool scripting examples? fax8 Linux - General 1 04-20-2006 04:29 AM
shell interface vs shell scripting? I'm confused jcchenz Linux - Software 1 10-26-2005 03:32 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 12:09 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration