LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-26-2013, 06:42 PM   #1
powerplyer
Member
 
Registered: Mar 2012
Posts: 31

Rep: Reputation: Disabled
Scripting & Pipe Help


Hello long time lurker and new to Linux.

I am using CentOs 6.4 and I need some help setting up a script.

Here is what I am looking to automate and below is a sample of the script I setup.

1) looking to pipe the output of the data to a variable file each time the script is run. For example currently I have to run ./erase.sh >> SNxxxxxxx.txt. However optimally I would like to run just ./erase.sh and have the script file look at the output from the script and grab the SNxxxxxxx and time stamp the output file automatically.

2)Can a run a script inside a script? I think I can but just making sure.
3)Is there a way to output to screen and pipe the information at the same not. Not a big deal just wondering.

Here is the sample script<erase.sh> ddcli is a command for the card.
#!/bin/bash
# rsync using variables

SOURCEDIR=/home/user/Desktop/


./ddcli -c 4 -list
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -health
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -format -eraseflash -s
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=FORMAT SECUREERASE complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -list
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -health
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

essentially this is a command to pull data from storage card I have.

Thank you in advance for the help.
 
Old 10-26-2013, 07:58 PM   #2
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
Quote:
Originally Posted by powerplyer View Post
Hello long time lurker and new to Linux.

I am using CentOs 6.4 and I need some help setting up a script.

Here is what I am looking to automate and below is a sample of the script I setup.

1) looking to pipe the output of the data to a variable file each time the script is run. For example currently I have to run ./erase.sh >> SNxxxxxxx.txt. However optimally I would like to run just ./erase.sh and have the script file look at the output from the script and grab the SNxxxxxxx and time stamp the output file automatically.
Quote:
Originally Posted by powerplyer View Post
2)Can a run a script inside a script? I think I can but just making sure.
yes
Quote:
Originally Posted by powerplyer View Post
3)Is there a way to output to screen and pipe the information at the same not. Not a big deal just wondering.
yes, there is a utility called tee, which does just this

Code:
echo "send to tee" | tee Log.log
cat Log.log
echo "send another to tee" | tee Log.log
cat Log.log
echo "send to tee" | tee -a Log.log
cat Log.log
echo "send another to tee" | tee -a Log.log
cat Log.log
notice what the -a did,
not much to read, but read anyway
Code:
man tee
Quote:
Originally Posted by powerplyer View Post
Here is the sample script<erase.sh> ddcli is a command for the card.
Code:
#!/bin/bash
# rsync using variables

SOURCEDIR=/home/user/Desktop/


./ddcli -c 4 -list
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -health
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -format -eraseflash -s
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=FORMAT SECUREERASE complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -list
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -health
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="
essentially this is a command to pull data from storage card I have.

Thank you in advance for the help.
please use code tags,, example
[code]
Your code here
[/code]


not much for me to say about that script
only that you might want to make it 'position independent'
that is, hardcode it to the location of ddcli
one way is to cd into the dir that contains ddcli
another is to use the full path, that assumes the ddcli script/program is hardcoded with relevant paths

also it will 'carry on regardless' of any failure, since I can't see ddcli, I don't know if you can make it 'smarter'

assuming ddcli uses exit codes ( with 0 meaning success and none-zero being failure )
Code:
em="=-=-=-=-=-=-=-=-=-=-=-=-=-="
./ddcli -c 4 -list \
    && echo "${em}Initial LIST complete${em}" \
    || ( echo "${em} some error msg ${em}";exit 1 )
that is a list construct
basically it is this
Code:
em="=-=-=-=-=-=-=-=-=-=-=-=-=-=" # makes life easier later
# feel free to make it longer
# EqualMinus="=-=-=-=-=-=-=-=-=-=-=-=-=-="
# if you use something like vim, then there is a nice feature
# example , note <ctrl+p> is the ctrl key + p :)
# echo "${E<ctrl+p>
# it will either 'complete' the keyword, or present a list ;)
# 
./ddcli -c 4 -list
if [[ "$?" == "0" ]]; then
    echo "${em}Initial LIST complete${em}"
else
    echo "${em} some error msg ${em}"
    exit 1
fi
some reading
http://www.tldp.org/LDP/Bash-Beginners-Guide/html/
http://www.tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/BashGuide
http://www.gnu.org/software/bash/manual/bashref.html

The tldp stuff is great, however there are some nasty bad habits in it,
The mywiki.wooledge does a very good job of 'fixing' those habits
 
Old 10-26-2013, 08:56 PM   #3
powerplyer
Member
 
Registered: Mar 2012
Posts: 31

Original Poster
Rep: Reputation: Disabled
Hi Firerat thank you for the quick reply. Sorry about not putting the code in a box...

I am still unclear about item 1. I think I get the jist of item 3. Again, sorry some of what you said is still above my head.

Code:
#!/bin/bash
# rsync using variables

SOURCEDIR=/home/user/Desktop/


./ddcli -c 4 -list
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -health
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -format -eraseflash -s
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=FORMAT SECUREERASE complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -list
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -health
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="
Here is a sample output from the file. Sorry if I was unclear but this is what I am trying accomplish

-- run all 5 commands (via the script) automatically output to a file called temp_output.txt
-- read the output and automatically get the serial number from the temp_output.txt
-- automatically rename the file temp_output to SN_SP24447758_<time stamp>.txt

I think my biggest question is how to automatically rename temp_output.txt to SN_SPxxxxxxxx_<timestamp>.txt. The problem is I do not know what to put in a script to read the file, get the board tracer number, rename that file with board tracer number and time stamp to a text file.

Here is a sample of the output from ddcli
Code:
WarpDrive Selected is NWD-RLP4-1860
------------------------------------------------------------------------
WarpDrive Information
------------------------------------------------------------------------
  WarpDrive ID                           : 3
  PCI Address                            : 00:81:00:00
  PCI Slot Number                        : 0x05
  PCIe Link Width                        : X8 Lane(s)
  PCIe Link Rate                         : 0x01 (5.0 Gbps)
  PCI DeviceId                           : 0x7E
  PCI SubSystem DeviceId                 : 0x507
  PCI SubSystem VendorId                 : 0x1000
  SAS Address                            : 500605B 00568B0A0
  Package Version                        : 10.110.04.00
  Firmware Version                       : 110.00.01.00
  Legacy BIOS Version                    : 110.00.00.00
  UEFI BSD Version                       : 07.18.05.03
  Chip Name                              : Nytro WarpDrive
  Board Name                             : NWD-RLP4-1860
  Board Assembly Number                  : L3-25586-00A
  Board Tracer Number                    : SP24447758
  RAID Support                           : YES
If you have any more insight that would be great. Please consider me a 2 year old when it comes to Linux.

Thanks again for your help

Last edited by powerplyer; 10-26-2013 at 08:58 PM.
 
Old 10-26-2013, 09:16 PM   #4
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
not clear when you get that output

Code:
#!/bin/bash
# rsync using variables

MyTemp=$(mktemp)
LogIt="|tee -a $MyTemp"
SOURCEDIR=/home/user/Desktop/


./ddcli -c 4 -list $Logit
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -health $Logit
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -format -eraseflash -s $Logit
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=FORMAT SECUREERASE complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -list $Logit
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddcli -c 4 -health $Logit
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

FileName="SN_$( awk '/Board Tracer Number/{print $NF;exit}' "$MyTemp" )_$( date +%Y-%m-%d-%H%M%S ).txt"
mv "$MyTemp" "$FileName"
Warning:
I have made no effort to use correct paths
I don't know if "Board Tracer Number" will appear more than once
( actually, just added the ;exit to the awk, so after first result it exits )
 
1 members found this post helpful.
Old 10-26-2013, 09:45 PM   #5
powerplyer
Member
 
Registered: Mar 2012
Posts: 31

Original Poster
Rep: Reputation: Disabled
you are very gracious for doing the script.

I get a SN_2013-10-260193116.txt (great) file but it is blank, also the filename did not pick up the board tracer number. Could it be because we are not piping the output to any file. I see the output on the screen. I grabbed all the output and attached the text file.

I made a minor modification to the script to comment out the eraseflash option. Just to test it out... The eraseflash option takes too long. I think we are almost there...
Attached Files
File Type: txt Terminal_Output.txt (25.1 KB, 18 views)
 
Old 10-26-2013, 10:00 PM   #6
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
well, if those ./ddcli commands do not print to stdout, then tee will not be able to append them

I'm blind as I can't see what ddcli does

if you see the relevant text 'fly by' when running the script it could that it is sending it to stderr

change LogIt="|tee -a $MyTemp" to
Code:
LogIt=" 2>&1 | tee -a $MyTemp"
that redirects stderr to stdout
tee should then be appending it to the tmpfile,
 
Old 10-26-2013, 10:19 PM   #7
powerplyer
Member
 
Registered: Mar 2012
Posts: 31

Original Poster
Rep: Reputation: Disabled
Quote:
well, if those ./ddcli commands do not print to stdout, then tee will not be able to append them

I'm blind as I can't see what ddcli does
basically ddcli is a utility to pull information from the card.
If I were to run ddcli -c 1 -list in terminal --> the utility selects the card 1 and displays the list (informational) output of the card (ie FW,and other device information)to the terminal window. If I want to save it to a file I would do a ddcli -c 1 -list >> list_output.txt, as an example. Like doing ls command it would output to termianl windows and if I did a ls >> dir.txt it would output the directory content to a file called dir.txt. BTW I under stand that doing a ">>" will continue to append to that one file.

Quote:
if you see the relevant text 'fly by' when running the script it could that it is sending it to stderr
I do see the revelent information fly by

Quote:
change LogIt="|tee -a $MyTemp" to
Code:
LogIt=" 2>&1 | tee -a $MyTemp"
that redirects stderr to stdout
tee should then be appending it to the tmpfile
Is there something information I can get to help further, please let me know?
Thank you
 
Old 10-26-2013, 10:26 PM   #8
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
ok, well I don't understand why the file was blank

lets keep it simple and eliminate tee ( it might not be installed? )

LogIt=">> $MyTemp"
 
Old 10-26-2013, 10:42 PM   #9
powerplyer
Member
 
Registered: Mar 2012
Posts: 31

Original Poster
Rep: Reputation: Disabled
I think we are almost there.

I modified the code slightly, I pointed to the $MyTemp. The output file is correct. The only problem is the output does not "fly by" on the screen. I can only see my echo commands.

Here is the modified code:

Code:
#!/bin/bash
# rsync using variables

MyTemp=$(mktemp)
LogIt=">> $MyTemp"
SOURCEDIR=/home/user/Desktop/LSI/6208


./ddoemcli -c 1 -list >> $MyTemp
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddoemcli -c 1 -health >> $MyTemp
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

#./ddoemcli -c 1 -format -eraseflash -s $Logit
#echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=FORMAT SECUREERASE complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddoemcli -c 1 -list >> $MyTemp
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

./ddoemcli -c 1 -health >> $MyTemp
echo "=-=-=-=-=-=-=-=-=-=-=-=-=-=Final HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-="

FileName="SN_$( awk '/Board Tracer Number/{print$NF;exit}' "$MyTemp" )_$( date +%Y-%m-%d-%H%M%S ).txt"
mv "$MyTemp" "$FileName"
Here is the output on the terminal file:
Code:
./1C_Securerase.sh
=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-=
=-=-=-=-=-=-=-=-=-=-=-=-=-=Initial HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-=
=-=-=-=-=-=-=-=-=-=-=-=-=-=Final LIST complete=-=-=-=-=-=-=-=-=-=-=-=-=-=
=-=-=-=-=-=-=-=-=-=-=-=-=-=Final HEALTH complete=-=-=-=-=-=-=-=-=-=-=-=-=-=
and here is the output of the file name and it has data.

SN_SP32611130_2013-10-26-203612.txt
 
Old 10-26-2013, 11:04 PM   #10
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
yes, the reason you do not see the ddcli script output is because it is being sent to the temp file

at the moment I assume tee is not installed,

another way of doing what tee does is to start tail in the background

Code:
MyTemp=$(mktemp)
tail -f $MyTemp &
tailPID=$!
.. rest of code..
kill $tailPID
mv "$MyTemp" "$FileName"
& puts tail -f into the background, it is still running

the $! is the PID of the last background process
we get that so we can 'cleanup' before the script exits
 
Old 10-26-2013, 11:22 PM   #11
powerplyer
Member
 
Registered: Mar 2012
Posts: 31

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Firerat View Post
yes, the reason you do not see the ddcli script output is because it is being sent to the temp file

at the moment I assume tee is not installed,

another way of doing what tee does is to start tail in the background

Code:
MyTemp=$(mktemp)
tail -f $MyTemp &
tailPID=$!
.. rest of code..
kill $tailPID
mv "$MyTemp" "$FileName"
& puts tail -f into the background, it is still running

the $! is the PID of the last background process
we get that so we can 'cleanup' before the script exits

PERFECT!!! Thank you very much for you help, I tied to rep you but it looks like you have helped many. Many thanks!!! We can mark this as [SOLVED].
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
pipe Q&A paarthiban Linux - Newbie 2 06-04-2013 12:12 AM
bash scripting sending "cat: write error: Broken pipe," rojoblandino Linux - Server 8 04-23-2010 10:23 AM
Bash Scripting: Pipe input to script vs. $1 jhwilliams Linux - Software 3 12-21-2007 10:54 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:41 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration