LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   process the output of one shell script in another one (https://www.linuxquestions.org/questions/linux-newbie-8/process-the-output-of-one-shell-script-in-another-one-4175474498/)

samasara 08-24-2013 02:38 AM

process the output of one shell script in another one
 
Hi dear users,

I just write a shell script that the output of that is something like "danger=0" or "danger=1" for each input that it gives. now i want to get this output of shell script and pass it to other shell script and then regards to the value of "danger" flush a rule from iptable. The problem is that how can i use this output and then do further processing on it. i did something like this but it seems there are some mistakes:
Code:

sh myscript // run the first shell script
while [$1=danger]

if danger = 1
iptables -A INPUT -i lo -j ACCEPT
if danger = 2
iptables -P OUTPUT ACCEPT

how could it be? I mean how could it be possible to get the output of first shell script and do further processing (flush iptable rules regards to each output of shell script) on the output of first shell script regards to each output that it produce?

Thanks a lot

druuna 08-24-2013 04:06 AM

I wonder if this can't be done more elegant (one script?), but without knowing the full context of what you are trying to accomplish I can't give you any advise on that.

About your question, have a look at this (script number 2)
Code:

#!/bin/bash

for THIS in $@    # $@ contains all arguments from input (script 1)
do
  case $THIS in
    danger=0) echo "iptables rule for 0" ;;
    danger=1) echo "iptables rule for 1" ;;
    danger=2) echo "iptables rule for 2" ;;
  esac
done

You can now do something like this:
Code:

script_2.sh $( script_1.sh )
This command first executes the green part, which will output danger=0 danger=1 etc. This output is then given to the blue part.

Here's a simple example run:
Code:

$ script_1.sh
danger=0
danger=1
danger=0
danger=2

$ script_2.sh $( script_1.sh )
iptables rule for 0
iptables rule for 1
iptables rule for 0
iptables rule for 2

These might come in handy:

HMW 08-24-2013 04:15 AM

Quote:

Originally Posted by samasara (Post 5014903)
Hi dear users,

I just write a shell script that the output of that is something like "danger=0" or "danger=1" for each input that it gives. now i want to get this output of shell script and pass it to other shell script and then regards to the value of "danger" flush a rule from iptable. The problem is that how can i use this output and then do further processing on it. i did something like this but it seems there are some mistakes:
Code:

sh myscript // run the first shell script
while [$1=danger]

if danger = 1
iptables -A INPUT -i lo -j ACCEPT
if danger = 2
iptables -P OUTPUT ACCEPT

how could it be? I mean how could it be possible to get the output of first shell script and do further processing (flush iptable rules regards to each output of shell script) on the output of first shell script regards to each output that it produce?

Thanks a lot

Hi!

Can't you just simply add some exit codes to your script, and then check the exit status from the other script?
Like so:

Code:

sh myscript // run the first shell script
while [$1=danger]

if danger = 1
iptables -A INPUT -i lo -j ACCEPT
exit 10
if danger = 2
iptables -P OUTPUT ACCEPT
exit 20

And then in your other script something like (pseudocode):

Code:

if [ $? == 10 ]; then # $? = check exit status from former command
  do stuff
elif [ $? == 20 ]; then
  do some other stuff
fi

Best of luck!
HMW

samasara 08-24-2013 10:36 AM

Quote:

Originally Posted by druuna (Post 5014931)
I wonder if this can't be done more elegant (one script?), but without knowing the full context of what you are trying to accomplish I can't give you any advise on that.

About your question, have a look at this (script number 2)
Code:

#!/bin/bash

for THIS in $@    # $@ contains all arguments from input (script 1)
do
  case $THIS in
    danger=0) echo "iptables rule for 0" ;;
    danger=1) echo "iptables rule for 1" ;;
    danger=2) echo "iptables rule for 2" ;;
  esac
done

You can now do something like this:
Code:

script_2.sh $( script_1.sh )
This command first executes the green part, which will output danger=0 danger=1 etc. This output is then given to the blue part.

Here's a simple example run:
Code:

$ script_1.sh
danger=0
danger=1
danger=0
danger=2

$ script_2.sh $( script_1.sh )
iptables rule for 0
iptables rule for 1
iptables rule for 0
iptables rule for 2

These might come in handy:


Hi dear user,
Thanks for your reply. My problem is that i do not know exactly if i echo some firewall rules does it really apply these rules? I mean when i say that danger=0) echo
Quote:

"iptables -A INPUT -i eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j REJECT"
does it really block the port 80? Or for example when i say danger=1) echo "halt"(system commands not iptable rules) does it really shut down the machine?Do you know i want these commands to be executed regards to the value of danger?
Really thanks for your kind and help

druuna 08-24-2013 11:16 AM

Quote:

Originally Posted by samasara (Post 5015078)
My problem is that i do not know exactly if i echo some firewall rules does it really apply these rules? I mean when i say that
Code:

danger=0) echo"iptables -A INPUT -i eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j REJECT"
does it really block the port 80? Or for example when i say danger=1) echo "halt"(system commands not iptable rules) does it really shut down the machine?Do you know i want these commands to be executed regards to the value of danger?

The echo "iptables rule for X" parts I used are just an example that is safe to use for testing. It does need to be replaced with the real command.

Here's another example that:
- writes to a logfile if danger=0
- adds an iptables rule if danger=1
- halts the system if danger=2
Code:

#!/bin/bash

for THIS in $@
do
  case $THIS in
    danger=0) echo "some message" >> /var/log/danger.log ;;
    danger=1) /sbin/iptables -A INPUT -i eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j REJECT ;;
    danger=2) /sbin/halt ;;
  esac
done

Don't blindly copy and paste the above script, only run it when you understand what it does!

I'm still thinking you are trying to do something the hard way. Why do you need 2 scripts? Can't you combine them? What is your overall goal.

You seem to be very new at (bash) shell scripting; Do have a look at the links I posted. I would recommend starting with the first one.

samasara 08-25-2013 01:35 AM

process the output of one shell script in another one
 
Quote:

Originally Posted by druuna (Post 5015097)
The echo "iptables rule for X" parts I used are just an example that is safe to use for testing. It does need to be replaced with the real command.

Here's another example that:
- writes to a logfile if danger=0
- adds an iptables rule if danger=1
- halts the system if danger=2
Code:

#!/bin/bash

for THIS in $@
do
  case $THIS in
    danger=0) echo "some message" >> /var/log/danger.log ;;
    danger=1) /sbin/iptables -A INPUT -i eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j REJECT ;;
    danger=2) /sbin/halt ;;
  esac
done

Don't blindly copy and paste the above script, only run it when you understand what it does!

I'm still thinking you are trying to do something the hard way. Why do you need 2 scripts? Can't you combine them? What is your overall goal.

You seem to be very new at (bash) shell scripting; Do have a look at the links I posted. I would recommend starting with the first one.

Hello dear,

I am just trying to know more about shell scripting and i am trying to study the links. Regards to my first script, it is just like this:

It will get data from a file then pass it to another program(written in cpp). I used something like this to run the whole thing that i want and get the latest output that i want:
Quote:

./myscript /path/to/*.text | path to my cpp program
by doing this for example for 2 inputs from text file i have 2 outputs like this:
Quote:

danger=0
danger=1
I put the whole
Quote:

./myscript /path/to/*.text | path to my cpp program
in a shell script to be run(and it is my first shell script). Then i try to communicate between this shell script and the second one. Now i am trying to know more about your code that you write. i have a problem with this:
Quote:

for THIS in $@
. Can i ask you to expalin more about this? I know that it contains arguments from input but i do not know exactly how really it works. Does it really get
Quote:

danger=0
danger=1
as input? I think second shell script does not exactly get
Quote:

danger=0
danger=1
as input?'

really thanks

druuna 08-25-2013 03:50 AM

Quote:

Originally Posted by samasara (Post 5015368)
I am just trying to know more about shell scripting and i am trying to study the links. Regards to my first script, it is just like this:

It will get data from a file then pass it to another program(written in cpp). I used something like this to run the whole thing that i want and get the latest output that i want:
Code:

./myscript /path/to/*.text | path to my cpp program
by doing this for example for 2 inputs from text file i have 2 outputs like this:
Code:

danger=0
danger=1

I put the whole
Code:

./myscript /path/to/*.text | path to my cpp program
in a shell script to be run(and it is my first shell script). Then i try to communicate between this shell script and the second one.

The above seems like a very elaborate way of doing things. Depending on your cpp/scripting knowledge, it must be possible to make one script (or cpp executable) that does all the things you describe. Lets put this aside for the moment and focus on the next part.

Quote:

Originally Posted by samasara
Now i am trying to know more about your code that you write. i have a problem with this:
Code:

for THIS in $@
Can i ask you to expalin more about this? I know that it contains arguments from input but i do not know exactly how really it works. Does it really get
Code:

danger=0
danger=1

as input? I think second shell script does not exactly get
Code:

danger=0
danger=1

as input?'

I'm not sure I understand what you mean by Does it really get danger=0 danger=1 as input. You mention that your first script does generate this output, if that is so then script_2.sh does get the input.
Code:

#!/bin/bash
# this is script_2.sh
for THIS in $@
do

  case $THIS in
    danger=0) echo "got danger=0 as input" ;;
    danger=1) echo "got danger=1 as input" ;;
  esac

done

The above example works as follows:

- The blue part is a for loop.
This loop uses the input given to the script (the $@ part) and parses this, one entry at the time. Each individual entry ($THIS) can be accessed within the do ... done part.

- The green part is a case statement.
It checks to see if the individual entry ($THIS) matches a pattern (the grey parts). If it finds a matching pattern then the brown part is executed. The brown parts can be just about anything you want. In the above example I used echo.

Hope this clears things up a bit.

samasara 08-25-2013 07:47 AM

process the output of one shell script in another one
 
Quote:

Originally Posted by druuna (Post 5015419)
The above seems like a very elaborate way of doing things. Depending on your cpp/scripting knowledge, it must be possible to make one script (or cpp executable) that does all the things you describe. Lets put this aside for the moment and focus on the next part.

I'm not sure I understand what you mean by Does it really get danger=0 danger=1 as input. You mention that your first script does generate this output, if that is so then script_2.sh does get the input.
Code:

#!/bin/bash
# this is script_2.sh
for THIS in $@
do

  case $THIS in
    danger=0) echo "got danger=0 as input" ;;
    danger=1) echo "got danger=1 as input" ;;
  esac

done

The above example works as follows:

- The blue part is a for loop.
This loop uses the input given to the script (the $@ part) and parses this, one entry at the time. Each individual entry ($THIS) can be accessed within the do ... done part.

- The green part is a case statement.
It checks to see if the individual entry ($THIS) matches a pattern (the grey parts). If it finds a matching pattern then the brown part is executed. The brown parts can be just about anything you want. In the above example I used echo.

Hope this clears things up a bit.

hi agian dear user,
I just study more and my problem solved. Really thanks for your kind and help. Just one point is that for linux command like halt to be applied i should write just halt not /sbin/halt,
Really really thanks.

druuna 08-25-2013 09:31 AM

Quote:

Originally Posted by samasara (Post 5015528)
I just study more and my problem solved. Really thanks for your kind and help.

You're welcome.

Quote:

Originally Posted by samasara
Just one point is that for linux command like halt to be applied i should write just halt not /sbin/halt.

I tend to include the full path for commands that are "powerful/dangerous" for safety reasons.

On Debian, RedHat and Slackware the halt command is found in /sbin. You don't mention the Linux distro you are using, so maybe your halt command is placed elsewhere.

samasara 10-02-2013 12:33 PM

get the last logs
 
Hi dear druuna,

I just have a file in .log format. it contains full of logs and each log contain 5 parts. each time my file will be updated and the last log come to the end of log file. i want to get the last two logs each time. i tried tail -n 2 , to get the last two logs, but it shows me just the last line of log. tails -f also do not show the whole log completely. How could it be? how can i get the last two logs each time?


Thanks a lot for your kind and help
Really wit for your reply

druuna 10-02-2013 12:59 PM

Quote:

Originally Posted by samasara (Post 5038726)
each log contain 5 parts. each time my file will be updated and the last log come to the end of log file. i want to get the last two logs each time. i tried tail -n 2 , to get the last two logs, but it shows me just the last line of log. tails -f also do not show the whole log completely. How could it be? how can i get the last two logs each time?

Without knowing the content of the log file in question and which parts you want to get, it is impossible to help you.

Post a relevant(!!) example of the log file and the expected output (both in [code] ... [/code] tags).

samasara 10-02-2013 03:16 PM

get the last logs
 
ok, thanks dear. this is a log file with two logs in it:

Quote:

--25814763-A--
[21/Jul/2013:10:45:18 -0400] 258a49e5-9691-4626-bac1-457ecb1e858d 127.0.0.1 8009 127.0.0.1 80

--25814763-B--
GET w7e/neQhmsdwu7imdb0etet/eT/hsvegbff/EH/niRAvLwGK_L/osLnBWcHRk5oGMI/tmLJFqSww/sSjS6KRJB.html?Settotzeertnl=%27pn+&8nafitm=74LuKUC5t0J&4ttNe=Anmsyusi6&Mf1g-vYqyx=elTTsw&Euoytxp$
Host: 28.99.169.38
Connection: close
Accept: */*
Accept-Charset: *
Accept-Encoding: *;q=0.7
Accept-Language: ondtmsih-rqjk1;q=0.6
Cache-Control: upy='aels'
Cookie2: $Version="388"
Date: Thu, 27 Apr 06 24:36:50 UTC
ETag: "kAPqiaNm18b1MKgZ"
If-Modified-Since: Thu, 18 May 06 01:58:48 GMT
If-Unmodified-Since: Fri, 19 Sep 08 10:20:47 CET
If-Match: *
If-None-Match: "mGN7VPxNspBNWF50vLeI"
Pragma: vthu=ifdlS4il
Authorization: Basic TnJ3MDY6UWVzc29sY2U=
Referer: http://www.hao8.de/0ehnqceo/segfto2z...f/tmdpe2En.avi
TE: deflate;q=0.7,deflate;q=0.1,deflate;q=0.2
User-Agent: rfno5rdhfNnssmascs
UA-CPU: MIPS
Via: FTP/1.8 200.207.173.114
Transfer-Encoding: wria
Warning: 540 www.plsshi.jpeg "rdomaoAwf2ahhud2t" "Fri, 15 Sep 06 13:24:25 CET"

--25814763-F--
--25814763-H--
Message: Access denied with code 406 (phase 2). Pattern match "(^[\"'`\xc2\xb4\xe2\x80\x99\xe2\x80\x98;]+|[\"'`\xc2\xb4\xe2\x80\x99\xe2\x80\x98;]+$)" at ARGS:Settotzeertnl. [file "/usr/local/apachhe/conf/samane_rules/SpiderLabs-owasp-modsecurity-crs-33612c6/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "64"] [id "981318"] [rev "2"] [msg "SQL Injection Attack: Common Injection Testing Detected"] [data "Matched Data: ' found within ARGS:Settotzeertnl: 'pn "] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.7"] [maturity "9"] [accuracy "8"] [tag "OWASP_CRS/WEB _ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"]
Apache-Error: [file "http_filters.c"] [line 262] [level 3] Unknown Transfer-Encoding: wria, referer: http://www.hao8.de/0ehnqceo/segfto2z...f/tmdpe2En.avi
Action: Intercepted (phase 2)
Stopwatch: 1374417991011250 5572 (- - -)
Stopwatch2: 1374417991011250 5572; combined=867, p1=10, p2=830, p3=0, p4=0, p5=26, sr=0, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.2 (http://www.modsecurity.org/).
Server: Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips DAV/2 PHP/5.4.12
Engine-Mode: "ENABLED"

--25814763-Z--

--25814763-A--
[21/Jul/2013:10:45:18 -0400] 0ee2fbb1-c51e-4e9c-a0fb-3caa6e5058a0 127.0.0.1 8009 127.0.0.1 80

--25814763-B--
GET Http://dvwa.com
Host: 28.99.169.38
Connection: close
Accept: */*
Accept-Charset: *
Accept-Encoding: *;q=0.7
Accept-Language: ondtmsih-rqjk1;q=0.6
Cache-Control: upy='aels'
Cookie2: $Version="388"
Date: Thu, 27 Apr 06 24:36:50 UTC
ETag: "kAPqiaNm18b1MKgZ"
If-Modified-Since: Thu, 18 May 06 01:58:48 GMT
If-Unmodified-Since: Fri, 19 Sep 08 10:20:47 CET
If-Match: *
If-None-Match: "mGN7VPxNspBNWF50vLeI"
Pragma: vthu=ifdlS4il
Authorization: Basic TnJ3MDY6UWVzc29sY2U=
Referer: http://www.hao8.de/0ehnqceo/segfto2z...f/tmdpe2En.avi
TE: deflate;q=0.7,deflate;q=0.1,deflate;q=0.2
User-Agent: rfno5rdhfNnssmascs
UA-CPU: MIPS
Via: FTP/1.8 200.207.173.114
Transfer-Encoding: wria
Warning: 540 www.plsshi.jpeg "rdomaoAwf2ahhud2t" "Fri, 15 Sep 06 13:24:25 CET"

--25814763-F--
--25814763-H--
Message: Access denied with code 406 (phase 2). Pattern match "(^[\"'`\xc2\xb4\xe2\x80\x99\xe2\x80\x98;]+|[\"'`\xc2\xb4\xe2\x80\x99\xe2\x80\x98;]+$)" at ARGS:Settotzeertnl. [file "/usr/local/apachhe/conf/samane_rules/SpiderLabs-owasp-modsecurity-crs-33612c6/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "64"] [id "981318"] [rev "2"] [msg "SQL Injection Attack: Common Injection Testing Detected"] [data "Matched Data: ' found within ARGS:Settotzeertnl: 'pn "] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.7"] [maturity "9"] [accuracy "8"] [tag "OWASP_CRS/WEB _ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"]
Apache-Error: [file "http_filters.c"] [line 262] [level 3] Unknown Transfer-Encoding: wria, referer: http://www.hao8.de/0ehnqceo/segfto2z...f/tmdpe2En.avi
Action: Intercepted (phase 2)
Stopwatch: 1374417991011250 5572 (- - -)
Stopwatch2: 1374417991011250 5572; combined=867, p1=10, p2=830, p3=0, p4=0, p5=26, sr=0, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.2 (http://www.modsecurity.org/).
Server: Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips DAV/2 PHP/5.4.12
Engine-Mode: "ENABLED"

--25814763-Z--


Assume that each time my log file will be updated and 1 new log will be created in it. Each time i want the complete two last log of log file. I need the complete part of two last logs.

Thanks a lot
best regards

chrism01 10-02-2013 06:51 PM

Please show from the above example, what you define as 'last 2 log records'.

Note that cmds such as head, tail, grep are all line oriented tools, so the last 2 records/lines are
Code:


--25814763-Z--

assuming that last blank line is an artefact of copy/paste and not actually in the log file.

If you want to retrieve entire blocks of text ie multiple lines/recs (as I suspect), then you'll need something like Perl or awk or possibly sed ...

samasara 10-02-2013 11:20 PM

process the output of one shell script in another one
 
Hi dear Chris. i have a log file with lots of logs that 2 sample of these logs are shown in my previous message. each time my logfile will be updated and one new log will be added to it. I just want each time two last log that are created in my logfile for further processing(the whole log from A-Z).i think with awk such a think could not be possible. how would it be?
Best regards

druuna 10-03-2013 02:41 AM

If I assume that the example posted are only the 2 blocks you are talking about then I do have questions:

1) Did you copy/paste the same block twice? Both blocks are the same and there's nothing unique in either of the 2 blocks.
2) Is there anything else in that log that is not related to the 2 blocks you are talking about?

As already asked by chrism01, we need to know exactly which parts in post #12 are the "two last logs" you are talking about. We can only guess and this might not give you the answer you want/need.

I've asked before, but will do so again: Please post a relevant part of the log you are talking about. This should include the 2 blocks you are talking about (make 1 block bold and the other italic and the possible other entries that are in that log.

Without detailed information we cannot help you.


All times are GMT -5. The time now is 07:26 AM.