LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-17-2012, 03:23 AM   #1
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Rep: Reputation: 0
Interesting awk


Fellas,

I trust and completly believe that there are many Unix Gurus on this forum and I am just beginner ! hoping to reach there soon

I need small help.
Code:
[user01@centos-flash rel-01]$find . -type f -print |xargs p4 have|awk 'not on {print $1}'
./hf_rel-01.0.14073_cca.sql - file(s) not on client.
./hf_rel-01.0.29.14656_ccr.sql.commit - file(s) not on client.
./hf_rel-01.0.28_ccr.sql - file(s) not on client.
./hf_rel-01.0.27_ccr.sql - file(s) not on client.
./hf_rel-01.0.29.14074_cca.sql.commit - file(s) not on client.
./hf_rel-01.0.29.14191_ccr.sql - file(s) not on client.
./hf_rel-01.0.29.14085_ccr.sql - file(s) not on client.
./hf_rel-01.0.29.12459_ccr.sh - file(s) not on client.
./hf_rel-01.0.12_cca.sql - file(s) not on client.
./hf_rel-01.0.29.12701_ccr.sql - file(s) not on client.
./hf_rel-01.0.29.14013_cca.sql.commit - file(s) not on client.
./14074_CCCB2PRD_20110514.out - file(s) not on client.
./hf_rel-01.0.12_cca.sh - file(s) not on client.
./hf_rel-01.0.29.14180_ccr.sql.commit - file(s) not on client.
I need to extract only name of file say hf_rel-01.0.14073_cca.sql so on and so forth..

I even did
Code:
find . -type f -print |xargs p4 have|awk 'not on {print $1}'|grep -v "not on client"
Code:
find . -type f -print |xargs p4 have|grep "not on"|awk -F'-' '{print $1}'
but had no success
How should I go about this?
 
Old 03-17-2012, 04:10 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

You've chosen a certain solution for your problem and are stuck with that, could you explain what it is you are trying to accomplish?
Quote:
Originally Posted by sysmicuser
$ find . -type f -print | xargs p4 have | awk 'not on {print $1}'
You seem to be looking for normal files in the current and subdirectories (the find . -type f -print part), the rest of the pipe (xargs and the awk part) doesn't make any sense.

Please tell us what your starting point is and what you want as end-result. Depending on what it is you want to accomplish the find command might not be needed at all.

Hope this helps.
 
1 members found this post helpful.
Old 03-17-2012, 04:42 AM   #3
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Original Poster
Rep: Reputation: 0
@drunna

What I trying to accomplish is a good question.

My intention is to look for all files which are not in perforce.
First part as your rightly said is find all files in current and sub directories

second part xargs p4 have means executing on each and every file found a simple comand called p4 have

Once we get output from p4 have all I do is search for that file which is not in perforce client.

It would give typical output as follows is file is not in perforce client

Code:
./hf_rel-01.0.14073_cca.sql - file(s) not on client.
But I need to further filter in such a way that I need to get only filename of those file which are not in client.

Hope I have now clarified?

So I want output such as
Code:
hf_rel-01.0.14073_cca.sql
 
Old 03-17-2012, 05:16 AM   #4
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

If I focus on the output of this part find . -type f -print |xargs p4 have, which I believe could produce lines like this ./hf_rel-01.0.14073_cca.sql - file(s) not on client. (of which you only want to print the filename) I would do something like this:
Code:
$ find . -type f -print | xargs p4 have | awk -F"[ /]" '/not on client/ { print $2 }'
The bold part looks for lines that contain not on client and will print the filename. You can use multiple field separators in awk (the -F"[ /]" part), in this case a space and the forward slash are the field separators.

Hope this helps.

Last edited by druuna; 03-17-2012 at 05:20 AM. Reason: typo: filed -> field
 
1 members found this post helpful.
Old 03-17-2012, 06:09 AM   #5
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Original Poster
Rep: Reputation: 0
@drunna bro.

No effect it is still the same

Code:
[user01@centos-flash rel-01]$find . -type f -print | xargs p4 have | awk -F"[ /]" '/not on client/ { print $2 }'
./hf_rel-01.0.14073_cca.sql - file(s) not on client.
./hf_rel-01.0.29.14656_ccr.sql.commit - file(s) not on client.
./hf_rel-01.0.28_ccr.sql - file(s) not on client.
./hf_rel-01.0.27_ccr.sql - file(s) not on client.
./hf_rel-01.0.29.14074_cca.sql.commit - file(s) not on client.
./hf_rel-01.0.29.14191_ccr.sql - file(s) not on client.
./hf_rel-01.0.29.14085_ccr.sql - file(s) not on client.
./hf_rel-01.0.29.12459_ccr.sh - file(s) not on client.
./hf_rel-01.0.12_cca.sql - file(s) not on client.
I am also looking let us see.
 
Old 03-17-2012, 06:23 AM   #6
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Original Poster
Rep: Reputation: 0
One more interesting this is that if I use stderr and stdout I cannot bloody use in all of the above commands

2>&1 | tee -a 1.log

here 1.log file after execution is of "0" size

That's not so cool..
Code:
[user01@centos-flash rel-01]$find . -type f -print | xargs p4 have | awk -F"[ /]" '/not on client/ { print $2 }' 2>&1 | tee -a 1.log
[user01@centos-flash rel-01]$ls -l 1.log 
-rw-r--r-- 1 user01 user01 0 Mar 17 21:21 1.log
[user01@centos-flash rel-01]$
also if I use instead of print $2, print FILENAME that doesn't work either....
 
Old 03-17-2012, 06:34 AM   #7
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

I can only assume that the output generated by p4 have might have something to do with it. It might not print the output to the stdout (and or stderr).

Without knowing how the p4 program/script works and how it generates output this will be hard to trouble-shoot.

Does the following work? (please post output of commands, within code tags):
Code:
$ find . -type f -print | xargs p4 have > outfile
$ head -3 outfile
$ file outfile
$ od -c outfile | head
BTW: You are working on a linux box and are using bash as shell?
 
1 members found this post helpful.
Old 03-17-2012, 06:46 AM   #8
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Original Poster
Rep: Reputation: 0
Yes working on a linux box and using bash as shell.
 
Old 03-17-2012, 06:48 AM   #9
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Original Poster
Rep: Reputation: 0
I am trying to do google but what the heck -perm -200 means??

---------- Post added 03-17-12 at 09:48 PM ----------

Yeah getting output for each command and sending it your private email.
 
Old 03-17-2012, 06:50 AM   #10
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Please post the output I asked for in this thread! I'm not going to do a 1 on 1 session!
 
Old 03-17-2012, 07:20 AM   #11
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Original Poster
Rep: Reputation: 0
I don't want either private 1 on 1 session, email was only to keep our security freaks happy!

I shall post on thread anyways!
 
Old 03-17-2012, 08:29 AM   #12
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Original Poster
Rep: Reputation: 0
Output of ccommand 1.
Quote:
find . -type f -print | xargs p4 have > outfile
Code:
[user01@centos-flash 1.9.0.4.6.1]$find . -type f -print | xargs p4 have > outfile
./hf_1.9.0.4.6.1.0.14073_cca.sql - file(s) not on client.
./hf_1.9.0.4.6.1.0.29.14656_ccr.sql.commit - file(s) not on client.
./hf_1.9.0.4.6.1.0.28_ccr.sql - file(s) not on client.
./hf_1.9.0.4.6.1.0.27_ccr.sql - file(s) not on client.
./hf_1.9.0.4.6.1.0.29.14074_cca.sql.commit - file(s) not on client.
./hf_1.9.0.4.6.1.0.29.14191_ccr.sql - file(s) not on client.
./hf_1.9.0.4.6.1.0.29.14085_ccr.sql - file(s) not on client.
./hf_1.9.0.4.6.1.0.29.12459_ccr.sh - file(s) not on client.
./hf_1.9.0.4.6.1.0.12_cca.sql - file(s) not on client.
./hf_1.9.0.4.6.1.0.29.12701_ccr.sql - file(s) not on client.
./hf_1.9.0.4.6.1.0.29.14013_cca.sql.commit - file(s) not on client.
Output of command 2
Quote:
head -3 outfile
Code:
[user01@centos-flash 1.9.0.4.6.1]$head -3 outfile
//glide/main/scm/hfscripts/1.9.0.4.6.1/hf_1.9.0.4.6.1.0.24_int.sh#1 - /app/scm/p4ws/scmbuild/scmtools/hfscripts/1.9.0.4.6.1/hf_1.9.0.4.6.1.0.24_int.sh
//glide/main/scm/hfscripts/1.9.0.4.6.1/hf_1.9.0.4.6.1.0.22_ccr.sh#1 - /app/scm/p4ws/scmbuild/scmtools/hfscripts/1.9.0.4.6.1/hf_1.9.0.4.6.1.0.22_ccr.sh
//glide/main/scm/hfscripts/1.9.0.4.6.1/hf_1.9.0.4.6.1.0.18_iop.sh#1 - /app/scm/p4ws/scmbuild/scmtools/hfscripts/1.9.0.4.6.1/hf_1.9.0.4.6.1.0.18_iop.sh
[user01@centos-flash 1.9.0.4.6.1]$
Output of comamnd 3
Quote:
file outfile
[code]
[user01@centos-flash 1.9.0.4.6.1]$file outfile
outfile: ASCII text
[user01@centos-flash 1.9.0.4.6.1]$
[code]


Output of command 4
Quote:
od -c outfile | head
Code:
[user01@centos-flash 1.9.0.4.6.1]$od -c outfile | head
0000000   /   /   g   l   i   d   e   /   m   a   i   n   /   s   c   m
0000020   /   h   f   s   c   r   i   p   t   s   /   1   .   9   .   0
0000040   .   4   .   6   .   1   /   h   f   _   1   .   9   .   0   .
0000060   4   .   6   .   1   .   0   .   2   4   _   i   n   t   .   s
0000100   h   #   1       -       /   a   p   p   /   s   c   m   /   p
0000120   4   w   s   /   s   c   m   b   u   i   l   d   /   s   c   m
0000140   t   o   o   l   s   /   h   f   s   c   r   i   p   t   s   /
0000160   1   .   9   .   0   .   4   .   6   .   1   /   h   f   _   1
0000200   .   9   .   0   .   4   .   6   .   1   .   0   .   2   4   _
0000220   i   n   t   .   s   h  \n   /   /   g   l   i   d   e   /   m
[user01@centos-flash 1.9.0.4.6.1]$


---------- Post added 03-17-12 at 11:30 PM ----------

Interesting to note that output and logfile are completely different !
 
Old 03-17-2012, 08:37 AM   #13
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

The output you see in post #1 and post #5 are not send to stdout by the p4 program. The output from command 1 and 2 show this (outfile is filled with other information).

Without knowing what p4 does I cannot help you. It (p4) probably manipulates file descriptors for its multiple output.

I would suggest changing p4 so that the output that is now send to screen (the ./hf_1.9.0.4.6.1.0.14073_cca.sql - file(s) not on client. lines) are send to a seperate file, which you can then use.

Hope this helps.
 
1 members found this post helpful.
Old 03-17-2012, 09:23 AM   #14
sysmicuser
Member
 
Registered: Mar 2010
Posts: 359

Original Poster
Rep: Reputation: 0
I was also thinking say for example if get output in stdout by means of script command and then exit and later parsing the file.

You are correct p4 doesn't send output to stdout,
I ran script and later parsed file and it gives exact file names !
Code:
awk -F"[ /]" '/not on client/ { print $2 }' <file_name>
certainly works.

Well I would like to ask you the most idiot question anyone would have ever asked. Sorry about my incompetent shell/unix skills

what is stdout and stderr

When any error is thrown does it means stderr and general output as stdout? what is exactly difference between them? how does unix os in intself what is error and what is output.

The reason I ask is whenever we do deployments I always capture any command out to a logfile by following command

2>&1 | tee -a <log_file>

does this means whatever is displayed on screen/terminal throw that stuff in <log_file> if thats the case it even throws build failed error in logfile should it not only throw stdout?

Please help me understand this.

Many Thanks.
 
Old 03-17-2012, 09:55 AM   #15
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

By default stdout (1) is used for normal output and stderr (2) is used for outputting error related messages. Both, by default, print their messages to screen.

A user or a script/program can manipulate this. If you use 2>&1 you tell bash to redirect the stderr (the 2) messages to stdout (the 1). Messages from both are now going through the same pipe.

Expanding on that: Why would you need/want that? When using this 2>&1 | tee -a <log_file>, redirecting to one pipe is needed because the tee command only captures stdout (man tee). If you would omit the 2>&1 part, normal output would be send to <log_file> and errors would go to the screen (stderr's default).

In the case of p4 I suspect one or more extra file descriptors ("pipes") to be created/used, which send output to screen (stderr and stdout are used for other output).

Here's the whole story: I/O Redirection
 
2 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Interesting awk files for ns2 mobile simulations unodior Linux - Software 10 05-27-2012 05:11 PM
[SOLVED] call awk from bash script behaves differently to awk from CLI = missing newlines titanium_geek Programming 4 05-26-2011 10:06 PM
[SOLVED] awk: how can I assign value to a shell variable inside awk? quanba Programming 6 03-23-2010 03:18 AM
LXer: Interesting new Ubuntu-derived, OS X-inspired distro, interesting revenue (yes, LXer Syndicated Linux News 0 05-01-2009 09:51 AM
shell command using awk fields inside awk one71 Programming 6 06-26-2008 05:11 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 08:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration