LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-11-2009, 04:19 PM   #1
amwink
Member
 
Registered: Mar 2005
Location: Connecticut/USA
Distribution: Ubuntu 9.04
Posts: 68

Rep: Reputation: 15
Replace a field for a whole line in the same file, with awk.


Dear programmers,

I'm trying to write a script with awk that would replace a field in a line for an entire line in the same file.

E.g. if the field $3 contains the value "57", replace it for the content of line #57 of the same file.

Any suggestion?

Last edited by amwink; 11-11-2009 at 05:15 PM. Reason: make a bit more clear
 
Old 11-11-2009, 07:32 PM   #2
lewc
Member
 
Registered: Nov 2009
Distribution: Gentoo, Slackware or Debian
Posts: 60
Blog Entries: 1

Rep: Reputation: 18
use sed and sh, why bother with awk when the commnd is only one line long from terminal
 
Old 11-11-2009, 07:52 PM   #3
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
Quote:
Originally Posted by amwink View Post
Dear programmers,

I'm trying to write a script with awk that would replace a field in a line for an entire line in the same file.

E.g. if the field $3 contains the value "57", replace it for the content of line #57 of the same file.

Any suggestion?
post what you have and what its not working. provide sample input and output. have you read awk docs?
 
Old 11-11-2009, 07:54 PM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
Quote:
Originally Posted by lewc View Post
use sed and sh, why bother with awk when the commnd is only one line long from terminal
why bother with sed or sh when awk is much simpler to understand and it does the job of sed+sh. ?
 
Old 11-12-2009, 03:36 AM   #5
lewc
Member
 
Registered: Nov 2009
Distribution: Gentoo, Slackware or Debian
Posts: 60
Blog Entries: 1

Rep: Reputation: 18
A single question. Does Awk allow you to do this in one line, if so continue with awk, otherwise just keep it simple
 
Old 11-12-2009, 03:51 AM   #6
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682
The OP asked for an AWK solution.

Such a contrived problem seems like it is a homework assignment. So the OP should post what he has done so far, and we can help with that.

AWK is usually the best tool to use when the datafile consists of lines of fields. In this example, one of the fields will be replaced with contents from another line based on the value of a certain field, so this involves more than regular expression pattern matching & replacing which is where SED excels.

Amwink: For questions about awk and grep, it is usually necessary to provide a short input sample, and what the output should look like. The awk command will need to know what delimiter is used for example.
 
Old 11-12-2009, 04:12 AM   #7
lewc
Member
 
Registered: Nov 2009
Distribution: Gentoo, Slackware or Debian
Posts: 60
Blog Entries: 1

Rep: Reputation: 18
:S sorry I must have misunderstood, I thought he anted to replace a field on a particular line
 
Old 11-12-2009, 04:14 AM   #8
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
Quote:
Originally Posted by lewc View Post
Does Awk allow you to do this in one line
of course.
 
Old 11-12-2009, 04:16 AM   #9
lewc
Member
 
Registered: Nov 2009
Distribution: Gentoo, Slackware or Debian
Posts: 60
Blog Entries: 1

Rep: Reputation: 18
I meant allow you to do this in one line of code, I know AWK operates on lines and fields, perhaps a rethink of my sentence structure
 
Old 11-12-2009, 04:19 AM   #10
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682
lewc: response to post #7

If I read the question correctly, if the third field of the fourth line, contains "57", then replace the third field with the contents of the entire record of the 57th line. I don't see how this would be useful, because you will be replacing a line (record) with N fields with a line with 2*N-1 fields. If $3 in a later line references an earlier line in the file, the number of records increases even more.

A general description of what the OP is trying to accomplish would be useful. Are the values in $3 of all the lines unique so that the number of fields in the output will be constant? What should happen if this is a constraint but is violated? Or does the input file have sections, such that you just process the first 50 lines and take the values from the next 50 lines?

Last edited by jschiwal; 11-12-2009 at 04:25 AM.
 
Old 11-12-2009, 04:39 AM   #11
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
Quote:
Originally Posted by lewc View Post
I meant allow you to do this in one line of code,
i say again, YES!
 
Old 11-12-2009, 01:04 PM   #12
amwink
Member
 
Registered: Mar 2005
Location: Connecticut/USA
Distribution: Ubuntu 9.04
Posts: 68

Original Poster
Rep: Reputation: 15
Thanks for all for the feedback.
Quote:
Originally Posted by ghostdog74 View Post
post what you have and what its not working. provide sample input and output. have you read awk docs?
Yes, I have been reading about awk for quite a while. This is actually the second time I face the same problem in disparate applications. The first time was for a less important project, that would relate 2 databases for use with GMT tools. I ended up not working in this project after all and just gave up. Although this time I solved already the problem by other means (see below), I would still like to learn the principles for doing this in awk.

Quote:
Originally Posted by jschiwal View Post
The OP asked for an AWK solution.

Such a contrived problem seems like it is a homework assignment. So the OP should post what he has done so far, and we can help with that.
[...]
Amwink: For questions about awk and grep, it is usually necessary to provide a short input sample, and what the output should look like. The awk command will need to know what delimiter is used for example.
It's not homework in the sense that I'm not taking classes anymore. It's for real life. The reason for not posting what I have is because I don't have anything to show so far. I couldn't even formulate a way to organise the code from the very beginning in terms of awk logic. It's not much a problem on syntax, but on the how organise the code, from which I could build the rest.

Quote:
Originally Posted by jschiwal View Post
[...] Or does the input file have sections, such that you just process the first 50 lines and take the values from the next 50 lines?
Precisely. The first section of the file contains coordinates of the vertices of a mesh in a 3D space (x,y,z), one vertex per line. The second section of the file contains a list of triangular faces, one face per line, each with the index of the 3 vertices that it is made of. I can separate the two parts in separate files if needed, but even in that case, I still wouldn't know how to replace a vertex index, say vertex #57, for its coordinates triplet on line #57 (x,y,z).

The task is to compute the area of each face using an analytic formula, and print the sum of all faces. The idea is that after replacement, I would have, in each line of the second part, 9 values corresponding to the coordinates of the vetices A, B and C of each face (xA,yA,zA,xB,yB,zB,xC,yC,zC), from which then awk itself could be used to compute some determinants (the area of the projections of each triangle in each of orthogonal planes, XY, YZ and XZ), then the area of each face the 3D space, then the sum for all faces of the mesh, all simple maths. I'm sure there are other ways to get the same using awk, but it always ends up having to search for a value found one point in the file for its match elsewhere in the same file or another.

It can also be performed in a number of different ways using different languages, and since yesterday when I opened the thread, I already wrote a bulky Matlab function that does the job, so the actual need is solved. However, I'm sure awk would do the trick in a more elegant way, with a much shorter code, faster to run, and I could also submit to a cluster (i.e. a computer grid) to do it faster in large scale here.

Thanks again!

Last edited by amwink; 11-12-2009 at 03:19 PM. Reason: minor edit to be more precise.
 
Old 11-13-2009, 06:51 AM   #13
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682
I would recommend downloading the source for the gawk package. Besides the usual make and make install targets, there is also a "make pdf" target to produce the book "GAWK: Effective AWK Programming". This book is produced from the .texi files used to produce the info files. The pdf (or ps) version is a very print worthy book and is an excellent guide for learning awk.

---

Your input file presents the references to the data before the data itself, which isn't read until the second section of the file. You would probably would produce your output document in the END{ ... } block.

Be sure to read the section on arrays in AWK. They are sparse associative arrays. You could build up one array in the first part of the file and then in the second section, enter values in a second array, and then in the END block, iterate through the first array, and use the value of $3 as the index of the second array.

Remember the AWK model. For each line of text, the awk script is executed. Since your file consists of two parts, you can use pattern matching (on the line of input) to determine which part of the file you are in. If the patterns of a line for parts are distint, your awk script could look like this:

BEGIN { ... }
/coordinates pattern/{
awk
commands
}
/faces pattern/{
awk
commands
}
END { awk commands to produce the output }

You can instead use ranges. Suppose that each section has a header, the the two parts have a blank line in between.
/coordinates header/,/^$/{ awk commands for coordinates section }
/faces section/,${ awk commands for faces section }
END {
process the values in your arrays and produce the output
}

Good Luck!

Last edited by jschiwal; 11-13-2009 at 06:54 AM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
awk to replace particular field vgr12386 Programming 9 06-05-2009 07:15 AM
replace a pattern with a line using sed/awk lokeshn05 Linux - Newbie 3 05-06-2009 03:01 PM
how to replace line of file with another line using awk in shell script amit_pansuria Programming 3 03-29-2009 09:43 AM
AWK: change a particular field in a csv file help help help!!!! haydar68 Programming 20 08-03-2008 01:10 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 02:47 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration