LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   Shell Script to replace specific columns on matched lines (http://www.linuxquestions.org/questions/programming-9/shell-script-to-replace-specific-columns-on-matched-lines-4175448018/)

axl718 01-31-2013 11:43 AM

Shell Script to replace specific columns on matched lines
 
Hi,
I have a data file that has many lines in it, that is space delimited. What I want to do is match lines in the file that have the word host in it and then i want to replace columns 3-6 with data that I have in another file. The other file only has the information that belongs on those specific lines. Any ideas?

TB0ne 01-31-2013 11:55 AM

Quote:

Originally Posted by axl718 (Post 4881481)
Hi,
I have a data file that has many lines in it, that is space delimited. What I want to do is match lines in the file that have the word host in it and then i want to replace columns 3-6 with data that I have in another file. The other file only has the information that belongs on those specific lines. Any ideas?

Ideas? Yes...write a script to do what you want. You can easily write such a script using bash, perl, or any number of other languages. The cut, sed, awk, and grep commands will probably all be used.

There is a link at the bottom of this post to a bash scripting tutorial...that's a good place to start. If you have specific problems/questions, please post them (along with samples of the input/output data), and we will be glad to help.

axl718 01-31-2013 12:00 PM

Well yeah I know write a script. I'm having trouble matching the lines and I'm having trouble replacing the columns. I am fairly new at this. I'm thinking awk or sed but I just don't know the right syntax to get it all working.

TB0ne 01-31-2013 12:33 PM

Quote:

Originally Posted by axl718 (Post 4881496)
Well yeah I know write a script. I'm having trouble matching the lines and I'm having trouble replacing the columns. I am fairly new at this. I'm thinking awk or sed but I just don't know the right syntax to get it all working.

Ok, but you still haven't given us any details to even BEGIN to try to help you. Just saying "having trouble" gives us nothing to work with.

Again, post samples of the data and what you've tried/written so far, and we'll be glad to assist. We need to see how both sets of input data look now, and what you want when you're done, and what you've written so far.

axl718 01-31-2013 12:44 PM

Ok here is an example of the data in file1 which has the new file size/date time information that I need in file2

file1:
22696 Jan 31 11:04 2013
23035 Jan 31 10:28 2013
22694 Jan 31 11:04 2013
23033 Jan 31 10:28 2013


etc....etc... repeats many times with different file sizes and times

the other file that i need to merge that data in to looks like this

file2
rwxrwxrwx 0/0 0 Dec 17 15:25 2002 ./
rwxrwxrwx 0/0 0 Dec 17 15:37 2002 ./blah/
rwxrwxr-x 0/3 0 Dec 17 15:37 2002 ./blah/home/
rwxrwxr-x 8654/8655 0 Dec 17 15:53 2002 ./blah/home/foo/
rw-rw-rw- 0/3 889 Jul 29 14:14 2002 ./blah/home/foo/hostlist
rwxrwxrwx 0/3 0 Dec 16 16:02 2002 ./comp0/
rwxrwxrwx 0/3 0 Dec 17 10:23 2002 ./comp0/etc/
rw-rw-rw- 0/3 477 Jul 29 14:14 2002 ./comp0/etc/tab
r--r--r-- 0/3 7329 Jul 29 14:14 2002 ./comp0/etc/bootinfo
rwxrwxrwx 0/3 0 Mar 26 11:07 2003 ./comp0/opt/
rwxrwxrwx 0/3 0 Mar 26 10:45 2003 ./comp0/opt/RARED/
rw-rw-r-- 0/0 133120 Dec 6 16:05 2002 ./comp0/opt/RARED/configs.tar
rwxrwxrwx 0/0 0 Dec 16 17:01 2002 ./comp1/
rwxrwxrwx 0/0 0 Dec 16 17:01 2002 ./comp1/etc/
r--r--r-- 0/0 5797 Dec 10 10:24 2002 ./comp1/etc/bootinfo
r--r--r-- 0/3 22483 Dec 16 15:25 2002 ./comp1/etc/hosts

Again a snippet it continues much longer than this.


I need to replace columns 3-6 with the data from above on the lines that match */etc/hosts. The data above lines up with the lines in the replacement file. Not sure if that helps understand what I'm doing. I really don't know awk or sed but thought they might be of use here somehow.

danielbmartin 01-31-2013 12:50 PM

Quote:

Originally Posted by axl718 (Post 4881481)
Hi,
I have a data file that has many lines in it, that is space delimited. What I want to do is match lines in the file that have the word host in it and then i want to replace columns 3-6 with data that I have in another file. The other file only has the information that belongs on those specific lines. Any ideas?

What happens to the lines in File1 which do not contain the word host?
Are they discarded? Are they written to the output file, unchanged?

What happens if there are more lines in File1 which contain the word host than there are lines in File2?

Daniel B. Martin

axl718 01-31-2013 12:56 PM

Quote:

Originally Posted by danielbmartin (Post 4881532)
What happens to the lines in File1 which do not contain the word host?
Are they discarded? Are they written to the output file, unchanged?

What happens if there are more lines in File1 which contain the word host than there are lines in File2?

Daniel B. Martin

The lines in file one that do not contain the word host remain in the file unchanged. There won't be more lines in file1 that contain the word host than there are lines in file2 because file2 was generated after the /etc/hosts file was updated for all the computers in the directory structure which is where file1 was generated from also. However I guess for a sanity check I could count the lines in file1 that match hosts and the lines in file1 and make sure the number is the same.

axl718 01-31-2013 12:59 PM

I did a sanity check and the number of lines that need to be fixed is 62 and the number of lines that I have data for is also 62.

danielbmartin 01-31-2013 01:02 PM

Quote:

Originally Posted by axl718 (Post 4881526)
I really don't know awk or sed but thought they might be of use here somehow.

You are on the right track. sed and awk are powerful general-use commands. sed is well-suited to text processing; awk is good for that and much more.

It will be helpful if you identify your two input files with names, even if the names are arbitrary such as File1 and File2.

It will be helpful if you give samples of your input files which contain both match and no-match cases.

It will be helpful if you give a sample of the desired output file which corresponds to the sample input files. It gives us a standard of comparison -- if we write a code snippet, we can judge its correctness.

Daniel B. Martin

axl718 01-31-2013 01:06 PM

Quote:

Originally Posted by danielbmartin (Post 4881546)
You are on the right track. sed and awk are powerful general-use commands. sed is well-suited to text processing; awk is good for that and much more.

It will be helpful if you identify your two input files with names, even if the names are arbitrary such as File1 and File2.

It will be helpful if you give samples of your input files which contain both match and no-match cases.

It will be helpful if you give a sample of the desired output file which corresponds to the sample input files. It gives us a standard of comparison -- if we write a code snippet, we can judge its correctness.

Daniel B. Martin

I gave samples of the input files above I will edit them and call them file1 and file2 here is an example of what i need the output to be.

rwxrwxrwx 0/0 0 Dec 17 15:25 2002 ./
rwxrwxrwx 0/0 0 Dec 17 15:37 2002 ./blah/
rwxrwxr-x 0/3 0 Dec 17 15:37 2002 ./blah/home/
rwxrwxr-x 8654/8655 0 Dec 17 15:53 2002 ./blah/home/foo/
rw-rw-rw- 0/3 889 Jul 29 14:14 2002 ./blah/home/foo/hostlist
rwxrwxrwx 0/3 0 Dec 16 16:02 2002 ./comp0/
rwxrwxrwx 0/3 0 Dec 17 10:23 2002 ./comp0/etc/
rw-rw-rw- 0/3 477 Jul 29 14:14 2002 ./comp0/etc/tab
r--r--r-- 0/3 7329 Jul 29 14:14 2002 ./comp0/etc/bootinfo
rwxrwxrwx 0/3 0 Mar 26 11:07 2003 ./comp0/opt/
rwxrwxrwx 0/3 0 Mar 26 10:45 2003 ./comp0/opt/RARED/
rw-rw-r-- 0/0 133120 Dec 6 16:05 2002 ./comp0/opt/RARED/configs.tar
rwxrwxrwx 0/0 0 Dec 16 17:01 2002 ./comp1/
rwxrwxrwx 0/0 0 Dec 16 17:01 2002 ./comp1/etc/
r--r--r-- 0/0 5797 Dec 10 10:24 2002 ./comp1/etc/bootinfo
r--r--r-- 0/3 22696 Jan 31 11:04 2013 ./comp1/etc/hosts

axl718 01-31-2013 01:58 PM

So far I have come up with these commands that will give me the file2 and also find the lines in file1 that need to be changed
ls -Rl | grep hosts | cut -d " " -f 5-8 > file2
sed -n '/hosts/p' file1 > linestobechanged

whizje 01-31-2013 02:02 PM

Interesting problem replace each match with next item from list.

danielbmartin 01-31-2013 02:39 PM

Quote:

Originally Posted by axl718 (Post 4881551)
... here is an example of what i need the output to be.
Code:

r--r--r-- 0/3 22696 Jan 31 11:04 2013 ./comp1/etc/hosts

Where did the 2013 come from?
It isn't in either input file.

Daniel B. Martin

axl718 01-31-2013 02:45 PM

It belongs in file1 I forgot to add that column. Sorry

danielbmartin 01-31-2013 02:58 PM

I changed the input files to make a better test.
File1 ...
Code:

22696 Jan 31 11:04 2013
23035 Feb 01 10:28 2013
22694 Feb 03 11:04 2013
23033 Feb 05 10:28 2013

File2 ...
Code:

rwxrwxrwx 0/0 0 Dec 17 15:25 2002 ./
rwxrwxrwx 0/0 0 Dec 17 15:37 2002 ./blah/
rwxrwxr-x 0/3 0 Dec 17 15:37 2002 ./1b_2a_2b_4a_4b_a11_a21_c11_c21_cndadj/home/
rwxrwxr-x 8654/8655 0 Dec 17 15:53 2002 ./blah/home/foo/
rw-rw-rw- 0/3 889 Jul 29 14:14 2002 ./blah/home/foo/hostlist
rwxrwxrwx 0/3 0 Dec 16 16:02 2002 ./comp0/
rwxrwxrwx 0/3 0 Dec 17 10:23 2002 ./comp0/etc/
rw-rw-rw- 0/3 477 Jul 29 14:14 2002 ./comp0/etc/tab
r--r--r-- 0/3 7329 Jul 29 14:14 2002 ./comp0/etc/bootinfo
rwxrwxrwx 0/3 0 Mar 26 11:07 2003 ./comp0/opt/
rwxrwxrwx 0/3 0 Mar 26 10:45 2003 ./comp0/opt/RARED/
rw-rw-r-- 0/0 133120 Dec 6 16:05 2002 ./comp0/opt/RARED/configs.tar
rwxrwxrwx 0/0 0 Dec 16 17:01 2002 ./comp1/
rwxrwxrwx 0/0 0 Dec 16 17:01 2002 ./comp1/etc/
r--r--r-- 0/0 5797 Dec 10 10:24 2002 ./comp1/etc/bootinfo
r--r--r-- 0/3 22483 Dec 16 15:25 2002 ./comp1/etc/hosts
r--r--r-- 0/3 22483 Dec 17 15:35 2002 ./comp1/etc/hosts
r--r--r-- 0/3 22483 Dec 18 15:45 2002 ./comp1/etc/hosts
r--r--r-- 0/3 22483 Dec 19 15:55 2002 ./comp1/etc/hosts

This code ...
Code:

awk 'BEGIN {while (getline < "'"$InFile1"'") a[++j]=$0};
    {if ($0 !~ /hosts/) {print}
  else
    {print $1,$2,a[++k],$8} }' $InFile2 > $OutFile

... produced this output file.
Code:

rwxrwxrwx 0/0 0 Dec 17 15:25 2002 ./
rwxrwxrwx 0/0 0 Dec 17 15:37 2002 ./blah/
rwxrwxr-x 0/3 0 Dec 17 15:37 2002 ./1b_2a_2b_4a_4b_a11_a21_c11_c21_cndadj/home/
rwxrwxr-x 8654/8655 0 Dec 17 15:53 2002 ./blah/home/foo/
rw-rw-rw- 0/3 889 Jul 29 14:14 2002 ./blah/home/foo/hostlist
rwxrwxrwx 0/3 0 Dec 16 16:02 2002 ./comp0/
rwxrwxrwx 0/3 0 Dec 17 10:23 2002 ./comp0/etc/
rw-rw-rw- 0/3 477 Jul 29 14:14 2002 ./comp0/etc/tab
r--r--r-- 0/3 7329 Jul 29 14:14 2002 ./comp0/etc/bootinfo
rwxrwxrwx 0/3 0 Mar 26 11:07 2003 ./comp0/opt/
rwxrwxrwx 0/3 0 Mar 26 10:45 2003 ./comp0/opt/RARED/
rw-rw-r-- 0/0 133120 Dec 6 16:05 2002 ./comp0/opt/RARED/configs.tar
rwxrwxrwx 0/0 0 Dec 16 17:01 2002 ./comp1/
rwxrwxrwx 0/0 0 Dec 16 17:01 2002 ./comp1/etc/
r--r--r-- 0/0 5797 Dec 10 10:24 2002 ./comp1/etc/bootinfo
r--r--r-- 0/3 22696 Jan 31 11:04 2013 ./comp1/etc/hosts
r--r--r-- 0/3 23035 Feb 01 10:28 2013 ./comp1/etc/hosts
r--r--r-- 0/3 22694 Feb 03 11:04 2013 ./comp1/etc/hosts
r--r--r-- 0/3 23033 Feb 05 10:28 2013 ./comp1/etc/hosts

Daniel B. Martin


All times are GMT -5. The time now is 06:40 PM.