LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-29-2014, 10:29 AM   #1
mfarch99
LQ Newbie
 
Registered: Mar 2012
Posts: 20

Rep: Reputation: Disabled
Extract data from a file using bash then use part of that data to rename the input fi


I have a file that has data that changes consistently. I need to make the file name unique and use a portion of the data that is always constant as part of the new filename. Basically, this is what is needed:
1) read the file line by line
2) find " 70 " in the first 5 columns
3) Once found, start extracting data that starts in column 12 for 18
4) Use that data found and place in a variable
5) take the variable and append to the end of the input file name.
6) result would be a new file name

I have the bash code to read a file line by line, but where I'm failing is gathering the data from the file to place in a variable.

As an example, the input file name would be RPT70
I need to have the result of this script create a file name that looks something like this:

RPT70.$variablename
 
Old 08-29-2014, 10:33 AM   #2
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,149

Rep: Reputation: 887Reputation: 887Reputation: 887Reputation: 887Reputation: 887Reputation: 887Reputation: 887
can you post your code so far so someone can help debug it.

i suggest looking into awk.
 
Old 08-29-2014, 10:35 AM   #3
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,211

Rep: Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612
Also a piece of the file would be helpful.
 
Old 08-29-2014, 01:02 PM   #4
mfarch99
LQ Newbie
 
Registered: Mar 2012
Posts: 20

Original Poster
Rep: Reputation: Disabled
Sorry, Here is a piece of the file

Sorry,
here is the code and a piece if the file:

#!/bin/bash
filename="mike.txt"
while read -r line
do
name=$line
echo "Name read from file - $name"

done < "$filename"



1SASSHIS8 CA WORKLOAD AUTOMATION SE CONTROL CARD EDIT 08/28/14 12:50 PAGE 1
0------------------------------------------------------------------------------------------------------------------------------------
0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 EDIT COMMENTS
....5....0....5....0....5....0....5....0....5....0....5....0....5....0....5....0----------------------------------------------------
070 142401230142401250 HIS8-09 NO CONTROL CARD ERRORS DETECTED.
70 142401230142401250 HIS8-18 CONTROL CARD EDIT FINAL RESULTS.
1SASSHIS8 CA WORKLOAD AUTOMATION SE CONTROL CARD EDIT 08/28/14 12:50 PAGE 2
0------------------------------------------------------------------------------------------------------------------------------------
0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 EDIT COMMENTS
....5....0....5....0....5....0....5....0....5....0....5....0....5....0....5....0----------------------------------------------------
- *** T O T A L S ***
0 001 CARDS READ.
0 001 CARDS ACCEPTED.
0 000 CARDS REJECTED.
-
-2471291 ...LOG RECORDS READ.
00011526 ...RECORDS EXTRACTED FOR --- 70
1

Last edited by mfarch99; 08-29-2014 at 01:03 PM.
 
Old 08-29-2014, 01:18 PM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Based on the data provided:

1. What is your definition of a column?
2. What would be the output based on the supplied data?
3. What does 'column 12 for 18' mean?
4. The string being searched for ' 70 ', does it have spaces and/or tabs either side of it?
 
Old 08-29-2014, 01:39 PM   #6
mfarch99
LQ Newbie
 
Registered: Mar 2012
Posts: 20

Original Poster
Rep: Reputation: Disabled
data

It is hard to identify the data in this thread. The file is very long. and about 133 columns wide. The data I'm looking for is in the file and is a parameter card representation. The information would look like this 70 142401050142401110. The first column or character would be a space followed by 70 then 8 spaces and then the data I need to retrieve. The data being retrieved changes. The only constant would be " 70 "

Last edited by mfarch99; 08-29-2014 at 01:40 PM.
 
Old 08-29-2014, 01:48 PM   #7
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 5,393

Rep: Reputation: 409Reputation: 409Reputation: 409Reputation: 409Reputation: 409
NO CONTROL CARD ERRORS DETECTED.

Is it 1 column or 5 ?
 
Old 08-29-2014, 08:40 PM   #8
mfarch99
LQ Newbie
 
Registered: Mar 2012
Posts: 20

Original Poster
Rep: Reputation: Disabled
Starting position

The data that wiil be searched starts in column(position) 1 of the string. All counts be off that position. I mentioned the columns, positions earlier in the thread. So basically the search string to lokk for starts in position 1 as a space the next 2 are numeric 70 and then the next two are spaces. Like this-- " 70 " without the quotes.

The data to be extracted would be on that same line but would start around 8 positions in and go for 12 positions. That data I need to extract to a variable and use as part of the new file name. An example was provided above. Thank you..

Last edited by mfarch99; 08-29-2014 at 08:44 PM.
 
Old 08-30-2014, 03:22 AM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Ok ... I think I am getting a better picture (I hope)

So as far as columns go we are simply looking at 1 column is 1 character (yes)

So to restate the question, we are searching for lines that start with a space followed by the number 70 and some more white space. We are trying to retrieve data from the the 8th
character on the line and the subsequent 12 characters.

Based on this the current data return no data, however I believe that due to you pasting directly within your question we have lost information about white space (in the future
if you place code or data in [code][/code] tags, this type of information will be preserved).

So i have edited the information to look as below:
Code:
1SASSHIS8 CA WORKLOAD AUTOMATION SE CONTROL CARD EDIT 08/28/14 12:50 PAGE 1
0------------------------------------------------------------------------------------------------------------------------------------
0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 EDIT COMMENTS
....5....0....5....0....5....0....5....0....5....0....5....0....5....0....5....0----------------------------------------------------
070 142401230142401250 HIS8-09 NO CONTROL CARD ERRORS DETECTED.
 70 142401230142401250 HIS8-18 CONTROL CARD EDIT FINAL RESULTS.
1SASSHIS8 CA WORKLOAD AUTOMATION SE CONTROL CARD EDIT 08/28/14 12:50 PAGE 2
0------------------------------------------------------------------------------------------------------------------------------------
0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 EDIT COMMENTS
....5....0....5....0....5....0....5....0....5....0....5....0....5....0....5....0----------------------------------------------------
- *** T O T A L S *** 
0 001 CARDS READ.
0 001 CARDS ACCEPTED.
0 000 CARDS REJECTED.
-
-2471291 ...LOG RECORDS READ.
00011526 ...RECORDS EXTRACTED FOR --- 70
1
Here we now see on line 6 we have the scenario where the following grep will pick up data we are looking for:
Code:
$ grep -E '^ 70 ' in_file
 70 142401230142401250 HIS8-18 CONTROL CARD EDIT FINAL RESULTS.
Assuming I am on the correct path here, would the data to be returned be:
Code:
301424012
 
Old 09-05-2014, 08:52 AM   #10
mfarch99
LQ Newbie
 
Registered: Mar 2012
Posts: 20

Original Poster
Rep: Reputation: Disabled
Sorry I did not het back to you sooner. This is what would be needed from that line 142401230142401250.
I know if I just do an awk '{print $2}' I can retrieve the data I want. But how to I place that data into a variable so I can use that to append to the end of the original file name.

Last edited by mfarch99; 09-05-2014 at 10:28 AM.
 
Old 09-05-2014, 11:33 AM   #11
mfarch99
LQ Newbie
 
Registered: Mar 2012
Posts: 20

Original Poster
Rep: Reputation: Disabled
Looks like I have what is needed. Thanks. Here is what I have as the final code. Crude but works:
#!/bin/bash

RPT70EXT=$(grep -E '^ 70 ' RPT70 | awk '{print $2}')

mv -f RPT70 RPT70.$RPT70EXT
 
Old 09-05-2014, 11:51 AM   #12
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Well that would be a grep and awk solution, not a bash one.

Also, awk can already match regular expressions, so grep is not needed.

Here are some other alternatives to return the required data:
Code:
# awk
awk '/^ 70 /{print $2}' RPT70

# sed
sed -rn 's/^ 70 *([^ ]*).*/\1/p' RPT70

# bash
#!/usr/bin/env bash

regex='^ 70 *([^ ]*)'

while IFS=$'\n' read -r line
do
	[[ "$line" =~ $regex ]] && echo "${BASH_REMATCH[1]}"
done<RPT70

# alternate bash
#!/usr/bin/env bash

while read -r num data _
do
	[[ "$num" == 70 ]] && echo "$data"
done<RPT70
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Python: Extract some data from one text file a.albreiki Programming 10 06-10-2014 01:27 AM
[SOLVED] extract data from file in using perl divyashree Programming 4 12-30-2012 05:57 AM
extract last few of line data from txt file irwinyeo Programming 7 12-20-2011 10:47 AM
How do I extract data from nmblookup in bash shell script? utahnix Linux - General 23 10-12-2009 05:28 AM
Bash: Data Input Verification Help! Vozx Programming 4 12-04-2005 08:23 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:38 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration