Huge Data Set Analysis, Shell Script to copy specific HEX Pairs into a separate file
I've got a huge data set that I'm working with, the data is dumped from another script into a file in real time and consists of 16 Groupings of 2 Hex Characters times 10 rows, then the sequence repeats with different values. This happens over and over again.. "Multi-GB a day."
Here's an example of what I'm looking at. Code:
5f 30 59 ed ea e1 8c 61 57 d7 36 5b e6 40 90 8c So if I break it down I know the following: - Each grouping is 10 rows long - Each grouping is 16 columns of HEX pairs wide + spaces - There is one blank line between every 10 rows - The specific HEX Pair I need to output is row 5 x column 7 - I need to copy the HEX pair out to a new file "so cat to a file (>)" - I need the output file to be historic, so in this case I don't want to overwrite it. (>>) So what I can infer: - I need to use something like the cut command to get the right column - I need another command to grab the correct line the first time - I need some sort of a loop that can increment the line grab by 11 each time it loops. --- Any suggestions? |
Code:
$ cat hex.pl |
Quote:
Sets a starting point of row 1 counts down to row 5 counts over to the correct grouping cuts that grouping then starts the entire process over again based on the next blank line Does that sound about right? |
If you look at the data, there seem to be repeating patterns that are not in sync with the 10-line formatting. Are you sure which data you need to extract?
|
Quote:
|
Or by using paragraph read:
Code:
#!/usr/bin/perl Quote:
|
Quote:
Both methods do work though so thank you both for a quick solution. |
It is indeed simple, and there are many ways to perform a task.
I chose what I thought might be more self-explanatory to you, where you could change the row and column easily. I personally also think some input validation is worthwhile, as well is being able to ignore garbage that might follow after the 16th col. But too each his own. Cheers. |
Well formed data can pardon a multitude of sins.
What if the "blank line" does indeed contain whitespace ???. |
A trivial fix:
if (/^$/) { if (/^\s*$/) { |
The bottom-line of this thread so-far is that:
In such an environment, therefore, it is very easy to make the rueful discovery that ... you're making things much harder on yourself than you actually needed to, a-n-d, you didn't even know it. Hey... "no harm, no foul!" If, say, "you cut your teeth on Microsoft Windows," where tools other than Visual Basic (heh...) are essentially non-existent, then you might be entirely unaccustomed to find yourself in the "embarrassment of riches" that is Unix/Linux. No problem... I am not making fun at your expense. Welcome aboard! Now you know "what all the fuss is about!" :) |
Quote:
|
All times are GMT -5. The time now is 07:25 AM. |