[SOLVED] Problems reformating file

SerZKO · 01-11-2018, 04:23 AM

I have a problem with file which was reformated during some operation and looks like this:

Code:

18450 0600060009                       25  0 768188519       768188519       0030020124 000 000   10
77019           73587                       25  0 73165           730740506       0160100051 029 000
74049                                       25  0 73950                           1040030047 030 000
14048           14048                       25  0 11833           11833                      022 000
18450 0600060027                       25  0 709948778       709948778       0010030064 000 000   10
                73330                       25  0 0050312875      77020                      000 000
18450 0600040020                       25  0 703820853       703820853       0030020125 000 000   10

And I need it to look like this

Code:

18450 0600060009                       25  0 768188519       768188519       0030020124 000 000   10
77019      73587                       25  0 73165           730740506       0160100051 029 000
74049                                  25  0 73950                           1040030047 030 000
14048      14048                       25  0 11833           11833                      022 000
18450 0600060027                       25  0 709948778       709948778       0010030064 000 000   10
           73330                       25  0 0050312875      77020                      000 000
18450 0600040020                       25  0 703820853       703820853       0030020125 000 000   10

What happened was that column B has shifted 5 places to the right.
I've tried to solve it with a simple oneliner

Code:

while read do printf

but the problem is how to read empty spaces into the variable. Then I've tried perl and substr, but it wouldn't work wll. It looks like sed would be a perfect tool for that, but I just don't know how to do it as any of column where number of digits is greater then 3 can be empty. Any help appreciated

Turbocapitalist · 01-11-2018, 04:46 AM

Perl or Awk would be a perfect match for this. Can you post the code you have so far so we can see where you are stuck?

syg00 · 01-11-2018, 05:14 AM

Based on that input, if col 40 != "25" delete (first) 5 spaces. I used sed.

SerZKO · 01-11-2018, 06:57 AM

Here is perl try

Code:

#!/usr/bin/perl -w
use strict;

my $infile  =  $ARGV[0];
my $outfile = 'fil.txt';
open my $in,'<',$infile or die "Could not open $infile : $!";
open my $out,'>',$outfile or die "Could not open $outfile : $!";


while (<>){
  chomp;
  my $spaces = substr($_,58,5); 
  if ( $spaces eq '     '){
    substr($_,58,5,'');
    $_ .= $spaces;
  }
  print $out "$_\n";
}
close $in;
close $out;

__END__

It resolves a lot of problems for me, but it seems that I need several itterations to get everything done, because after running a script I've found a row like this:

Code:

17921 087001000                            25  0 60846      60846      1010010014 000 000
                                      25  0 60846      60846      1010010014 000 000     
18450 060006002                            25  0 40976521   40976521   0030020013 000 000
77019     73320                            25  0 73130      702000623  0130060067 030 000

and that's where line begins with a empty spaces...

syg00 · 01-11-2018, 07:17 AM

As you said yourself, column B has apparently been shifted - why are you testing so far away for spaces ?. Especially when it is not always true as you found. See my suggestion above (with limited data admittedly).

danielbmartin · 01-11-2018, 09:33 AM

Post withdrawn... I misread post #1 and solved the wrong problem!

Daniel B. Martin

keefaz · 01-11-2018, 09:45 AM

Quote:

Originally Posted by syg00

Based on that input, if col 40 != "25" delete (first) 5 spaces. I used sed.

Using perl reg exp,
search and capture a number from start of line (digits+ ) or not
followed by 5 spaces
replace with captured (or with nothing)

ondoho · 01-11-2018, 12:57 PM

Quote:

Originally Posted by SerZKO

the problem is how to read empty spaces into the variable.

this will work if you put the IFS to an empty string:

Code:

oldifs="$IFS"
IFS=''
while read line; do
    echo "$line"
done <file
IFS="$oldifs"

assuming bash.

sundialsvcs · 01-11-2018, 01:09 PM

What I would do is to use a real programming language to read the space-delimited values from the string – a scanf()-type function or a regular-expression could do it easily – then write out the array of values to a new file in the format that you want – e.g. a printf()-type operation.

MadeInGermany · 01-11-2018, 02:02 PM

Quote:

Originally Posted by ondoho

this will work if you put the IFS to an empty string:

Code:

oldifs="$IFS"
IFS=''
while read line; do
    echo "$line"
done <file
IFS="$oldifs"

assuming bash.

This is how you normally must do it with a for loop.
Here you can set a temporary environment for the read command

Code:

while IFS= read line; do
    echo "$line"
done <file

Advantage: save/restore of IFS is not needed, and further commands in the while-do-done block get the standard IFS.

MadeInGermany · 01-11-2018, 02:43 PM

For the original challenge, a sed solution (of course also doable in perl)

Code:

sed '
  :L
  s/^\(.\{6\}\)[[:blank:]]\(.\{32\}[[:blank:]]\)/\1\2/
  tL
' file

It deletes a blank at position 7 until there is no blank at position 40

keefaz · 01-11-2018, 02:52 PM

I was thinking of something like

Code:

perl -pe 's/^(\d+)?\s{5}/$1/' file

Or perhaps:

Code:

perl -pe 's/^(?!\s{17})(\d+)?\s{5}/$1/' file

(it you don't want to shift lines that begin with more than 17 space characters)

MadeInGermany · 01-11-2018, 03:38 PM

Seeing post#4, where my previous solution stops when the position 7 gets non-blank.
So here is a further attempt to adjust position 40

Code:

sed '
  :L
  s/^\(.\{6\}\)[[:blank:]]\(.\{32\}[[:blank:]]\)/\1\2/
  tL
  s/^\(.\{39\}\)[[:blank:]]*/\1/
' file

syg00 · 01-11-2018, 04:35 PM

Seems like the OP doesn't like our suggestions and opened a duplicate thread. He's my offering.

Code:

sed -r '/^.{39}25/! s/      / /' file

Should use classes, but a quick-and-dirty ....

SerZKO · 01-12-2018, 04:11 AM

Hej guys,

Thanks for all help. But it was sed that done it best.

Thanks again !