LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 04-14-2011, 06:19 PM   #1
di11rod
Member
 
Registered: Jan 2004
Location: Austin, TEXAS
Distribution: CentOS 6.5
Posts: 207

Rep: Reputation: 32
shell question: pad end of each line with spaces to = 80 chars ??


I've searched the forums and the google looking for a means to do this and haven't found anything I can use.

I have a large file that looks like this:


Code:
18000034161828M850
18000034172676M850
98   093095
D    01   6H0NH    031111    18AH001HBS
18000032384257M850
18000032225047M850
18000032550899M850
18000032561615M850
18000032342743M850
I need to add spaces at the end of each line to ensure that every line has 80 chars before the carriage return. I was thinking something like this, but it doesn't do the right thing:

Code:
cat filename | sed -e 's/$/(bunch of spaces)/' | cut -c1-80 > filename2
Any ideas? I'm on fedora, so I can use awk, sed, bash, ksh, etc.

Appreciatively,

di11rod
 
Old 04-14-2011, 06:42 PM   #2
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,395
Blog Entries: 2

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
This might do the trick.
Code:
while read line; do printf "%-80s\n" $line; done < filename
--- rod.
 
Old 04-14-2011, 08:14 PM   #3
kurumi
Member
 
Registered: Apr 2010
Posts: 223

Rep: Reputation: 45
Ruby(1.9+)

Code:
$ ruby -i.bak -ne 'puts $_.chomp.ljust(80)' file
 
Old 04-15-2011, 12:09 AM   #4
di11rod
Member
 
Registered: Jan 2004
Location: Austin, TEXAS
Distribution: CentOS 6.5
Posts: 207

Original Poster
Rep: Reputation: 32
Smile

Thank you so much for these suggestions! I'll try them when I get to work tomorrow.

Appreciatively,

di11rod
 
Old 04-15-2011, 02:09 AM   #5
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 147Reputation: 147
Quote:
Originally Posted by theNbomr View Post
Code:
while read line; do printf "%-80s\n" $line; done < filename
Should work in AWK too

Code:
awk '{printf "%-80s\n", $1}' filename
 
Old 04-15-2011, 11:02 AM   #6
di11rod
Member
 
Registered: Jan 2004
Location: Austin, TEXAS
Distribution: CentOS 6.5
Posts: 207

Original Poster
Rep: Reputation: 32
Kurumi-

I think your suggestion just cuts the line to no greater than 80 chars. At least that's what the filename.bak file looks like.

Code:
$ ruby -i.bak -ne 'puts $_.chomp.ljust(80)' file
thenbomr-

Code:
while read line; do printf "%-80s\n" $line; done < onefile > twofile
Yours is the closest. It puts 80 chars at the end of each line, but some of my lines have spaces between elements, and your solution chops each at the space and puts those values on separate lines.

So, the lines that look like this come out fine as desired:

Code:
18000034002519M850
But the lines that look like this get broken up into separate lines:

Code:
98   098100
D    01   6H0NH    031111    18AH001HBS
Is there a way to tie the delimiter to an existing carriage return instead of a space?

Telengard-

The awk syntax doesn't work because it is referencing field one with the $1 part. On the lines with spaces, it breaks each value out to separate lines. If there were a way to avoid referencing the fields and treat the entire line as a field...

Thanks for these suggestions. If there's a way to tweak theNbomr's solution so it deals with the lines with spaces, then I'm golden.

Appreciatively,

di11rod
 
Old 04-15-2011, 11:39 AM   #7
di11rod
Member
 
Registered: Jan 2004
Location: Austin, TEXAS
Distribution: CentOS 6.5
Posts: 207

Original Poster
Rep: Reputation: 32
I apologize for being a lazy baby asking the programming forum to spoon-feed me a solution. Duh. I figured it out. I used sed to substitute the blanks for ':' and then used the awk syntax provided by Telengard, then used sed to change the colons back to spaces.

I love the elegance of command-line scripting. Here is the solution!


Code:
cat onefile | sed 's/ /:/g' | awk '{printf "%-80s\n", $1}' | sed 's/:/ /g' > twofile
Thanks for taking the time to help me out.

Appreciatively,

di11rod
 
Old 04-15-2011, 01:11 PM   #8
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 147Reputation: 147
Quote:
Originally Posted by di11rod View Post
The awk syntax doesn't work because it is referencing field one with the $1 part. On the lines with spaces, it breaks each value out to separate lines. If there were a way to avoid referencing the fields and treat the entire line as a field...
Sorry, that was my mistake. I was testing it that way and forgot to change it. I should have posted this:

Code:
foo$ awk '{printf "%-80s\n", $0}' onefile
18000034161828M850
18000034172676M850
98   093095
D    01   6H0NH    031111    18AH001HBS
18000032384257M850
18000032225047M850
18000032550899M850
18000032561615M850
18000032342743M850
foo$
Quote:
Originally Posted by di11rod View Post
Code:
cat onefile | sed 's/ /:/g' | awk '{printf "%-80s\n", $1}' | sed 's/:/ /g' > twofile
I'm quite certain it can be done much more elegantly and efficiently, if you are willing to put a little more time and study into it.

We can eliminate the UUOC. The sed program accepts a second argument for the file name to process. That gives us this:

Code:
foo$ sed 's/ /:/g' onefile | awk '{printf "%-80s\n", $1}' | sed 's/:/ /g'
18000034161828M850
18000034172676M850
98   093095
D    01   6H0NH    031111    18AH001HBS
18000032384257M850
18000032225047M850
18000032550899M850
18000032561615M850
18000032342743M850
foo$
Now combine that knowledge with what we learned from the corrected awk command above. The sed command is not needed at all; awk can do the entire job itself.

Code:
foo$ awk '{printf "%-80s\n", $0}' onefile
18000034161828M850
18000034172676M850
98   093095
D    01   6H0NH    031111    18AH001HBS
18000032384257M850
18000032225047M850
18000032550899M850
18000032561615M850
18000032342743M850
foo$
To demonstrate that the lines are really padded as you want I will use tr to turn all the spaces into hyphens.

Code:
foo$ awk '{printf "%-80s\n", $0}' onefile | tr ' ' '-'
18000034161828M850--------------------------------------------------------------
18000034172676M850--------------------------------------------------------------
98---093095---------------------------------------------------------------------
D----01---6H0NH----031111----18AH001HBS-----------------------------------------
18000032384257M850--------------------------------------------------------------
18000032225047M850--------------------------------------------------------------
18000032550899M850--------------------------------------------------------------
18000032561615M850--------------------------------------------------------------
18000032342743M850--------------------------------------------------------------
foo$
So here's my final answer on how to pad lines to 80 characters with spaces using AWK.

Code:
awk '{printf "%-80s\n", $0}' FILENAME
If the input file is named onefile and the output file is to be named twofile then it can be invoked thusly:

Code:
awk '{printf "%-80s\n", $0}' onefile > twofile
If you are processing your file with awk, then use the full capabilities of the AWK language to do as much of the work as possible. I can't think of a single reason to ever pipe sed into awk or the reverse.

Many more solutions are possible. Unix, and Linux by extension, is rife with text processing programs. Each has its own place and serves its own purpose. My own approach is to learn one at a time as well as I can before moving on to the next. AWK is my current obsession, but maybe soon it will be sed, ruby, or python.

If you feel your question has been answered then please consider using the thread tools to mark this thread solved. Otherwise please post back to let us know what problems remain.

HTH
 
1 members found this post helpful.
Old 04-15-2011, 02:48 PM   #9
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,395
Blog Entries: 2

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
Quote:
Originally Posted by di11rod View Post
Kurumi-

thenbomr-

Code:
while read line; do printf "%-80s\n" $line; done < onefile > twofile
Yours is the closest. It puts 80 chars at the end of each line, but some of my lines have spaces between elements, and your solution chops each at the space and puts those values on separate lines.

So, the lines that look like this come out fine as desired:

Code:
18000034002519M850
But the lines that look like this get broken up into separate lines:

Code:
98   098100
D    01   6H0NH    031111    18AH001HBS
Is there a way to tie the delimiter to an existing carriage return instead of a space?
Sorry, I don't know how I managed to post the wrong version. This seems to work.
Code:
while read line; do printf "%-80s\n" "$line"; done < filename
--- rod.
 
Old 04-16-2011, 05:57 AM   #10
AnanthaP
Member
 
Registered: Jul 2004
Location: Chennai, India
Distribution: UBUNTU 5.10 since Jul-18,2006 on Intel 820 DC
Posts: 627

Rep: Reputation: 137Reputation: 137
In awk.

if (len($0) < 80) printf "%-80s\n", $0 ;
else print $0
 
Old 04-16-2011, 07:07 AM   #11
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
Quote:
Originally Posted by AnanthaP View Post
In awk.

if (len($0) < 80) printf "%-80s\n", $0 ;
else print $0
what if the length is greater than 80 ? You don't need to answer, but that's the first thing I thought of when I saw this thread.
 
Old 04-16-2011, 10:36 PM   #12
kurumi
Member
 
Registered: Apr 2010
Posts: 223

Rep: Reputation: 45
Quote:
Originally Posted by di11rod View Post
Kurumi-

I think your suggestion just cuts the line to no greater than 80 chars. At least that's what the filename.bak file looks like.

Code:
$ ruby -i.bak -ne 'puts $_.chomp.ljust(80)' file
filename.bak is the original file. ljust() is not used for "cutting".
 
Old 04-16-2011, 11:54 PM   #13
AnanthaP
Member
 
Registered: Jul 2004
Location: Chennai, India
Distribution: UBUNTU 5.10 since Jul-18,2006 on Intel 820 DC
Posts: 627

Rep: Reputation: 137Reputation: 137
Quote:
I think your suggestion just cuts the line to no greater than 80 chars. At least that's what the filename.bak file looks like.
I think the OP didn't want to cut lines > 80 in length. So I just added the else part in awk.

Clearly, the end application must decide what to do with lines having length > 80. But you dont want to lose the additional bytes when coding a module to pad to length= 80.
 
Old 04-17-2011, 03:53 AM   #14
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,755

Rep: Reputation: 466Reputation: 466Reputation: 466Reputation: 466Reputation: 466
Set the IFS to newline to take care of lines with spaces, and simply echo any lines longer than 80 chars. Why use sed/awk/perl/ruby/python when bash can do it all?
Code:
( IFS=$'\n' ; while read line; do 
  if [[ ${#line} -gt 80 ]] ; then
    echo $line
  else
    printf "%-80s\n" "$line"
  fi
done < filename )
 
1 members found this post helpful.
Old 04-17-2011, 03:56 AM   #15
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 15

Rep: Reputation: 233Reputation: 233Reputation: 233
Quote:
Originally Posted by di11rod View Post
thenbomr-

Code:
while read line; do printf "%-80s\n" $line; done < onefile > twofile
Yours is the closest. It puts 80 chars at the end of each line, but some of my lines have spaces between elements, and your solution chops each at the space and puts those values on separate lines.
Perhaps $line should just be enclosed in double quotes to prevent it from expanding to multiple arguments:

Code:
while read line; do printf "%-80s\n" "$line"; done < onefile > twofile
Another way with bash:

Code:
SPACES='                                                                                '
while read LINE; do TEMP=$LINE$SPACES; echo "${TEMP:0:80}"; done < onefile > twofile
Placing in a script:

script.sh
Code:
#!/bin/bash
SPACES='                                                                                '
while read LINE; do TEMP=$LINE$SPACES; echo "${TEMP:0:80}"; done
Code:
bash script.sh < onefile > twofile
---- Update ----

To prevent chopping longer lines:

Code:
#!/bin/bash
SPACES='                                                                                '
while read LINE; do
    if [[ ${#LINE} -ge 80 ]]; then
        echo "$LINE"
    else
        TEMP=$LINE$SPACES
        echo "${TEMP:0:80}"
    fi
done

Last edited by konsolebox; 04-17-2011 at 04:00 AM.
 
  


Reply

Tags
awk, line, pad, printf


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
head adds chars to end of each line (Red Hat Enterprise Linux) CheckiSt Linux - General 13 04-14-2010 03:52 AM
Shell scripting: How to add characters at the end of the line Micro420 Programming 7 05-18-2007 01:56 AM
deleting all spaces at end of line in vi Tonatiuh Linux - General 4 08-16-2006 02:34 PM
got a syntax error which shows unexpected end of line when tried to run a shell scrip racer_mec Linux - Newbie 1 01-10-2005 01:43 AM


All times are GMT -5. The time now is 02:53 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration