Need a script to break a block of text on to different lines.

Thaidog · 12-15-2010, 07:53 AM

I have a clump of text that needs to be broke up:

gdbm Sat 07 Feb 2009 03:28:18 AM EST libattr Sat 07 Feb 2009 03:28:18 AM EST db4 Sat 07 Feb 2009 03:28:19 AM EST mktemp Sat 07 Feb 2009 03:28:19 AM EST keyutils Sat 07 Feb 2009 03:28:20 AM EST pcre Sat 07 Feb 2009 03:28:21 AM EST setserial Sat 07 Feb 2009 03:28:24 AM EST zlib Sat 07 Feb 2009 03:28:24 AM EST gawk Sat 07 Feb 2009 03:28:25 AM EST readline Sat 07 Feb 2009 03:28:26 AM EST rhpl Sat 07 Feb 2009 03:28:28 AM EST cracklib-dicts Sat 07 Feb 2009 03:28:37 AM EST setools Sat 07 Feb 2009 03:28:37 AM EST hal Sat 07 Feb 2009 03:28:38 AM EST which Sat 07 Feb 2009 03:28:39 AM EST

Is there a way to get everything after the EDT in the text to be moved to a new line?

druuna · 12-15-2010, 07:57 AM

Hi,

I'm guessing EDT is a typo and should be EST.

Quote:

sed 's/EST /EST\n/g' infile

Example output:

Code:

$ sed 's/EST /EST\n/g' infile 
gdbm Sat 07 Feb 2009 03:28:18 AM EST
libattr Sat 07 Feb 2009 03:28:18 AM EST
db4 Sat 07 Feb 2009 03:28:19 AM EST
mktemp Sat 07 Feb 2009 03:28:19 AM EST
keyutils Sat 07 Feb 2009 03:28:20 AM EST
pcre Sat 07 Feb 2009 03:28:21 AM EST
setserial Sat 07 Feb 2009 03:28:24 AM EST
zlib Sat 07 Feb 2009 03:28:24 AM EST
gawk Sat 07 Feb 2009 03:28:25 AM EST
readline Sat 07 Feb 2009 03:28:26 AM EST
rhpl Sat 07 Feb 2009 03:28:28 AM EST
cracklib-dicts Sat 07 Feb 2009 03:28:37 AM EST
setools Sat 07 Feb 2009 03:28:37 AM EST
hal Sat 07 Feb 2009 03:28:38 AM EST
which Sat 07 Feb 2009 03:28:39 AM EST

Hope this helps

GrapefruiTgirl · 12-15-2010, 07:58 AM

If you echo that chunk of text into sed, and use sed to change "EDT" to "EDT\n" (to insert a newline after each EDT) it then looks like this:

Code:

gdbm Sat 07 Feb 2009 03:28:18 AM EST
libattr Sat 07 Feb 2009 03:28:18 AM EST
db4 Sat 07 Feb 2009 03:28:19 AM EST
mktemp Sat 07 Feb 2009 03:28:19 AM EST
keyutils Sat 07 Feb 2009 03:28:20 AM EST
pcre Sat 07 Feb 2009 03:28:21 AM EST
setserial Sat 07 Feb 2009 03:28:24 AM EST
zlib Sat 07 Feb 2009 03:28:24 AM EST
gawk Sat 07 Feb 2009 03:28:25 AM EST
readline Sat 07 Feb 2009 03:28:26 AM EST
rhpl Sat 07 Feb 2009 03:28:28 AM EST
cracklib-dicts Sat 07 Feb 2009 03:28:37 AM EST
setools Sat 07 Feb 2009 03:28:37 AM EST
hal Sat 07 Feb 2009 03:28:38 AM EST
which Sat 07 Feb 2009 03:28:39 AM EST

Note that I used "EST" rather than "EDT", since I see no instances of "EDT" in the input.

I did exactly this:

Code:

echo "all that text" | sed 's/EST /EST\n/g'

GrapefruiTgirl · 12-15-2010, 07:59 AM

@ Druuna - nice one, I like that method!

grail · 12-15-2010, 08:15 AM

Well I think the sed is easier, but:

Code:

awk 'BEGIN{FS="EST ";OFS="EST\n"}$1=$1' file

Thaidog · 12-15-2010, 08:33 AM

Thanks I used:
sed 's/EDT /EDT\n/g' server.txt >> newfile.txt

and that worked with one problem... every incomplete string that was broke to a second line in the first file breaks to a second line in the new file... for instance:

[root@server tyler2]# head server.txt.bak
bgcc Tue 07 Apr 2009 09:24:38 AM EDT setup Tue 07 Apr 2009 09:24:39 AM EDT
audit-libs Tue 07 Apr 2009 09:24:47 AM EDT e2fsprogs Tue 07 Apr 2009 09:24:48
AM EDT expat Tue 07 Apr 2009 09:24:48 AM EDT keyutils-libs Tue 07 Apr 2009
09:24:48 AM EDT libselinux Tue 07 Apr 2009 09:24:48 AM EDT libstdc++ Tue 07
Apr 2009 09:24:49 AM EDT mingetty Tue 07 Apr 2009 09:24:49 AM EDT iproute Tue
07 Apr 2009 09:24:49 AM EDT net-tools Tue 07 Apr 2009 09:24:51 AM EDT psmisc
Tue 07 Apr 2009 09:24:53 AM EDT sysfsutils Tue 07 Apr 2009 09:24:54 AM EDT
file Tue 07 Apr 2009 09:24:54 AM EDT perl-Filter Tue 07 Apr 2009 09:24:56 AM
EDT python Tue 07 Apr 2009 09:24:58 AM EDT rpmdb-CentOS Tue 07 Apr 2009
09:25:03 AM EDT dbus-glib Tue 07 Apr 2009 09:25:04 AM EDT tar Tue 07 Apr 2009

Now looks like:

[root@server tyler2]# head server.txt
bgcc Tue 07 Apr 2009 09:24:38 AM EDT
setup Tue 07 Apr 2009 09:24:39 AM EDT
audit-libs Tue 07 Apr 2009 09:24:47 AM EDT
e2fsprogs Tue 07 Apr 2009 09:24:48
AM EDT
expat Tue 07 Apr 2009 09:24:48 AM EDT
keyutils-libs Tue 07 Apr 2009
09:24:48 AM EDT
libselinux Tue 07 Apr 2009 09:24:48 AM EDT

Instead of:

[root@server tyler2]# head server.txt
bgcc Tue 07 Apr 2009 09:24:38 AM EDT
setup Tue 07 Apr 2009 09:24:39 AM EDT
audit-libs Tue 07 Apr 2009 09:24:47 AM EDT
e2fsprogs Tue 07 Apr 2009 09:24:48 AM EDT
expat Tue 07 Apr 2009 09:24:48 AM EDT
keyutils-libs Tue 07 Apr 2009 09:24:48 AM EDT
libselinux Tue 07 Apr 2009 09:24:48 AM EDT

Any way to fix that?

chickenjoy · 12-15-2010, 08:45 AM

I'm not good in sed but what if you first issue a command to convert them all into just one gigantic line. Then run the original sed command.

btw I don't know how to do the first part as well

markush · 12-15-2010, 08:48 AM

Well, you must not save the textfile with newlines. That means the text must be one single line. I don't know which editor you're using, but you should take "autowrap" off.

Markus

colucix · 12-15-2010, 08:59 AM

Code:

awk '{sub(/$/," "); gsub(/EDT */,"EDT\n"); printf "%s",$0}' file

druuna · 12-15-2010, 09:26 AM

Hi,

Although I would prefer colucix' solution, here's another option:

Quote:

echo $(cat infile) | sed 's/EST /EST\n/g'

grail · 12-15-2010, 10:00 AM

And just cause its fun:

Code:

awk 'BEGIN{RS="EDT *";ORS=" EDT\n"}$1=$1' file

Thaidog · 12-16-2010, 05:04 PM

Quote:

Originally Posted by colucix

Code:

awk '{sub(/$/," "); gsub(/EDT */,"EDT\n"); printf "%s",$0}' file

Thanks guys! I ended up using this one. The original problem was that I had a script to get patch info off of several servers:

Code:

#!/bin/ksh
for SERVER in `cat server.lst`
do
ssh -n -o Batchmode=yes -o ConnectTimeout=30 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null $SERVER "rpm -qa --qf %{NAME}"\t"%{INSTALLTIME:date}"\n" > /tmp/patch.out;cat /tmp/patch.out | mail -s $SERVER thaidogl@email.com"
done

But this returned ti garbbeled output you saw for each email. I was thinking it might be good to add in the awk statement somehow but some servers are reporting in EST and others in EDT so I guess I would have to throw an if statement in there somewhere which could get a little complex. Still maybe a good idea for a ton of servers...

druuna · 12-17-2010, 03:30 AM

Hi,

An if statement isn't needed to tackle the EDT / EST problem. Both can be checked and replaced by one statement.

Using the solution I gave in post #10 as starting point, it will now look like this:

Code:

echo $(cat infile) | sed 's/\(E[DS]T \)/\1\n/g'

The sed statement will now look for EST or EDT and replace accordingly (using back-referencing). Example run based on the data you provided in post #6 (some EDT's are changed to EST):

Code:

$ cat infile
bgcc Tue 07 Apr 2009 09:24:38 AM EDT setup Tue 07 Apr 2009 09:24:39 AM EDT
audit-libs Tue 07 Apr 2009 09:24:47 AM EST e2fsprogs Tue 07 Apr 2009 09:24:48
AM EDT expat Tue 07 Apr 2009 09:24:48 AM EDT keyutils-libs Tue 07 Apr 2009
09:24:48 AM EST libselinux Tue 07 Apr 2009 09:24:48 AM EDT libstdc++ Tue 07
Apr 2009 09:24:49 AM EDT mingetty Tue 07 Apr 2009 09:24:49 AM EST iproute Tue
07 Apr 2009 09:24:49 AM EDT net-tools Tue 07 Apr 2009 09:24:51 AM EDT psmisc
Tue 07 Apr 2009 09:24:53 AM EDT sysfsutils Tue 07 Apr 2009 09:24:54 AM EST
file Tue 07 Apr 2009 09:24:54 AM EDT perl-Filter Tue 07 Apr 2009 09:24:56 AM
EDT python Tue 07 Apr 2009 09:24:58 AM EST rpmdb-CentOS Tue 07 Apr 2009
09:25:03 AM EDT dbus-glib Tue 07 Apr 2009 09:25:04 AM EST tar Tue 07 Apr 2009

$ echo $(cat infile) | sed 's/\(E[DS]T \)/\1\n/g'
bgcc Tue 07 Apr 2009 09:24:38 AM EDT 
setup Tue 07 Apr 2009 09:24:39 AM EDT 
audit-libs Tue 07 Apr 2009 09:24:47 AM EST 
e2fsprogs Tue 07 Apr 2009 09:24:48 AM EDT 
expat Tue 07 Apr 2009 09:24:48 AM EDT 
keyutils-libs Tue 07 Apr 2009 09:24:48 AM EST 
libselinux Tue 07 Apr 2009 09:24:48 AM EDT 
libstdc++ Tue 07 Apr 2009 09:24:49 AM EDT 
mingetty Tue 07 Apr 2009 09:24:49 AM EST 
iproute Tue 07 Apr 2009 09:24:49 AM EDT 
net-tools Tue 07 Apr 2009 09:24:51 AM EDT 
psmisc Tue 07 Apr 2009 09:24:53 AM EDT 
sysfsutils Tue 07 Apr 2009 09:24:54 AM EST 
file Tue 07 Apr 2009 09:24:54 AM EDT 
perl-Filter Tue 07 Apr 2009 09:24:56 AM EDT 
python Tue 07 Apr 2009 09:24:58 AM EST 
rpmdb-CentOS Tue 07 Apr 2009 09:25:03 AM EDT 
dbus-glib Tue 07 Apr 2009 09:25:04 AM EST 
tar Tue 07 Apr 2009

Hope this helps.

grail · 12-17-2010, 04:46 AM

Ditto with awk:

Code:

awk 'BEGIN{RS="E[DS]T *"}{ORS=" "RT"\n"}$1=$1' file