LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-31-2013, 09:42 PM   #1
azheruddin
Member
 
Registered: Dec 2011
Posts: 83
Blog Entries: 1

Rep: Reputation: Disabled
wanted to make file content in required format.


Hi,

I do have one file which contains some data in format as....

pqrs
abcd
efgh
xyz
....

so i wanted to make that everything should be in a single line as below format like ..

'pqrs','abcd','efgh','xyz',.....

means each word should be in a single line with seprator as , and should be quoated with single quoate.

i was trying with sed but single quoate creating a problem..i guess single quote doesnt work properly with sed utility..

so any other way to make in this format...
any other way to make this...
 
Old 01-31-2013, 10:13 PM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,424

Rep: Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823
Show us your attempt so we may assist in a solution?
 
Old 01-31-2013, 10:23 PM   #3
azheruddin
Member
 
Registered: Dec 2011
Posts: 83
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
so the file contents are..

123.gzip
234.gzp
356.gzip

and iam trying to do like

'123.gzip','234.gzip','356.gzip'

by command
sed -e 's/[0-9]*$/'&'/g' filename

but not working..
 
Old 01-31-2013, 10:47 PM   #4
sangfroid
LQ Newbie
 
Registered: Mar 2012
Posts: 10

Rep: Reputation: Disabled
hmm...

try this one..
awk '{printf "\x27%s\x27,", $1}' data

where data is the file that contains your list..

was an interesting one..
 
Old 01-31-2013, 10:47 PM   #5
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,500

Rep: Reputation: 415Reputation: 415Reputation: 415Reputation: 415Reputation: 415
Have ...
Code:
123.gzip
234.gzp
356.gzip
Want ...
Code:
'123.gzip','234.gzip','356.gzip'
Try this ...
Code:
sed "s/^/'/; s/$/'/" $InFile |paste -s -d","
Daniel B. Martin

Last edited by danielbmartin; 01-31-2013 at 10:52 PM. Reason: Simplify the code, slightly.
 
Old 01-31-2013, 10:57 PM   #6
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,500

Rep: Reputation: 415Reputation: 415Reputation: 415Reputation: 415Reputation: 415
Wink Minor nitpick

Quote:
Originally Posted by sangfroid View Post
hmm...

try this one..
awk '{printf "\x27%s\x27,", $1}' data

where data is the file that contains your list..

was an interesting one..
OP asked for this...
Code:
'123.gzip','234.gzp','356.gzip'
Your awk delivered this ...
Code:
'123.gzip','234.gzp','356.gzip',
... which has an unwanted trailing comma.

Daniel B. Martin
 
Old 01-31-2013, 11:05 PM   #7
azheruddin
Member
 
Registered: Dec 2011
Posts: 83
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
Thumbs up

Thanks! Sangfriod its worked out
can you please elaborate awk '{printf "\x27%s\x27,", $1}' .How it works out...
or redirect please redirect me to the reference page

Thanks Again!
 
Old 02-01-2013, 12:18 AM   #8
sangfroid
LQ Newbie
 
Registered: Mar 2012
Posts: 10

Rep: Reputation: Disabled
x27 is the hex code for '. Basically, it is going through each of the line of the file and printing 'line_content (variable $1) and a comma (,) and again apostrophe.... it is doing that thing for each and every line.
 
Old 02-01-2013, 03:32 AM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,424

Rep: Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823Reputation: 2823
Here is an all sed:
Code:
sed -r "s/.*/'&'/;:a N;s/(.*)\n(.*)/\1,'\2'/;ta" file
Probably can be done a little better as my sedfu is not great.
 
Old 02-01-2013, 07:37 AM   #10
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
The 's'ubstitute command in sed cannot, by default, operate on newlines due to the way it processes input. sed takes each line in turn into its pattern buffer minus the newline, and applies the given expressions to it, then empties the buffer before grabbing the next line. There are never any newline characters in the buffer for it to operate on.

Multi-line editing requires use of 'N' and/or the various hold buffer commands. These commands can put multiple lines into the pattern buffer at the same time, with newlines between them, allowing you to target them.

See the relevant sections of the grymoire and the sedfaq:

Here are a few useful sed references:
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sed1line.txt
http://www.catonmat.net/series/sed-one-liners-explained


BTW, here's an ed solution for you too. Since it's a full-fledged text editor, it's able to do many multi-line operations more easily than sed.

Code:
printf '%s\n' "%s/.*/'&'/" '%-1s/$/,/' '%j' 'w newfile.txt' | ed -s oldfile.txt
The first 's' expression surrounds every line with quotemarks. The second 's' adds a comma to the end of every line except the last. The third 'j'oins all lines together. Finally it 'w'rites the modified buffer to a file, which can be the same as the input file (just leave the filename off).

You can insert a '%p' (print all) command to view the entire file at any step.

How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)
 
Old 02-01-2013, 08:57 AM   #11
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,500

Rep: Reputation: 415Reputation: 415Reputation: 415Reputation: 415Reputation: 415
Test results and questions

These tests ...
Code:
echo; echo "Method of LQ Newbie sangfroid"
awk '{printf "\x27%s\x27,", $1}' $InFile > $OutFile
echo "OutFile ..."; cat $OutFile; echo "END"

echo; echo "Method of LQ Member danielbmartin"
sed "s/^/'/; s/$/'/" $InFile |paste -s -d"," > $OutFile
echo "OutFile ..."; cat $OutFile; echo "END"

echo; echo "Method of LQ Guru grail"
sed -r "s/.*/'&'/;:a N;s/(.*)\n(.*)/\1,'\2'/;ta" $InFile > $OutFile
echo "OutFile ..."; cat $OutFile; echo "END"

echo; echo "Method of LQ Guru David the H."
printf '%s\n' "%s/.*/'&'/" '%-1s/$/,/' '%j' 'w newfile.txt' | ed -s $InFile > $OutFile
echo "OutFile ..."; cat $OutFile; echo "END"
Produced these results ...
Code:
Method of LQ Newbie sangfroid
OutFile ...
'123.gzip','234.gzp','356.gzip',END

Method of LQ Member danielbmartin
OutFile ...
'123.gzip','234.gzp','356.gzip'
END

Method of LQ Guru grail
OutFile ...
'123.gzip','234.gzp','356.gzip'
END

Method of LQ Guru David the H.
OutFile ...
END
Question 1) Why did sangfroid's solution omit a linefeed? Is this a bug?
Question 2) Why did David the H.'s solution fail?

Daniel B. Martin

Last edited by danielbmartin; 02-01-2013 at 08:58 AM. Reason: Minor cosmetic improvements
 
Old 02-01-2013, 09:32 AM   #12
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
Hmm. What does the input look like, exactly?

I tried it on multiple inputs, and they all did just fine, except for one file that had dos-format newlines (I'd forgotten I'd set it as such for a previous test). It worked just fine too, after I converted it back.

Let me play with it.

Edit: Ah, my ed command writes directly to an output file, which I named "newfile.txt". You're trying to redirect the non-existant stdout to a different file.

Change the final command to "w $OutFile" and drop the redirection, or alternately to "%p" to have it print the buffer to stdout instead.

Last edited by David the H.; 02-01-2013 at 09:43 AM. Reason: follow-ups
 
Old 02-01-2013, 09:43 AM   #13
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
To put a multiline input into a single line you can use echo. In this way you are a step ahead in order to apply a simple sed solution, e.g.
Code:
echo $(<file) | sed -r "s/^|$/'/g; s/ /','/g"
However this fails if the original lines contain blank spaces, since they will be changed to ',' as well.

Regarding the awk solution you can easily avoid the trailing comma if you add the separator before the string, except for the first iteration. This is easily done by means of a C-like conditional expression:
Code:
awk 'BEGIN{ q = "\x27" } { s ? s = s q "," q $0 : s = q $0 } END { print s q }' file
This takes care of inner blank spaces, since no substitution is made and the characters or strings are added sequentially.
 
Old 02-01-2013, 09:53 AM   #14
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
BTW, as for Q1, the output doesn't have any newlines because sangfroid's printf format string doesn't have any. It's a byproduct of the concatenation process. You can tack on an 'END{ print }' to have it insert a final one.
 
Old 02-01-2013, 01:57 PM   #15
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,500

Rep: Reputation: 415Reputation: 415Reputation: 415Reputation: 415Reputation: 415
Improved solution

I "polished" my code and now have this ...
Code:
sed "s/.*/'&'/" $InFile |paste -sd, > $OutFile
This is the most concise of all solutions offered in this thread.
It's also the most readable, though readability is subjective.

I really like paste because, in one swoop, it ...
- inserts commas where they are wanted
- removes linefeeds where they are not wanted.

Daniel B. Martin

Last edited by danielbmartin; 02-02-2013 at 10:51 AM. Reason: Tighten the code, slightly
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
WAVE file meets the format specs necessary to make an audio CD out of it? stf92 Linux - Software 7 04-21-2012 09:50 AM
separating content from format for desktop users? nglbrkr Linux - Desktop 3 11-21-2009 06:43 AM
vmlinuz file format problems - redhat kernel source tree & make tsnider Linux - Kernel 1 04-05-2008 10:45 AM
vmlinuz file format problems - redhat kernel source tree & make tsnider Red Hat 1 03-25-2008 12:26 PM


All times are GMT -5. The time now is 03:06 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration