LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 10-25-2010, 04:34 AM   #1
jimieee
Member
 
Registered: Aug 2003
Location: London, UK
Distribution: Debian and Fedora for play and RHEL + Solaris for work
Posts: 172

Rep: Reputation: 15
Using sed to search and replace backwards


Hi All,

I'm trying to use sed to search and replace backwards. The problem is that I have a shell script that is required to put commas into big numbers. For example

9999999 as 9,999,999

I've tried a few things, but none seem to work:

Code:
$ echo 9999999 | sed -e 's/\([0-9]\{3\}\)/,\1/g'
,999,9999

$ echo 9999999 | sed -e 's/\([0-9]\{3\}\)$/\1,/g' -e 's/\([0-9]\{3\}\)/\1,/g'
999,999,9,

$ echo 9999999 | sed -e 's/\([0-9]\{3\}\)$/\1,/g' -e 's/\([0-9]\{3\}\)/,\1/g'
,999,9999,

$ echo 9999999 | sed -e 's/\([0-9]\{3\}\)$/,\1/g' -e 's/\([0-9]\{3\}\)/,\1/g'
,9999,,999

$ echo 9999999 | sed -e 's/\([0-9]\{3\}\)$/,\1/g' -e 's/\([0-9]\{3\}\)/,\1/g'
It would be much easier if I could search backwards! For example Bash parameter substitution style:

Code:
$ echo 9999999 | sed -e 's%\([0-9]\{3\}\)%,\1%g'
Or may be someone has a better way to do this altogether...
 
Old 10-25-2010, 04:44 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,508

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Keep it simple:
Code:
printf "%'d\n" 9999999
 
Old 10-25-2010, 05:04 AM   #3
jimieee
Member
 
Registered: Aug 2003
Location: London, UK
Distribution: Debian and Fedora for play and RHEL + Solaris for work
Posts: 172

Original Poster
Rep: Reputation: 15
This doesn't seem to work for me.

Code:
printf "%'d\n" 9999999
9999999
What version of bash is this that you are using? This is mine...

Code:
$ bash --version
GNU bash, version 3.00.16(1)-release (i386-pc-solaris2.10)
Copyright (C) 2004 Free Software Foundation, Inc.
 
Old 10-25-2010, 05:09 AM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,508

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
It should depend from your current locale, in particular from the LC_NUMERIC variable. For example, the POSIX locale does not have a thousands separator, while en_US or en_GB should work. What is the output of the locale command (without arguments it should list your current setup)?

In any case something like:
Code:
env LC_NUMERIC=en_GB printf "%'d\n" 9999999
should work.

Last edited by colucix; 10-25-2010 at 05:13 AM.
 
Old 10-25-2010, 05:21 AM   #5
jimieee
Member
 
Registered: Aug 2003
Location: London, UK
Distribution: Debian and Fedora for play and RHEL + Solaris for work
Posts: 172

Original Poster
Rep: Reputation: 15
Code:
LC_NUMERIC="C"
It still doesn't work as you described though:

Code:
$ env LC_NUMERIC=en_GB printf "%'d\n" 9999999
'd

$ LC_NUMERIC=en_GB printf "%'d\n" 9999999
9999999
 
Old 10-25-2010, 05:27 AM   #6
jimieee
Member
 
Registered: Aug 2003
Location: London, UK
Distribution: Debian and Fedora for play and RHEL + Solaris for work
Posts: 172

Original Poster
Rep: Reputation: 15
I've found out how to do this in sed now:

Code:
sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'
Thanks to command line fu

http://www.commandlinefu.com/command...or-within-pipe
 
Old 10-25-2010, 05:34 AM   #7
jimieee
Member
 
Registered: Aug 2003
Location: London, UK
Distribution: Debian and Fedora for play and RHEL + Solaris for work
Posts: 172

Original Poster
Rep: Reputation: 15
I've marked this as 'solved' although I would still be interested to see how I can make the printf solution work and how to actually get sed to search backwards (if this is possible).
 
Old 10-25-2010, 05:42 AM   #8
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550
Not sure how exactly to ask sed to search backwards, however here's a workaround for reading a variable backwards:

The `rev` tool:
Code:
root@reactor: var=123456
root@reactor: echo "$var" | rev
654321
So you could do:
Code:
echo "$var" | rev | sed '/blah blah blah/' | rev
However the solution in post #6 looks nice and neat meanwhile. Try this for fun sometime too:
Code:
tac some-file | rev
The locale-based printf doesn't work for me either. My default LC_NUMERIC is "POSIX" but I tried "C", "en_GB" and "en_US" and none changed the output.

Last edited by GrapefruiTgirl; 10-25-2010 at 05:47 AM.
 
Old 10-25-2010, 06:00 AM   #9
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,508

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Glad to see you found a solution. Regarding the printf issue, does the same happen in awk?
Code:
echo 9999999 | env LC_NUMERIC=en_US awk '{printf "%'\''d\n", $1}'
 
Old 10-25-2010, 06:07 AM   #10
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550
@ colucix,

post #9 above does work for me; and, I've discovered that while my shell's `printf` does not work for this, I can use /bin/printf and it works:
Code:
root@reactor: LC_NUMERIC=en_US /bin/printf "%'d\n" 9999999
9,999,999
root@reactor:
 
Old 10-25-2010, 07:13 AM   #11
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,508

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
@ GrapefruiTgirl,

thank you for reporting that. I also tried on a Solaris machine and I cannot make it to work in any way (even with en_US locale set for all applications). I suspect the "%'d" format has been introduced in a C language specification which has not been adopted by all systems. Maybe, better to stick with the regular expression solution!
 
Old 10-25-2010, 07:33 AM   #12
jimieee
Member
 
Registered: Aug 2003
Location: London, UK
Distribution: Debian and Fedora for play and RHEL + Solaris for work
Posts: 172

Original Poster
Rep: Reputation: 15
I can see that there are a number of ways to achieve this.

I wanted something quick and easy in sed, or something simple using printf.

I am aware of the more convoluted methods using perl and awk, for example:

Code:
echo "123456" | perl -nwe 'chomp; print reverse($_) . "\n"' | sed -e 's/\([0-9]\{3\}\)/\1,/g' | perl -nwe 'chomp; print reverse($_) . "\n"' | sed -e 's/^,//'
Or just pure perl

Code:
$ echo "12345679" | perl -nwe 'chomp; $reverse=reverse($_); $reverse =~ s/(\d{3})/$1,/g; $reverse =~ s/,$//; print reverse($reverse) . "\n"'
12,345,679
It's not very satisfactory though, because I can't see a good reason why one shouldn't be able to search/replace in reverse order with sed.
Sorry if that makes me sound ignorant - I don't mean to be!


My results with awk:

Code:
$ echo 9999999 | env LC_NUMERIC=en_US awk '{printf "%'\''d\n", $1}'
9999999
tac doesn't exist in my environment (I'm using Solaris), but I guess I could do the same thing with perl...


Using /bin/printf
Code:
$ LC_NUMERIC=en_GB /bin/printf "%'d\n" 9999999
'd
May be there's something specific to Solaris that I'm missing here? It's good to know that it works in (I assume this is where you're testing it) Linux!
 
Old 10-25-2010, 07:45 AM   #13
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,508

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Quote:
Originally Posted by jimieee View Post
May be there's something specific to Solaris that I'm missing here?
I just checked on the GNU coreutils and GNU C library documentation and it clearly states the format specifier for thousands separator is a GNU specific extension. So there is no way to let it work with native solaris utilities. Here is the relevant link.
 
1 members found this post helpful.
Old 10-25-2010, 08:14 AM   #14
jimieee
Member
 
Registered: Aug 2003
Location: London, UK
Distribution: Debian and Fedora for play and RHEL + Solaris for work
Posts: 172

Original Poster
Rep: Reputation: 15
It's a pity.

(Generally) you don't want to actually update the value in your variable - as you might want to go on an use it for mathematical purposes. Most tools will interpret the value as a string if it contains characters other than [0-9].

Being able to printf the thousands separator would be the perfect solution... May be I need to ask my SA to install GNU printf...
 
Old 10-25-2010, 09:37 AM   #15
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,564

Rep: Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939
Thought I would give you a prettied up version of your sed too, only slightly shorter:
Code:
sed -r ':a s/([0-9]+)([^,]{3})/\1,\2/;ta'
-r is your friend when getting rid of pesky back slashes
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
sed - testing multiline search and replace webhope Programming 11 05-13-2010 02:21 PM
sed multi-line search/replace woes djmm Programming 8 03-17-2009 05:25 AM
sed search replace tomerbd1 Linux - General 9 04-10-2008 04:31 AM
sed question for search and replace jakev383 Linux - General 8 05-05-2007 05:40 AM
sed search & replace sharathkv25 Programming 2 03-07-2007 10:16 AM


All times are GMT -5. The time now is 12:37 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration