LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-19-2013, 05:45 PM   #1
TheBigMing
Member
 
Registered: Dec 2008
Location: east anglia
Distribution: SuSE, antiX
Posts: 36

Rep: Reputation: 7
sed "expression #1, char 2: extra characters after command"


I'm perfectly happy with regex's but years ago I gave up trying to make sed work and, from that time on, sorted everything from within vim. So, for instance, here's some output from inxi:

Code:
$ inxi -plu         
Partition: ID: / size: 15G used: 4.1G (29%) fs: ext4 dev: /dev/sda7 
           label: N/A uuid: 78a25913-bc21-47de-8bdf-ef97ddb124fc
           ID: /home size: 436G used: 108G (25%) fs: ext3 dev: /dev/sda9 
           label: N/A uuid: b8521e14-6947-4f42-a593-c90a5cfc99b5
           ID: /media/sda4 size: 22G used: 4.4G (21%) fs: ext4 dev: /dev/sda4 
           label: SAMSUNG_REC uuid: a3d8a489-1f02-457b-a15d-f5b5339e69c1
           ID: swap-1 size: 6.45GB used: 0.00GB (0%) fs: swap dev: /dev/sda10 
           label: N/A uuid: 4e3201c9-fdb1-4858-bfe1-60eb365ce6b1
Or, at least, that's what it appears to be on the screen. Put it into less and it looks like this:

Code:
      1 ESC[1;34mPartition:ESC[0;37m ESC[1;34mID:ESC[0;37m / ESC[1;34msize:ESC[0      1 ;37m 15G ESC[1;34mused:ESC[0;37m 4.1G (29%) ESC[1;34mfs:ESC[0;37m ext4 
      1 ESC[1;34mdev:ESC[0;37m /dev/sda7 
      2 ESC[1;34m          ESC[0;37m ESC[1;34mlabel:ESC[0;37m N/A ESC[1;34muuid:      2 ESC[0;37m 78a25913-bc21-47de-8bdf-ef97ddb124fc
      3 ESC[1;34m          ESC[0;37m ESC[1;34mID:ESC[0;37m /home ESC[1;34msize:
      3 ESC[0;37m 436G ESC[1;34mused:ESC[0;37m 108G (25%) ESC[1;34mfs:ESC[0;37m       3 ext3 ESC[1;34mdev:ESC[0;37m /dev/sda9
Now, normally, I'm trying to strip the coding out & vim's perfect for that. This time I decided I rather liked the coded output but I did want the report in 4 blocks of 2 lines per block.

No problem, into vim:

Code:
  7 ^[[1;34m          ^[[0;37m ^[[1;34mID:^[[0;37m swap-1 ^[[1;34msize:^[[0;37m     6.45GB ^[[1;34mused:^[[0;37m 0.00GB (0%) ^[[1;34mfs:^[[0;37m swap ^[[1;34mde    v:^[[0;37m /dev/sda10 
  8 ^[[1;34m          ^[[0;37m ^[[1;34mlabel:^[[0;37m N/A ^[[1;34muuid:^[[0;37m     4e3201c9-fdb1-4858-bfe1-60eb365ce6b1
  9 ^[[0m
~                                                                               
: g/label/s/$/\r/g
and:

Code:
$ cat sedtst
Partition: ID: / size: 15G used: 4.1G (29%) fs: ext4 dev: /dev/sda7 
           label: N/A uuid: 78a25913-bc21-47de-8bdf-ef97ddb124fc

           ID: /home size: 436G used: 108G (25%) fs: ext3 dev: /dev/sda9 
           label: N/A uuid: b8521e14-6947-4f42-a593-c90a5cfc99b5

           ID: /media/sda4 size: 22G used: 4.4G (21%) fs: ext4 dev: /dev/sda4 
           label: SAMSUNG_REC uuid: a3d8a489-1f02-457b-a15d-f5b5339e69c1

           ID: swap-1 size: 6.45GB used: 0.00GB (0%) fs: swap dev: /dev/sda10 
           label: N/A uuid: 4e3201c9-fdb1-4858-bfe1-60eb365ce6b1
So, what's my problem? Well, having knocked this out I thought it should be easy enough to do in sed:

Code:
$ inxi -plu | sed g/label/s/$/\r/g
sed: -e expression #1, char 2: extra characters after command
$  sed g/label/s/$/\r/g sedtest
sed: -e expression #1, char 2: extra characters after command
and so on double inverted commas, single, escapes the lot - nada!

Now I remember why I gave up on sed all those years ago - but I must be missing something!?

----- and I shudder to think what it would make of something like this!
Code:
 16 
 17 But we didn~@~Yt stop there..
 18 
 ~                                                                               
: /�~@~Y/s//'/
 
Old 03-19-2013, 06:16 PM   #2
whizje
Member
 
Registered: Sep 2008
Location: The Netherlands
Distribution: Slackware64 current
Posts: 592

Rep: Reputation: 140Reputation: 140
Code:
bash-4.2$ cat inxi.txt |sed '/label/a \
\'
Partition: ID: / size: 15G used: 4.1G (29%) fs: ext4 dev: /dev/sda7 
           label: N/A uuid: 78a25913-bc21-47de-8bdf-ef97ddb124fc

           ID: /home size: 436G used: 108G (25%) fs: ext3 dev: /dev/sda9 
           label: N/A uuid: b8521e14-6947-4f42-a593-c90a5cfc99b5

           ID: /media/sda4 size: 22G used: 4.4G (21%) fs: ext4 dev: /dev/sda4 
           label: SAMSUNG_REC uuid: a3d8a489-1f02-457b-a15d-f5b5339e69c1

           ID: swap-1 size: 6.45GB used: 0.00GB (0%) fs: swap dev: /dev/sda10 
           label: N/A uuid: 4e3201c9-fdb1-4858-bfe1-60eb365ce6b1

bash-4.2$
 
Old 03-19-2013, 10:38 PM   #3
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,254

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
whizje has provided the solution, but I am curious what the 'g' at the start of the line does in vim?
 
Old 03-20-2013, 05:40 AM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
'g', in vim (and ed), is the global command, i.e. run the following command(s) on every line that matches the given regex. sed doesn't have a global command because it always operates sequentially anyway. global operation is the default.

(Actually, sed does have a 'g' command, but it's a different operation, getting text from the hold buffer.)

From things like this, it seems to me that the OP's trouble with sed is that he's expecting its syntax and operation to be almost exactly like vim. But it's not. While there are many similarities due to their shared development history, there are also many differences, due to the way the programs operate.

vim is a full text editor, which means that the entire file is first loaded into a memory buffer, then individual lines are processed, in a random-access way, using address flags. But sed is the "stream editor", meaning that it processes one line at a time, in one direction only, from the first to the last. sed's addressing system is there simply to match lines for processing as they pass by.

Because of this, real multi-line editing in sed, particularly if you need to backtrack and access previous lines, can be tricky to do and usually requires the use of the hold buffer to store text temporarily for later access. But for simple one-line-at-a-time work, it's very easy and efficient.

Here's my list of few useful sed references:
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt
http://www.catonmat.net/series/sed-one-liners-explained

Although the OP may have seen some of them before, it might be good to try again in light of (hopefully) clearer understanding.


And as I briefly mentioned, ed is another option if you want vim-like addressing in a lighter package, and with easy scripting ability.

How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)


Finally, there's the ex command, which (in the gnu version at least) is a scriptable version of vim, i.e. it gives it an ed-like interface (equivalent to "vim -es").

Last edited by David the H.; 03-20-2013 at 05:45 AM. Reason: small addition
 
1 members found this post helpful.
Old 03-20-2013, 11:00 AM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,254

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
Thanks David, I personally have always just used '%'
 
Old 03-20-2013, 12:15 PM   #6
TheBigMing
Member
 
Registered: Dec 2008
Location: east anglia
Distribution: SuSE, antiX
Posts: 36

Original Poster
Rep: Reputation: 7
@grail - Me being lazy I'm afraid. This is from a log file and, as you can see, it occupies a single line:

Code:
4 Upgrade: user-management-antix:i386 (0.1.5^L 0.1.5.1)'\n' libglapi-mesa:i386 (8.0.3-1^L 8.
  0.4-2)^L gnome-keyring:i386 (3.4.1-4^L 3.4.1-5)^L libgtk2.0-common:i386 (2.24.10-1^L 2.24.
  10-2)^L exit-antix:i386 (0.1.1.3^L 0.1.1.5)^L mc-data:i386 (4.8.3-3^L 4.8.3-5)^L ttf-dejav
  u-core:i386 (2.33-2^L 2.33-3)^L libswscale2:i386 (0.8.3-4^L 0.8.3-7)^L libstdc++6:i386 (4.
  .7.1-2^L 4.7.1-7)^L bzip2:i386 (1.0.6-3^L 1.0.6-4)^L
OK - that's a single line with only one change to make 200 odd times and the regex:
Code:
: /)^L/s//)\r/g
would work fine with it. The g at the end of the line means the expression finds a line that contains the pattern for which it's searching and does whatever it has to for every occurrence of the pattern on that line. Then it stops.

But this next example is different:
Code:
  2 ~@~\We removed functionality because ~@~Xthat  
  3 approach is broken,~@~Y for degrees of broken from 
  4 ~@~Xit is a security hole~@~Y all the way to ~@~Xit does not   
  5 conform to the new style we are using.~@~Y~@~]  
  6 This ~@~Xattribution~@~Y obviously didn~@~Yt go down
Here there are many lines, and line 6 has 2 occurences of the pattern. In this case. the g at the start of the line searches every line but only acts on the first incident of the pattern on each line. This would would work on my original example, the pattern searched for occurred only once every two lines but it would fail to work on line 6 (above). The pattern we need here is:

Code:
: g/�~@~Y/s//'/g
a g at each end means that every line is searched & every incident dealt with. And, bcause I'm using vim, all I do is flip through my history, find a regex that more or less matches my reqirements and edit it. They may have been sweated over to start off with but thyey're all boiler plates now.

dmk

Last edited by TheBigMing; 03-20-2013 at 04:54 PM.
 
Old 03-20-2013, 12:44 PM   #7
TheBigMing
Member
 
Registered: Dec 2008
Location: east anglia
Distribution: SuSE, antiX
Posts: 36

Original Poster
Rep: Reputation: 7
@David the H - Aha we've met before. Thanks for your reply, I'll certainly chase up your rfeferences. I know about ex & ed - one or the other forms the basis of the vim 'command line' - but I've never used either in anger. I use awk quite often, which I like, but it would be nice to have something a more lightweight.

I suppose I'll have to bite the bullet & come to terms with sed - whether I will or not's another matter, but thanks for your help.

dmk
 
Old 03-20-2013, 01:24 PM   #8
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,254

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
The sed regex should work exactly the same and as stated, since sed already will process every line in the file, the 'g' at the start is no longer necessary (even if it were an option)
 
Old 03-20-2013, 08:05 PM   #9
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,241

Rep: Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325
Re regex differences; in fact I'd never assume any 2 tools have EXACTLY the same regex engines, even though certain symbols definitions are pretty constant.
Try http://regex.info/book.html if you want the full skinny; highly recommended
 
Old 03-26-2013, 01:19 PM   #10
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
I see another issue with the above.

vim does some funky stuff with newlines, converting them to carriage returns when the text is loaded in a buffer. I had to learn, for example, that the LHS of the s command uses '\n' to match newlines, but the RHS uses '\r' to insert them.

The other programs don't have this behavior, and always use the default '\n'. So in sed you have to use '\n' on the RHS to insert a newline.

Also, sed's pattern buffer will have no newlines to match in it unless you've previously told it to combine multiple lines together (through the N, G, or H commands). Only then can you use '\n' on the LHS to match them. This can sometimes be quite complex to do correctly, another reason why I don't recommend sed for multi-line processing.


@grail: '%' is a simple range address, equal to '1,$', which does do the job much of the time. But sometimes you need to run a command globally while also using a matching pattern. e.g. if you want to delete every line that contains the pattern 'foo' you'd have to use ":g/foo/d".

'g' can also accept a range itself, so you can limit the deletions to the first 20 lines with ':1,20g/foo/d'. Finally, in ed at least, it can be used to run multiple commands at once.

Last edited by David the H.; 03-26-2013 at 01:21 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] sed gives :sed: -e expression #1, char 1: unknown command: `'' samasat Linux - Newbie 10 06-09-2012 06:31 PM
[SOLVED] Sed giving "extra characters after command" error theonislair Programming 3 06-09-2012 04:42 AM
Need help filtering out sed's "special" characters fatsheep Programming 2 11-09-2006 05:54 PM
sed error message:extra characters after the command wmh830621 Programming 4 08-14-2006 08:13 PM
sed error message: extra characters after the command. nano_mag Linux - General 3 05-15-2005 02:00 AM


All times are GMT -5. The time now is 04:21 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration