LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-16-2014, 12:13 PM   #1
michaelgg13
LQ Newbie
 
Registered: Apr 2014
Posts: 11

Rep: Reputation: Disabled
using SED to modify a file, help!


As a linux/unix noob trying to teach myself how to use the command line, I am confused about using sed.

Can someone give me an example of some sed commands that

1) removes leading blanks spaces in a sentence
2) removes trailing blank spaces in a sentence
3) removes unnecessary blanks between words( so intstead of HI____my name is, it should be HI_my name is. _ representing blank spaces)
4) deletes blank lines

I will use these commands to print the old file to a new file which I do know how to do.

Thanks for any pointers!
 
Old 04-16-2014, 12:30 PM   #2
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,876
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
You should try to accomplish this, or whatever attempts you have tried, please post what you've done. Which is to say that it's more appropriate to give it a try and show at least that you've attempted it and are not just listing a bunch of items you want someone else to do for you.

Suggestions as to how to proceed:
- Look at the manpage for sed, it will tell you the semantics of the command.
- Google for sed examples, and you can google for very specific examples; likely some of which will match exactly what you want.
- Know that what sed does for you is to take in either from a file or from a pipe and then outputs to stdout.

That last item is a good tip to consider, and I think very helpful to avoid personal "oops" occurrences.

Taking in from a file is, to me; normal. But you can also channel the output of another command into sed; for instance a grep of a pattern from a file. Say the file contains many lines of varying content, but you can find some smaller group of those lines which you care about by using grep. You can pipe the output of that grep into sed. Next sed will do the edits you want; such as a search and replace of patterns, and search/replace can search for pattern and replace with nothing, thus erasing the pattern. And it sends that output to stdout, or your console. You can then choose to redirect that output to some other file. The result is that you make some sed command which you're not sure of, and believe me I do it that way, because I always have to google and figure out what I need to do to accomplish my sed goal, I don't use it everyday. And the beauty I see is that it won't overwrite your input file, causing that form of personal "oops" which I mentioned before.
 
1 members found this post helpful.
Old 04-16-2014, 12:32 PM   #3
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
Learn some regex. For your task you need to know that '^' matches the beginning of the line, '$' matches the end of the line, and you can match a minimum and maximum number of the previous character with '{min,max}'.

An example for removing more than one blank spaces between words:
Code:
echo 'test test   test' | sed -r 's/ {2,}/ /g'
You need the '-r' for extended regex which allows use of '{min,max}' and others.

Also see:
http://www.grymoire.com/Unix/Regular.html
 
Old 04-16-2014, 12:48 PM   #4
michaelgg13
LQ Newbie
 
Registered: Apr 2014
Posts: 11

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by rtmistler View Post
You should try to accomplish this, or whatever attempts you have tried, please post what you've done. Which is to say that it's more appropriate to give it a try and show at least that you've attempted it and are not just listing a bunch of items you want someone else to do for you.

Suggestions as to how to proceed:
- Look at the manpage for sed, it will tell you the semantics of the command.
- Google for sed examples, and you can google for very specific examples; likely some of which will match exactly what you want.
- Know that what sed does for you is to take in either from a file or from a pipe and then outputs to stdout.

That last item is a good tip to consider, and I think very helpful to avoid personal "oops" occurrences.

Taking in from a file is, to me; normal. But you can also channel the output of another command into sed; for instance a grep of a pattern from a file. Say the file contains many lines of varying content, but you can find some smaller group of those lines which you care about by using grep. You can pipe the output of that grep into sed. Next sed will do the edits you want; such as a search and replace of patterns, and search/replace can search for pattern and replace with nothing, thus erasing the pattern. And it sends that output to stdout, or your console. You can then choose to redirect that output to some other file. The result is that you make some sed command which you're not sure of, and believe me I do it that way, because I always have to google and figure out what I need to do to accomplish my sed goal, I don't use it everyday. And the beauty I see is that it won't overwrite your input file, causing that form of personal "oops" which I mentioned before.
ok so they would be

1)sed 's/^ *//g'
2)sed 's/ *$//'
3)sed 's/ */\ /g'
4)sed '/^$/d'

How would you put all of these in a file and run them together? This is where I cannot seem to find much. I want to place those commands in a file run the file and save the results to another file.

Last edited by michaelgg13; 04-16-2014 at 01:05 PM.
 
Old 04-16-2014, 01:18 PM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,999

Rep: Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190
Have you had a look at the -f option in the man page.

Also look here for more details:

http://www.grymoire.com/Unix/Sed.html
 
Old 04-16-2014, 01:21 PM   #6
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,876
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
That would be a script, such as a bash script. Which is a similar, but almost entirely different topic.

Bash Beginner Guide
Advanced Bash Scripting Guide
Bash Scripting for Dummies and Geniuses

My guidance points (and check that last link it's my blog entry on the topic), are to stay fundamental, understand that a "script" is essentially the same as commands you can run from the command prompt. Therefore if you can "type" it, you can "script" it.

To do these sed commands the way you want, you should write them as part of a script:
Code:
#!/bin/bash
# comment line - this is my script to do what I want
sed-command-1
sed-command-2
 
Old 04-16-2014, 01:38 PM   #7
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
Quote:
Originally Posted by michaelgg13 View Post
ok so they would be

1)sed 's/^ *//g'
2)sed 's/ *$//'
3)sed 's/ */\ /g'
4)sed '/^$/d'

How would you put all of these in a file and run them together? This is where I cannot seem to find much. I want to place those commands in a file run the file and save the results to another file.
3 is NOT correct, unless it does what you want it to do. Here's what I get:
Code:
bash-4.2$ echo 'test test   test' | sed 's/ */\ /g'
 t e s t t e s t t e s t
This one isn't as easy as the others, so that's why I posted the answer.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to modify binary file by using combination command (dd, hexdump, sed, ...)??? bvdyi Linux - Newbie 2 11-07-2013 01:28 AM
[SOLVED] using sed to modify a file and name expansion forethinker Linux - Newbie 7 03-20-2012 03:34 AM
how to modify xml file using sed/awk akhand jyoti Linux - Newbie 3 11-29-2011 02:47 PM
modify a file that contains \\ using sed or awk j-me Linux - Newbie 7 05-14-2010 08:30 AM
modify file access & modify timestamps i2itstud Linux - General 1 05-20-2003 03:34 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 08:12 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration