LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 06-05-2009, 08:35 AM   #1
dlugasx
Member
 
Registered: Dec 2008
Location: Germany/Poland
Distribution: CentOS / Debian / Solaris / RedHat
Posts: 250

Rep: Reputation: 18
How to remove string in the text file ? Bash script


Hello

I have a text file which include:

Code:
https://myserver/xa/if?took1=324234234345345345
https://myserver/xa/if?took1=546456456456
https://myserver/xa/if?took1=45645654665768678
https://myserver/xa/if?took1=678678678678
https://myserver/xa/if?took1=67867867867
https://myserver/xa/if?took1=678678678678
How can I remove with awk this string
Code:
https://myserver/xa/if?took1=
I mean, this string should be removed from each line and save in another file.

help...
 
Old 06-05-2009, 08:50 AM   #2
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,028
Blog Entries: 5

Rep: Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790
Code:
awk -F/ '{print $5}' file |awk -F= '{print $2}' >newfile
Where "file" is the name of the file that contains the original variables and "newfile" is the newfile that contains the output.
 
Old 06-05-2009, 08:58 AM   #3
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,028
Blog Entries: 5

Rep: Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790
Of course it is more common to do this kind of operation (replacement/deletion) with sed:

Code:
sed -e s/https:"\/""\/"myserver"\/"xa"\/"if?took1=// file >newfile
For this particular line sed is a bit of a bear because / has is used for subtitution:
sed -e s/pattern/newpattern/

Since your pattern includes several / characters you have to do the bizarre quoting and escaping seen in the live above so it knows which ones are literal and which are part of the sed script.
 
Old 06-05-2009, 08:59 AM   #4
dlugasx
Member
 
Registered: Dec 2008
Location: Germany/Poland
Distribution: CentOS / Debian / Solaris / RedHat
Posts: 250

Original Poster
Rep: Reputation: 18
Quote:
Originally Posted by jlightner View Post
Code:
awk -F/ '{print $5}' file |awk -F= '{print $2}' >newfile
Where "file" is the name of the file that contains the original variables and "newfile" is the newfile that contains the output.
where can I put the string in this command ?

If this address will change for example


Code:
https://myserver/xa/if?took1=  >   https://yourserver/xa/if?took1=


what should I do ?


thanks for fast answer
 
Old 06-05-2009, 09:04 AM   #5
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
Code:
awk -F = '{print $1}' filename
This prints everything before "="

Change $1 to $2 to print everything after the "=".

Code:
awk -F = '{print $1}' filename > new filename
To write output of command to a new file
 
Old 06-05-2009, 09:13 AM   #6
dlugasx
Member
 
Registered: Dec 2008
Location: Germany/Poland
Distribution: CentOS / Debian / Solaris / RedHat
Posts: 250

Original Poster
Rep: Reputation: 18
Thanks a lot !

This is a big help for me !



greetings from Germany !
 
Old 06-05-2009, 09:24 AM   #7
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,028
Blog Entries: 5

Rep: Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790
This is why I mentioned sed - it is more common to do something like this:

Code:
sed -e s/myserver/yoursever/ file >newfile
Since you're only replacing one string you don't have to worry about all the rest of the line (special characters etc that I mentioned in earlier post).

Last edited by MensaWater; 06-05-2009 at 09:37 AM.
 
Old 06-05-2009, 09:38 AM   #8
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Quote:
Originally Posted by jlightner View Post
Of course it is more common to do this kind of operation (replacement/deletion) with sed:

Code:
sed -e s/https:"\/""\/"myserver"\/"xa"\/"if?took1=// file >newfile
For this particular line sed is a bit of a bear because / has is used for subtitution:
sed -e s/pattern/newpattern/

Since your pattern includes several / characters you have to do the bizarre quoting and escaping seen in the live above so it knows which ones are literal and which are part of the sed script.
Actually there's no need to resort to backslash escapes here. You can use any character as the separator in sed, not just the slash. Try changing it to something like '|' instead, for example.

Code:
sed -e 's|https://myserver/xa/if?took1=||' file >newfile
You also don't need to quote each character separately. Quote the entire phrase in single quotes instead, and the whole string will be passed to sed literally.
 
Old 06-05-2009, 09:49 AM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Quote:
Originally Posted by pixellany View Post
Code:
awk -F = '{print $1}' filename
This prints everything before "="

Change $1 to $2 to print everything after the "=".

Code:
awk -F = '{print $1}' filename > new filename
To write output of command to a new file
To expand on this, awk works on the concept of fields. $1 is field one of the input string, $2 is field 2, etc. Usually the field separator is a space, but the '-F =' changes the separator to the equals sign. So $1 is everything before the equals sign, and $2 is everything after it.

Follow it up with the command to print only the fields you want to see and Bob's your uncle, as the Brits say.


You can learn a lot more about how to use sed and awk on this page. It's well worth taking the time to read (and work through) them.
 
Old 06-05-2009, 12:40 PM   #10
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by jlightner View Post
Of course it is more common to do this kind of operation (replacement/deletion) with sed:
replacement and substitution operation is common string manipulation task but not necessary the tool used. since the input data has a structured format, it is obvious using fields and field delimiter is the better and "neater" approach.
Code:
awk -F"=" '{print $NF}'
you do not have to meddle with escaping slashes and stuffs like that.
 
  


Reply

Tags
awk, remove, string


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
bash script- text file manip- delete everything before/after string justin99 Programming 9 11-20-2014 04:16 AM
How to replace string pattern with multi-line text in bash script? brumela Linux - Newbie 6 04-21-2011 07:56 AM
help with bash script: remove * to the last . in string drkstr Linux - Software 3 04-25-2006 05:54 PM
remove part of string in bash script crewblunts Programming 2 03-16-2006 06:54 PM
bash-script: output text between two ocurrences of a specific string isl01jbe Programming 1 06-17-2004 03:36 PM


All times are GMT -5. The time now is 02:19 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration