How to remove string in the text file ? Bash script

dlugasx · 06-05-2009, 08:35 AM

Hello

I have a text file which include:

Code:

https://myserver/xa/if?took1=324234234345345345
https://myserver/xa/if?took1=546456456456
https://myserver/xa/if?took1=45645654665768678
https://myserver/xa/if?took1=678678678678
https://myserver/xa/if?took1=67867867867
https://myserver/xa/if?took1=678678678678

How can I remove with awk this string

Code:

https://myserver/xa/if?took1=

I mean, this string should be removed from each line and save in another file.

help...

jlightner · 06-05-2009, 08:50 AM

Code:

awk -F/ '{print $5}' file |awk -F= '{print $2}' >newfile

Where "file" is the name of the file that contains the original variables and "newfile" is the newfile that contains the output.

jlightner · 06-05-2009, 08:58 AM

Of course it is more common to do this kind of operation (replacement/deletion) with sed:

Code:

sed -e s/https:"\/""\/"myserver"\/"xa"\/"if?took1=// file >newfile

For this particular line sed is a bit of a bear because / has is used for subtitution:
sed -e s/pattern/newpattern/

Since your pattern includes several / characters you have to do the bizarre quoting and escaping seen in the live above so it knows which ones are literal and which are part of the sed script.

dlugasx · 06-05-2009, 08:59 AM

Quote:

Originally Posted by jlightner

Code:

awk -F/ '{print $5}' file |awk -F= '{print $2}' >newfile

Where "file" is the name of the file that contains the original variables and "newfile" is the newfile that contains the output.

where can I put the string in this command ?

If this address will change for example

Code:

https://myserver/xa/if?took1=  >   https://yourserver/xa/if?took1=

what should I do ?

thanks for fast answer

pixellany · 06-05-2009, 09:04 AM

Code:

awk -F = '{print $1}' filename

This prints everything before "="

Change $1 to $2 to print everything after the "=".

Code:

awk -F = '{print $1}' filename > new filename

To write output of command to a new file

dlugasx · 06-05-2009, 09:13 AM

Thanks a lot !

This is a big help for me !

greetings from Germany !

jlightner · 06-05-2009, 09:24 AM

Quote:

https://myserver/xa/if?took1= > https://yourserver/xa/if?took1=

This is why I mentioned sed - it is more common to do something like this:

Code:

sed -e s/myserver/yoursever/ file >newfile

Since you're only replacing one string you don't have to worry about all the rest of the line (special characters etc that I mentioned in earlier post).

David the H. · 06-05-2009, 09:38 AM

Quote:

Originally Posted by jlightner

Of course it is more common to do this kind of operation (replacement/deletion) with sed:

Code:

sed -e s/https:"\/""\/"myserver"\/"xa"\/"if?took1=// file >newfile

For this particular line sed is a bit of a bear because / has is used for subtitution:
sed -e s/pattern/newpattern/

Since your pattern includes several / characters you have to do the bizarre quoting and escaping seen in the live above so it knows which ones are literal and which are part of the sed script.

Actually there's no need to resort to backslash escapes here. You can use any character as the separator in sed, not just the slash. Try changing it to something like '|' instead, for example.

Code:

sed -e 's|https://myserver/xa/if?took1=||' file >newfile

You also don't need to quote each character separately. Quote the entire phrase in single quotes instead, and the whole string will be passed to sed literally.

David the H. · 06-05-2009, 09:49 AM

Quote:

Originally Posted by pixellany

Code:

awk -F = '{print $1}' filename

This prints everything before "="

Change $1 to $2 to print everything after the "=".

Code:

awk -F = '{print $1}' filename > new filename

To write output of command to a new file

To expand on this, awk works on the concept of fields. $1 is field one of the input string, $2 is field 2, etc. Usually the field separator is a space, but the '-F =' changes the separator to the equals sign. So $1 is everything before the equals sign, and $2 is everything after it.

Follow it up with the command to print only the fields you want to see and Bob's your uncle, as the Brits say.

You can learn a lot more about how to use sed and awk on this page. It's well worth taking the time to read (and work through) them.

ghostdog74 · 06-05-2009, 12:40 PM

Quote:

Originally Posted by jlightner

Of course it is more common to do this kind of operation (replacement/deletion) with sed:

replacement and substitution operation is common string manipulation task but not necessary the tool used. since the input data has a structured format, it is obvious using fields and field delimiter is the better and "neater" approach.

Code:

awk -F"=" '{print $NF}'

you do not have to meddle with escaping slashes and stuffs like that.