I'd like a sed command to replace all decimal values greater than 0.5 with nothing in a tab delimited text file.


EX: Input text- 0.8765 0.301 0.5 0.11 (note, for simplicity these values are space delim)
This is what I'd like back- 0.301 0.11
There would be one tab before 0.301 and two tabs after it. This way when you paste the text into an excel spreadsheet, there are empty cells where you deleted values.

I thought the sed command would be something like:

sed s/0\.[5-9].*[[:space:]]//g

But this will delete everything.

What I need is something that will start deleting when it sees 0\.[5-9] and stop when it reaches a tab. I know you can use [^character]+ to do this, but it doesn't seem to work with a TAB as the character.

sed /0\.[5-9][^character]+//g


Another idea I had would be to have sed replace from 0.[5-9] to 0. and replace with a tab + 0.
But I also cant get [^0\.]+ to work as it only works with single characters.



Any ideas?
Reallly need this command for my work.
Thanks-

First, your criteria seems to have very little to do with tabs---you need to recognize decimal numbers.

First, the regex for a decimal number with any number of digits, value >or= 0.5 and < 1.0, followed by any non-numeric character:

To keep from deleting the non-digit, put it in a "backreference" \(....\), then recall it using \1

So:

Not tested---if you post a larger example of the file contents, we can help test.

Here is a text example. I included the same text in an attached text file.


0.1972 0.07161 0.06874 0.1313 0.1499 0.197710473 0.071610311 0.071297537 0.07466101
0.04915 0.02215 0.02114 0.08588 0.02757 0.049525059 0.022147094 0.021650466 0.023604862


Your sed command:
sed 's/0\.[5-9]\+\([^[:digit]]\)/\1/'

Gave me this error:
sed: -e expression #1, char 25: unterminated `s' command

Attached Files

2x9_decimal-values_tab-delim.txt (178 Bytes, 17 views)

Hi,

pixellany made a typo in his command, it should be [:digit:], not [:digit] (mind the missing : at the end).

I don't get which output you want. You already provided an infile example, could you post the desired output that goes with that example?

Hi,

Your first request (replace all decimal values greater than 0.5 with nothing) and your example don't go together. There are no numbers larger then 0.5 (0.1 is biggest).

Wrong example or should that be 0.05 or ...?

Anyway, this does what you originally requested:

sed 's/0\.[5-9][0-9]*//' infile

I dope hope this is what you are looking for.....

Wow, my bad.

I actually just figured it out. My sed command is:


sed 's/0\.[5-9][0-9]*\([^0-9]\)/\1/g'

Only problem is that it doesn't replace decimal strings in the last field. It would be nice, but I think I can just paste a dummy column at the end, or add a tab or something.


Thanks for your help. Couldn't of figured this out otherwise.

Eeeek!!! another mistake......

Your file does not have anything larger than 0.5, so the command is not going to work!!!!!

I found more problems---here is what seems to work (I created the test file "dec"):

Note that I had to change to extended regexes and use the alternation operator so it would also match the end of the line.

The other glaring error in my previous code is that it would only match 1 or more occurences of [5-9]---what's needed is [5-9] followed by any digit.

Great minds work in the same channels!!! My code takes care of the end of the line, also.

if you have 0.55 0.25 0.501 0.95 0.55 delimited by tabs
the this will keep the tabs and eliminate the ones under .5 :

sed 's/0*\.[0-4][0-9]*//g' old file > newfile

the output is 0.55 0.501 0.95 0.55

and for 0.25 0.25 0.301 0.95 0.55

the output is: 0.95 0.55

The TABS stay in so Excel will see them.

Fred

1 members found this post helpful.

Thanks pixellany, works perfect!


sed -r 's/0\.[5-9][0-9]*([^0-9]|$)/\1/g'

I'm really new to unix/sed, so pardon my ignorance, but:
What does the |$ mean/ literally do?
And why did the backreference (/...)/ work before, but with the addition of |$ it doesn't and (...) must be used instead?

The overall logic is to find the desired number pattern followed by any character that is:
not a number ..... [^0-9]
OR
the end of the line ..... $

The | is the "alternation operator"---a fancy way of saying "or". This operator works only with extended regexes, so I switched to that mode using the -r flag. Once I did this, then the backreference uses (...) instead of \(...\).

If you are now confused, you are a member of a large community....

Good tutorials here:
http://www.grymoire.com/Unix/

and here:
http://tldp.org
Look for the Bash Guide for Beginners and the Advanced Bash Scripting Guide (ABS)

I think that the version from flewis777 is actually good enough for the sample data we have seen so far (except that the logic is backwards---you want to delete the numbers that are 0.5 and larger)---i.e. as long as you don't have anything except numbers between the tabs, my version is overkill.

deleted

@OP, you are using the wrong choice of tool. Use awk, where you can compare numbers. don't need to waste time creating complicated regex..
Further, there are things that the proposed regex solutions might not catch. What if you have say 09.155. That, theoretically is 9.155. Might not happen with your case, BUT nothing is impossible.

0 members found this post helpful.

With all due respect, SED is an entirely appropriate solution for this. No doubt there are some arguments to favor AWK---especially if the problem is more generalized---but that does not make the SED solution incorrect.

In our motorcycle shop--almost 50 years ago--we had a mantra: "If it works, it's OK." Still true today---and will continue to be true for all time.

I would be hard-pressed to think of a computing problem that has only one solution.......

1 members found this post helpful.