sed to strip out chat log time index

texasone · 10-01-2011, 05:22 PM

I'm using weechat-curses as my irc client and I'm trying to create a way to alert me to when something is said. This is what I have as my script:

Code:

#!/bin/bash 
tail -0f /home/user/.weechat/logs/irc.server.##channel.weechatlog | while read LINE 
do
  echo $LINE
  echo $LINE | festival --tts 
  sleep 0.75 
done

This would be an example of the log output:

Code:

2011-10-01 18:04:53         @nick   aight

The problem is that the "2011-10-01 18:04:53" gets spoken and really draws away from the purpose of using festival. I tried to strip it out with the sed code:

Code:

sed "s/^[0-9]{4}-[0-9]{2}-[0-9]{2}[ ]*[0-9]{2}:[0-9]{2}:[0-9]{2}//"

The problem is that this doesn't give me any output. Anyone have any ideas?

crts · 10-01-2011, 05:37 PM

Hi,

this can be done easily with sed. Even easier with awk:

Code:

awk '{$1="";$2=""}1'

If you are limited to sed:

Code:

sed 's/[^[:blank:]]* *[^[:blank:]]* *//'

Hope this helps.

[EDIT]
BTW, the reason why your solution is not working is because you have to either use escapes like '\{' and '\}' or you need to invoke 'sed' with the '-r' option to enable extended RegEx:

Code:

sed -r 's/<your RegEx>//'

texasone · 10-01-2011, 06:47 PM

Okay, when I use

Code:

tail /path/to/log/file |awk '$1="";$2="";$3""'|xargs echo

It works perfectly, however, it fails within the code of the script

Code:

tail -0f /home/user/.weechat/logs/irc.server.##channel.weechatlog | awk '$1="";$2="";$3""' | while read LINE

is the use of either sed or awk before piping in the "while read LINE" preventing it from reading the line? OR is "tail -0f" causing the issue?

crts · 10-01-2011, 07:57 PM

Quote:

Originally Posted by texasone

Okay, when I use

Code:

tail /path/to/log/file |awk '$1="";$2="";$3""'|xargs echo

It works perfectly, however, it fails within the code of the script

Code:

tail -0f /home/user/.weechat/logs/irc.server.##channel.weechatlog | awk '$1="";$2="";$3""' | while read LINE

is the use of either sed or awk before piping in the "while read LINE" preventing it from reading the line? OR is "tail -0f" causing the issue?

I made a typo in my post. Try this awk instead:

Code:

awk '{$1="";$2=""}1'

I tested it with the following little script:

Code:

#!/bin/bash
# file: sh.sh
{ sleep 1;for i in {1..5};do echo "2011-10-01 18:04:53         @nick   aight${i}">>file; sleep 1; done } &
tail -0f file |awk '{$1="";$2="";}1'

and the output was:

Code:

$ ./sh.sh
  @nick aight1
  @nick aight2
  @nick aight3
  @nick aight4
  @nick aight5

This looks fine so far but I also could not make it work with espeak (I am not familiar with festival), yet.

crts · 10-01-2011, 08:30 PM

Well, I found the *error*. The awk waits to process the complete file before it flushes its output through the pipe. Since there is no EOF the input is stuck in the middle of the pipe. Try this instead:

Code:

#!/bin/bash
{ sleep 1;for i in {1..5};do echo "2011-10-01 18:04:53         @nick   aight${i}">>file; sleep 1; done } & # simulate input

while read line;do
      echo "$line"|awk '{$1="";$2="";}1' |festival --tts
done < <(tail -0f file)

texasone · 10-01-2011, 08:51 PM

ctrs: I'll have to give that code a try. I ended up resorting to using "script -c". It works but I went back to the sed statement because of my laziness.
I'm probably going to spend the next little while cleaning it up to get rid of logins, logouts, and shorten those long paragraphs down to like 14 or so words.

Code:

script -c "tail -0f /home/user/.weechat/logs/irc.server.#channel.weechatlog  | sed 's/[^[:blank:]]* *[^[:blank:]]* *//'" /dev/null | while read LINE


do 
   echo $LINE
   echo $LINE | festival --tts
   sleep 0.75
done

Thanks for the help

texasone · 10-01-2011, 10:34 PM

Stupid question. had nothing to do with the Original Topic

crts · 10-02-2011, 12:30 AM

Quote:

Originally Posted by texasone

ctrs: I'll have to give that code a try. I ended up resorting to using "script -c". It works but I went back to the sed statement because of my laziness.
I'm probably going to spend the next little while cleaning it up to get rid of logins, logouts, and shorten those long paragraphs down to like 14 or so words.

Code:

script -c "tail -0f /home/user/.weechat/logs/irc.server.#channel.weechatlog  | sed 's/[^[:blank:]]* *[^[:blank:]]* *//'" /dev/null | while read LINE


do 
   echo $LINE
   echo $LINE | festival --tts
   sleep 0.75
done

Thanks for the help

Ok, don't mean to sound harsh but this is bad. I am not familiar with script but from what I have read so far in the man-page this is not its main purpose. Have another look at post #5. The solution is to not pipe 'tail -0f' into awk. Instead let 'tail -0f' feed the loop only and then modify the variable with awk.

If you want to use sed and are willing to replace festival with espeak then you could do something like this:

Code:

tail -0f file  | sed -u  's/[^[:blank:]]* *[^[:blank:]]* *//'|espeak

That's it. It doesn't need a loop. The -u option tells sed to output immediately; without buffering. 'espeak' also does process the input line by line rather than waiting for the whole input to finish like festival does. However, espeak's speech quality is not as good as festival's.

David the H. · 10-02-2011, 04:06 AM

What's with all this complicated messing around with sed and awk anyway?

Code:

echo "@${LINE##*@}"

String manipulation in bash

crts · 10-02-2011, 08:41 AM

Quote:

Originally Posted by David the H.

What's with all this complicated messing around with sed and awk anyway?

Code:

echo "@${LINE##*@}"

String manipulation in bash

Yeah, I did not see that right at the beginning and then I got caught up with that pipe issue. That is, which programs can deal with neverending input and which cannot. So, apparently awk and festival cannot deal with it.