There might be more elegant ways but maybe something like this could be an idea.
First run something like 'tail -f logfile >> tempoutput&'
That will append anything that gets added to the logfile to a tempfile. All you have to do in your script is a loop which periodically checks the tempfile and then clears the contents.
Code:
#!/bin/bash
tail -f logfile >> tempoutput&
while [ 1 ]
do
grep ERROR tempoutput
cat /dev/null > tempoutput
sleep 1
done