[SOLVED] problem with line buffering using here document in bash.

bartonski · 01-22-2010, 08:59 AM

I keep time sheet entries at work in an sqlite database called 'timesheet'.

I have a shell script called 'today' which queries for all timesheet entries which are less than 24 hours old; it looks like this:

Code:

#!/bin/bash
echo "select datetime(start_time, 'unixepoch', 'localtime'), datetime(end_time, 'unixepoch', 'localtime'), project, site, duration from timesheet where start_time > strftime('%s', 'now', '-1 day', 'localtime');" | sqlite3 ~/.timesheet/timesheet.db

In an effort to make it more readable, I've put the query inside a here document:

Code:

#!/bin/bash
cat | sqlite3 ~/.timesheet/timesheet.db << query
select
        datetime(start_time, 'unixepoch', 'localtime')
        , datetime(end_time, 'unixepoch', 'localtime')
        , project
        , site
        , duration
from
        timesheet
where
        start_time > strftime('%s', 'now', '-1 day', 'localtime');
query

The problem is that the second shell script doesn't return to the bash prompt after it runs... it requires me to press 'enter' before I get my prompt back. I can't figure out why.

Ok... I just solved my problem by re-directing the here document directly to sqlite3, rather than using 'cat |':

Code:

#!/bin/bash
sqlite3 ~/.timesheet/timesheet.db << query
select
        datetime(start_time, 'unixepoch', 'localtime')
        , datetime(end_time, 'unixepoch', 'localtime')
        , project
        , site
        , duration
from
        timesheet
where
        start_time > strftime('%s', 'now', '-1 day', 'localtime');
query

but it's still not clear to me why the former wasn't working.

rn_ · 01-22-2010, 09:12 AM

I believe it's because the << is being applied to the sqllite command and not the cat command, but when the script starts, cat has to do something to pass the control to the sqllite cmd, which is the extra enter that you have to press. If you try ths: cat << query | sqllite.... the script won't wait for an enter. The better solution as you already found out was to remove cat completely. Another solution that I always use is just use the echo command as in your original script, just break it into multiple lines; the shell will handle it.

Code:

echo "
select stmt ...
....
....
" | sqllite....

bartonski · 01-22-2010, 04:48 PM

Thanks, rn. That makes perfect sense...

as such, I think that I could also have done this:

Code:

#!/bin/bash
(cat | sqlite3 ~/.timesheet/timesheet.db) << query
select
        datetime(start_time, 'unixepoch', 'localtime')
        , datetime(end_time, 'unixepoch', 'localtime')
        , project
        , site
        , duration
from
        timesheet
where
        start_time > strftime('%s', 'now', '-1 day', 'localtime');
query

... although there's no reason to.