Bash scripting: parsing a text file character-by-character

Completely Clueless · 08-11-2009, 08:41 AM

Hi all,

Is there a way to process individual characters one-by-one from a text file in Bash, or is that hoping for a little too much from this lovable old clunker?

Thanks,

cc

PMP · 08-11-2009, 09:16 AM

Not the exact solution of your problem but might take you to the right path

Code:

sed 's/\(.\)/\1\n/g' <file>

ntubski · 08-11-2009, 09:30 AM

Code:

~$ help read
read: read [-ers] [-u fd] [-t timeout] [-p prompt] [-a array] [-n nchars] [-d delim] [name ...]
...
    If -n is supplied with a non-zero NCHARS argument, read returns after NCHARS
    characters have been read.

catkin · 08-11-2009, 10:10 AM

Quote:

Originally Posted by Completely Clueless

lovable old clunker

So ya seein' the light, huh?

Praise be and pass the pipe!

Completely Clueless · 08-11-2009, 10:25 AM

Quote:

Originally Posted by catkin

So ya seein' the light, huh?

Praise be and pass the pipe!

Hello Charles,

Well I'm simply more realistic about Bash's capabilities now I know how it arose and what for.
This task is something I could jot down on the back of a fag packet in C, but that was very many years ago and I can't recall the relevant functions. ISTR C has a getchar() function which does this. I need a Bash equivalent that I can run in an 'until' loop until the next character read from the file is EOF.
It's annoying that so much I want to do in Linux is usually something I've yet to learn!

gnashley · 08-11-2009, 11:01 AM

Code:

while read -n 1 ; do
# do something
done < file

catkin · 08-11-2009, 12:01 PM

Quote:

Originally Posted by Completely Clueless

This task is something I could jot down on the back of a fag packet in C, but that was very many years ago and I can't recall the relevant functions. ISTR C has a getchar() function which does this. I need a Bash equivalent that I can run in an 'until' loop until the next character read from the file is EOF.
It's annoying that so much I want to do in Linux is usually something I've yet to learn!

Ah, the delights of steep learning curves!

Solutions already posted show how to do what you want in bash; if you want to do a non-blocking read -- simple in C -- then things get a little more "interesting"

ISTR that getchar() and putchar() are actually C #define macros that hide an underlying buffer for convenience and performance but that also comes from "very many years ago" so take it with a pinch of salt.

Completely Clueless · 08-11-2009, 12:02 PM

Quote:

Originally Posted by gnashley

Code:

while read -n 1 ; do
# do something
done < file

That looks promising. I'll give it a try and report back later. Thanks.

Completely Clueless · 08-11-2009, 06:03 PM

Okay, well we're off and running, but a few teething problems. This is the relevant code fragment as it stands after some tinkering:

Code:

#!/bin/bash
#process a text file character-by-character
# the variable '$char' is the individual file character under test

x=1
while read -n$x char
do

   echo -n "iteration number $x:"
   echo " $char"
   x=$[ $x + 1 ]



done < textfile.txt

The problem is that I'm incrementing the -n of 'read' using the variable $x each time the loop iterates, but each time read functions, it's reading longer and longer strings from the file each time around. I only want it to fetch *one* character at a time to test on; kind of like a pointer moving up an array one element at a time.

How do I tell 'read' to just fetch one character each time, but every iteration make it take the *next* character and so on in sequence until the end of the file is reached?
Thanks, guys.

cc

ghostdog74 · 08-11-2009, 07:49 PM

Quote:

Originally Posted by Completely Clueless

Hi all,

Is there a way to process individual characters one-by-one from a text file in Bash, or is that hoping for a little too much from this lovable old clunker?

Thanks,

cc

use awk

Code:

awk 'BEGIN{FS=""}{ 
  for(i=1;i<=NF;i++){ print "do something with "$i }
}' file

ta0kira · 08-11-2009, 08:15 PM

Quote:

Originally Posted by catkin

ISTR that getchar() and putchar() are actually C #define macros that hide an underlying buffer for convenience and performance but that also comes from "very many years ago" so take it with a pinch of salt.

(taking the tangent...) It doesn't make a lot of sense to use an input buffer in that manner, anyway. In C you could read in a line of text, then process it a character at a time from an array. If the objective was to react to keyboard input immediately, the read system call would be more appropriate (after changing the terminal settings) with a requested size of 1. I think the objective of getchar is to avoid incrementing the buffer by an entire line, or to simulate read when one is predominantly using text input.
Kevin Barry

gnashley · 08-12-2009, 01:57 AM

Code:

#!/bin/bash
#process a text file character-by-character
# the variable '$char' is the individual file character under test

count=1
while read -n1 char
do

   echo -n "iteration number $count:"
   echo " $char"
   (( count++ ))

done < textfile.txt

Completely Clueless · 08-12-2009, 04:18 AM

Excellent work, Gnashley! Works like a charm.

catkin · 08-12-2009, 09:07 AM

In case you're interested in why Gnashley's solution works ...

textfile.txt is opened only once, by the indirection operator < on the while; do; <commands>; done compound command.

I guess you were thinking it was the read command that opened textfile.txt when you coded while read -n$x char