LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 08-11-2009, 09:41 AM   #1
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 810

Rep: Reputation: 68
Bash scripting: parsing a text file character-by-character


Hi all,

Is there a way to process individual characters one-by-one from a text file in Bash, or is that hoping for a little too much from this lovable old clunker?

Thanks,

cc
 
Old 08-11-2009, 10:16 AM   #2
PMP
Member
 
Registered: Apr 2009
Location: ~
Distribution: RHEL, Fedora
Posts: 381

Rep: Reputation: 58
Not the exact solution of your problem but might take you to the right path
Code:
sed 's/\(.\)/\1\n/g' <file>
 
Old 08-11-2009, 10:30 AM   #3
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,543

Rep: Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879
Code:
~$ help read
read: read [-ers] [-u fd] [-t timeout] [-p prompt] [-a array] [-n nchars] [-d delim] [name ...]
...
    If -n is supplied with a non-zero NCHARS argument, read returns after NCHARS
    characters have been read.
 
Old 08-11-2009, 11:10 AM   #4
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
Quote:
Originally Posted by Completely Clueless View Post
lovable old clunker
So ya seein' the light, huh? Praise be and pass the pipe!
 
Old 08-11-2009, 11:25 AM   #5
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 810

Original Poster
Rep: Reputation: 68
Quote:
Originally Posted by catkin View Post
So ya seein' the light, huh? Praise be and pass the pipe!
Hello Charles,

Well I'm simply more realistic about Bash's capabilities now I know how it arose and what for.
This task is something I could jot down on the back of a fag packet in C, but that was very many years ago and I can't recall the relevant functions. ISTR C has a getchar() function which does this. I need a Bash equivalent that I can run in an 'until' loop until the next character read from the file is EOF.
It's annoying that so much I want to do in Linux is usually something I've yet to learn!
 
Old 08-11-2009, 12:01 PM   #6
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,775

Rep: Reputation: 481Reputation: 481Reputation: 481Reputation: 481Reputation: 481
Code:
while read -n 1 ; do
# do something
done < file
 
Old 08-11-2009, 01:01 PM   #7
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
Quote:
Originally Posted by Completely Clueless View Post
This task is something I could jot down on the back of a fag packet in C, but that was very many years ago and I can't recall the relevant functions. ISTR C has a getchar() function which does this. I need a Bash equivalent that I can run in an 'until' loop until the next character read from the file is EOF.
It's annoying that so much I want to do in Linux is usually something I've yet to learn!
Ah, the delights of steep learning curves!

Solutions already posted show how to do what you want in bash; if you want to do a non-blocking read -- simple in C -- then things get a little more "interesting"

ISTR that getchar() and putchar() are actually C #define macros that hide an underlying buffer for convenience and performance but that also comes from "very many years ago" so take it with a pinch of salt.
 
Old 08-11-2009, 01:02 PM   #8
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 810

Original Poster
Rep: Reputation: 68
Quote:
Originally Posted by gnashley View Post
Code:
while read -n 1 ; do
# do something
done < file
That looks promising. I'll give it a try and report back later. Thanks.
 
Old 08-11-2009, 07:03 PM   #9
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 810

Original Poster
Rep: Reputation: 68
Okay, well we're off and running, but a few teething problems. This is the relevant code fragment as it stands after some tinkering:

Code:
#!/bin/bash
#process a text file character-by-character
# the variable '$char' is the individual file character under test

x=1
while read -n$x char
do

   echo -n "iteration number $x:"
   echo " $char"
   x=$[ $x + 1 ]



done < textfile.txt
The problem is that I'm incrementing the -n of 'read' using the variable $x each time the loop iterates, but each time read functions, it's reading longer and longer strings from the file each time around. I only want it to fetch *one* character at a time to test on; kind of like a pointer moving up an array one element at a time.

How do I tell 'read' to just fetch one character each time, but every iteration make it take the *next* character and so on in sequence until the end of the file is reached?
Thanks, guys.

cc
 
Old 08-11-2009, 08:49 PM   #10
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by Completely Clueless View Post
Hi all,

Is there a way to process individual characters one-by-one from a text file in Bash, or is that hoping for a little too much from this lovable old clunker?

Thanks,

cc
use awk

Code:
awk 'BEGIN{FS=""}{ 
  for(i=1;i<=NF;i++){ print "do something with "$i }
}' file
 
Old 08-11-2009, 09:15 PM   #11
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by catkin View Post
ISTR that getchar() and putchar() are actually C #define macros that hide an underlying buffer for convenience and performance but that also comes from "very many years ago" so take it with a pinch of salt.
(taking the tangent...) It doesn't make a lot of sense to use an input buffer in that manner, anyway. In C you could read in a line of text, then process it a character at a time from an array. If the objective was to react to keyboard input immediately, the read system call would be more appropriate (after changing the terminal settings) with a requested size of 1. I think the objective of getchar is to avoid incrementing the buffer by an entire line, or to simulate read when one is predominantly using text input.
Kevin Barry
 
Old 08-12-2009, 02:57 AM   #12
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,775

Rep: Reputation: 481Reputation: 481Reputation: 481Reputation: 481Reputation: 481
Code:
#!/bin/bash
#process a text file character-by-character
# the variable '$char' is the individual file character under test

count=1
while read -n1 char
do

   echo -n "iteration number $count:"
   echo " $char"
   (( count++ ))

done < textfile.txt
 
Old 08-12-2009, 05:18 AM   #13
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 810

Original Poster
Rep: Reputation: 68
Excellent work, Gnashley! Works like a charm.
 
Old 08-12-2009, 10:07 AM   #14
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
In case you're interested in why Gnashley's solution works ...

textfile.txt is opened only once, by the indirection operator < on the while; do; <commands>; done compound command.

I guess you were thinking it was the read command that opened textfile.txt when you coded while read -n$x char
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH: Truncating file names with special character tboss888 Programming 2 12-18-2008 08:28 PM
how to delete nth character in a text file? xiawinter Linux - Software 3 05-13-2008 11:50 AM
View Chinese character in text file using vi ufmale Red Hat 1 02-17-2008 02:05 PM
How to store text(strings) in a 2D character array reading from a text file(C++) bewidankit Programming 3 02-14-2008 08:08 AM
Bash scripting - add a character to a line. welby Programming 1 01-14-2004 11:09 AM


All times are GMT -5. The time now is 08:19 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration