I have a bash shell script that spits out some long lines to the terminal screen, and seems to do so with no issues, but if the output is redirected (">") to a file, the lines in the file are all wrapped at 80 characters using hard returns.

I copy any of these long lines (in this case, 98 characters) when the output is displayed on the terminal screen with no redirection, and then "echo" it with a redirection ">" to a named text file.

When I open the file this time, the line seems to be reproduced accurately, with the full text on one line (no wrapping).

I've tried adding "COLUMNS=98" and "stty cols 98" to the script, but this makes no difference. All this is annoying because my terminal profile (Initial Terminal Size/Columns) is 132, not 80.

Where is the routine responding to the redirection getting its number of columns from?

I'm sure this is just a head space issue on my part, but I'm getting older, so that's not a big surprise.

Thanks for any help.

Some questions that come to mind:
Is the redirect on the command line? That is: script > file (In which case, it should work just like the echo, I'd think)
Or is it being done within the script?

If the latter, please show us the code.

How are you "opening" the file?

What do you mean by "hard returns"?

Is your terminal on a Desktop, or is it a remote connection? (ssh or PuTTy-like)

Edit: simulpost

So you're doing "script > outputfile" or are you doing the redirect within the script?

Does doing "script |tee outputfile" make a difference?

At a guess it may be your TERM variable that is doing this. If you type "echo $TERM" what does it show?

Does putting the script in cron and letting it run in background make a difference?

https://www.computerhope.com/unix/ufold.htm

see if that works.

Shouldn't that be a pipe into fold rather than a redirect?

Interestingly I don't see an issue like the one OP described when I just redirect into a file without doing your fold syntax.

pipping sends it to stdout whereas redirection sends it to another file.

He wanted something to over ride the 80 chars limit fold provides for that.

with my terminal stretched out to accommodate for length.

Not exactly correct, piping makes the output of one command the input of the one that follows.

Redirect (with ">") changes where stdout goes. That can be another "file" but not usually another command.

I wasn't questioning the ">" at end of line - I was questioning the ">" before "fold" you wrote. Normally it would be a pipe there. Otherwise I'd expect "> fold" to just create a file named "fold" which didn't seem to be what you intended.

Thanks for pointing that out, frist time using it, and well .. long story short,
would be the correct way to get text in one file into another at the given length. but still it'd probably be to no avail, I have no idea what his script is doing.


so in lue of me sticking my foot in my mouth any further.

Wow - thank you all for the quick responses. I'll respond in the order they showed up.

Sean: the redirect is being done within the script. The actual line is:
./dist $OriginalFile > 'ShowDist_'$OriginalFile'.dst'

The program dist looks at several sequential $OriginalFiles and creates a sort of text graph of the distribution of byte values within the file. The output is forty lines or so, each consisting of 94 characters per line. Each such "graph" displays just fine on the terminal when the
'> ShowDist_'$OriginalFile'.dst'
part of the command is eliminated, but the file has only 80 chars on each line.

All previous files created in this manner are deleted in the beginning of the script with [rm ShowDist_*.dst], and this works fine, so it is the redirect itself which creates/opens the file.

The "hard returns" are hex 0a bytes (confirmed using the bless hex editor).

The terminal and all files are on a local machine - no remote stuff.

Mensa: I'm doing the redirect within the script as described above. I suspect you're on to something with the use of tee; I hadn't thought to try that, but it produces interesting (at least to me, but then I'm easily amused) results. Changing the original line in the script to
./dist $OriginalFile |tee 'ShowDist_'$OriginalFile'.dst'
causes each line on BOTH the screen display and output file to be wrapped at 80 characters per line. So apparently whatever is causing my problem is also evident when tee is used. I have the feeling this is what's known as a clue, but no solution comes to mind.

The output of [echo $TERM] is xterm-256color

BW-userx: After studying your sample for a moment I realized you likely meant:
./longlines | fold -w 100 -s > getlines

I assumed that would cause getlines to have each number on a separate line, because of the -s, but it is still one long line in the getlines file.

Thinking (well, maybe not that well) that I could use this, I added
./dist $OriginalFile | fold -w96 -s > 'ShowDist_'$OriginalFile'.dst'
to the script, but that still resulted in the truncated/wrapped lines. I also tried it with a higher -w120, but the -w seems to have no effect here. This suggests that I may be generating something that actually has the hard returns, but in such a way that only the terminal itself seems to ignore them: but that's impossible, right?

So, I'm still befuddled, but maybe after a good dinner ...

Once again, thanks for the suggestions; hopefully, my answers to the questions will prompt some more ideas.

you want it all in one big line within the new file?
I am getting a distink difference from echo'ing a really really long one liner.

Thanks for the response and the example - but my system seems to react differently.

If I execute my [ ./dist $OriginalFile ] (or your examples) - whether from within my shell script or directly from the command line, all is well and the full length lines are displayed.

If, however, I execute [ ./dist $OriginalFile | fold -w 110 -s ], again whether from within the shell script or directly from the command line (and note there is no redirection here), I get lines that are wrapped at 80 characters. So the redirection thing is likely not part of the issue. Nor, apparently, is the fact that the command is being run from within a shell script. I guess that constitutes progress, so thanks.

I can only conclude that there is apparently some setting somewhere that limits line lengths to 80 characters when feeding program output (stdout ??) into the redirect or pipe command. I'll have to think of something else that takes stdout to try and confirm this, but I've examined the actual program output with bless and don't spot any extraneous line feeds - but they are easy to spot after the file has gone through a pipe or a redirection.

Repetitive debugging is way more convenient now than it was in the days of punched cards, but the reasons for needing to debug seem to have been working hard to keep up.

But thanks again - maybe after a long sleepless night, a light will go off ...

What if you change your TERM variable temporarily:

export OTERM=$TERM
TERM=xterm

run your script

TERM=$OTERM

Or alternatively set TERM within your script before it does the line output.

Hi Mensa ...

Thanks again, but I just tried resetting TERM as you suggested, both from the command line and within the script, and got the same results. And, yes, I confirmed the value before and after each test.

In my last post (#11) I said "I can only conclude that there is apparently some setting somewhere that limits line lengths to 80 characters when feeding program output (stdout ??) into the redirect or pipe command." Based on my results (post #9) of your suggestion to try tee (post #3), I realized last night that tee should join the redirect and pipe commands as those that are subject to this behavior on my system.

In my original post, I mentioned that adding "COLUMNS=98" and "stty cols 98" to the script made no difference. I also tried these from the command line, but a line length of 80 remains solidly in place for some reason, even though [ echo $COLUMNS ] reports what it was told.

For my immediate purposes, I modified my dist utility to generate an output file from within using a command line switch, which does the trick. Since you are apparently a mystery buff, you'll appreciate that - as Archie and Nero would - I still want to get to the bottom of this. I'm planning to locate another box with a different flavor of Linux to see what happens, but for the moment I guess I'll try to figure out string theory since it seems to be an easier problem.

Thanks again, though. I appreciate all the time and suggestions.

What terminal Emul are you using? try using it in a different one. Mine line wraps depending on how long I have it stretched out. Logic states so you can read it without having to use a scroll bar.

term emulator: terminator.

Now I feel like Sisyphus, bashing some big rock up and over a never ending 80 column hill.

I tried all of the sequences I could think of in Gnome Terminal (which I normally use), xterm, uxterm and, after a quick trip to synaptic, terminator. I experienced the same behavior in each of those.

But again, thanks.