For example I want a file to be processed by sed, and then overwrite the file with sed's output. I would try this:

but it doesn't work as expected, instead it empties the file (I am thinking that as the first byte comes out of sed, it overwrites the whole file and sed has nothing more to do). How can I make this work?

See if your version of sed has the -i option. What that is supposed to do it edit the file "in place". So, your command would be: sed -i '<regex goes here>' myfile

That's good, but what if I use something other than sed that doesn't have an "in-place" option?

Is there a specific reason why you have to edit the original file ?
Why not have the script make a copy of the file, edit the copy, and then rename both files when done.

--ET

Is there the real possibility that you'll be using something other than sed (which has the -i option)?
Maybe provide us with something closer to a real example of what you plan to do, and we could give better advice.

Sed's -i option can create a backup file as well, before doing the editing:

where the .BAK can be whatever extension you want to tack onto the backup filename.

You may also be able to use AWK to read the file, do some processing, and use > to direct the result to a new file..

Sasha

PS -- another idea is to cat the entire original file into a variable, then echo the variable through the processing tool (sed, awk) and then > back to the original file. A warning though: I'm not sure if there's a limit to how much stuff you can stick into a single variable; it may be 65535 bytes, or some other limit. However, in the case that the file is too large for a single variable, then cat it line by line into an array, then process the array one by one and > back to the original file.

Is this similar to what you are asking ?
--ET

I don't understand that, I'm not very good with shell scripts.

I know that you can make a copy of the file, but in my situation you have to run multiple instances of editing programs over the same file to achieve the final relult, and that way you would have to have two copies and swap them every time, and then if you insert a command in the middle of the sequence you would have to tediously modify the rest of the file.

No you wouldn't have to "have two copies" or "tediously modify the rest of the file".

What you (or rather your script/program doing the editing) would do is copy the existing (ie current) file to {filename}.new, modify the {filename}.new, mv {filename} to {filename}.sav, and then mv the {filename}.new to {filename}.
And to avoid more than one process attempting to modify the file at the same time. You can incorporate "lockfile" into your scripts.

This is essentially what the command "visudo" does with /etc/sudoers.

--ET

So that will work because sed's modified output won't affect cat?

It still would be nice to have a command that would buffer the output and then write it all at the very end.

Which OS is this working on?

For me it bears the same result as
... an empty file.

I can reproduce that on x86_64, z and SPARC with
Slackware, RHEL and Solaris 10.


Cheers,
Tink

Execute it from a script, file name must be specified as a variable and It will work:

Hmmmm

Hmmmm ... nope. Well ... on one of the three platforms
it does what you say - on two it results in an empty
file. Interesting - have to see WHY :}

[edit]
Corrrection: on the 2nd and subsequent runs it produces an
empty file on all three platforms. Very interesting.[/edit]


Cheers,
Tink

It's not a reliable method. When executed repeatedly, It can result blank files. My guess, is that the problem goes from piping, and not closing properly the bite streams.

Just for fun I will try to make a little C program that will store up all it's input from stdin in an array, and when it gets an EOF it will dump everything it stored up and exit.