LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   How to make a command write to the file it's reading from? (https://www.linuxquestions.org/questions/linux-newbie-8/how-to-make-a-command-write-to-the-file-its-reading-from-779990/)

MTK358 01-05-2010 08:32 AM

How to make a command write to the file it's reading from?
 
For example I want a file to be processed by sed, and then overwrite the file with sed's output. I would try this:

Code:

sed '<regex goes here>' myfile > myfile
but it doesn't work as expected, instead it empties the file (I am thinking that as the first byte comes out of sed, it overwrites the whole file and sed has nothing more to do). How can I make this work?

nuwen52 01-05-2010 08:38 AM

See if your version of sed has the -i option. What that is supposed to do it edit the file "in place". So, your command would be: sed -i '<regex goes here>' myfile

MTK358 01-05-2010 09:05 AM

That's good, but what if I use something other than sed that doesn't have an "in-place" option?

slacker_et 01-05-2010 09:13 AM

Is there a specific reason why you have to edit the original file ?
Why not have the script make a copy of the file, edit the copy, and then rename both files when done.

--ET

GrapefruiTgirl 01-05-2010 09:19 AM

Quote:

Originally Posted by MTK358 (Post 3814933)
That's good, but what if I use something other than sed that doesn't have an "in-place" option?

Is there the real possibility that you'll be using something other than sed (which has the -i option)?
Maybe provide us with something closer to a real example of what you plan to do, and we could give better advice.

Sed's -i option can create a backup file as well, before doing the editing:

Code:

sed -i.BAK '/regex/' myfile
where the .BAK can be whatever extension you want to tack onto the backup filename.

You may also be able to use AWK to read the file, do some processing, and use > to direct the result to a new file..

Sasha

PS -- another idea is to cat the entire original file into a variable, then echo the variable through the processing tool (sed, awk) and then > back to the original file. A warning though: I'm not sure if there's a limit to how much stuff you can stick into a single variable; it may be 65535 bytes, or some other limit. However, in the case that the file is too large for a single variable, then cat it line by line into an array, then process the array one by one and > back to the original file.

slacker_et 01-05-2010 09:30 AM

Quote:

Originally Posted by MTK358 (Post 3814933)
That's good, but what if I use something other than sed that doesn't have an "in-place" option?

Is this similar to what you are asking ?
Code:

#!/bin/ksh



ls -lid ${0}*

sed '3a#echo blah blah blah' $0 >$0.new
chmod +x $0.new

mv $0 $0.sav
mv $0.new $0
ls -lid ${0}*

exit 0

--ET

MTK358 01-05-2010 10:28 AM

Quote:

Originally Posted by slacker_et (Post 3814962)
Is this similar to what you are asking ?
Code:

#!/bin/ksh



ls -lid ${0}*

sed '3a#echo blah blah blah' $0 >$0.new
chmod +x $0.new

mv $0 $0.sav
mv $0.new $0
ls -lid ${0}*

exit 0

--ET

I don't understand that, I'm not very good with shell scripts.

I know that you can make a copy of the file, but in my situation you have to run multiple instances of editing programs over the same file to achieve the final relult, and that way you would have to have two copies and swap them every time, and then if you insert a command in the middle of the sequence you would have to tediously modify the rest of the file.

slacker_et 01-07-2010 08:47 AM

Quote:

Originally Posted by MTK358 (Post 3815046)
I don't understand that, I'm not very good with shell scripts.

I know that you can make a copy of the file, but in my situation you have to run multiple instances of editing programs over the same file to achieve the final relult, and that way you would have to have two copies and swap them every time, and then if you insert a command in the middle of the sequence you would have to tediously modify the rest of the file.

No you wouldn't have to "have two copies" or "tediously modify the rest of the file".

What you (or rather your script/program doing the editing) would do is copy the existing (ie current) file to {filename}.new, modify the {filename}.new, mv {filename} to {filename}.sav, and then mv the {filename}.new to {filename}.
And to avoid more than one process attempting to modify the file at the same time. You can incorporate "lockfile" into your scripts.

This is essentially what the command "visudo" does with /etc/sudoers.

--ET

kofucii 01-07-2010 12:34 PM

Code:

cat myfile | sed 'rexeg' > myfile

MTK358 01-07-2010 01:53 PM

Quote:

Originally Posted by kofucii (Post 3817788)
Code:

cat myfile | sed 'rexeg' > myfile

So that will work because sed's modified output won't affect cat?

It still would be nice to have a command that would buffer the output and then write it all at the very end.

Tinkster 01-07-2010 02:12 PM

Quote:

Originally Posted by kofucii (Post 3817788)
Code:

cat myfile | sed 'rexeg' > myfile

Which OS is this working on?

Code:

$ cat silly
This is just a silly
little test silly file to
check whether silly sed and cat
silly do work in the way described
by kofucii  ...

$ cat silly | sed 's/silly/clever/g' > silly
$ cat silly
$

For me it bears the same result as
Code:

sed 's/silly/clever/g' silly > silly
... an empty file.

I can reproduce that on x86_64, z and SPARC with
Slackware, RHEL and Solaris 10.


Cheers,
Tink

kofucii 01-07-2010 02:23 PM

Quote:

Originally Posted by Tinkster (Post 3817898)
Which OS is this working on?

Code:

$ cat silly
This is just a silly
little test silly file to
check whether silly sed and cat
silly do work in the way described
by kofucii  ...

$ cat silly | sed 's/silly/clever/g' > silly
$ cat silly
$

For me it bears the same result as
Code:

sed 's/silly/clever/g' silly > silly
... an empty file.

I can reproduce that on x86_64, z and SPARC with
Slackware, RHEL and Solaris 10.


Cheers,
Tink

Execute it from a script, file name must be specified as a variable and It will work:
Code:

#!/bin/bash
FILE=myfile
cat $FILE | sed "s/foo/bar/g" > $FILE


Tinkster 01-07-2010 02:41 PM

Quote:

Originally Posted by kofucii (Post 3817914)
Execute it from a script, file name must be specified as a variable and It will work:
Code:

#!/bin/bash
FILE=myfile
cat $FILE | sed "s/foo/bar/g" > $FILE


Hmmmm

Code:

$ cat silly test.sh
This is just a silly
little test silly file to
check whether silly sed and cat
silly do work in the way described
by kofucii silly ...

#!/bin/bash
FILE=silly
cat $FILE | sed "s/foo/bar/g" > $FILE
$
$
$ sh test.sh
$ cat silly       
$

Hmmmm ... nope. Well ... on one of the three platforms
it does what you say - on two it results in an empty
file. Interesting - have to see WHY :}

[edit]
Corrrection: on the 2nd and subsequent runs it produces an
empty file on all three platforms. Very interesting.[/edit]


Cheers,
Tink

kofucii 01-07-2010 02:49 PM

It's not a reliable method. When executed repeatedly, It can result blank files. My guess, is that the problem goes from piping, and not closing properly the bite streams.

MTK358 01-07-2010 02:57 PM

Just for fun I will try to make a little C program that will store up all it's input from stdin in an array, and when it gets an EOF it will dump everything it stored up and exit.


All times are GMT -5. The time now is 01:46 AM.