How to replace newline pattern in file by other newline pattern in a shell script
I have several (vhdl) files containing a pattern with newline characters that I need to replace by another pattern that also contains newline characters.
I start with something like: Code:
port ( Code:
port ( As I need to do this (very) often I want to use a shell script. I tried: 1. Code:
#!/bin/sh Code:
sed: -e expression #1, char 5: unterminated `s' command Code:
#!/bin/sh File remains unchanged 3. Code:
#!/bin/sh Code:
unexpected newline or end of string Code:
#!/bin/sh Displays the unchanged testfile Are there any suggestions how I can automate the pattern replacement? Code:
$ sed --version |
A multi-line pattern is recognized only if you store all the lines into the pattern space. Using the following sed command, basically you store all the lines into the pattern space separated by newline, perform the substitution and finally print the result:
Code:
sed -n '1h; 1!H; g; s/ A : in std_logic_vector(width - 1 downto 0);\n Z : out std_logic;/ A : in std_logic_vector(width - 1 downto 0);\n B : in std_logig;\n Z : out std_logic;/g; $p' testfile |
Wow, thank you. It seems to work. Until now I only tested a small file. The final file really is quite big (more than 50,000 lines). I will see how it works for it.
I only have one remark for others following my attempt. To perform the action in place (i.e. directly inside the file) you have to add 'i' as parameter: Code:
sed -ni '1h; 1!H; g; s/foo\nbar/foo\nfoobar\nfoo/g; $p' ./testfile Code:
foo Code:
foo |
Just to give an update:
I applied the method to the 50,000 lines file. It took about 15 minutes on a Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz but at least it works. |
The sed code is slow because it applies a 's' command at each cycle and so uses 50000 of them.
Either one of these should be faster; they both read all of the file into the pattern space and then use just one 's' command. Code:
sed -in '1h; 1!H; ${g; s/foo\nbar/foo\nfoobar\nfoo/g;p}' ./testfile |
I tried the two proposals and both work nearly in zero time. Thank you.
|
large files
Hi,
though your problem is solved I'd like to add another solution to address the large file "issue". Code:
sed -i '/foo/ ! {/bar/ {x; /foo/ i\ As for the large file "issue": It is not really an issue. Suppose every line contains 100 characters. with 50000 lines the filesize would be appr. 5MB. This is still negligible. So there is nothing wrong with the previous solutions. In fact, Kenhelm's is probably the fastest. So consider the above code just as complementary "backup" in the highly unlikely case that you might have to process a file of several hundred MB's in size. PS: You might want to change the single quotes to double quotes and use variables for the patterns 'foo' and 'bar'. This way you keep the 'sed' readable if the pattern is big as in your case. You also gain flexibility. |
Quote:
Code:
foo bar Code:
#!/bin/sh |
Quote:
BTW, this Code:
sed -i "/$START/ ! {/$END/ {x; /$BEGINNING/ i\$INSERTION x}};h" $FILE Code:
sed -i "/$START/ ! {/$END/ {x; /$BEGINNING/ i\ Here is how you use the whole pattern in memory alternative with multiline patterns: Code:
START="foo bar\nfoo" |
Ah - ok thank you. I just misunderstood what you wrote and was already suspicious.
|
All times are GMT -5. The time now is 09:43 AM. |