[SOLVED] inserting lines from a file into another file as the nth line

vincix · 08-17-2018, 07:44 PM

Basically what I'm trying to find out is to see if the following expression can have a file as the input and as the result to be output into another file:

Code:

sed '3~4 i string' file.txt

So instead of "string" I'd like to have lines from a certain file.

Code:

sed -e '4~4 e cat input.txt' file.txt

This doesn't do what I'm expecting it to do. After the 4th line, it inserts the whole "input.txt", and that's it. So I guess I'd need a loop. So I tried:

Code:

for i in input.txt; do sed -e '4~4 e cat $i file.txt'

Which is, of course, even worse, as it's going to insert the whole input.txt after each 4th line.

I also tried this:

Code:

while read line; do sed -e "4~4 i $line" file.txt; done < input.txt

Which inserts the same line (taken at a time) in the whole file, after each 4th line, and then it begins again with the second line from input.txt and so, again, I end up with a huge number of lines.

I'm not sure how I'm supposed to approach this problem.

scasey · 08-17-2018, 10:15 PM

I think we need a better explanation of what you're trying to accomplish.

Best would be to show us the contents (samples of the data) of both files and what you want the final result to be.
Perhaps sed is not the right tool for the job.

vincix · 08-18-2018, 02:42 AM

The first expression, sed '3~4 i string' file.txt, introduces the string "string" each 4th line, starting with the 3rd line. Instead of "string" I'd like to take the lines from a certain file and introduce them into the second file in the same way (starting with the third line and then adding the lines every 4th line).
First file:

Code:

Warum hast du mich nie gefragt,
worum es in diesem Film geht?
Und warum hab ich es dir eigentlich nie erzählt?

second file (destination):

Code:

1
00:02:44,040 --> 00:02:45,960
Why haven't you ever asked me

2
00:02:46,200 --> 00:02:49,240
what is this film about?

3
00:02:53,720 --> 00:02:58,800
And why haven't I ever told you, anyway?

The merge:

Code:

1
00:02:44,040 --> 00:02:45,960
Warum hast du mich nie gefragt,
Why haven't you ever asked me

2
00:02:46,200 --> 00:02:49,240
worum es in diesem Film geht?
what is this film about?

3
00:02:53,720 --> 00:02:58,800
Und warum hab ich es dir eigentlich nie erzählt?
And why haven't I ever told you, anyway?

syg00 · 08-18-2018, 03:10 AM

Wrong tool.
I'd probably use awk as it enables the detection of file change on read, but a shell script to read and keep count of records (or use modulo) then getline from the inserts should be simple enough to test.

vincix · 08-18-2018, 03:13 AM

It seems weird that I can do it so easily with a simple string with sed, but I can't do it when the input are several lines from a file

Not that I have anything against awk, I'm trying to progress in that respect too, but I'm also trying to understand more advanced sed editing.

syg00 · 08-18-2018, 03:45 AM

sed is a stream editor, as I think I've pointed out before. It has a hold buffer and looping constructs, but doesn't have the logic capabilities sometimes needed.
A shifting spanner does lot of jobs, but sometimes you really need a hammer.

pan64 · 08-18-2018, 03:45 AM

you need to understand how sed works at all. And you will understand better why is it not really suitable for this task (first of all because sed cannot read input from one file and put lines into the other. sed is a stream editor, it works on a single stream [only]). Learning advanced sed editing is a good idea, but will not help you in this specific case (although there can be a pure sed solution).
You ought to try another tool which can handle this situation. awk/perl/python/c/java or whatever you prefer.

Kenhelm · 08-19-2018, 10:28 AM

GNU sed has the 'R' command. It queues a line of a file to be read and inserted into the output stream at the end of the current cycle, or when the next input line is read. Because the inserted lines don't enter the pattern space they can't be edited or manipulated.
https://www.gnu.org/software/sed/man...-Commands.html

Code:

sed '2~4 R file1.txt' file2.txt

1
00:02:44,040 --> 00:02:45,960
Warum hast du mich nie gefragt,
Why haven't you ever asked me

2
00:02:46,200 --> 00:02:49,240
worum es in diesem Film geht?
what is this film about?

3
00:02:53,720 --> 00:02:58,800
Und warum hab ich es dir eigentlich nie erzählt?
And why haven 't I ever told you, anyway?

MadeInGermany · 08-19-2018, 01:03 PM

Even the (standard-)shell is quite good for this.

Code:

#!/bin/sh
cnt=1
while IFS= read f1
do
  echo "$f1"
  if [ $((cnt+=1)) -eq 4 ]
  then
    cnt=1
    IFS= read f2 <&3
    echo "$f2"
  fi
done <file.txt 3<insert.txt

The while-done block is fed from file.txt via stdin (file descriptor 0) and from insert.txt via file descriptor 3 (1 and 2 are reserved for stdout and stderr).

vincix · 08-19-2018, 01:45 PM

Great! Thank you, Kenhelm, for showing me such an easy solution with sed.
That doesn't mean that I'm not taking into consideration what syg00 and pan64 said. They do have a point.

It's nice to know that I can also do it with bash. I'll have to digest it first

A script you suggested on another thread I understood after 6-7 months

) Not that it was too complicated, but it was good homework. I kept returning to it from time to time until I kind of got it

syg00 · 08-19-2018, 06:28 PM

Quote:

Originally Posted by vincix

Great! Thank you, Kenhelm, for showing me such an easy solution with sed.

Indeed. I'm sure I've glossed over that at some time and thought "what the hell would I use that for ..."

danielbmartin · 08-20-2018, 10:07 AM

With InFile1 ...

Code:

Warum hast du mich nie gefragt,
worum es in diesem Film geht?
Und warum hab ich es dir eigentlich nie erzählt?

... and InFile2 ...

Code:

1
00:02:44,040 --> 00:02:45,960
Why haven't you ever asked me

2
00:02:46,200 --> 00:02:49,240
what is this film about?

3
00:02:53,720 --> 00:02:58,800
And why haven't I ever told you, anyway?

... this sed ...

Code:

sed '2~4 R $InFile1' $InFile2 > $OutFile

... produced this OutFile ...

Code:

1
00:02:44,040 --> 00:02:45,960
Why haven't you ever asked me

2
00:02:46,200 --> 00:02:49,240
what is this film about?

3
00:02:53,720 --> 00:02:58,800
And why haven't I ever told you, anyway?

This result does not match that shown in post #8.

Please advise.

Daniel B. Martin

.

vincix · 08-20-2018, 10:19 AM

Well, yeah, because $InFile1 is going to be interpreted literally under the simple quotes. Why are you using variables in the first place, anyhow? Just use the files themselves, like in post #8. I'm not sure how sed handles bash variables. As far as I can tell, sed doesn't find anything in $InFile1, so there's nothing to output.

pan64 · 08-20-2018, 10:20 AM

you ought to learn about quotation....
Also try to use set -xv to see what's really happening during execution.
Also you can try www.shellcheck.net to catch problems.

And if you still not found:
between 'single quotes' the variables are not evaluated, so sed will see (and wanted to use as filename) $InFile1 as is.

danielbmartin · 08-20-2018, 10:36 AM

Thank you, pan64 and vincix, for the correction. Though I ought to know better, I often get tripped up with quotation.

Daniel B. Martin

.