LinuxQuestions.org - [SOLVED] Bash script to merge files together (given as a comma separated string)

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Bash script to merge files together (given as a comma separated string) (https://www.linuxquestions.org/questions/linux-newbie-8/bash-script-to-merge-files-together-given-as-a-comma-separated-string-905144/)

Bash script to merge files together (given as a comma separated string)

Hello everyone, I'm sorry if this sounds trivial!

I have a $file variable that contains a comma separated list of files

e.g.
'../file1,/somedir/anotherfile.txt,./file'

and I'd like to merge them into a single file, say in ./tempmerged
Having the files appended sequentially is fine.
How can I do this?

Thanks a lot!

I am not sure I understand ... you want to merge a variable into a file?

Or are you asking how to remove the commas so the variable contains a single path?

Sorry I'll reformulate the question.

Imagine you have this variable (It comes from the arguments with a getopts)

files = 'path/to/file1,path/to/file2,path/to/file3'

file1 contains the following lines:
line1
line2

file2 contains the following lines:
line3
line4

file3 contains the following lines:
line5
line6

I'd like to obtain a new file that contains the following lines
line1
line2
line3
line4
line5
line6

Hi,

I hope I understood your question:

Code:

#!/bin/bash

arr=$(echo $1 | tr "," "\n")

cat $arr > tempmerged

You basically replace the commas with newlines, create an array from the resulting substrings and pass them all as filenames into cat.

By the way, this took me longer to figure out than I care to admit.

Quote:

Originally Posted by derstephen (Post 4483163)

Hi,

I hope I understood your question:

Code:

#!/bin/bash

arr=$(echo $1 | tr "," "\n")

cat $arr > tempmerged

I guess I'm not the only person that actually spends their own personal time learning how to do things just so they can help other people, now I don't feel like such a loser, lol.

Quote:

Originally Posted by dudeman41465 (Post 4483169)

I guess I'm not the only person that actually spends their own personal time learning how to do things just so they can help other people, now I don't feel like such a loser, lol.

Yeah, I need every motivation I can think of to learn pretty much anything, so really I'm only helping people for my own personal gain.

Well as it is already in a variable and assuming bash:

Code:

cat ${file//,/ } > tempmerged

Thanks a lot for the solutions.

I'm particularly fascinated by this

Quote:

cat ${file//,/ } > tempmerged

I really can't see the rationale behind.

Learning how to do things to help other people is awesome. It's still something new you learn.

^^^
I would also like to know how that works. I've heard of bracket expansion of variables but I'm not sure about the stuff inside it. A sed expression perhaps?

Grail's solution is very simply an application of parameter expansion. It replaces all of the commas in the string with spaces, allowing the shell to word-split the result into individual filenames.

Notice that this is one of the rare instances where you do not want to quote the variable, otherwise no word-splitting would occur. And of course if the filenames already have spaces, then we'll have to use a different technique (see below).

Incidentally, when working with lists of things, like filenames, it's usually recommended to use arrays instead. That way each element in the list can be accessed individually, as well as collectively.

http://mywiki.wooledge.org/BashGuide/Arrays

Ideally you should rewrite whatever it is that generates that variable so that it sets an array instead. If you can't do that, then you can split it into an array easily, using the same basic technique that grail provided.

Code:

$ string='file1.txt,file2.txt,file3.txt,file4.txt'



$ list=( ${string//,/ } )



$ echo "[${list[0]}] [${list[1]}] [${list[2]}] [${list[3]}]"

[file1.txt] [file2.txt] [file3.txt] [file4.txt]



$ cat "${list[@]}"

To split the string on commas if the filenames themselves have spaces in them, you'll have to change the IFS environment variable instead, which defines which characters the shell uses for word-splitting.

Code:

$ string='file 1.txt,file 2.txt,file 3.txt,file 4.txt'



$ oldIFS=$IFS        #store the original value (space,tab,newline by default)

$ IFS=','        #tell the shell to split only on commas



$ list=( $string )



$ IFS=$oldIFS        #return IFS to the original value



$ echo "[${list[0]}] [${list[1]}] [${list[2]}] [${list[3]}]"

[file 1.txt] [file 2.txt] [file 3.txt] [file 4.txt]



$ cat "${list[@]}"

When you get a chance, read through the entire BashGuide above. It will teach you all the basic concepts you need to know for good scripting.

@David - thanks for helping with the explanation :)