Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I'm totally new to Linux and this website. I was wondering if anyone had or could help me create a shell script that would merge two files from two different directories and then have that new merged file in a third differnt directory.
The merged file would need to eliminate duplicates and sort the contents. Any suggestions? Thanks.
Click here to see the post LQ members have rated as the most helpful post in this thread.
file1 and file2 are the files which you want to merge sort will be the option that will sort | uniq is the option that will only take the unique info basically avoiding duplicate things in both the files and > will redirect the output to file3 which will be the file that will get created after merger.
If you want to do it for files from different directories you just need to do this:
I have a suggestion. Provide some details. We aren't mind-readers here.
What kind of files? What kind of contents? What should the merged file look like? What is the purpose or context behind merging them? What's the directory structure, or will some method will be needed to locate the files first? Is this a one-time thing, or something that you will need to automate? Etc. etc.
At the very least you should provide some sample input, and a sample of the desired output.
Edit: @T3RM1NVT0R. "sort -u" alone does the same thing as "sort | uniq".
Last edited by David the H.; 05-14-2011 at 07:19 PM.
David the H. - This is a homework problem one of my friends has. He asked me to look into a way to solve this problem his professor gave them. Any and all solutions would be greatly appreciated, thanks. He needs a shell script that would satisfy this:
We have two set of files, let's say they are in directory a and directory b. Let's say the file names are a1 a2 a3 a4 ... and b1 b2 b3 b4. (I thought having the same name might be confusing, so maybe having different names is better. So, you will need to line up a1 with b1, a2 with b2 ... to create c1, c2 ... File contents are different but they need to have overlapping parts. They also have spaces in them (more than one char space).
What you'll need to do is to create a 3rd set of files in directory c where files from directory a and b are merged. You will need to compare each line from file1 of directory a with file1 of directory b and write it into file1 of directory c. Meanwhile, you'll need to merge the lines, eliminate the blanks which are more than a single char space, and eliminate the duplicates and sort this content from left to write in the output file. This should work for multiple lines and multiple files (so you need to automate this). Let's consider that each line is a fixed length--i.e. at most 80 char for input files and 160 for the output files.
For example:
file1 of a: file1 of b:
ffffff eee ccc r 12 ddd fff k ccc bbbb zzz nnn eeeee aaaaaaaaa 3
file1 of c:
12 3 aaaaaaaaa bbbb ccc ddd eee eeeee fff ffffff k nnn r zzz
Well, nobody is going to provide all the answers to homework questions, but we can help guide you through areas where you're having difficulty.
Your first order of business is to break the script down into steps and figure out how to do each one individually. Then you can assemble the working commands up into a script that controls the flow.
So start by determining how to merge two files and work from there. A simple cat+sort won't work here, since you have to actually merge lines. You'll probably need to use awk instead, which is a field-based tool, or something else with advanced editing ability like perl (but have a look at other available commands first).
The actual filenames shouldn't be a big problem; the script just needs to know where to find each one. I personally would look into creating two lists of files and storing them in arrays with matching index numbers. then you should be able to easily process everything with a single loop over those index numbers.
this is what I think will work as long as the newly combined file has an even number of characters and all characters are in one line. Let me know what you think and if there is another piece of code that would solve the even # of characters problem. Thanks again for all your input.
#!/bin/bash
# all files must contain an even number of characters or else this shell will error out
y=1
echo "Input the # of file(s) in each directory"
read x
while [ $y -le $x ]
do
filea="/dira/a"$y
fileb="/dirb/b"$y
filetemp="/temp/temp"$y
filec="/dirc/c"$y
This is a good example of how to properly get help with homework questions here at LQ. As per the LQ Rules;
Quote:
Do not expect LQ members to do your homework - you will learn much more by doing it yourself.
Kudos to you for your honesty and the manner in which you have related the queries.
I do suggest that you use/wrap vbcode tags # or quote to post your data or long lists. This will make your posts cleaner therefore easier to read.
FYI: I suggest that you look at 'How to Ask Questions the Smart Way' so in the future your queries provide information that will aid us in diagnosis of the problem or query.
Just a few links to aid you to gaining some understanding. I would start at 4,5 &6 while the other links will enhance your experience;
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.