Trouble with script
 

Hi,

I am having a problem with a script I am writing. It is a simple kind of script which joins a few files, spits out a csv file and then I call a perl script to convert it to html.

The following 3 lines are giving me trouble:
#I know I could do the sort all in the join line - but I find this more readable
sort -t, -k2 /tmp/proxmox/nmap_net0_join > /tmp/proxmox/sorted_nmap_net0_join
sort -t, -k1 /tmp/proxmox/qmlist | sed 's/ /,/g' > /tmp/proxmox/sorted_qmlist
join -t, -11 -22 /tmp/proxmox/sorted_qmlist /tmp/proxmox/sorted_nmap_net0_join >/tmp/proxmox/statusinfo 2>/tmp/proxmox/log

If I execute each line individually in a command prompt the statusinfo file is generated correctly. However if I execute the script I get a "join: file 1 is not in sorted order" exception in the log. What am I missing?

If your script has just those 3 lines, then there should not be any difference from running the 3 lines manually. (Easy to try it...) My guess is that the issue is with something else in the script.

you can try to run your script with -x (bash -x <srcipt>) and check the output

The bash -x option was new for me. I haven't solved the problem yet, but now I have more info to work with. I will sprinkle the script with echos and other output so I know what is happening at that stage.

Thanks!

Try bash -xv or
at the top of the script itself and just call it direct

Thanks Chris! I actually did set it in my script "#!/bin/bash -vx" and that worked fine. Still not working as expected though. Maybe let me explain my issue again.

Let's say I have 2 files:
file1:
234,up
235,up
236,down
237,up
238,down
239,up

file2:
AB,34X,236
BB,36N,237
AC,56X,238
DB,39X,239
EC,39L,240
FR,19M,241
NH,99P,242
EN,21Q,242

I want to join them on the first col of file1 and 3rd of file2 (even if there is no match) and have output like this:

234,up,NA,NA
235,up,NA,NA
236,down,AB,34X
237,up,BB,36N
238,down,AC,56X
239,up,DB,39X
240,NA,EC,39L
241,NA,NH,99P
242,NA,EN,21Q

(NA where there is no matching data). I have tried with simple left and right joins (using -a and -e options) as well as initial g/awk attempts (don't really graps awk yet). not working as expected yet. Any thoughts?

The idea of the -xv was to see what was causing the error reported in your first post.
Just run it with those cmds in a script and post the results; we can't guess, we need to see what you see,

From an awk point of view:

1. Read first file into an array with field one as index (this of course assumes that it will not be overwritten, ie all field one entries are uniq)

2. Search file two and compare indexes to third field, from here you have two sub options

a. Print data in format required

b. Append data to array at index

Part 'b' the requires you to output the array in the END clause, but the benefit here is you can then sort it by the indexes so they appear
in numerical order.

Here is a good awk reference. if you need further help let us know?

Chris and Grail - thanks for the help. I got too busy in the past few days with other issues. Hope to have more time to try your advice soon.

Thanks!
Gilad