Need help in bash scripting

unclesamcrazy · 03-18-2015, 03:10 AM

I have two files which has exact same number of lines.
I want first line of first file should be filename of new file and content of this new file should be first line of second file.
Then second line of first file should be filename of again new file and content of this new file should be second line of second file.
then third line of first file should be filename of again new file and content of this new file should be third line of second file.
and so on...
I am trying to do it using for loop but I am not able to create two for loops.
This is what I have done

Code:

IFS=$'\n'
var=$(sed 's/\"http\(.*\)\/\(.*\).wav\"\,\".*/\2/g' 1797.csv) # filenames of all files
var2=$(sed 's/\"http\(.*\)\/\(.*\).wav\"\,\"\(.*\)\"$/\3/g' 1797.csv) # contents of all files
for j in $var;
do
#Here I do not know how to use $var2
done

Please help.

pan64 · 03-18-2015, 04:33 AM

so you need to use indices:

Code:

total=${#var[*]}
for (( i=0; i<$total; i++))
#use ${var[$i]} and ${var2[$i]}

unclesamcrazy · 03-18-2015, 05:19 AM

I tried to use this using sed one liner but I am not able to do this. I have used this command

Code:

sed -e "s/\"http\(.*\)\/\(.*\).wav\"\,\"\(.*\)\"$/echo \3 > \2.txt/g" 1797.csv | bash

1797.csv contains

Quote:

"http://domain.com/f1/f2/f3/4f5438b5d7.wav","description1"
"http://domain.com/f1/f2/f3/4f5438e5d7.wav","description2"
"http://domain.com/f1/f2/f3/4f5438z5d7.wav","description3"

I want create files
4f5438b5d7.txt with text description1
4f5438e5d7.txt with text description2
4f5438z5d7.txt with text description3

Please help to achieve this task.

grail · 03-18-2015, 06:46 AM

I thought you said there were 2 files? You appear to only use one in your examples. Could you maybe try and describe the problem again?

unclesamcrazy · 03-18-2015, 07:20 AM

I created separate two files to use in BASH script but I was not able to do it either using shell script so I scraped the procedure.
Originally, there is only one file 1797.csv which has almost 100000 lines like above. There are two columns. One is url and other is description.
Every url contains 10 digit .wav filename.
I need to create individual text files whose filename will be filename of wav file and whose content will be description of that .wav file.

pan64 · 03-18-2015, 07:54 AM

I would use awk

Code:

awk ' { here comes your script } ' 1797.csv

and in your script you will have three variables, the filename, url and description. It looks really easy to implement, but I would like to ask you to show us a few lines (as example, with fake data probably) just to better understand your needs.

unclesamcrazy · 03-18-2015, 10:17 AM

Too much tried in awk, not able to write it.
I am using for loop but not able to succeed, here is my for loop script.

Quote:

IFS="
"
var=$(sed 's/\"http$.*$\/$.*$.wav\"\,\".*/\2/g' 1797.csv)
for i in $var
do
lst=$(sed 's/\"http$.*$\/$.*$.wav\"\,\"$.*$\"$/\3/g' 1797.csv)
IFS="
"
for j in $lst
do
echo $i
done
echo $j
done

I am trying to print, first stored value of variable $var and first stored value of variable $lst then
second stored value of variable $var and second stored value of variable $lst
but it prints all values of variable $lst then all values of variable $var.
Please help to print first from each then second from each then third and so on...

veerain · 03-18-2015, 12:32 PM

My try at it:

Code:

cat 1797.csv | awk 'BEGIN {FS=","} { fname=$1; sub(/.+\//, "", fname); sub(/.wav"/, ".txt", fname); fdesc=$2; sub(/^"/, "", fdesc); sub(/"$/, "", fdesc); fexec="/bin/echo "; sub(/$/, fdesc " > " fname, fexec); system(fexec); }'

grail · 03-18-2015, 08:26 PM

I think we could make it a little simpler:

Code:

awk -F'[",]+' '{print $3 > gensub(/^.*\/|.wav$/,"","g",$2)".txt"}' 1797.csv

And as for bash:

Code:

while IFS=, read -r file_name desc
do
  file_name=${file_name##*/}
  file_name=${file_name.*}.txt
  desc=${desc//"/}

  echo "$desc" > "$file_name"
done<1797.csv

Neither of the above check that a file name has been previously created, but I'll leave that to you.

unclesamcrazy · 03-30-2015, 06:01 AM

I was trying to do this using awk array and gsub but I am facing error. Can you please help to solve it using gsub and awk array.

Quote:

awk -F',\"' '{print $2 > gsub(/^.*\/|\.wav.*/,"")}' 1797.csv

It is not giving any error but I am not able to produce files using gsub

I have tried this command but same condition.

Quote:

awk -F',\"' '{var=$(gsub(/^.*\/|\.wav.*/,""))};{print $2 > $var2".txt"}' 1797.csv

It is producing files but files are blank, it looks like print $2 is not echoed inside files.

Please help.

grail · 03-30-2015, 09:41 AM

You need to look at the examples provided again. We are not using gsub but rather gensub. gsub returns the number of changes made whereas gensub returns the altered string, which is what you want.

unclesamcrazy · 03-31-2015, 06:24 AM

Thank you grail for your explanation.
awk is a beauty. I always wanted to learn awk but it is really tough.
I have tried using bash array, I don't know why I was using two loops, I have tried to do with single loop. It is working(for now).

Quote:

#!/bin/bash
IFS=$'\n'
ARR=( $(cat 1797.csv) )
for i in "${ARR[@]}"
do
var=$(echo $i | sed 's/\"http$.*$\/$.*$.wav\"\,\".*/\2/g')
var2=$(echo $i | sed 's/\"http$.*$\/$.*$.wav\"\,\"$.*$\"$/\3/g')
echo $var2 > $var.txt
done

But your solution is sweet, it is portable. I wish I could write awk one liners.
Can you please help me to do this task using awk array (one liner)?
Thanks

grail · 03-31-2015, 09:55 AM

So does the awk solution I provided not meet your criteria?

Also the bash script I provided should also do the same and does not need the sed lines at all.

If there is new criteria for either you will need to explain?

unclesamcrazy · 04-01-2015, 01:51 AM

No your scripts have done their job successfully. i was trying to learn by doing same task with different methods.
I want to learn awk but tutorials are quite tough which are suggested by experts here.

grail · 04-01-2015, 09:18 AM

Sorry I am a little confused, I have provided an awk script, but you want another one to do the same thing?