[SOLVED] How to read a CSV file in a shell script

mhashemi · 03-07-2019, 09:30 AM

How to read a CSV file in a shell script.

Mort Hashemi

michaelk · 03-07-2019, 09:47 AM

Welcome to LinuxQuestions.

You should be able to find examples by searching the web. Using IFS=, you can use a loop and read each field into a separate variable or an array. Here is an old example.

https://www.cyberciti.biz/faq/unix-l...rated-cvsfile/

There isn't a standard that defines a field so depending on your data format reading a CSV file using bash may not be as simple as the above example.

mhashemi · 03-07-2019, 10:34 AM

Thank you michaelk, I tried the script that you suggested, but IFS loop does not return any values. Any idea why?

for file_name in $search_dir/*.idx
do
oldifs=$ifs
while ifs=, read a1 a2 a3 a4 a5;
do
echo $a1 $a2 $a3 $a5
done < file_name

ifs=$oldifs
done

michaelk · 03-07-2019, 10:57 AM

Code:

for file_name in $search_dir/*.idx
do
oldifs=$ifs
while ifs=, read a1 a2 a3 a4 a5;
do
echo $a1 $a2 $a3 $a5
done < file_name  <--- You missed the $ in the variable name.

ifs=$oldifs
done

Some syntax errors but it depends on what shell you are using and assume bash. You can check your script using shellcheck.net.
Make sure you define search_dir.

Code:

#!/bin/bash

for file_name in $search_dir/*.idx
do

  while ifs=, read a1 a2 a3 a4 a5;
  do
    echo $a1 $a2 $a3 $a4 $a5
  done < "$file_name" 

done

mhashemi · 03-07-2019, 11:32 AM

I tried it with $ and without. Also tried enclosed in ("). None of them worked.

pan64 · 03-07-2019, 11:49 AM

ifs should be UPPERCASE

mhashemi · 03-07-2019, 12:35 PM

Thank you pan64. I change ifs to IFS. still not going through loop. Looks like that the "while IFS=,...." not reading $file_name. $file_name has the path and filename:
echo $file_name give me this: "/home/admin/sps/CBD27.idx" the file CBD27.idx is a CSV file.

pan64 · 03-07-2019, 12:47 PM

I would say insert set -xv at the beginning and you will see what's happening Probably you will find the reason (this will not solve anything, just display a lot of logs)

mhashemi · 03-07-2019, 02:08 PM

Thank you pan64. It is working now.