LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 08-19-2008, 09:47 AM   #1
imkornhulio
Member
 
Registered: Aug 2008
Posts: 54

Rep: Reputation: 15
Need help create a bash script to edit CSV File


i need to edit 4th field in the CSV file (i need to subtract 5 from whatever number is their in the fourth field, if the number is negative then replace it with 0)
The separator is ,

Please help
 
Old 08-19-2008, 09:52 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Can you post some lines of the CSV file as example? What have you tried until now?
 
Old 08-19-2008, 09:52 AM   #3
imkornhulio
Member
 
Registered: Aug 2008
Posts: 54

Original Poster
Rep: Reputation: 15
Online,World,20080819,0,1200M05,045468004016,N,,42,94,,,,,,N
Online,World,20080819,13,1200M055,045468004023,N,,42,94,,,,,,N
Online,World,20080819,0,1200M06,045468004030,N,,42,94,,,,,,N
Online,World,20080819,67,1200M065,045468004047,N,,42,94,,,,,,N

i tried to use awk but i am not that familiar with it

Last edited by imkornhulio; 08-19-2008 at 10:18 AM.
 
Old 08-19-2008, 10:24 AM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Using awk you can do something like this:
Code:
awk -F, '                                                              
BEGIN{OFS=","}
{if ($4 < 5)
    $4 = 0;
else
    $4 = $4 - 5
print}' file
I think the code is self-explaining. I put it on multiple lines to make it more clear. If you're looking for a solution in pure bash, well... it is a bit more complicated.
 
Old 08-19-2008, 10:36 AM   #5
imkornhulio
Member
 
Registered: Aug 2008
Posts: 54

Original Poster
Rep: Reputation: 15
is it possible to do it in Bash. i am not familiar on how to use awk.
Can i use awk in bash as a command and create a script?

Last edited by imkornhulio; 08-19-2008 at 10:42 AM.
 
Old 08-19-2008, 11:11 AM   #6
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Quote:
Originally Posted by imkornhulio View Post
Can i use awk in bash as a command and create a script?
Of course. You can use awk in a shell script as any other external command (I use the term external commands here to distinguish between them and the shell built-ins). Depending on what you're trying to do, you may need something like this:
Code:
#/bin/bash
awk -F, 'BEGIN{OFS=","}{if ($4 < 5) $4 = 0; else $4 = $4 - 5; print}' /path/to/input/file > /path/to/output/file
 
Old 08-19-2008, 11:35 AM   #7
imkornhulio
Member
 
Registered: Aug 2008
Posts: 54

Original Poster
Rep: Reputation: 15
works like a charm

Thanks a lot
 
Old 08-19-2008, 11:53 AM   #8
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
You're welcome!

Here is a bash solution, too
Code:
#!/bin/bash
declare -a array

# Store the original input field separator
OLD_IFS=$IFS

while read line
do
  # Load the comma separated fields into array
  IFS=','
  array=(`echo "$line"`)
  
  # Do arithmetics on 4th element
  if [ ${array[3]} -lt 5 ] ; then
     array[3]=0
  else
     array[3]=$((${array[3]} - 5))
  fi

  # Print a "comma separated" array
  printf "%s," ${array[@]} | sed 's/,$//'
  echo
done < testfile

# Restore the original input field separator
IFS=$OLD_IFS
this does not use awk, but uses sed as external command to strip out the trailing comma when printing the content of the array. Maybe there is a better way to do.
 
Old 08-26-2008, 07:40 AM   #9
imkornhulio
Member
 
Registered: Aug 2008
Posts: 54

Original Poster
Rep: Reputation: 15
I used the awk script and ran it in the cron job. but when i run the cron job the output file shows 0bytes. When i run it manually it runs fine. Can someone help me resolve this problem. I changed the file rights and the directory rights and gave them both 777 permission. NEED HELP!!!

Thanks

Last edited by imkornhulio; 08-26-2008 at 09:46 AM.
 
Old 08-27-2008, 05:04 AM   #10
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
The crontab has a very limited environment, different from what you get upon login. The PATH is usually limited to /bin:/usr/bin, so that a good rule is to specify the absolute path of all the commands in the script. You can also look at the mail of the user owning the crontab to see if there is any error message. On most systems the cron daemon sends the standard output and the standard error to the user's mail, unless they are redirected to a file. Feel free to post the whole script and the crontab entry to let us take a look at what the problem is.
 
Old 08-29-2008, 09:51 AM   #11
imkornhulio
Member
 
Registered: Aug 2008
Posts: 54

Original Poster
Rep: Reputation: 15
It worked out great. I put in full path in the script and worked like a charm.

Thanks a lot
 
Old 02-04-2009, 02:04 PM   #12
imkornhulio
Member
 
Registered: Aug 2008
Posts: 54

Original Poster
Rep: Reputation: 15
help

Quote:
Originally Posted by colucix View Post
Of course. You can use awk in a shell script as any other external command (I use the term external commands here to distinguish between them and the shell built-ins). Depending on what you're trying to do, you may need something like this:
Code:
#/bin/bash
awk -F, 'BEGIN{OFS=","}{if ($4 < 5) $4 = 0; else $4 = $4 - 5; print}' /path/to/input/file > /path/to/output/file
the script doesn't work if the separator is "|"

awk -F, 'BEGIN{OFS="|"}{if ($4 < 5) $4 = 0; else $4 = $4 - 5; print}' /path/to/input/file > /path/to/output/file

Please help
 
Old 02-04-2009, 11:37 PM   #13
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Quote:
Originally Posted by imkornhulio View Post
awk -F, 'BEGIN{OFS="|"}{if ($4 < 5) $4 = 0; else $4 = $4 - 5; print}' /path/to/input/file > /path/to/output/file
This code works for me in respect of your modification. You have changed the Output Field Separator but maybe you meant the Input Field Separator, didn't you?! If this is the case, just restore the OFS variable and modify the value of the option -F
Code:
awk -F"|" 'BEGIN{OFS=","}{if ($4 < 5) $4 = 0; else $4 = $4 - 5; print}' /path/to/input/file > /path/to/output/file
In an awk program the input field separator is simply FS. The -F option is just a quick shortcut for the assignment of FS. If you want to explicitly assign it within the awk code, just add it to the BEGIN section:
Code:
awk 'BEGIN{FS="|"; OFS=","}{if ($4 < 5) $4 = 0; else $4 = $4 - 5; print}' /path/to/input/file > /path/to/output/file
 
Old 02-05-2009, 10:23 AM   #14
imkornhulio
Member
 
Registered: Aug 2008
Posts: 54

Original Poster
Rep: Reputation: 15
Thanks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
CSV file - Duplicate records need merging | BASH? lmedland Programming 21 12-10-2010 05:01 AM
create chart from csv file wolfipa Linux - Software 6 01-31-2008 07:59 PM
Bash script to read file and alter lines depending on create date ChristianHein Linux - General 13 08-04-2007 05:39 AM
Bash script to edit text file snowman81 Linux - Desktop 2 01-10-2007 03:33 PM
bash script - remove header row from csv file pljvaldez Programming 5 08-30-2006 11:05 AM


All times are GMT -5. The time now is 07:34 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration