Share your knowledge at the LQ Wiki.
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 01-14-2015, 09:07 PM   #1
Registered: Aug 2009
Posts: 349

Rep: Reputation: 16
perl one liner to delete line in csv file if one columns is toobig

I have a little bash script that cats out a file and tells me if there is a line
where the 11th column has more than 6 characters in it.
It emails me where there is a bad line in a file - bead meaning that it will break a
donwstream process.

anyhow when i get the email saying that there is a bad file i just log in to the pc via
vpn and the I sed out the lines from the file that I get in the email. The bad lines are
always in danny.csv not danny1.csv
It has been the same lines killing the downstream process for a few weeks, so i put the "sed -i's" into
the script and it does it automagically.

for i in danny.csv danny1.csv
cat /come/and/play/with/$i | perl -ne 'print if length((split /,/)[10]) > 6' | mail -s "danny.csv bad line"

#it would be nice to find a perl change the file in place
sed -i '/D,642,0642,UBF,EVL,,M,,S,S,FOREVER,213,213,/d' /come/and/play/with/us/danny.csv
sed -i '/D,642,0642,UBF,EVL,,M,,S,S,QSP-U=C,4,4,/d' /come/and/play/with/us/danny.csv

However when a new line gets put into this file, I am going to have to log in and take out the line.
SO I have been trying to write a perl one liner that will edit the file in place, like sed, and make a
backup of the file. I just need a perl one liner that will delete any line where the 11th columns has more
than 6 characters in it.
perl -p -i.bak -e 's/\,\w{7}\,//g - which does not work.
I tried something like this:

perl -nle 'print if /\,\w{7}\,/' /come/and/play/with/us/danny.csv
but that does not catch the QSP-U=C and it catches more lines than just the
FOREVER. for a solutinog I need to focus on the the 11th column.
Old 01-15-2015, 01:47 AM   #2
LQ Guru
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 10,098

Rep: Reputation: 2989Reputation: 2989Reputation: 2989Reputation: 2989Reputation: 2989Reputation: 2989Reputation: 2989Reputation: 2989Reputation: 2989Reputation: 2989Reputation: 2989
I do not really understand it, but first please use [code] and [/code] instead of [CODE].
you have already a working sed, why do you want rewrite it perl?
in your perl script you ought to use the same regexp (as in your sed).
Old 01-15-2015, 08:38 AM   #3
Senior Member
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,714

Rep: Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280
First, sed doesn't do in-place edits either. It uses a workfile, then at the end, it deletes the original file and renames the workfile.

You can do the same thing within perl, or use an mv to move the output to the same name as the original file (this is easier for perl one liner commands).


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Perl One Liner : Capturing more than one matched regex per line and printing uncle-c Programming 6 01-22-2012 05:16 AM
[SOLVED] Perl one liner - problem with printing captured regex on new line uncle-c Programming 2 01-18-2012 06:15 AM
Map 1 CSV's columns to matching columns in another CSV 2legit2quit Programming 7 10-27-2011 09:53 AM
Batch manipulating CSV columns and files in Perl script briana.paige Linux - Newbie 1 07-14-2009 12:02 PM
Perl question: delete line from text file with duplicate match at beginning of line mrealty Programming 7 04-01-2009 07:46 PM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 02:20 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration