LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-04-2016, 10:25 AM   #1
kmkemp
LQ Newbie
 
Registered: Oct 2016
Posts: 5

Rep: Reputation: Disabled
grep & count patterns in a set of files, then print to a csv file


Hi all,

I am trying to come up with a lines of code that will report all the file names in a directory in column a and then look inside each file, count and report a pattern using grep in column b of a tab delimitated file. I am very close but need a little help.

Here's what I have:
for f in joined/*/*.fastq; do ls $f; grep "@M02849" $f | wc -l; done >> joined_out.txt

Here's the output I get in joined_out.txt:
joined/17.control.aw.T0.050615_R1/fastqjoin.join.fastq
119537
joined/17.control.aw.T0.050615_R1/fastqjoin.un1.fastq
15549
joined/17.control.aw.T0.050615_R1/fastqjoin.un2.fastq
15549
...... many more lines

Here's what I want to be printed to joined_out.txt:
joined/17.control.aw.T0.050615_R1/fastqjoin.join.fastq 119537
joined/17.control.aw.T0.050615_R1/fastqjoin.un1.fastq 15549
joined/17.control.aw.T0.050615_R1/fastqjoin.un2.fastq 15549


Many thanks from a microbial ecologist/wannabe bioinformatician!
 
Old 10-04-2016, 11:02 AM   #2
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,208

Rep: Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611
Something like:

Code:
for f in joined/*/*.fastq; do echo -n `ls $f`; echo -n " "; grep "@M02849" $f | wc -l; done >> joined_out.txt
Probably a prettier, more succinct way to do it, but this will work.
 
1 members found this post helpful.
Old 10-04-2016, 11:02 AM   #3
allend
Senior Member
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 4,603

Rep: Reputation: 1509Reputation: 1509Reputation: 1509Reputation: 1509Reputation: 1509Reputation: 1509Reputation: 1509Reputation: 1509Reputation: 1509Reputation: 1509Reputation: 1509
Perhaps
Code:
for f in joined/*/*.fastq; do grep -cH "@M02849" "$f" | sed "s/:/ /"; done >> joined_out.txt
will do want you want.

Last edited by allend; 10-04-2016 at 11:09 AM.
 
1 members found this post helpful.
Old 10-04-2016, 12:20 PM   #4
kmkemp
LQ Newbie
 
Registered: Oct 2016
Posts: 5

Original Poster
Rep: Reputation: Disabled
Thanks!

Very helpful! That would have taken me all afternoon to trouble shoot.

Both of those options worked and I will use this code frequently during my sequence processing!
 
Old 10-04-2016, 12:54 PM   #5
MadeInGermany
Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 428

Rep: Reputation: 187Reputation: 187
The loop scales over very many arguments and is easily extendable.
If this is not required, here is a quick alternative
Code:
grep -cH "@M02849" joined/*/*.fastq | sed "s/:/ /" > joined_out.txt
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to print lines in csv file if 1 csv column field = "text". There are 10 column (;) in csv file nexuslinux Linux - Newbie 9 04-22-2016 11:35 PM
Awk to Count Multiple patterns in a huge file reach.sree@gmail.com Programming 6 06-08-2012 01:04 PM
[SOLVED] grep many files in multiple directories using patterns from a file francy_casa Linux - Newbie 4 04-12-2012 08:49 AM
[SOLVED] Search multiple patterns & print matching patterns instead of whole line Trd300 Linux - Newbie 29 03-05-2012 07:41 PM
Find/grep/wc command to find matching files, print filename and word count dbasch Linux - Newbie 10 09-14-2009 05:55 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 04:07 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration