LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-12-2018, 07:49 AM   #1
Puneeth_Linux
LQ Newbie
 
Registered: Nov 2018
Posts: 2

Rep: Reputation: Disabled
Question Trying to perform 2 operations on a single source file (huge data) to gain performance.


Hi All,

I am trying to perform 2 operations on a single Linux command. The operations are :
1) Sending data from source file into a new target file
2) Counting the number of records in the source file.

For example -
Source file - a.txt
target file - b.txt

cat a.txt > b.txt ;
cat a.txt | wc - l

In the above example, I will need to extract data from a.txt twice to perform the 2 operations. However, in my case, the source file is very huge. So for better performance, I am trying to find a way to extract data from source file just once and perform both the operations.

Please help.

Thanks,
Puneeth
 
Old 11-12-2018, 07:52 AM   #2
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
The tee command to the rescue! It splits output into two streams, one going to a file, the other to tee’s output.
Code:
cat a.txt | tee b.txt | wc -l
 
1 members found this post helpful.
Old 11-12-2018, 08:11 AM   #3
Puneeth_Linux
LQ Newbie
 
Registered: Nov 2018
Posts: 2

Original Poster
Rep: Reputation: Disabled
Thumbs up Worked !!

Hi berndbausch,

Thanks for your quick reply. Worked like a charm.
 
Old 11-12-2018, 11:09 AM   #4
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.9.2009
Posts: 5,727

Rep: Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211
Also (although still two commands)
Code:
cp a.txt b.txt; wc -l a.txt
cat is unnecessary for either process.
cat'ing a file and redirecting the output is a very inefficient way to make a copy
wc takes a filename as input. see man wc
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Perform operations based on numbers in plain text file. jmgibson1981 Linux - Newbie 11 03-31-2018 10:08 AM
LXer: How to perform search operations in Vim LXer Syndicated Linux News 0 02-10-2017 05:39 PM
LXer: Android and Linux gain drivers for huge touchscreens LXer Syndicated Linux News 0 03-27-2014 04:50 AM
How to perform mathematical operations on linux terminal? pratikjajal Linux - General 9 01-27-2014 12:52 AM
how to perform floating point operations in linux kernel? raulapati Programming 1 04-07-2011 06:25 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:17 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration