LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 09-10-2015, 06:55 PM   #1
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Rep: Reputation: Disabled
Program to split stdin to multiple output streams?


Does anyone know of a program to split stdin into multiple output streams? By "split" I really mean "stripe", where lines are striped accross the output files.
Something like this:
cat file | command 4 1> 1.out 2> 2.out 3> 3.out 4> 4.out
 
Old 09-10-2015, 09:49 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,329

Rep: Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745
If you just want to split a file into multiple pieces, there is in fact a cmd called 'split' that does that.
If you actually meant something else, please give more context/example.
 
Old 09-11-2015, 12:06 AM   #3
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by chrism01 View Post
If you just want to split a file into multiple pieces, there is in fact a cmd called 'split' that does that.
If you actually meant something else, please give more context/example.
Im specifically looking for a way to split the output using a round-robbin approach, similar to a stripe, where each successive line goes to the next file in the sequence.
I can do each output, one at a time by using the awk command with a 'NR%value' to determine which group a record falls into, but it requires multiple passes.
Was looking for a way to split the input into output files so that each output file get's the same number of lines (+- one) to account for odd line counts.

Line 1 -> file 1
Line 2 -> file 2
Line 3 -> file 3
etc...
Line N -> file 1
Line N+1 -> file 2
etc...
 
Old 09-11-2015, 12:18 AM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,013

Rep: Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102
Quote:
Originally Posted by usao View Post
I can do each output, one at a time by using the awk command with a 'NRvalue' to determine which group a record falls into, but it requires multiple passes.
No it doesn't. Awk will handle all this in one pass if you direct the print to a filename (as a string) - simply use the modulo result as a suffix for example.
 
Old 09-11-2015, 12:37 AM   #5
usao
Member
 
Registered: Dec 2011
Location: Chandler, AZ
Posts: 286

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by syg00 View Post
No it doesn't. Awk will handle all this in one pass if you direct the print to a filename (as a string) - simply use the modulo result as a suffix for example.
Can you give me an example? I don't see how to do it in one pass.
 
Old 09-11-2015, 12:41 AM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,013

Rep: Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102Reputation: 4102
Code:
echo -e "1\n2\n3\n4\n5" | awk '{print > "file"NR%2}'

Last edited by syg00; 09-11-2015 at 12:43 AM. Reason: LQ deleting percent sign
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Distribute lots of files output by program across multiple locations (dirs, drives) muahahaha9001 Programming 2 10-02-2013 05:40 PM
gzip: stdin: Input/output error,what does it mean? ayongying Linux - Newbie 11 03-03-2010 01:13 AM
help with c program to read each line from text file, split line , process and output gkoumantaris Programming 12 07-01-2008 01:38 PM
Send ALL output streams to file? Nuvious Linux - General 1 04-22-2007 03:08 PM
Can't have multiple audio streams Dillbilly Linux - General 3 03-05-2006 11:16 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:40 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration