LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 07-03-2010, 05:16 PM   #1
fernandomm
LQ Newbie
 
Registered: May 2010
Posts: 2

Rep: Reputation: 0
Shell script: stdin redirection + multiple curl process issues


Hello,

I have a script that redirects stdin to a "log file" ( like this http://tldp.org/LDP/abs/html/redircb.html ). This script calls multiple curl threads with & ( running on background ).

The issue that i'm having is that sometimes 2 curl processes write on the same line. Example:

How it should be:

Code:
This is a log line from curl 1
This is a log line from curl 2
How it looks like sometimes ( it is random but happens daily ):

Code:
This is a This is a log line from curl 2
Or something like this:

Code:
This is a log line from This is a log line from curl 2
Do you have any ideas of how i could fix this problem? I am not sure if locking the file for write would help, as it could reduce performance.


Thanks!
 
Old 07-04-2010, 08:10 AM   #2
bgoodr
Member
 
Registered: Dec 2006
Location: Oregon
Distribution: RHEL[567] x86_64, Ubuntu 17.10 x86_64
Posts: 221

Rep: Reputation: 36
Quote:
Originally Posted by fernandomm View Post
The issue that i'm having is that sometimes 2 curl processes write on the same line.
The standard output for the script is inherited by the child processes (the curl processes), and they do not know anything about each other, and the output will be interleaved by characters and not by lines.

The cleanest way to address this is to redirect to specific, separate log files using the ">" operator. I think C/C++ applications that are designed to run as daemons use the syslog facility (command line interface I believe is the "logger" program), but that won't help you much if you don't want to write into root-controlled logging directory such as underneath /var/log. Hence my suggestion to write them to separate log files and figure out some other way to collate them. Scanning the curl man page indicates no way to get curl to write out date/time stamps on each line it emits, so collating is going to be problematic anyway.

bg
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell script stdout, stderr and stdin solo9300 Linux - General 6 12-29-2009 12:33 AM
read from stdin in a shell script bujecas Linux - General 3 07-03-2009 04:46 PM
How to create a shell script that reads something from stdin? ricky_ds Programming 2 05-17-2005 05:06 AM
shell script output redirection goral.j Programming 3 01-27-2005 05:34 AM
I/O redirection in a shell script gladeiator Programming 1 11-27-2003 01:35 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:47 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration