LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-15-2021, 03:32 AM   #1
Sxing
LQ Newbie
 
Registered: Nov 2021
Location: Hangzhou, China
Distribution: ArchLinux, Ubuntu
Posts: 13

Rep: Reputation: Disabled
Question Why is my C program output not in the same order as when I redirect it to a file ?


It's a multi-threading program synchronized with semaphore. According to my knowledge, those three thread should print one by one, at least in this way.
And if I redirect it to a file, it seems the lines are organized by thread and the print line in father thread is repeated 3 times ???
I am a rookie in multi-threading programming. please help me.

compiled with gcc -pthread on Ubuntu x86/64
Code:
#include <sys/sem.h>
#include <stdio.h>
#include <sys/types.h>
#include <stdlib.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <fcntl.h>
#include <semaphore.h>
#include <sys/stat.h>
#include <time.h>
#include <sys/wait.h>

int main()
{
    int pipe_fd[2];
    int status = 20;
    char buffer[4096]; //会初始化为0吗?Y
    pid_t pid1, pid2, pid3;
    sem_t *pipe_mutex;
    pipe_mutex = sem_open("pipe_mutex", O_CREAT | O_RDWR, 0666, 0);

    printf("init\n");

    if ((pid1 = fork()) == 0)
    {

        printf("child1 init.\n");
        // fflush(stdout);
        while (status >= 0)
        {
            sem_wait(pipe_mutex);
            fprintf(stdout, "child1 start writing\n");
            // fflush(stdout);
            // status = write(pipe_fd[1], buffer, 128);
            fprintf(stdout, "child1 write %d\n", status);
            // fflush(stdout);
            // sleep(1);
            sem_post(pipe_mutex);
            status--;
        }
        exit(0);
    }
    else if ((pid2 = fork()) == 0)
    {

        printf("child2 init.\n");
        // fflush(stdout);
        while (status >= 0)
        {
            sem_wait(pipe_mutex);
            fprintf(stdout, "child2 start writing\n");
            // fflush(stdout);
            // status = write(pipe_fd[1], buffer, 2047);
            fprintf(stdout, "child2 write %d\n", status);
            // fflush(stdout);
            // sleep(1);
            sem_post(pipe_mutex);
            status--;
        }

        exit(0);
    }
    else if ((pid3 = fork()) == 0)
    {

        printf("child3 init.\n");
        // fflush(stdout);
        while (status >= 0)
        {
            sem_wait(pipe_mutex);
            fprintf(stdout, "child3 start writing\n");
            // fflush(stdout);
            // status = write(pipe_fd[1], buffer, 4200);
            fprintf(stdout, "child3 write %d\n", status);
            // fflush(stdout);
            // sleep(1);
            sem_post(pipe_mutex);
            status--;
        }

        exit(0);
    }
    else
    {
        // getchar(); //这个语句在这里的作用是什么?type anything and enter to start
        sem_post(pipe_mutex);
        wait(0);
        wait(0);
        wait(0);
        // pipe_left_size(pipe_fd, pipe_mutex); //这两个函数的作用又是什么?
        printf("end\n");
        // sem_unlink("pipe_mutex");
    }

    return 0;
}

in a file:
init
child1 init.
child1 start writing
child1 write 20
child1 start writing
child1 write 19
child1 start writing
child1 write 18
child1 start writing
child1 write 17
child1 start writing
child1 write 16
child1 start writing
child1 write 15
child1 start writing
child1 write 14
child1 start writing
child1 write 13
child1 start writing
child1 write 12
child1 start writing
child1 write 11
child1 start writing
child1 write 10
child1 start writing
child1 write 9
child1 start writing
child1 write 8
child1 start writing
child1 write 7
child1 start writing
child1 write 6
child1 start writing
child1 write 5
child1 start writing
child1 write 4
child1 start writing
child1 write 3
child1 start writing
child1 write 2
child1 start writing
child1 write 1
child1 start writing
child1 write 0
init
child2 init.
child2 start writing
child2 write 20
child2 start writing
child2 write 19
child2 start writing
child2 write 18
child2 start writing
child2 write 17
child2 start writing
child2 write 16
child2 start writing
child2 write 15
child2 start writing
child2 write 14
child2 start writing
child2 write 13
child2 start writing
child2 write 12
child2 start writing
child2 write 11
child2 start writing
child2 write 10
child2 start writing
child2 write 9
child2 start writing
child2 write 8
child2 start writing
child2 write 7
child2 start writing
child2 write 6
child2 start writing
child2 write 5
child2 start writing
child2 write 4
child2 start writing
child2 write 3
child2 start writing
child2 write 2
child2 start writing
child2 write 1
child2 start writing
child2 write 0
init
child3 init.
child3 start writing
child3 write 20
child3 start writing
child3 write 19
child3 start writing
child3 write 18
child3 start writing
child3 write 17
child3 start writing
child3 write 16
child3 start writing
child3 write 15
child3 start writing
child3 write 14
child3 start writing
child3 write 13
child3 start writing
child3 write 12
child3 start writing
child3 write 11
child3 start writing
child3 write 10
child3 start writing
child3 write 9
child3 start writing
child3 write 8
child3 start writing
child3 write 7
child3 start writing
child3 write 6
child3 start writing
child3 write 5
child3 start writing
child3 write 4
child3 start writing
child3 write 3
child3 start writing
child3 write 2
child3 start writing
child3 write 1
child3 start writing
child3 write 0
init
end


in terminal:
init
child2 init.
child2 start writing
child2 write 20
child2 start writing
child2 write 19
child2 start writing
child2 write 18
child2 start writing
child2 write 17
child2 start writing
child2 write 16
child2 start writing
child2 write 15
child2 start writing
child2 write 14
child2 start writing
child2 write 13
child2 start writing
child2 write 12
child2 start writing
child2 write 11
child2 start writing
child2 write 10
child2 start writing
child2 write 9
child2 start writing
child2 write 8
child2 start writing
child2 write 7
child2 start writing
child2 write 6
child2 start writing
child2 write 5
child2 start writing
child2 write 4
child2 start writing
child2 write 3
child2 start writing
child2 write 2
child2 start writing
child2 write 1
child2 start writing
child2 write 0
child1 init.
child1 start writing
child1 write 20
child1 start writing
child1 write 19
child1 start writing
child1 write 18
child1 start writing
child1 write 17
child1 start writing
child1 write 16
child1 start writing
child1 write 15
child1 start writing
child1 write 14
child1 start writing
child1 write 13
child3 init.
child1 start writing
child3 start writing
child1 write 12
child1 start writing
child1 write 11
child1 start writing
child1 write 10
child3 write 20
child1 start writing
child3 start writing
child1 write 9
child3 write 19
child1 start writing
child3 start writing
child1 write 8
child3 write 18
child1 start writing
child3 start writing
child1 write 7
child3 write 17
child1 start writing
child3 start writing
child3 write 16
child1 write 6
child3 start writing
child1 start writing
child3 write 15
child1 write 5
child3 start writing
child1 start writing
child3 write 14
child1 write 4
child3 start writing
child1 start writing
child3 write 13
child1 write 3
child3 start writing
child1 start writing
child3 write 12
child1 write 2
child3 start writing
child1 start writing
child3 write 11
child1 write 1
child3 start writing
child1 start writing
child3 write 10
child1 write 0
child3 start writing
child3 write 9
child3 start writing
child3 write 8
child3 start writing
child3 write 7
child3 start writing
child3 write 6
child3 start writing
child3 write 5
child3 start writing
child3 write 4
child3 start writing
child3 write 3
child3 start writing
child3 write 2
child3 start writing
child3 write 1
child3 start writing
child3 write 0
end
 
Old 11-15-2021, 03:48 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 17,202

Rep: Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822
this is not multithreading, but multitasking. -pthread is useless here (actually it is required because of the semaphore, but it is still not a multithreaded app).
(Multithreading) multitasking means the three children will run parallel, therefore the output can be mixed. And occasionally they are not mixed, it just depends on the kernel and the execution.
 
Old 11-15-2021, 04:47 AM   #3
Sxing
LQ Newbie
 
Registered: Nov 2021
Location: Hangzhou, China
Distribution: ArchLinux, Ubuntu
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
this is not multithreading, but multitasking. -pthread is useless here (actually it is required because of the semaphore, but it is still not a multithreaded app).
(Multithreading) multitasking means the three children will run parallel, therefore the output can be mixed. And occasionally they are not mixed, it just depends on the kernel and the execution.
Thx for the reply. I noticed the lines are mixed in a different way between executions but are in different dimensions: In terminal every line is mixed. In the file each task’s output is mixed but lines from different tasks are not.
Also, why is the line of printf in the shared code have four outputs in a file but one in a terminal?
 
Old 11-15-2021, 04:59 AM   #4
Sxing
LQ Newbie
 
Registered: Nov 2021
Location: Hangzhou, China
Distribution: ArchLinux, Ubuntu
Posts: 13

Original Poster
Rep: Reputation: Disabled
why doesn't my reply appear?
 
Old 11-15-2021, 05:01 AM   #5
Sxing
LQ Newbie
 
Registered: Nov 2021
Location: Hangzhou, China
Distribution: ArchLinux, Ubuntu
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Sxing View Post
why doesn't my reply appear?
test
 
Old 11-15-2021, 05:19 AM   #6
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 17,202

Rep: Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822
I guess it depends on the load of the system and other resources (like ram, cache). In general the order of those output lines is unpredictable. It also may depend on the internal buffers used by the app (for example printf) and the timings (for example if fork needs more time than the full loop).
 
Old 11-15-2021, 05:30 AM   #7
Sxing
LQ Newbie
 
Registered: Nov 2021
Location: Hangzhou, China
Distribution: ArchLinux, Ubuntu
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
I guess it depends on the load of the system and other resources (like ram, cache). In general the order of those output lines is unpredictable. It also may depend on the internal buffers used by the app (for example printf) and the timings (for example if fork needs more time than the full loop).
That's alright. Thanks for your help anyway! *😄
 
Old 11-15-2021, 06:10 AM   #8
onebuck
Moderator
 
Registered: Jan 2005
Location: Summer Midwest USA, Central Illinois, Winter Central Florida
Distribution: Slackware®
Posts: 13,763
Blog Entries: 37

Rep: Reputation: 2946Reputation: 2946Reputation: 2946Reputation: 2946Reputation: 2946Reputation: 2946Reputation: 2946Reputation: 2946Reputation: 2946Reputation: 2946Reputation: 2946
Moderator Response

Quote:
Originally Posted by Sxing View Post
why doesn't my reply appear?
Your duplicate posts went to moderation control.
 
Old 11-15-2021, 10:08 AM   #9
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,579

Rep: Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097
Quote:
Originally Posted by Sxing View Post
It's a multi-threading program synchronized with semaphore. According to my knowledge, those three thread should print one by one, at least in this way.
And if I redirect it to a file, it seems the lines are organized by thread and the print line in father thread is repeated 3 times ???
This is almost certainly due to stdio buffering. Output to a terminal is by default line-buffered, i.e., the buffer is flushed after each newline character. Output to a file defaults to block-buffered, where the buffer is flushed whenever a full block (typically 4K) is collected.

You can use the setlinebuf() call to force an output stream to be line-buffered regardless of the device. See `man serlinebuf` for details.
 
Old 11-15-2021, 06:47 PM   #10
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 9,431
Blog Entries: 4

Rep: Reputation: 3375Reputation: 3375Reputation: 3375Reputation: 3375Reputation: 3375Reputation: 3375Reputation: 3375Reputation: 3375Reputation: 3375Reputation: 3375Reputation: 3375
You have "forked" several different processes, all of which are producing output that is "directed to the terminal." However, because they are "several different processes," each of them has its own STDIN, STDERR, STDOUT streams, to which each of them are writing independently. Yes, the various outputs are ultimately being gathered to the same physical destination – whether it be a terminal screen or a disk file – but it is unpredictable as to the order in which the various events might occur. (Likewise, the possibility of "buffering" which you also have no control over.)

I note that your source-code is peppered with wait() calls, as though you were trying to use these to achieve the "expected" output.

Your rule-of-thumb must be that the order-of-events of multitasking or even multithreaded operations is "UNPREDICTABLE."
 
1 members found this post helpful.
Old 11-15-2021, 10:14 PM   #11
Sxing
LQ Newbie
 
Registered: Nov 2021
Location: Hangzhou, China
Distribution: ArchLinux, Ubuntu
Posts: 13

Original Poster
Rep: Reputation: Disabled
Thanks for replying. I kind of understand that it's the buffering cause the differences.
When I use the setlinebuf() call, the output in the file behaves the same as the terminal.
There's still one thing I'm confused about.
I think the buffering should only affect the order of lines. But the “printf(”init\n”);” prints 1 line when line-buffered, however, 3 lines when block-buffered. Is it a feature of Linux (merging the duplicate lines printed by the shared part of several tasks when line-buffered)?
 
Old 11-15-2021, 11:45 PM   #12
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,579

Rep: Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097Reputation: 2097
Quote:
Originally Posted by Sxing View Post
But the “printf(”init\n”);” prints 1 line when line-buffered, however, 3 lines when block-buffered. Is it a feature of Linux (merging the duplicate lines printed by the shared part of several tasks when line-buffered)?
That should not be happening. Pipe the output through "hexdump -c", and/or run "hexdump -c" on the output file, to see exactly the characters being sent.
 
Old 11-16-2021, 05:17 AM   #13
Sxing
LQ Newbie
 
Registered: Nov 2021
Location: Hangzhou, China
Distribution: ArchLinux, Ubuntu
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by rknichols View Post
That should not be happening. Pipe the output through "hexdump -c", and/or run "hexdump -c" on the output file, to see exactly the characters being sent.
This may be due to my ignorance, but I don’t understand how does that help?
As you can see, there are 3 "init\n"(not child* init) in a (block-buffered)file but 1 "init\n" in a terminal.
In other words, the total numbers of rows are different.

Also, is my semaphore code completely useless? I haven't noticed any of semaphore's functions behaving at runtime.

Last edited by Sxing; 11-16-2021 at 05:21 AM.
 
Old 11-16-2021, 06:16 AM   #14
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 17,202

Rep: Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822Reputation: 5822
the 3 init lines are tricky, the forked processes may inherit the content of the buffer too (because that was not flushed).
From the other hand the semaphores are completely useless. But in some cases (when you have something else in between sem_wait and sem_post) it can be useful.
 
Old 11-16-2021, 06:41 AM   #15
Sxing
LQ Newbie
 
Registered: Nov 2021
Location: Hangzhou, China
Distribution: ArchLinux, Ubuntu
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
the 3 init lines are tricky, the forked processes may inherit the content of the buffer too (because that was not flushed).
From the other hand the semaphores are completely useless. But in some cases (when you have something else in between sem_wait and sem_post) it can be useful.
I thought that placing the “printf” between the sem_wait() and sem_post() would make 3 children to be printing in order.
For example, when child1 runs into printf, child 2 and child 3 will be waiting for releasing of semaphore, which will cause child 1 to not print sequentially. If I was right, the output should be like:
child1 write
child2 write
child3 write
or
child1 write
child3 write
child2 write
or something similar
but not
child 1 write
child 1 write
child 1 write
but they are obviously not
So, where is the error in my description?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Can I set the cups output order to always print in reverse order Thane Ubuntu 3 07-09-2018 10:49 PM
Logi Sales Manager on Ncurses (invoice, invoicing, orders, order, sale order, sales order...)? Xeratul Linux - Software 0 03-25-2017 03:45 PM
Why am I not able to redirect using htaccess. It says too many redirect rules. swathi.akkineni Linux - Newbie 1 07-31-2015 04:20 PM
How to make output redirect to a file and at the same time display on consloe? chinabenjamin66 General 1 05-01-2013 11:33 PM
Why the output files (trace file e.g. out.tr) are not same for the same tcl script ? askerwhat Ubuntu 1 06-03-2011 11:46 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:03 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration