fork and shared memory

multiplex22 · 03-06-2015, 02:18 PM

Quote:

Originally Posted by smallpond

No, they don't. Because part of an address is the PID, which specifies what address mapping to use. The PID is different between the two tines of the fork.

My example shows they do. Can you prove your hypo or give something worthwhile?

Look at it again before you say no.

multiplex22 · 03-06-2015, 02:22 PM

Quote:

Originally Posted by ntubski

Never heard the term "chain array", is it like a linked list?

You can't "share memory between forked processes" in a portable way. Windows doesn't even have the concept of "forked processes".

There is a pthreads implementation for Windows. https://www.sourceware.org/pthreads-win32/

Threads are not forks, I wonder which docs you're reading...

Virtual memory.

well there is certainly a lot of name mangling in them...

pthread_atfork

man fork
man clone
/usr/include/fork.h

All use, thread and fork in the same content.

im on droid so I attached a doc.

multiplex22 · 03-06-2015, 02:28 PM

Quote:

Originally Posted by johnsfine

If multiple threads (rather than multiple processes) is a better fit for your requirements, then use multiple threads.

If there is some portability issue with native threading (though that is hard to understand compared to the greater portability issues of using multiple processes) use some more portable version of multi-threading, such as OMP or Intel's (free and largely portable) TBB.

Sounds logical. Im not trying to cause any trouble. Just looking for some answers. Gotta be sure this is done right.

multiplex22 · 03-06-2015, 02:30 PM

Quote:

Originally Posted by mdooligan

In your sample code all the shm* calls are commented out. Wasn't this a question of shared memory?

Child inherits copies what the parent had at fork(). Even if the addresses *look* the same, they are not. The resemblance is confusing until you understand this. Your eyebrows might look just like your dad's, but they are not your dad's eyebrows.

I'm no guru, but shared memory lives outside any process. You can close the starting process, go make tea, go to work, go on vacation, get home, open a different program 2 weeks later, and link to the shared mem chunk unless someone explicitly closed it or the PC rebooted.

Nice, but callocs are useless.

multiplex22 · 03-06-2015, 02:31 PM

Quote:

Originally Posted by johnsfine

At some point please take the time to read the answers you already received. Otherwise, what was the point of asking the question.

Already have. What was your question? Dont mind if I take a crack at it.

ntubski · 03-06-2015, 03:26 PM

Quote:

Originally Posted by multiplex22

pthread_atfork

man fork
man clone
/usr/include/fork.h

All use, thread and fork in the same content.

pthread docs mention fork() mainly because they have to specify what happens when you fork a process that contains multiple threads.

The rest of the confusion is simply implementation details of Linux (that is, both fork() and thread creation are implemented using the same underlying system call (clone(), but with different arguments)).

Quote:

im on droid so I attached a doc.

From that doc

Quote:

Originally Posted by multiplex22

What we know:

The fork makes an exact copy of the program being executed at the call of fork. In data changes in the parent after the call to fork, this data remains in the parent process. The child will "not share any" changes in data that is exit flags, data updates (i.e. long lists), etc.

To remedy the data sharing we can either use shared memory or pipes to write data to the child.

The Curios:

What confuses me is I find constantly suggestions of pthread referring to itself as a fork. The syntax gets mangled among documentations and headers. pthread suggests it uses itself as a fork. If so, how is it suppose to share its data?

clone is the solution I am looking for. It suggests it is nothing more than a fork, or exactly uses fork, but shares the stack space with the parent.

You're a bit confused here, on Linux both fork() and pthreads are implemented on top of clone(). On other Unixes (e.g. OpenBSD) there is no clone() so they are implemented differently. On Windows, there is no fork(), there is no clone(), and pthreads must (obviously) be implemented differently.

Quote:

The Problem:

I want to use fork, not clone or pthread. I think this is some misundersantding. It is stated in documentation that clone is Linux specifix. pthread is just the same issues as being Linux specific. If I compile the same code in Windows in portability, I will have to rebuild the entire code.

My goal is something like A) gathering a large amount of Data, B) printing that data. This data will be manipulated, updated, allocated, appended constantly. Maybe one thread (fork) is fine for now.

How to share the data?

If you want to be portable to Windows, you can't use fork(). Period.

As I mentioned earlier, pthreads is not Linux specific (p stands for POSIX) and there is a Windows implementation (or you can look at johnsfine's suggestion of some other portable thread implementations).

Quote:

Originally Posted by multiplex22

My example shows they[string addresses] do [point to the same place].

Your example simply demonstrates the behaviour described and explained in johnsfine's post #10. See also Virtual Memory, again.

Also, still wondering what a "chain array" is.

johnsfine · 03-07-2015, 06:08 AM

Quote:

Originally Posted by multiplex22

What was your question?

I was talking about the question you asked and then you apparently asked again:

If I understood you correctly, your question was how/why are addresses the same but contents different in some variables in parent vs. child after a fork.

I thought you asked that, then I and others answered it, then you seemed to be asking the same question a different way (showing the symptom of the contents being different and apparently looking for an explanation).

multiplex22 · 03-10-2015, 02:38 PM

I imagine that makes since. Thanks for sharing. Ill have to deal with this fork issue. Im still wondering how im gonna share my calloc data from 4 megs of libraries to a fork process or something. Jeesh, I guess Ill have to dig deeper or stick with clone.

A chain array (list) is a dynaimic, allocated memory segment linked to other memory segments using pointers:

struct element {
Data
void *next;
};

no limitations other than menory itself regarding anything with size.

Quote:

Originally Posted by ntubski

pthread docs mention fork() mainly because they have to specify what happens when you fork a process that contains multiple threads.

The rest of the confusion is simply implementation details of Linux (that is, both fork() and thread creation are implemented using the same underlying system call (clone(), but with different arguments)).

From that doc

You're a bit confused here, on Linux both fork() and pthreads are implemented on top of clone(). On other Unixes (e.g. OpenBSD) there is no clone() so they are implemented differently. On Windows, there is no fork(), there is no clone(), and pthreads must (obviously) be implemented differently.

If you want to be portable to Windows, you can't use fork(). Period.

As I mentioned earlier, pthreads is not Linux specific (p stands for POSIX) and there is a Windows implementation (or you can look at johnsfine's suggestion of some other portable thread implementations).

Your example simply demonstrates the behaviour described and explained in johnsfine's post #10. See also Virtual Memory, again.

Also, still wondering what a "chain array" is.

jacoblee · 03-10-2015, 11:03 PM

I don't know what do you mean.

sundialsvcs · 03-11-2015, 08:03 AM

Here's a quick run-down of what "processes" (versus "threads") actually are:

A process is an instance of a running program. Each program that is running ... even if it's multiple instances of the same program ... is "a process."

A process "owns" two resources in particular: its set of now-open file handles, and its virtual memory space.

Virtual memory is what allows each program to refer to "memory" and to see only its, private, memory. Two programs can refer to, say, "address $123456," and store anything that they like there, and there will be no conflict between them because every one of them has its own private memory. (Unless each of them opens the same "shared-memory segment.")

Processes can consist of multiple "threads." Each "thread" created by a process, lives in the same "process" context, referencing the same virtual memory and the same file-handles (and, shared-memory segments).

When a Unix/Linux process calls fork(), a new process is created. This process inherits some things (but, not everything ...) from its ancestor. It inherits a copy of the virtual-memory space of the ancestor ... but, it is not the same space. The initial state of the new process is more-or-less a duplicate of the ancestor's state at the time that fork() was called, but the two processes are independent.

The Microsoft Windows operating system does not have an exact corollary to Unix/Linux fork(). The underlying design of these two operating systems is not the same. Nor is the threading-model exactly the same. Although implementations of the pthread model exist for this environment, it is not exactly the same. Nor is the implementation of shared-memory exactly the same.

rtmistler · 03-11-2015, 08:19 AM

Quote:

Originally Posted by sundialsvcs

Here's a quick run-down of what "processes" (versus "threads")

Good summary. I think things are getting off track here.

Quote:

Originally Posted by jacoblee

I don't know what do you mean.

Welcome to LQ, this was your first post. Any chance you had a question, or were you merely reading this thread and getting confused about the discussion points? Don't worry if you were by the way, I'm sort of saying here that I think the original question/point, cited next, has been lost.

Quote:

Originally Posted by multiplex22

Like in clone, I want to share memory between forked processes. ... Are there other ways?

Following this, people have discussed and even cited examples of shared memory use. I can further cite pipe examples but I'm not sure that is exactly what you're looking for. Hey if that works: Using PIPES for Interprocess Communications . Otherwise, the concept brought for before by me states that you can create a shared memory segment and use it across multiple processes, and an example link was provided. Just not understanding where there's a fork issue to deal with, instead it's just properly understanding scope and ownership or access of system and common resources.

SoftSprocket · 03-11-2015, 08:36 AM

Quote:

Originally Posted by multiplex22

I imagine that makes since. Thanks for sharing. Ill have to deal with this fork issue. Im still wondering how im gonna share my calloc data from 4 megs of libraries to a fork process or something. Jeesh, I guess Ill have to dig deeper or stick with clone.

A chain array (list) is a dynaimic, allocated memory segment linked to other memory segments using pointers:

struct element {
Data
void *next;
};

no limitations other than menory itself regarding anything with size.

The structure you are referring to is a linked list. It is worth using the common terminology so others know what you mean. An array is a contiguous area of memory where as a list is not. Because it is contiguous link pointers aren't required with arrays.

If you have a linked list of data that you wish to share between processes pthreads will work quite nicely. Whatever means you use you'll need to synchronize access.

Code:

void* synchronized_list_access_pop ();
void synchronized_list_access_push (void*);

struct thread_args {
    void* (*pop)();
    void (*push)(void*);
};

struct thread_args* ta = malloc (sizeof *ta);
ta->pop = synchronized_list_access_pop;
ta->push = synchronized_list_access_push;

pthread_create (&tid, start_function, NULL, ta);

genss · 03-11-2015, 09:39 AM

@multiplex22
man shm_overview
(or just use pthreads)

Quote:

Originally Posted by ntubski

Also, still wondering what a "chain array" is.

from the code, a linked list

ntubski · 03-11-2015, 10:34 AM

Quote:

Originally Posted by SoftSprocket

If you have a linked list of data that you wish to share between processes pthreads will work quite nicely. Whatever means you use you'll need to synchronize access.

By "share between processes" you mean "share between threads", yes?

SoftSprocket · 03-11-2015, 11:13 AM

Quote:

Originally Posted by ntubski

By "share between processes" you mean "share between threads", yes?

Yes.