Hi all,

I am trying to figure something out that has been a mystery to me for some time now. Sorry if the post gets a little long, but being a geek, information is my thing I am not opposed to R-ing the F-ing M, but I don't even know where to start so some very specific links could answer my question, but please, no conceptual info. I would like low level, or colloquially stated "nitty gritty," information.

Okay, let's see if I can articulate my problem.

Here is my question:

When one program communicates with itself or another program, how does it do it?

While this seems trivial on the surface, ala "it just does," that answer is a bit short for me. I would like to know a bit more, and to do so I think I should lay out what I know about the subject so you can get a good idea where I am coming from.

Ok, here is what I know:

Certain programs, such as ftp, various flavors of sql, ssh, etc all listen to a tcp/ip port. This is fairly straight forward. I understand tcp/ip, sockets, and the like. If I need to get information from a tcp/ip aware program, I just send requests to the port it is listening on. This I understand.

But, lets say my computer does not have the tcp/ip stack installed (just for the sake of this question, it would never happen, well, maybe some secret computing...but I digress) and I want to get information from another program that has run or is running, and have my program manipulate or parse that information. How would I do it? Does a program write information to a predetermined memory address that everyone has previously agreed upon? Is there a standard location where output is stored (stdin and stdout is for human use I believe)? Does each programmer decide where he/she wants it to be held for his/her program? If so, how does one go about finding that location?

I am willing to invest a good measure of personal time solving this mystery, but I have to know where to start. I am looking for a VERY detailed and technical answer. I hope the good people of this forum can help. Please, if you can give me ANY information I would appreciate it. I can't afford many books right now, so web links would be best.

Thanks for your time reading this, and long answers are good answers.

Raspis

Here goes, I'm slightly new(ish) at this so things could be wrong, but heres the jist of it. First I'll talk quickly about pipes because I know those better and they ARE in fact a type of process communication:

So you know about the 3 major file descriptors, stdin, stdout, stderr. They are all in an open state for each process, and each process I believe recieves new descriptors although I'm rusty and they could in fact be inherited from the parent. Doesnt matter. What does matter is that when your program is called with output piped to it from another program, what will happen before your program is invoked is the stdin file descriptor is closed, and a new descriptor is open - feeding your program the input of another programs output. I wish I could word it better but to be honest I haven't done much programming with it, only read the concepts.

Moving on to IPC, we can have signals. When a process is running, it can recieve a signal from the OS or another program. I'm even rustier here, I've only done the simplest parent/child signalling in Perl so I'll leave a better explanation to someone else. But basically you can set listener functions to react to various signals. A well known signal in Linux is the ctrl-C signal - SIGKILL. This signal cant be reprogrammed by you - when the OS sends you this signal, your program will quit, like it or not. You can set up more graceful exits for when you recieve "nicer" kill signals, I believe signal 15 - SIGTERM is a signal you might recieve first (say...when linux is shutting down). You better act quick (break from long loops, close files and descriptors, serialize data, etc...) because if you dont, you'll probably recieve the KILL signal which is like executing your process.

I found this document on the web, but I swear I had a much better one bookmarked somewhere. I cant find it now. Oh well. Heres one: http://www.comptechdoc.org/os/linux/...pgsignals.html


EDIT: I'd look deeper into programming with processes and threads, this will help you learn many important concepts.

Lowpro2k3,

Thanks for the response, and I think I understand what you are saying. I now have a bit more to go on. If nothing else I can search terms you have mentioned: pipes, file descriptor, etc. I must say I never gave signals much thought (being all uninformed) other than the kill -9 type of command. You have given me a step further, and I thank you.

Anyone else have any other information or links?

Raspis

Sure, take a look at this:
http://www.ecst.csuchico.edu/~beej/guide/ipc/

Perfect_Circle,

Wow, thanks for that link! That helps a lot! As with most things preceeding an exclamation point, I still need just a bit more. While that site is excellent and written as interestingly as I think such things can be, it only answers part of my question.

To take a bit of it and paraphrase (the section about SIGTERM in the signals section) If one wants to stop a process, one sends a signal SIGTERM to stop the process. Where does that SIGTERM signal "go" Now, I might be asking questions that are platform dependent, but I would like to know. Say a program is compiled with a SIGTERM to be "triggered", what does the system ie., processor/memory do to make it happen? I am sure the data flow of this would be unreadable by humans without translation, but it has to reside somewhere in memory or processor. right?

Suppose I have process A. I tell the "system" to start and run process A. If I do so without specifics, process A will run where ever the "system" has room for it (by room I mean address space, processor interrupt, etc) Now say I tell the "system" to start process B, but only if process A has returned a bit of data. Process B will run in its own space, different from process A. I presuppose this to be true. Where exactly (such as memory space) does process A write that bit of information, and how does process B know how and where to get it? PID is one way, but it is still a higher level than the answer I am seeking.

Maybe a question I should ask is "How does one find ALL the information ie., what memory space, processor job, etc.) that a PID is running?

I might be asking questions that could best be answered "Go get a Computer Science degree and you will be taught this information." If I am, could anyone please point me in the right direction to find such information on the web? Although I would have LOVED to go to college for CompSci I did not have the resources available to do so. Besides, anyone with a true thirst for knowledge and the interweb can find out most anything, right?

Thanks again for all your patience

Raspis

Hello,

For inter-process communication you could use a named-pipe / fifo method. I believe it was meantioned above, but I am by no means a programmer so I was not 100% sure if the suggestions above talked about the same thing.

Do a man for "fifo" and google named pipes.

Michael

It's funny that you mention it, I'm getting a Computer science and Informatics degree next week and I cannot answer you that, because I'm not a Kernel developer. I know some stuff about processes but the only project I had and had to do with low-level OS programming was about the minix time scheduler.

There is a thing in Computer Science called abstract (I think). This may not be the right word, English isn't my native language.
The guy designing transistors may need to know the physical attributes of metals.
A kernel developer needs to know how to implement signals and system calls for the signals.
An ordinary programmer needs only to know how to use those system calls and a secretary only needs to know that you trigger the internet explorer by double-clicking on the icon.

What you are asking depends heavily on the implementation. There are basic principles in OS development but signals may not be implemented the same way in minix, linux, FreeBSD or Solaris. I may be able to provide you some e-books about the linux (it's for the 2.2) and minix kernel. Just mail me. You'll also find linux low-level stuff in The Linux Documentation Project I'm pretty interested in some low-level stuff myself but until now I didn't have the time to dig a bit further. Maybe we can dig into the linux kernel this summer together.

I'm sure Richard Stallman had this in mind when he founded GNU back in 1983:
Why Free Software Needs Free Documentation

In Greece there are only public Universities and they are for free for everyone. You just give exams and according to the results and a list of choices you provide, you go to the University. At least as an undergraduate student, you don't pay fees and if your parents income is low, they provide you a place to live and food for free also. It's a shame that in the US something that in Europe is considered obvious, (free education and free medical care for everyone) is considered to be a radical left opinion. I was reading one of Micheal Moore's books and he described public schools being in an awful condition. Here 95% of the children go to public schools.

If you really have thirst for knowledge nothing can stop you from learning.

Perfect_Circle,

Thanks for the post again.

I would agree with you that most do not need to know how an OS works, but if I am to truly understand, I feel the need to know. I would be happy to take part in any project that would further knowledge (and document) how things work in Linux. As for the e-books you mentioned, I accept your offer. You have mail, Sir.

You must have sent the mail to someone else. I did not get any mail at all.....

When you're sitting around waiting for events to trigger program actions, you've entered an event-driven design. I haven't actually given it the thought you're putting into it ( ), but think of higher-level programming - GUIs, Event Listeners (in Java), javascript (onClick="functionCall"), etc...

Usually when you do GUI programming your old design methodologies of "instruct user to do this...process...now instruct user to do this" are changed. Now your waiting for the user to do one of many things, like create a new firefox tab, right-click the desktop, etc...

What does this have to do with signals? A little bit... Writing this out helps me visualize too So you're programming your applications to perform actions when some event occurs. Right now I'm trying to clear up exactly what you're asking. It seems you're asking the more specific question "how are signals delivered properly to a process", rather than the more generic "in which ways can multiple processes communicate".

I'll quickly answer the generic one by saying "with pipes, fifo's and signals". Assuming you knew that I'll try to answer "how signals are delivered", I started to try and explain it, but I'm just regurgitating words and info from the various tutorials, <signal.h>, <unistd.h> and others.

I would say spend some time with C (only a little bit, brush up on those function pointers!), then start toying with

<unistd.h>
<sys/types.h>
<errno.h>
<signal.h>

and many others. Programming will answer so many of these questions that would take us a long time to write up. Feel free to wait for more advice, I dont mind, heck I'll read it too. But I just think that the theory presented combined with hands on coding will quickly answer all your current questions (and open a whole boatlod of new ones )

Sorry Raspis21, I meant to actually give you much better guidance than just telling you to write code, but it is getting WAY to late for me to be awake on message boards! I hope you understand. I'll follow this thread because it resparked my interest in working with threads which somehow died off earlier this summer.

Good luck

Oh, and thanks for the link perfect_circle!