Hey guys! I installed Ubuntu 5.10 on my laptop the other day to use for an operating systems course that I am taking. I am a Linux newbie, but everything seemed to go ok. Anyways, I have this assignment to do that I've been spending many hours on and can't seem to get anywhere. The question is as follows:

POSIX defines a standard thread package in the context of the C programming language. Several manufacturers provide a POSIX thread package as a user library along with their C programming facilities. If you have a system available to you that supports threads, then design and implement a thread program so that one thread reads a file, while a second thread writes the data to another file.

My textbook does not talk much about POSIX except for saying that a POSIX file is "a named sequential collection of bytes." It also gives an example Linux program that opens two files, and then copies the contents of one file into the other one byte at a time using the POSIX interface. I assume that this is similar to the answer except that the solution must use two threads. Here is the code that the book gives:

#include <stdio.h>
#include <fcntl.h>

int main() {
int inFile, outFile;
char *inFileName = "in_test";
char *outFileName = "out_test";
int len;
char c;

inFile = open(inFileName, O_RDONLY);
outFile = open(outFileName, O_WRONLY);
// Loop through the input file
while((len = read(inFile, &c, 1)) > 0)
write(outFile, &c, 1);
// close files and quit
close(inFile);
close(outFile);
}

I have installed the Anjuta IDE and searched around the Internet for some POSIX/pthread examples. I'm not sure how to approach this. How can I get the two threads to work together? All the examples that I've found just use the "pthread_create" and then call some useless function that doesn't do much. Any assistance is greatly appreciated!

Google is your friend(TM). Just by searching for linux pthreads examples I was able to find this excellent tutorial on POSIX threads:
http://tinyurl.com/6a22l

I have come across that link before as well as many others. The part I don't understand is how to set it up. In the example I provided, it seems like one character is being read at a time from the file and then it is written to the other file by reference. All the Posix/Pthread examples that I have come across show a separate function that handles the threads. If I am to create two new threads, how can I "syncronize" them to do their tasks of reading and writing together? Pardon my ignorance, but we are only in the second chapter of this online operating systems course and there was no mention of how much C programming would be involved. We never even formally learned C! The first programming course was a basic primer in C++ and the second and third were all in Java. I've completed the other problems ok but this one is really confusing me and the structure of the course is not helping things.

Hi,

I think that you may be confusing pthreads (POSIX threads) with the POSIX standards. Your sample program is simply showing how a POSIX complient program performs file I/O. It is not showing you anything having to do with multi-threading. pthreads allows you to run multi-threaded applications, it is also an interface that has been "blessed" by the POSIX consortium which means that it is POSIX complient interface. POSIX compliency simply means that if you take a source program written to the POSIX standards and compile it on another machine that also supports the POSIX standards, then your program will compile and load without any porting modifications.

Now, are you trying to learn about the POSIX standards, or are you trying to learn about POSIX complient multi-threading?

I believe I am trying to learn about the latter. I know that I need to create a multithreaded program, but I'm not sure how to do it so that one process is reading the file while another is writing it. All of the example sites, such as http://www.llnl.gov/computing/tutorials/pthreads/, shows how to create a thread which calls a separate function. I don't know if I need two separate functions, a control structure, or what.

Following the examples I've found online, I have come up with the following code:

#include <stdio.h>
#include <pthread.h>
#include <fcntl.h>


void *parseFile(void *); // function prototype


int main(void)
{
int ret1;
int t = 0;

pthread_t firstThread;

ret1 = pthread_create (&firstThread, NULL, parseFile, (void *)t);

pthread_exit(NULL);

return 0;
}

void *parseFile(void *data) {
char *inFileName = "in_test";
char *outFileName = "out_test";
int inFile;
int outFile;
int len;
char c;

outFile = open(outFileName, O_WRONLY);
inFile = open(inFileName, O_RDONLY);
while((len = read(inFile, &c, 1)) > 0)
write(outFile, &c, 1);
close(inFile);
pthread_exit(NULL);
}

The program works correctly but I don't think that this is they way the intructions say to do it. I think I am only creating one thread here instead of two.

Your example still really does not do anything "multi-threaded". You are thinking things through serially, multi-threaded programming provides you with a mechanism for thinking your problem through in parallel.

A very classic and simple example:

Supposed you had to create the server side of a client/server application, and the server needed to support more than one client at a time. How would you design such a server? Using a serial mode of thinking you would have the server service one client's complete transaction before moving on to service the next client, all from within a single monolithic process. Using a parallel mode of thinking you would spawn off an autonomous thread that would completely serve all of the transactions for a single client, and allow the main "overseer" process go back and wait for additional clients to send requests. Each new client gets it's own "service thread" that is dedicated to serving that one client. Each service thread gets it's own CPU slice based upon what algorithms have been programmed into the overall OS's scheduler. That usually means either the currently running thread gives up the CPU based upon a time slice maximum, or based upon whenever it needs to wait for some sort of I/O to occur, or a combination of both.

So, tell me how you would architect your reader/writer application to take advantage of parallel processing? At this point let's not even worry about the actual syntax of the code needed - let's just focus on the "higher level" design. What sorts of conditions could come up that could cause either your reader or your writer to process incorrect information? What techniques do you think need to be implemented from preventing those errors.

Once you have a workable and understandable design then you can begin to look at how to use the available toolset to accomplish that task.

Do I make sense? Is this helpful?

Thanks for your quick and detailed reply, rstewart. I think that I understand the concept of multithreading ok, but am having trouble visualizing how to approach the problem. The examples that I find online show silly things like two threads executing together that output text on the screen. I'm not sure how to have one thread interface with the other for a read/write operation. I assume that as the reader thread comes across a character, it sends it to the writer thread to output it to the file. This is the part that I am unclear about. Is there some kind of buffer that needs to be used? My knowledge of C is somewhat limited as I have never been formally taught it and it's been some time since I took C++. Pardon my ignorance and frustration, but I have been working on this problem for many hours over the past few days and it is due tonight at 11PM EST.

No problem. I will try and help you to understand what is involved.

First off, what exactly is the assignment? I presume that it is for some sort of class that you are taking, so I won't just provide you with an answer. I want to help you learn, so you will need to work with me for an answer. Once I know what it is that you are trying to create, then I can formulate questions and provide understandings into how it can be accomplished.

Okay?

I really appreciate your willingness to assist me. I do want to learn the material myself and don't expect anyone to provide me with the answer as that will not help me in the long run. As I mentioned previously, the problem is for an operating systems class that I am taking. The original problem is available in my first post. I need to create a multi-threaded program that does the same thing as what is shown in the code that I provided (read from a file and write the contents to another file.)

Okay cool, very simple problem.

First off you have two tasks that can be run in parallel. The first task is a task that reads data from the input file into a data buffer, and the second task takes that data and writes it to the output file.

Do you understand why I broke the problem down that way?

I think so. We essentially have a reader component and a writer component that are two distinct tasks running together. The buffer will be the common interface between the two.

Exactly!

Okay now for the next step.

We know we will need a global buffer in order to pass data between the two threads. Using pthreads this is easy. You simply declare/define an unsigned character buffer in global data space, ie outside of any function. The size of the data buffer can be up to you, but I suggest that you use a size of 1024 bytes.

You also know that files are usually not created in evenly divisible portions of your internal buffer size so you will probably at some point in time wind up with reading a partially full buffer (a buffer that does not contain 1024 bytes of data). So, you will also need some mechanism to communicate this information from your reader thread (which knows how much data was actually read) to your writer thread (which is waiting to be told how much data to be written), so you will also need some sort of variable that the reader thread writes to and your writer thread reads from. This variable will contain the amount of data that needs to be written by your writer thread.

Are you with me so far?

Yes. I understand the need for a global buffer but I never worked with one before. I realize the need for it to be global. Will the other variable be a character for the reader thread to write to and the writer thread to read from?

No, not a character it as 8 bits would be insufficient to hold any number larger than 255 while our buffer will be able to hold 1024 bytes. Just define an integer variable (32 bit, a bit of overkill but it is a value easily manipulated by the CPU).

Defining data in global memory is very easy, look at the following:

The buffer will be the actual area of memory that will be used to hold the character data being transfered from the input file to the output file. The other global variable will be used to inform the writer thread how much data needs to be written.