no of times write sys call is called while saving a file

chaitanyajun12 · 04-14-2011, 12:26 PM

Hi,

If the file to be written on to disk is 5MB for example,
how many times the write system call is called.

When we call write system call from user space like ..

write(fd, buf, 5000000);

In this case how many times the write system call is called ?

hope I am clear.

Chaitanya.

dwhitney67 · 04-14-2011, 01:05 PM

My guess is 1.

Try strace-ing your application.

chaitanyajun12 · 04-14-2011, 02:23 PM

i mean while writing it on to the disk ..

is it possible for write to be executed only once for writing the entire 5MB file onto disk ?

waiting for a reply.

Thanks in advance ..

Chaitanya.

dwhitney67 · 04-14-2011, 03:07 PM

From the man-page for write():

Code:

...
       The  write() function shall attempt to write nbyte bytes from the buffer pointed to by buf
       to the file associated with the open file descriptor, fildes.
...
       If  write()  is interrupted by a signal before it writes any data, it shall return -1 with
       errno set to [EINTR].

       If write() is interrupted by a signal after it successfully writes  some  data,  it  shall
       return the number of bytes written.

       If the value of nbyte is greater than {SSIZE_MAX}, the result is implementation-defined.
...
        * If  the O_NONBLOCK flag is clear, a write request may cause the thread to block, but on
          normal completion it shall return nbyte.
...
RETURN VALUE
       Upon successful completion, write() and pwrite() shall return the number of bytes actu‐
       ally written to the file associated with fildes. This number shall never be  greater  than
       nbyte. Otherwise, -1 shall be returned and errno set to indicate the error.
...

Thus, as I stated earlier, it is possible that write() will be accomplished with one sys call; however your "mileage" may vary depending on other external circumstances. In summary, always check the return value from write(), and decide what you wish to do.

Code:

size_t bytesWritten = 0;

while (bytesWritten < bufSize)
{
   ssize_t rtn = write(fd, buf + bytesWritten, bufSize - bytesWritten);

   if (rtn > 0)
   {
      bytesWritten += rtn;
   }
   else if (rtn < 0)
   {
      if (errno != EINTR && errno != EAGAIN)
      {
         // grave error!
         perror("write failed for reasons beyond my control!");
         break;
      }
   }
}

Nominal Animal · 04-14-2011, 05:05 PM

dwhitney67's answer is spot on.

In fact, the interruptions (and thus "short" reads and writes) are quite rare, so much so that many programmers ignore them altogether -- which of course leads to strange data loss problems if those interruptions do happen.

The precise answer is that if you call

Code:

write(fd, buf, 5000000);

in your code, it will result in one system call -- that's why these are called the low-level I/O functions --, but it does not mean that all 5000000 bytes of buf have been written to fd; you need to check the return value to find out. The write() function will return the number of bytes actually written, or -1 if none (and then errno has the reason).
If you want to write all 5000000 bytes, you need to use a loop, similar to dwhitney67's example code.

However, if you call

Code:

elements_written = fwrite(buffer, element_size, element_count, stream);

then you will see a varying number of write() system calls. Some C libraries may do it in one call, others may use a separate call for each element. That is an implementation detail internal to the C library used. To find out how your current C library does that, just run a test program via strace like dwhitney67 said in his previous post. For example, run

Code:

strace ./program

and look at the output.

chaitanyajun12 · 04-15-2011, 01:07 AM

The number of bytes to be written completely depends on the buffer size obviously .. I think, I am right here.

If we declare the buffer size to be size 5MB, then the complete data would be written to the fd in one system call to write().
The buffer size we declare is completely in our hands while using low-level I/O functions of course.

But while using high-level I/O functions buffer size will be a problem. In this case, will depend on system's buffer size? Am I correct?

So, how can we determine the system's buffer size ?

Chaitanya.

Nominal Animal · 04-15-2011, 02:24 AM

Quote:

Originally Posted by chaitanyajun12

In this case, will depend on system's buffer size?

There is no system buffer. POSIX does state that a write of up to PIPE_BUF bytes to a pipe is atomic, and that PIPE_BUF is at least 512 bytes, but that's it. The type of the descriptor and its flags determine exactly what happens when you write().

If the descriptor is a normal blocking one, the kernel will put the calling thread to sleep until either all of the data has been consumed, or something important happens. Important happenings are basically any errors related to that operation, and delivery of signals (to handlers installed without the SA_RESTART flag).

If the descriptor has the O_NONBLOCK flag set, write() will return a short count if the kernel cannot send all data immediately to the receiver. For sockets, the amount of data that can be in flight at once depends on the connection type and settings. For pipes, the kernel will generally take as much of the write() as the receiver is ready to read, aligned upwards to a full page (and thus at least one page). For files, the amount the kernel can immediately take depends on the amount of RAM available.

The Linux kernel does handle data in page-sized chunks. You can find out the page size via

Code:

#include <unistd.h>

long page_size = sysconf(_SC_PAGESIZE);

Most I/O in Linux also goes thorough the page cache, since Linux always uses otherwise unused RAM to cache the results of I/O operations, but it is not a system buffer per se. If you want to clear the caches (for example, to see how much RAM is actually in use), you can always quite safely run

Code:

sudo sh -c 'sync ; echo 3 > /proc/sys/vm/drop_caches'

which clears the page cache and inodes and dentries cached from filesystem accesses.

Note that stdio.h is a C library "addition" on top of the low-level I/O. fwrite(), fprintf(), fputs(), fputc() and other functions in stdio.h are implemented by the C library, and usually end up calling write(). There is an inherent buffer mechanism in the FILE streams, and you can modify that via setvbuf(), but it is just something the C library provides for you, it has nothing to do with the kernel.

Nominal Animal · 04-15-2011, 02:32 AM

Quote:

Originally Posted by chaitanyajun12

But while using high-level I/O functions buffer size will be a problem.

You can turn of buffering for any FILE stream via

Code:

setvbuf(stream, NULL, _IONBF, 0);

I don't know of a portable way to determine the current buffer size for a FILE stream, but you can always explicitly set a buffer size and either full (_IOFBF) or line (_IOLBF) buffering using

Code:

setvbuf(stream, NULL, _IOFBF, size_in_bytes);

The C library code will do the allocation automatically on next read or write operation.