return value of write(2)

Jerry Mcguire · 05-07-2010, 05:27 AM

Hi all,

I am using write(2) to append records to a file, and there are several similar processes that append to the same file.

Code:

int fd = open( "my.txt", O_RDWR|O_CREAT|O_APPEND|O_SYNC, (mode_t)0664 );
...
size_t iW = write( fd, &myrecord, sizeof(myrecord) );
if (iW == sizeof(myrecord) {
  // I find peace
  fsync( fd );
}
else if (iW == 0) {
  // don't panic. decide what to do.
}
else if (iW == -1) {
  // don't panic. check errno to see what to do.
}
else {
  // don't panic. may be I should.
}

I'm wondering if it is possible that write(2) writes an incomplete record to the file before another write(2) occurs.

The man page says write(2) will return:
the number of bytes written; or
0, indicating nothing written; or
-1, indicating error.

Should I worry about write(2) writing an incomplete record?

also, do I need fsync(2) despite I have the O_SYNC flag set when the fd is open?

Thanks for teaching.

Sergei Steshenko · 05-07-2010, 05:55 AM

Quote:

Originally Posted by Jerry Mcguire

...
Should I worry about write(2) writing an incomplete record?
...

What are you trying ultimately to do ? Think of, say, a database application not actually updating data on disk because of 'write' error.

wje_lq · 05-07-2010, 08:19 AM

If the return value from write() is not -1, then the following applies.

If the return value is equal to the number of bytes you wished to write, then there is obviously no problem presented here.

If it is fewer than the number of bytes you wished to write, then the reason has nothing to do with whether a physical write has actually occurred. It happens because you've run out of space to write data to that file, and the returned number shows how many bytes you successfully wrote. An example is zero, which means the space was full before you even started to write.

wje_lq · 05-07-2010, 02:30 PM

Let me expand on that: this refers only to synchronous writes. If you're writing asynchronously, then you might get fewer bytes written. In that case, you need to loop back until all of them have been written.

Jerry Mcguire · 05-09-2010, 09:23 PM

Hi Sergei,

I am try to do the followings,
1) I have an app that receives data from TCP socket and writes to a file for bufferring. Data in this file is then processed by several computational processes to extract information. (i.e. One-Write-Many-Read).

The file organization is records of 2-byte-length-descriptor + the actual binary data. There will be 350MB in this file per day. The first app uses writev(2). The computational processes uses read(2) starting off reading the 2-byte len, and then read again in a loop for the rest of the binary data.

2) These extracted information are then written to a centralized log file, and this log file will be read by another process so that appropriate information will be delivered to various clients over TCP sockets. (i.e. Many-Write-Many-Read).

The file organization is variable-length records separated by newline character, so that fputs() and fgets() can be easily used, and this log can be `cat` for examination.

I do use file for inter-process communication a lot. I worry about write or writev not finishing a complete record and then another write kicks in. Any advice please. Thanks.

----
Hi Wje,

I do only synchronous write to ensure records written to file are permanent. If the target runs out of space, the problem is more than just an incomplete record. I can deal with that later. Thanks.