open with delayed creation

Originally Posted by Geist Even if you don't have access to this library, you have access to your own code. So if the library wants to write a file immediately, then simply don't call the library functions until you want to. That should work, right? Keep your own copy of whatever you want to ultimately write in memory and only call the library function when you want to.

when my code does, eventually, call that library code, there will still be a period of time between when it calls open() and when it calls the first write().

i'm thinking to create an empty file distinct to the target file, though not the same exact name as the target file, and lock on that file to avoid concurrently doing this part in my code. that will at least be a start. locks will need to be added to other code, too.

Originally Posted by Skaperen when my code does, eventually, call that library code, there will still be a period of time between when it calls open() and when it calls the first write(). i'm thinking to create an empty file distinct to the target file, though not the same exact name as the target file, and lock on that file to avoid concurrently doing this part in my code. that will at least be a start. locks will need to be added to other code, too.

I don't see how you will be able to do anything then.
This makes it seem like you do one call to the library, which then doesn't return until it writes the file, which entails opening it first for some time while it does some operation that takes some time.

I can't think of a reliable, non crazy way to use a second thread to somehow balance the time between the library call starts (and creates the file) and finishes its prepwork to write to it.

There's so many variables.
Why does it even matter?

Security?
Well then encrypt the data first before you send it to the library and then decrypt it in a post processing step.

yeah, there's no hard definitive solution under the I/O model Posix/Unix/Linux uses. there's only designing around the problem to be sure no issues can happen in the first place. and that might mean having some way to lock against double uses of the file before some like first write or file completion.

there is a related problem. it's related because these programs will often take all day to complete the file due to the nature of what they are doing. consider opening a file for appending. now consider 2 processes opening the very same file for appending.

if the kernel (of any POSIX OS) were changed to work as i described, what would break? any applications? any libraries? is it something that would be harder to implement had the kernel done it this way since its first creation?

Zero-length files are valid and useful. They indicate the absence of something. The "test -s" command checks for non-zero-length files. I have used it many times.

I am still not sure why you want to avoid zero-length files.
Ed

zero length files have value where intended. i am trying to avoid them where they are not intended.

We posted the solution: create a temporary file, write to it, and then rename the temporary file to the desired name.
Ed

... and create it under /tmp or similar ie somewhere that gets cleaned out by the OS if it gets rebooted.

...or, as I said back in post #9, open() it with O_TMPFILE: though that is a Linux specific solution and is not portable.

Originally Posted by EdGr We posted the solution: create a temporary file, write to it, and then rename the temporary file to the desired name. Ed

the contrary case is something that runs for hours and hours collecting some statistical data that needs to be reviewed periodically all day. and this data needs to not be lost if the system reboots, eliminating the use of /tmp.

Originally Posted by GazL ...or, as I said back in post #9, open() it with O_TMPFILE: though that is a Linux specific solution and is not portable.

you sure did. and i overlooked it thinking it was something different. that does look like a doable approach. i'll have to make sure there is a lock to prevent duplicate attempts. but this is certainly a new, good, perspective on the problem.

Well, it's an option, but after reading your later posts (#20 specifically), IMO you're approaching the problem from the wrong end. Catch SIGTERM/SIGINT and give it a handler that will remove an empty file before exiting; then you can simply rely on open(filename, O_CREAT|O_EXCL) semantics to provide serialisation.

i'm trying to work out a way where an empty file doesn't matter. i can't reveal the code at that level.