i would like to be able to open a file without actually creating it until the first write is performed. is there a way to do that without having a 2nd process doing that, yet get the file descriptor?

Can you explain a bit why you are trying to do this?

If you are looking at 'hiding' it from other processes until it has some data in it, there are various solns depending on the rules/circumstance that apply.

> is there a way to do that without having a 2nd process doing that

How could a second process help in your problem?

No, you can't have a file descriptor without a corresponding file (using the broad *nix definition of "file").

And note that you typically don't keep open-for-writing file handles open anyway. Typically, you'd build the file contents in memory, open the file for writing, write immediately, close immediately.

2 members found this post helpful.

a 2nd process could have a pipe connected to it and delay the open() until there is data to write. BTDT.

so there is never a time when the file exists but is empty.

Yes. Simply don’t open the file until you’re ready to write to it...

1 members found this post helpful.

i have never written any code in C that builds the whole file content in memory before writing it. but i have had to do that a few times in Python. my C programs always did a appropriate write when the data to be written was available. i often has cases where data was buffered and no more that one buffer was written (small file) so in those cases, sure, it did end up collection the whole file. but it could have had large data and written many buffers.

One common approach is to first create the file with a temporary name, then rename it once it is "ready". You often see this with file downloaders, where they'll add a .partial suffix until the file is complete.

Failing that, you can look at the O_TMPFILE flag of open(2).

1 members found this post helpful.

Well I'm sorry to hear that you haven't written C since the 80s, but...

Is your current target platform one where memory constraints would dictate an approach like this?

Would your current target platform perform better with a few large buffers or many small ones? Hint: does the target platform have a CPU cache?

Also keep in mind that on some platforms, sporadically writing many small files would create more disk fragmentation than writing a single large file in one operation.

Finally, if you're keeping the file open and locked throughout the lifetime of the application, well, that's not the way you're supposed to do it on *nix. Locking is supposed to be done only when necessary.

I have written code that creates a temporary file, writes to it, closes it, and renames the temporary file over an existing file. The key feature is that the update appears atomic. I am guessing that Skaperen really wants atomic update.
Ed

which means i will need to modify the library i will be calling to do the processing and output. i don't have source. it's not free.

Could you please explain X in this XY-problem?

1 members found this post helpful.

I though that was my line ...

1 members found this post helpful.

In the application you can have a framework that opens the file at the first writing. For example awk:
This writes all users with a zsh login shell to outfile.
If no user got a zsh then outfile is not created at all.