What prevents the implementation of sendfile() from socket to file?
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Well, why isn't possible remove the dependence on mmap(), at least when working with sockets? Have someone tried this and found out that it's slower than, for example, use mmaped area as a buffer for read() on socket (although sendfile(file,socket) is faster than open(file), mmap(file), write(socket,mmaped_ptr))?
Originally posted by jlliagre Maybe one reason is that sendfile name implies it is sending a file, and a socket descriptor is related to anything but a file ...
I'm asking about technical difficulties preventing the implementation of sendfile (socket, file). I don't think naming is a problem - if it could be done with syscall named something like recvfile(), I would be happy.
Distribution: Solaris 11.4, Oracle Linux, Mint, Debian/WSL
Posts: 9,789
Rep:
Nothing forbids you to improve sendfile and propose to the linux kernel maintainers this new feature ...
I think that it would be a bad idea anyway, as it would break compatibility with other O/Ses which implement it as a library function, and have conformed to the Linux semantics (AFAIK sendfile is unknow to POSIX or other Unix standardization groups).
Moreover, sendfile justification was to improve performance by using zero copy I/O operations, which is something efficient while mmap is used for the source, but meaningless when the source is remote and its data is arriving at an unpredictible rate from the network.
So this enhancement would have added too high level features in a system call, with few if any performance gain.
I think my idea is not new. When kernel maintainers decided to make sendfile(file,socket), I think they probably (or even certainly) considered reverse variant, and found it unacceptable (as have been said, "with few if any performance gain").
I wondered if someone can point out this considerations. I'm interested in quantitative values ("how many percent will performance rise or fall with this method") but I think it can be found only if someone has already implemented it and measured.
Another answer on my questions would probably be the explanation why it's obvious that there won't be any performance gain. I'm not sure that it's
Quote:
meaningless when the source is remote and its data is arriving at an unpredictible rate from the network
because it is be zero-copy I/O too, and might be fast.
For example, when one uses {open(file), mmap(file), write(socket,mmaped_ptr)}, there are 1 syscall more and, if file is large, mmap() can fail because of insufficient memory (and if so, one should remap some portions of file, adding count of syscalls), but AFAIK sendfile() uses only one page at a time.
Distribution: Solaris 11.4, Oracle Linux, Mint, Debian/WSL
Posts: 9,789
Rep:
Correct, sendfile is hopefully buffer size bounded.
Just to clarify, do you want a function able to receive from a socket, and in that case limited to write to a local file file descriptor, or do you want the function to be orthogonal and work too when both ends are sockets ?
Distribution: Solaris 11.4, Oracle Linux, Mint, Debian/WSL
Posts: 9,789
Rep:
Ok, so that would be something that receivefile, as you suggested.
Perhaps a reason nobody implemented it, is that sendfile is mainly used by servers where each improvement in performance matters (a server may process a bunch of concurrent requests), while that receivefile would be for clients, which are usually more bounded by network throughput, and processing few simultaneaous requests anyway.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.