LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-18-2004, 12:52 AM   #1
shy
Member
 
Registered: Dec 2002
Location: Russia
Distribution: ASP linux
Posts: 94

Rep: Reputation: 15
Question What prevents the implementation of sendfile() from socket to file?


My manpage sais
Quote:
Presently the descriptor from which data is read cannot correspond to a socket, it must correspond to a file which supports mmap()-like operations.
Does the word presently mean "someday it will be changed" or not?
And if it's not, then what does prevent it?
 
Old 11-18-2004, 01:22 AM   #2
itsme86
Senior Member
 
Registered: Jan 2004
Location: Oregon, USA
Distribution: Slackware
Posts: 1,246

Rep: Reputation: 59
Because it depends on mmap() which doesn't work with sockets.
 
Old 11-18-2004, 01:51 AM   #3
shy
Member
 
Registered: Dec 2002
Location: Russia
Distribution: ASP linux
Posts: 94

Original Poster
Rep: Reputation: 15
Well, why isn't possible remove the dependence on mmap(), at least when working with sockets? Have someone tried this and found out that it's slower than, for example, use mmaped area as a buffer for read() on socket (although sendfile(file,socket) is faster than open(file), mmap(file), write(socket,mmaped_ptr))?

Last edited by shy; 11-18-2004 at 01:52 AM.
 
Old 11-18-2004, 01:02 PM   #4
shy
Member
 
Registered: Dec 2002
Location: Russia
Distribution: ASP linux
Posts: 94

Original Poster
Rep: Reputation: 15
So far, noone knows? What a pity...
 
Old 11-18-2004, 02:32 PM   #5
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris 11.4, Oracle Linux, Mint, Debian/WSL
Posts: 9,789

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
Maybe one reason is that sendfile name implies it is sending a file, and a socket descriptor is related to anything but a file ...
 
Old 11-19-2004, 12:27 AM   #6
shy
Member
 
Registered: Dec 2002
Location: Russia
Distribution: ASP linux
Posts: 94

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by jlliagre
Maybe one reason is that sendfile name implies it is sending a file, and a socket descriptor is related to anything but a file ...
I'm asking about technical difficulties preventing the implementation of sendfile (socket, file). I don't think naming is a problem - if it could be done with syscall named something like recvfile(), I would be happy.
 
Old 11-19-2004, 11:09 AM   #7
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris 11.4, Oracle Linux, Mint, Debian/WSL
Posts: 9,789

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
Nothing forbids you to improve sendfile and propose to the linux kernel maintainers this new feature ...
I think that it would be a bad idea anyway, as it would break compatibility with other O/Ses which implement it as a library function, and have conformed to the Linux semantics (AFAIK sendfile is unknow to POSIX or other Unix standardization groups).

Moreover, sendfile justification was to improve performance by using zero copy I/O operations, which is something efficient while mmap is used for the source, but meaningless when the source is remote and its data is arriving at an unpredictible rate from the network.

So this enhancement would have added too high level features in a system call, with few if any performance gain.
 
Old 11-19-2004, 02:35 PM   #8
shy
Member
 
Registered: Dec 2002
Location: Russia
Distribution: ASP linux
Posts: 94

Original Poster
Rep: Reputation: 15
I think my idea is not new. When kernel maintainers decided to make sendfile(file,socket), I think they probably (or even certainly) considered reverse variant, and found it unacceptable (as have been said, "with few if any performance gain").
I wondered if someone can point out this considerations. I'm interested in quantitative values ("how many percent will performance rise or fall with this method") but I think it can be found only if someone has already implemented it and measured.
Another answer on my questions would probably be the explanation why it's obvious that there won't be any performance gain. I'm not sure that it's
Quote:
meaningless when the source is remote and its data is arriving at an unpredictible rate from the network
because it is be zero-copy I/O too, and might be fast.
For example, when one uses {open(file), mmap(file), write(socket,mmaped_ptr)}, there are 1 syscall more and, if file is large, mmap() can fail because of insufficient memory (and if so, one should remap some portions of file, adding count of syscalls), but AFAIK sendfile() uses only one page at a time.
 
Old 11-19-2004, 03:04 PM   #9
aluser
Member
 
Registered: Mar 2004
Location: Massachusetts
Distribution: Debian
Posts: 557

Rep: Reputation: 43
This is obvious, but sendfile() from a socket tied to a foreign host into a local file would risk filling up your disk
 
Old 11-20-2004, 05:06 AM   #10
shy
Member
 
Registered: Dec 2002
Location: Russia
Distribution: ASP linux
Posts: 94

Original Poster
Rep: Reputation: 15
Actually, sendfile() has 4th parameter (count) - the number of bytes you would like to send, so I don't think there are problems with that.

Last edited by shy; 11-20-2004 at 05:08 AM.
 
Old 11-20-2004, 11:09 AM   #11
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris 11.4, Oracle Linux, Mint, Debian/WSL
Posts: 9,789

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
Correct, sendfile is hopefully buffer size bounded.
Just to clarify, do you want a function able to receive from a socket, and in that case limited to write to a local file file descriptor, or do you want the function to be orthogonal and work too when both ends are sockets ?
 
Old 11-20-2004, 12:00 PM   #12
shy
Member
 
Registered: Dec 2002
Location: Russia
Distribution: ASP linux
Posts: 94

Original Poster
Rep: Reputation: 15
Just socket->file is ok for me
 
Old 11-20-2004, 01:26 PM   #13
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris 11.4, Oracle Linux, Mint, Debian/WSL
Posts: 9,789

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
Ok, so that would be something that receivefile, as you suggested.

Perhaps a reason nobody implemented it, is that sendfile is mainly used by servers where each improvement in performance matters (a server may process a bunch of concurrent requests), while that receivefile would be for clients, which are usually more bounded by network throughput, and processing few simultaneaous requests anyway.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how can I create a socket file? enigma82 Linux - Software 8 06-28-2011 08:59 PM
How to operate socket with file interface wangjinyi Programming 4 11-27-2005 11:31 PM
Hash File Implementation jwstric2 Programming 3 04-06-2005 11:58 AM
what is a socket file?? augustus123 Linux - Distributions 3 12-06-2004 10:41 PM
Unix socket programming - how to sent a file Scrag Programming 9 03-28-2004 09:54 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 03:17 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration