LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 07-05-2016, 06:49 PM   #1
david_8274
Member
 
Registered: Jun 2013
Location: California
Distribution: Ubuntu, Fedora
Posts: 75

Rep: Reputation: Disabled
Sending large data through socket programming


Hi all,

I am implementing a program on an embedded Linux system that uses socket to send/receive large data (as large as 10M) through Ethernet. In the user space code, I use the typical socket programming procedure to first establish the connection, then use the 'write' system call to send data out through the socket. My question is, if I want to send a big blob of data, say, 10M, should I first segment the data into smaller blocks then make the 'write' system call multiple times, or should I just make the 'write' system call one time by passing the entire 10M buffer to it? How is this type of big blob data send/receive usually handled in socket programming?

Thanks,
Wei
 
Old 07-06-2016, 09:39 AM   #2
mknirsch
Member
 
Registered: Feb 2003
Location: Germany
Distribution: SLES, SuSe, Ubuntu, HPUX
Posts: 38

Rep: Reputation: 18
Hi Wei,
you gain nothing but overhead when segmenting your data. The MTU size determine the number of bytes sent.
Provide a suitable rx buffer size in order to avoid further delays. Use tcp to rx all packages in the right order.

Regards

Martin
 
Old 07-06-2016, 05:21 PM   #3
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 8,626
Blog Entries: 4

Rep: Reputation: 2999Reputation: 2999Reputation: 2999Reputation: 2999Reputation: 2999Reputation: 2999Reputation: 2999Reputation: 2999Reputation: 2999Reputation: 2999Reputation: 2999
Given that physical-RAM sizes today are routinely measured in giga-bytes, this reasoning, today, is probably quite sound: an application program probably can "request 10+ contiguous megabytes," and get it, and find that all of that storage is actually, physically, "in RAM, right now."

Now, if you knew that you might be communicating with a device that is more memory-constrained, or you felt the need to provide a "progress bar," and so-on, you might well choose to transfer the data in "chunks," and I am of the opinion that you would not be "wrong" in deciding to do that. It's an engineering decision that is "up to you, the Engineer."
 
Old 07-09-2016, 05:57 PM   #4
mknirsch
Member
 
Registered: Feb 2003
Location: Germany
Distribution: SLES, SuSe, Ubuntu, HPUX
Posts: 38

Rep: Reputation: 18
Hi,
No doubt it's the engineers decision. The NIC and it's driver however control the amount of data sent at a time - not depending on the amount of data you provide. In case your "receiving side" is busy swapping it even shrinks the tcp window. Using standard linux you have little to no influence on this unless you write own drivers on the tx and rx side.

Ragards Martin
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to transfer hexadecimal data from client to server(socket programming)..? vktafs Programming 1 11-05-2012 06:50 PM
Socket programming: sending and receiving msgs back and forth ninja123 Linux - Networking 4 05-20-2010 07:30 AM
problems receving data in socket programming stephenwalter Programming 5 09-30-2005 12:58 AM
Gettting ENOTCONN when sending data, but the socket was opened. trevelluk Programming 3 05-12-2005 11:43 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 03:41 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration