LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 07-30-2011, 08:24 AM   #1
Ronayn
Member
 
Registered: Jan 2006
Posts: 55

Rep: Reputation: 0
Question Socket read into a bit bucket?


How do I get a TCP socket to read into a bit bucket (like /dev/null)?

Background/My Constraints:
In one program, I have a socket that uses 3 pre-allocated buffers to read into. I use another program to process the contents of a full buffer -- but that may take awhile, so its possible for the first program to fill up all 3 buffers. When that happens, I still need to perform a read (as I am being fed a continuous flow of data), but I just want to read into a bit bucket since all my buffers are full.

Is this possible? Or do I have to use some kind of stack buffer and keep reading into that until I can get a free buffer. It just seems wierd that there doesnt seem to be a way to tell a socket to perform a read, but in cases where there is no buffer (lets say the pointer is NULL) just read into /dev/null.
 
Old 07-30-2011, 08:48 AM   #2
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
So, allocate a 4th buffer into which your socket can read, and simply don't use the data that goes there. I think you're over-thinking the concept a bit.

--- rod.
 
Old 07-30-2011, 09:06 AM   #3
Ronayn
Member
 
Registered: Jan 2006
Posts: 55

Original Poster
Rep: Reputation: 0
Smile

Quote:
Originally Posted by theNbomr View Post
So, allocate a 4th buffer into which your socket can read, and simply don't use the data that goes there. I think you're over-thinking the concept a bit.

--- rod.
LOL. I *knew* I was going to get a response like this. Thanks, but that wont work. My buffers are gigabytes in size and they are created from shared memory (both constraints I have to work under), so the number I can create is limited (as my system has only 4gig of memory).

Again, I am suprised that there (apparently) is no Linux feature to read from a socket, into a bit bucket -- this just seems like something so common, you'd expect it to be a feature of the sockets read call. Anyhow, as I said in my original post, I know how to accomplish what I want with a smaller stack buffer -- it just grates me because its an inelegant solution to a very simple problem.

C'mon socket coders, I *know* you must have some secret to doing this besides a "4th buffer"!
 
Old 07-30-2011, 11:15 AM   #4
dwhitney67
Senior Member
 
Registered: Jun 2006
Location: Maryland
Distribution: Kubuntu, Fedora, RHEL
Posts: 1,541

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
Why not just disconnect (ie. close) the socket, and then re-connect at a later time, if needed.

Btw, what in pray tell are you using several gigabyte buffers for? Had you provided more background about your project and why you necessitate such large buffers things might make better sense, but for now, it seems to me that you have "issues" with your app's design.
 
Old 07-30-2011, 12:00 PM   #5
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
And, why does the bit-bucket buffer have to be so spacious and special? A few KBytes from the heap can be re-used ad nauseum. After all, you're throwing away all of the data.

--- rod.
 
Old 08-01-2011, 06:40 AM   #6
Ronayn
Member
 
Registered: Jan 2006
Posts: 55

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by dwhitney67 View Post
Why not just disconnect (ie. close) the socket, and then re-connect at a later time, if needed.

Btw, what in pray tell are you using several gigabyte buffers for? Had you provided more background about your project and why you necessitate such large buffers things might make better sense, but for now, it seems to me that you have "issues" with your app's design.
I cant just disconnect and reconnect because the data source is a (almost) constant stream. There is some downtime between the discrete messages, but we are talking microseconds at best. Constantly closing and opening a socket would be inefficient.

If I could explain in detail why I am using those buffers I would, but I cant, sorry. My constraints are as I listed, and that's the best background I can give.
 
Old 08-01-2011, 06:50 AM   #7
Ronayn
Member
 
Registered: Jan 2006
Posts: 55

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by theNbomr View Post
And, why does the bit-bucket buffer have to be so spacious and special? A few KBytes from the heap can be re-used ad nauseum. After all, you're throwing away all of the data.

--- rod.
Hey Rod,

Sorry, but you arent getting it. A bit bucket is neither spacious nor special -- its a concept that is represented (in one form) as /dev/null in linux. Unfortunately, I dont know how to get a char pointer (not a file pointer!) to /dev/null. So, I dont think that it can be used for what I am looking for.

Yes, I could use a local buffer and read a few KBytes, but that is grossly inefficient for the massive amount of data I am reading in. All those calls cost time/cpu resources! I should be able to make a socket function call to "read" in X bytes of data off of the sockets internal buffer into some internal bit bucket. I quoted "read" because it could be just an increment on the internal socket buffers pointer -- but I've no clue how to do that.

I hope I have made my question clearer.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
token bucket algorithm vs Leaky bucket algorithm xeon123 Linux - Networking 2 03-26-2007 04:57 AM
cannot read data at server socket, though client socket sends it jacques83 Linux - Networking 0 11-15-2005 01:58 PM
can't start ALSA (output goes to the bit bucket) yggdrasilos Linux - Software 2 01-31-2004 09:32 AM
C, read 8 bit octal numbers, convert to 24 bit binary bamalabs Programming 3 01-20-2004 09:59 AM
Overflow on /dev/null, please empty the bit bucket. little_ball Slackware 17 01-15-2004 07:34 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:36 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration