Hi,

I am writing a small http proxy/tunnel and I am having trouble with knowing when there is data to be received on the socket.

I have tried to determine the amount of data to be received by looking after the "content-length" field in the http header but sometimes the server doesn't send one.

I am also having a lot of troubles with the chunked packages.

Anyone here who knows his way around HTTP and can help me shed some light on this? All I need to know is how much data to receive. I really don't want to rely on timeouts while using my sockets cause that really slows things down to make it impossible to actually use the proxy.

The proxy is really simple, it doesn't do anything with the packages, just sends them back and forth.

Thanks,

Hi

What language are you programming in?

Also, you may want to check out the NON-BLOCKING mode of your stream (open connection), that way, you can read from the stream, and if there is no data, the call returns immediately.

I am using python and no, it does not. The call waits for a timeout if there is no data.

Found with Google:

See http://www.amk.ca/python/howto/sockets/

Hope this helps.

No, it does not. The problem is not the blocking part of the socket. The problem is that HTTP 1.1 uses persistent connections by default (unlike HTTP 1.0) and therefore I cannot rely on the function to return when there is no more data on the socket to be read.