LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   [python] reading non-blocking sockets with makefile (http://www.linuxquestions.org/questions/programming-9/%5Bpython%5D-reading-non-blocking-sockets-with-makefile-707542/)

Ephracis 02-25-2009 06:22 PM

[python] reading non-blocking sockets with makefile
 
I am trying to create a small proxy in python.

I started out trying to figure out how makefile works and now I think I finally got it. It is a really powerful way to work with sockets. However, I was a little bit disappointed that the connection was so slow. I did a little debugging, printing out every new line I got from the remote server and noticed that after the last line there was a long pause before the code continued. I guess that has something to do with my sockets not being non-blocking (correct me if I'm wrong here).

Of course I don't want slow code so I tried to make the sockets non-blocking which in turn gave me a lot of errors like "resource temporarily unavailable", "operation now in progress" and such. So I choose to use select to check whenever my socket was ready to write and read.

The problem here is that I don't really know how to use select with makefile. My guess was this:

1: create socket
2: connect to server
3: set socket to non-blocking
4: create file object using makefile
5: use select to check if the socket is ready
6: use readlines or writelines on the file object

However when readlines get executed I still get "Resource temporarily unavailable" (same happens if I use "for line in <fileobject>" to go through each line, it happens after the last line is received).

The code looks something like this:
Code:

def connect_to_server(self, host, port, datagram_out):
        self.remote_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.remote_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
               
        print "debug: connecting to server"
        self.remote_socket.connect((host, port))
        self.remote_socket.setblocking(0)
               
        print "debug: calling send-to-server"
        self.send_to_server(datagram_out)
               
        print "debug: calling recv-from-server"
        datagram_in = self.recv_from_server()
               
        return datagram_in
               
                       
def send_to_server(self, data):
               
        print "debug: looking if the server is ready to receive"
        read_sockets, write_sockets, error_sockets = select.select([], [self.remote_socket], [])
               
        # creating file object
        remote_fd = write_sockets[0].makefile('w', 0)
       
        print "debug: sending data to server"
        remote_fd.writelines(data)
               
        remote_fd.close()
               
def recv_from_server(self):
               
        print "debug: looking if the server is ready to send"
        read_sockets, write_sockets, error_sockets = select.select([self.remote_socket], [], [])
       
        # creating file object
        remote_fd = read_sockets[0].makefile('r', 0)
               
        # read server response
        print "debug: fetching server response"
        datagram_in = remote_fd.readlines()
               
        #datagram_in = list()
        #i = 0
        #for line in remote_fd:
        #        i += 1
        #        print "debug:                got line #" + str(i) + " of length " + str(len(line))
        #        datagram_in.append(line)
        print "debug: response from server fetched"
               
        # close
        self.remote_socket.close()
        remote_fd.close()

        return datagram_in

Here's the output I get:
Quote:

debug: connecting to server
debug: calling send-to-server
debug: looking if the server is ready to receive
debug: sending data to server
debug: calling recv-from-server
debug: looking if the server is ready to send
debug: fetching server response
Traceback (most recent call last):
File "./server.py", line 147, in <module>
main()
File "./server.py", line 143, in main
client.boot('', int(sys.argv[1]))
File "./server.py", line 56, in boot
response = self.connect_to_server(remote_host, int(remote_port), data)
File "./server.py", line 73, in connect_to_server
datagram_in = self.recv_from_server()
File "./server.py", line 103, in recv_from_server
datagram_in = remote_fd.readlines()
File "/usr/lib/python2.5/socket.py", line 440, in readlines
line = self.readline()
File "/usr/lib/python2.5/socket.py", line 372, in readline
data = recv(1)
socket.error: (11, 'Resource temporarily unavailable')
Here's the output when I use the for loop instead of readlines():
Quote:

debug: connecting to server
debug: calling send-to-server
debug: looking if the server is ready to receive
debug: sending data to server
debug: calling recv-from-server
debug: looking if the server is ready to send
debug: fetching server response
debug: got line #1 of length 17
debug: got line #2 of length 37
debug: got line #3 of length 106
debug: got line #4 of length 46
debug: got line #5 of length 33
debug: got line #6 of length 22
debug: got line #7 of length 20
debug: got line #8 of length 25
debug: got line #9 of length 2
debug: got line #10 of length 15
Traceback (most recent call last):
File "./server.py", line 147, in <module>
main()
File "./server.py", line 143, in main
client.boot('', int(sys.argv[1]))
File "./server.py", line 56, in boot
response = self.connect_to_server(remote_host, int(remote_port), data)
File "./server.py", line 73, in connect_to_server
datagram_in = self.recv_from_server()
File "./server.py", line 107, in recv_from_server
for line in remote_fd:
File "/usr/lib/python2.5/socket.py", line 455, in next
line = self.readline()
File "/usr/lib/python2.5/socket.py", line 372, in readline
data = recv(1)
socket.error: (11, 'Resource temporarily unavailable')
Note that it is after line #10 that the pause is when I don't use non-blocking sockets.


All times are GMT -5. The time now is 02:11 AM.