LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-25-2009, 06:22 PM   #1
Ephracis
Senior Member
 
Registered: Sep 2004
Location: Sweden
Distribution: Ubuntu, Debian
Posts: 1,109

Rep: Reputation: 49
[python] reading non-blocking sockets with makefile


I am trying to create a small proxy in python.

I started out trying to figure out how makefile works and now I think I finally got it. It is a really powerful way to work with sockets. However, I was a little bit disappointed that the connection was so slow. I did a little debugging, printing out every new line I got from the remote server and noticed that after the last line there was a long pause before the code continued. I guess that has something to do with my sockets not being non-blocking (correct me if I'm wrong here).

Of course I don't want slow code so I tried to make the sockets non-blocking which in turn gave me a lot of errors like "resource temporarily unavailable", "operation now in progress" and such. So I choose to use select to check whenever my socket was ready to write and read.

The problem here is that I don't really know how to use select with makefile. My guess was this:

1: create socket
2: connect to server
3: set socket to non-blocking
4: create file object using makefile
5: use select to check if the socket is ready
6: use readlines or writelines on the file object

However when readlines get executed I still get "Resource temporarily unavailable" (same happens if I use "for line in <fileobject>" to go through each line, it happens after the last line is received).

The code looks something like this:
Code:
def connect_to_server(self, host, port, datagram_out):
	self.remote_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
	self.remote_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
		
	print "debug: connecting to server"
	self.remote_socket.connect((host, port))
	self.remote_socket.setblocking(0)
		
	print "debug: calling send-to-server"
	self.send_to_server(datagram_out)
		
	print "debug: calling recv-from-server"
	datagram_in = self.recv_from_server()
		
	return datagram_in
		
			
def send_to_server(self, data):
		
	print "debug: looking if the server is ready to receive"
	read_sockets, write_sockets, error_sockets = select.select([], [self.remote_socket], [])
		
	# creating file object
	remote_fd = write_sockets[0].makefile('w', 0)
	
	print "debug: sending data to server"
	remote_fd.writelines(data)
		
	remote_fd.close()
		
def recv_from_server(self):
		
	print "debug: looking if the server is ready to send"
	read_sockets, write_sockets, error_sockets = select.select([self.remote_socket], [], [])
	
	# creating file object
	remote_fd = read_sockets[0].makefile('r', 0)
		
	# read server response
	print "debug: fetching server response"
	datagram_in = remote_fd.readlines()
		
	#datagram_in = list()
	#i = 0
	#for line in remote_fd:
	#	i += 1
	#	print "debug:		got line #" + str(i) + " of length " + str(len(line))
	#	datagram_in.append(line)
	print "debug: response from server fetched"
		
	# close
	self.remote_socket.close()
	remote_fd.close()

	return datagram_in
Here's the output I get:
Quote:
debug: connecting to server
debug: calling send-to-server
debug: looking if the server is ready to receive
debug: sending data to server
debug: calling recv-from-server
debug: looking if the server is ready to send
debug: fetching server response
Traceback (most recent call last):
File "./server.py", line 147, in <module>
main()
File "./server.py", line 143, in main
client.boot('', int(sys.argv[1]))
File "./server.py", line 56, in boot
response = self.connect_to_server(remote_host, int(remote_port), data)
File "./server.py", line 73, in connect_to_server
datagram_in = self.recv_from_server()
File "./server.py", line 103, in recv_from_server
datagram_in = remote_fd.readlines()
File "/usr/lib/python2.5/socket.py", line 440, in readlines
line = self.readline()
File "/usr/lib/python2.5/socket.py", line 372, in readline
data = recv(1)
socket.error: (11, 'Resource temporarily unavailable')
Here's the output when I use the for loop instead of readlines():
Quote:
debug: connecting to server
debug: calling send-to-server
debug: looking if the server is ready to receive
debug: sending data to server
debug: calling recv-from-server
debug: looking if the server is ready to send
debug: fetching server response
debug: got line #1 of length 17
debug: got line #2 of length 37
debug: got line #3 of length 106
debug: got line #4 of length 46
debug: got line #5 of length 33
debug: got line #6 of length 22
debug: got line #7 of length 20
debug: got line #8 of length 25
debug: got line #9 of length 2
debug: got line #10 of length 15
Traceback (most recent call last):
File "./server.py", line 147, in <module>
main()
File "./server.py", line 143, in main
client.boot('', int(sys.argv[1]))
File "./server.py", line 56, in boot
response = self.connect_to_server(remote_host, int(remote_port), data)
File "./server.py", line 73, in connect_to_server
datagram_in = self.recv_from_server()
File "./server.py", line 107, in recv_from_server
for line in remote_fd:
File "/usr/lib/python2.5/socket.py", line 455, in next
line = self.readline()
File "/usr/lib/python2.5/socket.py", line 372, in readline
data = recv(1)
socket.error: (11, 'Resource temporarily unavailable')
Note that it is after line #10 that the pause is when I don't use non-blocking sockets.
 
  


Reply

Tags
python, sockets


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Reading the ip address from a packet raw sockets linuxtest Programming 2 12-10-2008 02:43 PM
How to use poll() with non-blocking sockets tnjones Programming 3 10-12-2008 09:55 AM
Problem w/ sockets, fdopen, and blocking. 95se Programming 3 11-17-2005 12:38 PM
Non blocking sockets in Perl ivanatora Programming 3 08-19-2005 02:14 AM
Set/Unset Non-Blocking sockets? zer0python Programming 2 01-04-2004 01:31 AM


All times are GMT -5. The time now is 09:59 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration