I have a very basic implementation of the http.server.HTTPServer i'm spinning up in Python 3.6. I'm using it to authenticate OAuth2 against the google Analytics API. As is standard, i've created a simple handle to receive the OAuth2 grant key and am using the serve_forever() method. However, the thread used to create the serve forever is hanging, and it seems to be stuck deep in the Python 3.6 implementation.
Interestingly, I don't think my code matters at all in this case. Look at the call stack that is being reported by listeners I setup for debugging:
service_action started
service_action complete
service_action started
service_action complete
service_action started
service_action complete
service_action started
service_action complete
"socketserver.BaseServer.serve_forever : self._handle_request_noblock()"
"socketserver.BaseServer._handle_request_noblock : self.get_request()"
"socketserver.BaseServer._handle_noblock_request : self.process_request()"
"socketserver.BaseServer.process_request : self.finish_request()"
"socketserver.BaseServer.finish_request : self.RequestHandlerClass(request, client_address, self) # request == <socket.socket fd=1828, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 8080), raddr=('127.0.0.1', 57504)>, client_address == ('127.0.0.1', 57504), self == <MDGCoreLib.Utilities.HttpServer.Server.Server object at 0x0000026304E3FC88>"
"socketserver.BaseRequestHandler.__init__ : set self.client_address"
"socketserver.BaseRequestHandler.__init__ : set self.server"
"socketserver.BaseRequestHandler.__init__ : setup()"
"socketserver.StreamRequestHandler.setup : self.connection = self.request"
"socketserver.BaseRequestHandler.__init__ : self.handle()"
"server.BaseHttpRequestHandler.handle : self.close_connection = True"
"server.BaseHTTPServer.handle_one_request : self.raw_requestline = self.rfile.readline(65537)"
"socket.readinto : self._checkClosed()"
"socket.readinto : return self._sock.recv_into(b) #b == <memory at 0x0000026304E38C48>"
"server.BaseHTTPHandler.handle_one_request : if len(self.raw_requestline) > 65536"
"server.BaseHTTPHandler.handle_one_request : if not self.raw_requestline"
"server.BaseHTTPHandler.handle_one_request : if not self.parse_request()"
"server.BaseHTTPHandler.handle_one_request : if not hassattr(self, mname)"
#My handler runs
*"OAuthGrantRequestHandler : self.send_response"
127.0.0.1 - - [27/Aug/2018 14:35:16] "GET /?(REDACTED) HTTP/1.1" 200 -
Request handler completed*
#My handler finishes
Parent thread : joining Thread 2
"server.BaseHTTPHandler.handle_one_request : flushing wfile to socket finishing request"
"socketserver.BaseRequestHandler.__init__ : self.finish()"
"socketserver.StreamRequestHandler.finish : "
"socketserver.BaseServer.process_request : self.shutdown_request"
service_action started
service_action complete
"socketserver.BaseServer.serve_forever : self._handle_request_noblock()"
"socketserver.BaseServer._handle_request_noblock : self.get_request()"
"socketserver.BaseServer._handle_noblock_request : self.process_request()"
"socketserver.BaseServer.process_request : self.finish_request()"
"socketserver.BaseServer.finish_request : self.RequestHandlerClass(request, client_address, self) # request == <socket.socket fd=1840, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 8080), raddr=('127.0.0.1', 57505)>, client_address == ('127.0.0.1', 57505), self == <MDGCoreLib.Utilities.HttpServer.Server.Server object at 0x0000026304E3FC88>"
"socketserver.BaseRequestHandler.__init__ : set self.client_address"
"socketserver.BaseRequestHandler.__init__ : set self.server"
"socketserver.BaseRequestHandler.__init__ : setup()"
"socketserver.StreamRequestHandler.setup : self.connection = self.request"
"socketserver.BaseRequestHandler.__init__ : self.handle()"
"server.BaseHttpRequestHandler.handle : self.close_connection = True"
"server.BaseHTTPServer.handle_one_request : self.raw_requestline = self.rfile.readline(65537)"
"socket.readinto : self._checkClosed()"
"socket.readinto : return self._sock.recv_into(b) #b == <memory at 0x0000026304E38C48>"
"server.BaseHTTPHandler.handle_one_request : if len(self.raw_requestline) > 65536"
"server.BaseHTTPHandler.handle_one_request : if not self.raw_requestline"
"server.BaseHTTPHandler.handle_one_request : if not self.parse_request()"
"server.BaseHTTPHandler.handle_one_request : if not hassattr(self, mname)"
#My Handler Runs
"OAuthGrantRequestHandler : self.send_response"
127.0.0.1 - - [27/Aug/2018 14:35:18] "GET /favicon.ico HTTP/1.1" 200 -
Request handler completed
#My Handler Finishes
"server.BaseHTTPHandler.handle_one_request : flushing wfile to socket finishing request"
"socketserver.BaseRequestHandler.__init__ : self.finish()"
"socketserver.StreamRequestHandler.finish : "
"socketserver.BaseServer.process_request : self.shutdown_request"
service_action started
service_action complete
"socketserver.BaseServer.serve_forever : self._handle_request_noblock()"
"socketserver.BaseServer._handle_request_noblock : self.get_request()"
"socketserver.BaseServer._handle_noblock_request : self.process_request()"
"socketserver.BaseServer.process_request : self.finish_request()"
"socketserver.BaseServer.finish_request : self.RequestHandlerClass(request, client_address, self) # request == <socket.socket fd=2008, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 8080), raddr=('127.0.0.1', 57506)>, client_address == ('127.0.0.1', 57506), self == <MDGCoreLib.Utilities.HttpServer.Server.Server object at 0x0000026304E3FC88>"
"socketserver.BaseRequestHandler.__init__ : set self.client_address"
"socketserver.BaseRequestHandler.__init__ : set self.server"
"socketserver.BaseRequestHandler.__init__ : setup()"
"socketserver.StreamRequestHandler.setup : self.connection = self.request"
"socketserver.BaseRequestHandler.__init__ : self.handle()"
"server.BaseHttpRequestHandler.handle : self.close_connection = True"
"server.BaseHTTPServer.handle_one_request : self.raw_requestline = self.rfile.readline(65537)"
"socket.readinto : self._checkClosed()"
"socket.readinto : return self._sock.recv_into(b) #b == <memory at 0x0000026304E38C48>"
The program '[54321] Python @ tcp://localhost:55098/?legacyUnitTest' has exited with code -1 (0xffffffff).
The above is everything from the thread running the "serve_forever()" method, and you can see the parent thread is waiting to join where output reads Parent thread : joining Thread 2
. My handler, get encountered and completed twice when the output is "OAuthGrantRequestHandler : self.send_response"
and
Request handler completed
: but then is never called again before the internal python library hangs at socket.readinto : return self._sock.recv_into(b)
while trying to process the last request.
Anyone had this Python hangup before, and is there anyway around it? I'm kind of at a loss as its hanging between server_forever() loops where my handler/code is encountered.
UPDATE 1
So comparing fiddler traces to the requests in the HTTPServer I see there are only two HTTPRequest, but the HTTPServer.serve_forever()
attempts to read a third request which is where it hangs. I've also verified (you can see it in the output above) that the "shutdown_request()" method and underlying methods are being called to shutdown the socket. So why does the serve_forever
loop continue to pass the check below and call self._handle_request_noblock() if it hasn't recieved a new request:
ready = selector.select(poll_interval)
if ready:
self._handle_request_noblock()
Had pretty much the same issue, so after digging through socketserver.StreamRequestHandler
implementation I found how to set timeout so that those erroneous read attempts would eventually unblock.
class MyRequestHandler(BaseHTTPRequestHandler):
timeout = 2 # This is what makes the difference (seconds)
def do_GET(self):
...
User contributions licensed under CC BY-SA 3.0