python 3.x - Strange delays of Twisted-based socket-server (python3) -
my project socket server interacting remote hardware. 2 things:
1) collection , processing of telematics packages sent remote hardware
2) transfer of commands received via external api
we using python3 along twisted + mongodb storage.
there 2 entities working twisted:
- deviceprotocol handles permanent connections remote hardware devices
- helperprotocol acts broker between api , hardwareprotocol (receives commands api via local tcp, , transfer deviceprotocol via fabric class)
code example
class deviceprotocol(protocol): def __init__(self, factory): self.factory = factory def connectionmade(self): """ change object status in db set objects collection in factory """ def connectionlost(self, reason): """ change object status in db del objects collection in factory """ def datareceived(self, data): recv_def = defer.deferred() recv_def.addcallback(self.pasre_data) recv_def.adderrback(self.log_error) recv_def.callback(data) def log_error(self, data): self.log.warning("something happens, %s", data) def parse_data(self, data): """ calculate response , call def send_data() manipulations data """ def send_data(self, data): self.transport.write(data) class helperprotocol(protocol): def __init__(self, factory): self.factory = factory def connectionmade(self): self.log.info("connection api") def datareceived(self, data): self.data = json.loads(data.decode('utf-8')) remote_device = self.data['remote_device'] data_pack = self.data['data_pack'] self.factory.objects[remote_device].send_data(data_pack) # send data remote device self.transport.write('ok'.encode('utf-8')) # send ok response remote api service def connectionlost(self, reason): self.log.info("api connection terminated") class protocolfactory(factory): objects = {} def __init__(self, type): self.clients = {} self.type = type def buildprotocol(self, addr): if self.type == "device": return deviceprotocol(self) if self.type == "api": return helperprotocol(self) endpoint = tcp4serverendpoint(reactor, port, interface) endpoint.listen(protocolfactory("device")) endpoint_api = tcp4serverendpoint(reactor, another_port, interface) endpoint_api.listen(protocolfactory("api")) reactor.run()
the actual problem first test 50 devices(deviceprotocol) , 10(helperprotocol) simultaneous connections helper got perfect result without delays. when added 20 more devices, system has gone nuts:
api receives data , passes helper on tcp connection
helper event connectionmade after 1 minute , datareceived event 1 more minute later (it couple of milliseconds 50 devices)
helper through factory sends command deviceprotocol
server’s 1.5 ghz core loaded 1 - 1.2%, there no consumption of ram either.
what’s going on, guys?
Comments
Post a Comment