python 3.x - Strange delays of Twisted-based socket-server (python3) -


my project socket server interacting remote hardware. 2 things:

1) collection , processing of telematics packages sent remote hardware

2) transfer of commands received via external api

we using python3 along twisted + mongodb storage.

there 2 entities working twisted:

  • deviceprotocol handles permanent connections remote hardware devices
  • helperprotocol acts broker between api , hardwareprotocol (receives commands api via local tcp, , transfer deviceprotocol via fabric class)

code example

class deviceprotocol(protocol):      def __init__(self, factory):         self.factory = factory       def connectionmade(self):         """         change object status in db         set objects collection in factory         """      def connectionlost(self, reason):         """         change object status in db         del objects collection in factory         """      def datareceived(self, data):         recv_def = defer.deferred()         recv_def.addcallback(self.pasre_data)         recv_def.adderrback(self.log_error)         recv_def.callback(data)      def log_error(self, data):         self.log.warning("something happens, %s", data)      def parse_data(self, data):         """         calculate response , call def send_data()         manipulations data         """      def send_data(self, data):         self.transport.write(data)    class helperprotocol(protocol):      def __init__(self, factory):         self.factory = factory       def connectionmade(self):         self.log.info("connection api")      def datareceived(self, data):         self.data = json.loads(data.decode('utf-8'))         remote_device = self.data['remote_device']         data_pack = self.data['data_pack']         self.factory.objects[remote_device].send_data(data_pack) # send data remote device         self.transport.write('ok'.encode('utf-8')) # send ok response remote api service       def connectionlost(self, reason):         self.log.info("api connection terminated")    class protocolfactory(factory):     objects = {}      def __init__(self, type):         self.clients = {}         self.type = type      def buildprotocol(self, addr):         if self.type == "device":             return deviceprotocol(self)         if self.type == "api":             return helperprotocol(self)   endpoint = tcp4serverendpoint(reactor, port, interface) endpoint.listen(protocolfactory("device"))  endpoint_api = tcp4serverendpoint(reactor, another_port, interface) endpoint_api.listen(protocolfactory("api"))  reactor.run() 

the actual problem first test 50 devices(deviceprotocol) , 10(helperprotocol) simultaneous connections helper got perfect result without delays. when added 20 more devices, system has gone nuts:

  • api receives data , passes helper on tcp connection

  • helper event connectionmade after 1 minute , datareceived event 1 more minute later (it couple of milliseconds 50 devices)

  • helper through factory sends command deviceprotocol

server’s 1.5 ghz core loaded 1 - 1.2%, there no consumption of ram either.

what’s going on, guys?


Comments

Popular posts from this blog

php - Permission denied. Laravel linux server -

google bigquery - Delta between query execution time and Java query call to finish -

python - Pandas two dataframes multiplication? -