hortonworks data platform - Storm Topology deployment using pre deployed jar -
we have jar contains 6 topologies. deploy these topologies 6 separate calls using
/bin/storm jar $local_jar $topology_class $topology_name $ps_env $zk_quorum -c nimbus.host=$nimbus_host $storm_config_args
looking @ log output, each time topology submitted jar uploaded nimbus i.e there 6 lines this
9937 [main] info o.a.s.stormsubmitter - uploaded topology jar assigned location:...
i want avoid multiple uploading of jar. have tried uploading jar via scp , placing in @ "uploadedjarlocation" on nimbus node ( once). changing deployment code use following each of topologies.
nimbusclient = nimbusclient.getconfiguredclient(storm_conf); client = nimbusclient.getclient(); ... client.submittopology(topologyname, uploadedjarlocation, jsonconf, topology.buildtopology());
this has sped things , seems work fine want ask
is safe approach, can safely reference uploadedjarlocation pre-uploaded nimbus via scp?
are there alternative methods avoid multiple jar upload?
i know stormsubmitter.submitjar alternative have found slow.
Comments
Post a Comment