Stikked
2017-08-02 23:24:51.583 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Client session timed out, have not heard from server in 13334ms for sessionid 0x35d394c995f0070, closing socket connection and attempting
2017-08-02 23:24:51.683 o.a.s.s.o.a.c.f.s.ConnectionStateManager main-EventThread [INFO] State change:
2017-08-02 23:24:51.684 o.a.s.u.StormBoundedExponentialBackoffRetry timer [WARN] WILL SLEEP FOR 1001ms (NOT
2017-08-02 23:24:52.138 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.1.215/20.0.1.215:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:24:58.810 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Client session timed out, have not heard from server in 7127ms for sessionid 0x35d394c995f0070, closing socket connection and attempting
2017-08-02 23:24:58.967 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.3.97/20.0.3.97:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:25:05.640 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Client session timed out, have not heard from server in 6729ms for sessionid 0x35d394c995f0070, closing socket connection and attempting
2017-08-02 23:25:06.687 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:06.687 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1001ms (NOT
2017-08-02 23:25:06.952 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.3.217/20.0.3.217:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:25:07.688 o.a.s.s.o.a.c.ConnectionState timer [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:489) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:237) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:226) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:223) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:216) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:175) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:32) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.zookeeper.Zookeeper.existsNode(Zookeeper.java:164) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.zookeeper.Zookeeper.mkdirsImpl(Zookeeper.java:197) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.zookeeper.Zookeeper.mkdirs(Zookeeper.java:189) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.cluster.ZKStateStorage.set_ephemeral_node(ZKStateStorage.java:144) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.cluster.StormClusterStateImpl.supervisorHeartbeat(StormClusterStateImpl.java:419) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.daemon.supervisor.timer.SupervisorHeartbeat.run(SupervisorHeartbeat.java:85) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.StormTimer$1.run(StormTimer.java:190) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:83) ~[storm-core-1.1.0.jar:1.1.
2017-08-02 23:25:07.688 o.a.s.u.StormBoundedExponentialBackoffRetry timer [WARN] WILL SLEEP FOR 1003ms (NOT
2017-08-02 23:25:07.689 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:07.689 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1002ms (NOT
2017-08-02 23:25:08.691 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:08.691 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1004ms (NOT
2017-08-02 23:25:09.123 o.a.s.d.s.Slot SLOT_6700 [WARN] SLOT 6700: HB is too old 31000 >
2017-08-02 23:25:09.123 o.a.s.d.s.Container SLOT_6700 [INFO] Killing
2017-08-02 23:25:09.695 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:09.695 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1012ms (NOT
2017-08-02 23:25:10.708 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:10.708 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1029ms (NOT
2017-08-02 23:25:11.737 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [WARN] Connection attempt unsuccessful after 20054 (greater than max timeout of 20000). Resetting connection and trying again with a new
2017-08-02 23:25:12.507 o.a.s.s.o.a.z.ZooKeeper Curator-Framework-0 [INFO] Session: 0x35d394c995f0070
2017-08-02 23:25:12.507 o.a.s.s.o.a.z.ZooKeeper Curator-Framework-0 [INFO] Initiating client connection, connectString=20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.
2017-08-02 23:25:12.507 o.a.s.s.o.a.z.ClientCnxn main-EventThread [INFO] EventThread shut
2017-08-02 23:25:12.508 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.3.217/20.0.3.217:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:25:12.508 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl Curator-Framework-0 [ERROR] Background operation retry gave
org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:99) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:728) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:857) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:12.509 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl Curator-Framework-0 [ERROR] Background retry gave
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:838) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:19.181 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.217:2181) [INFO] Client session timed out, have not heard from server in 6673ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:25:19.281 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.1.215/20.0.1.215:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:25:25.949 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.217:2181) [INFO] Client session timed out, have not heard from server in 6668ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:25:26.050 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.3.97/20.0.3.97:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:25:26.050 o.a.s.u.StormBoundedExponentialBackoffRetry timer [WARN] WILL SLEEP FOR 1006ms (NOT
2017-08-02 23:25:27.510 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:27.510 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1001ms (NOT
2017-08-02 23:25:28.512 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:28.512 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1003ms (NOT
2017-08-02 23:25:29.515 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:29.515 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1006ms (NOT
2017-08-02 23:25:30.521 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:30.521 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1009ms (NOT
2017-08-02 23:25:31.531 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [ERROR] Connection timed out for connection string (20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm) and timeout (15000) / elapsed
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:835) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:31.531 o.a.s.u.StormBoundedExponentialBackoffRetry Curator-Framework-0 [WARN] WILL SLEEP FOR 1025ms (NOT
2017-08-02 23:25:32.556 o.a.s.s.o.a.c.ConnectionState Curator-Framework-0 [WARN] Connection attempt unsuccessful after 20819 (greater than max timeout of 20000). Resetting connection and trying again with a new
2017-08-02 23:25:32.817 o.a.s.s.o.a.z.ZooKeeper Curator-Framework-0 [INFO] Session: 0x0
2017-08-02 23:25:32.817 o.a.s.s.o.a.z.ZooKeeper Curator-Framework-0 [INFO] Initiating client connection, connectString=20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.
2017-08-02 23:25:32.817 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-EventThread [INFO] EventThread shut
2017-08-02 23:25:32.817 o.a.s.s.o.a.c.f.s.ConnectionStateManager Curator-Framework-0 [INFO] State change:
2017-08-02 23:25:32.818 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Opening socket connection to server 20.0.3.97/20.0.3.97:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:25:32.818 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl Curator-Framework-0 [ERROR] Background operation retry gave
org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:99) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:728) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:857) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:32.818 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl Curator-Framework-0 [ERROR] Background retry gave
org.apache.storm.shade.org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:838) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) ~[storm-core-1.1.0.jar:1.1.
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.
at java.lang.Thread.run(Thread.java:745) [?:1.8.
2017-08-02 23:25:39.491 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Client session timed out, have not heard from server in 6673ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:25:39.591 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Opening socket connection to server 20.0.3.217/20.0.3.217:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:25:46.261 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Client session timed out, have not heard from server in 6670ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:25:46.361 o.a.s.u.StormBoundedExponentialBackoffRetry timer [WARN] WILL SLEEP FOR 1008ms (NOT
2017-08-02 23:25:46.361 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Opening socket connection to server 20.0.1.215/20.0.1.215:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:25:53.034 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Client session timed out, have not heard from server in 6673ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:25:54.135 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Opening socket connection to server 20.0.3.97/20.0.3.97:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:26:00.808 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Client session timed out, have not heard from server in 7673ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:26:00.908 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-SendThread(20.0.3.97:2181) [INFO] Opening socket connection to server 20.0.3.217/20.0.3.217:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:26:02.372 o.a.s.s.o.a.c.ConnectionState timer [WARN] Connection attempt unsuccessful after 29816 (greater than max timeout of 20000). Resetting connection and trying again with a new
2017-08-02 23:26:07.680 o.a.s.s.o.a.z.ZooKeeper timer [INFO] Session: 0x0
2017-08-02 23:26:07.680 o.a.s.s.o.a.z.ClientCnxn Curator-Framework-0-EventThread [INFO] EventThread shut
2017-08-02 23:26:07.680 o.a.s.s.o.a.z.ZooKeeper timer [INFO] Initiating client connection, connectString=20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.
2017-08-02 23:26:07.681 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.3.217/20.0.3.217:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:26:14.354 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.217:2181) [INFO] Client session timed out, have not heard from server in 6673ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:26:14.455 o.a.s.u.StormBoundedExponentialBackoffRetry timer [WARN] WILL SLEEP FOR 1025ms (NOT
2017-08-02 23:26:14.455 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.3.97/20.0.3.97:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:26:21.128 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.217:2181) [INFO] Client session timed out, have not heard from server in 6672ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:26:21.228 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.1.215/20.0.1.215:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:26:27.901 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.217:2181) [INFO] Client session timed out, have not heard from server in 6673ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:26:29.001 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.3.217/20.0.3.217:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:26:30.482 o.a.s.s.o.a.c.ConnectionState timer [WARN] Connection attempt unsuccessful after 28110 (greater than max timeout of 20000). Resetting connection and trying again with a new
2017-08-02 23:26:34.772 o.a.s.s.o.a.z.ZooKeeper timer [INFO] Session: 0x0
2017-08-02 23:26:34.772 o.a.s.s.o.a.z.ClientCnxn timer-EventThread [INFO] EventThread shut
2017-08-02 23:26:34.772 o.a.s.s.o.a.z.ZooKeeper timer [INFO] Initiating client connection, connectString=20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.
2017-08-02 23:26:34.773 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.97:2181) [INFO] Opening socket connection to server 20.0.3.97/20.0.3.97:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:26:41.445 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.97:2181) [INFO] Client session timed out, have not heard from server in 6673ms for sessionid 0x0, closing socket connection and attempting
2017-08-02 23:26:41.545 o.a.s.s.o.a.z.ClientCnxn timer-SendThread(20.0.3.97:2181) [INFO] Opening socket connection to server 20.0.3.217/20.0.3.217:2181. Will not attempt to authenticate using SASL (unknown
2017-08-02 23:26:41.545 o.a.s.d.s.DefaultUncaughtExceptionHandler timer [ERROR] Error when processing
java.lang.RuntimeException: org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for
at org.apache.storm.utils.Utils.wrapInRuntime(Utils.java:1531) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.zookeeper.Zookeeper.existsNode(Zookeeper.java:167) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.zookeeper.Zookeeper.mkdirsImpl(Zookeeper.java:197) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.zookeeper.Zookeeper.mkdirs(Zookeeper.java:189) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.cluster.ZKStateStorage.set_ephemeral_node(ZKStateStorage.java:144) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.cluster.StormClusterStateImpl.supervisorHeartbeat(StormClusterStateImpl.java:419) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.daemon.supervisor.timer.SupervisorHeartbeat.run(SupervisorHeartbeat.java:85) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.StormTimer$1.run(StormTimer.java:190) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:83) ~[storm-core-1.1.0.jar:1.1.
Caused by: org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for
at org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:99) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:51) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:237) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:226) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:223) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:216) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:175) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:32) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.zookeeper.Zookeeper.existsNode(Zookeeper.java:164) ~[storm-core-1.1.0.jar:1.1.
... 7
2017-08-02 23:26:41.545 o.a.s.u.Utils timer [ERROR] Halting process: Error when processing an
java.lang.RuntimeException: Halting process: Error when processing an
at org.apache.storm.utils.Utils.exitProcess(Utils.java:1773) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.daemon.supervisor.DefaultUncaughtExceptionHandler.uncaughtException(DefaultUncaughtExceptionHandler.java:29) ~[storm-core-1.1.0.jar:1.1.
at org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:104) ~[storm-core-1.1.0.jar:1.1.
2017-08-02 23:26:41.547 o.a.s.d.s.Supervisor Thread-7 [INFO] Shutting down supervisor
2017-08-03 00:04:07.459 o.a.s.z.Zookeeper main [INFO] Staring ZK
2017-08-03 00:04:07.460 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl main [INFO]
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:host.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:java.version=1.8.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:java.vendor=Oracle
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:java.home=/usr/local/share/jdk1.8.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:java.class.path=/mnt/storm/lib/storm-core-1.1.0.jar:/mnt/storm/lib/slf4j-api-1.7.21.jar:/mnt/storm/lib/objenesis-2.1.jar:/mnt/storm/lib/log4j-over-slf4j-1.6.6.jar:/mnt/storm/lib/storm-rename-hack-1.1.0.jar:/mnt/storm/lib/minlog-1.3.0.jar:/mnt/storm/lib/kryo-3.0.3.jar:/mnt/storm/lib/servlet-api-2.5.jar:/mnt/storm/lib/log4j-api-2.8.jar:/mnt/storm/lib/disruptor-3.3.2.jar:/mnt/storm/lib/log4j-core-2.8.jar:/mnt/storm/lib/asm-5.0.3.jar:/mnt/storm/lib/log4j-slf4j-impl-2.8.jar:/mnt/storm/lib/reflectasm-1.10.1.jar:/mnt/storm/lib/ring-cors-0.1.5.jar:/mnt/storm/lib/clojure-1.7.0.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:java.library.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:java.io.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:java.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:os.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:os.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:os.version=3.13.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:user.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:user.
2017-08-03 00:04:07.466 o.a.s.s.o.a.z.ZooKeeper main [INFO] Client environment:user.
2017-08-03 00:04:07.467 o.a.s.s.o.a.z.ZooKeeper main [INFO] Initiating client connection, connectString=20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.
2017-08-03 00:04:07.493 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Opening socket connection to server 20.0.3.217/20.0.3.217:2181. Will not attempt to authenticate using SASL (unknown
2017-08-03 00:04:07.555 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Socket connection established to 20.0.3.217/20.0.3.217:2181, initiating
2017-08-03 00:04:07.562 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.217:2181) [INFO] Session establishment complete on server 20.0.3.217/20.0.3.217:2181, sessionid = 0x35d394c995f00b7, negotiated timeout =
2017-08-03 00:04:07.565 o.a.s.s.o.a.c.f.s.ConnectionStateManager main-EventThread [INFO] State change:
2017-08-03 00:04:07.576 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl Curator-Framework-0 [INFO] backgroundOperationsLoop
2017-08-03 00:04:07.579 o.a.s.s.o.a.z.ZooKeeper main [INFO] Session: 0x35d394c995f00b7
2017-08-03 00:04:07.579 o.a.s.s.o.a.z.ClientCnxn main-EventThread [INFO] EventThread shut
2017-08-03 00:04:07.580 o.a.s.z.Zookeeper main [INFO] Staring ZK
2017-08-03 00:04:07.580 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl main [INFO]
2017-08-03 00:04:07.580 o.a.s.s.o.a.z.ZooKeeper main [INFO] Initiating client connection, connectString=20.0.3.97:2181,20.0.1.215:2181,20.0.3.217:2181/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.
2017-08-03 00:04:07.581 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.97:2181) [INFO] Opening socket connection to server 20.0.3.97/20.0.3.97:2181. Will not attempt to authenticate using SASL (unknown
2017-08-03 00:04:07.583 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.97:2181) [INFO] Socket connection established to 20.0.3.97/20.0.3.97:2181, initiating
2017-08-03 00:04:07.586 o.a.s.s.o.a.z.ClientCnxn main-SendThread(20.0.3.97:2181) [INFO] Session establishment complete on server 20.0.3.97/20.0.3.97:2181, sessionid = 0x15d394c995000b3, negotiated timeout =
2017-08-03 00:04:07.586 o.a.s.s.o.a.c.f.s.ConnectionStateManager main-EventThread [INFO] State change:
2017-08-03 00:04:07.613 o.a.s.l.Localizer main [INFO] Reconstruct localized resource:
2017-08-03 00:04:07.615 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-372f9e95-76b8-4043-bfde-656b901590fb.jar.
2017-08-03 00:04:07.616 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-9a1f0c57-843f-49f3-8c6b-f697dada5a8e.jar.
2017-08-03 00:04:07.616 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-49313312-290d-40a8-8f81-f4fae39d1e62.jar.
2017-08-03 00:04:07.616 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-bf594487-8e2c-41ee-8cbf-4dc575d784a8.jar.
2017-08-03 00:04:07.616 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-36bf992d-924b-4aeb-9a78-b3a9152e1432.jar.
2017-08-03 00:04:07.616 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-d3c97a19-a8f0-4c6d-8257-1c1a222a5a88.jar.
2017-08-03 00:04:07.617 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-3db75f0d-3dc3-4679-99d1-e2a3769be2b4.jar.
2017-08-03 00:04:07.617 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-7f02d98a-1649-4b07-b0e0-3a7cd68818e2.jar.
2017-08-03 00:04:07.617 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-518829d3-8bfe-4ece-bbbe-be05a26ce13d.jar.
2017-08-03 00:04:07.617 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-355a663f-c450-4e86-bb0d-c2f9ea6bd20f.jar.
2017-08-03 00:04:07.617 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-ba69705d-6eeb-40cd-919d-794e909f2b0b.jar.
2017-08-03 00:04:07.617 o.a.s.l.Localizer main [INFO] add localized in dir found: /mnt/storm/storm-local/supervisor/usercache/ubuntu/filecache/files/dep-storm-kinesis-1.1.0-116d7c3d-e584-494f-805d-3dff5827559b.jar.
2017-08-03 00:04:07.624 o.a.s.d.s.Supervisor main [INFO] Starting supervisor for storm version '1.1.
2017-08-03 00:04:07.624 o.a.s.d.s.Supervisor main [INFO] Starting Supervisor with conf {storm.messaging.netty.min_wait_ms=100, storm.zookeeper.auth.user=null, storm.messaging.netty.buffer_size=5242880, storm.exhibitor.port=8080, pacemaker.auth.method=NONE, ui.filter=null, worker.profiler.enabled=false, ui.http.creds.plugin=org.apache.storm.security.auth.DefaultHttpCredentialsPlugin, topology.bolts.outgoing.overflow.buffer.enable=false, supervisor.supervisors.commands=[], logviewer.cleanup.age.mins=10080, topology.tuple.serializer=org.apache.storm.serialization.types.ListDelegateSerializer, drpc.port=3772, topology.max.spout.pending=null, topology.transfer.buffer.size=1024, logviewer.port=8000, worker.childopts=-Xmx2048m, topology.component.cpu.pcore.percent=10.0, storm.daemon.metrics.reporter.plugins=[org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter], drpc.childopts=-Xmx768m, nimbus.task.launch.secs=120, logviewer.childopts=-Xmx128m, storm.zookeeper.servers=[20.0.3.97, 20.0.1.215, 20.0.3.217], topology.disruptor.batch.timeout.millis=1, storm.messaging.transport=org.apache.storm.messaging.netty.Context, storm.messaging.netty.authentication=false, topology.kryo.factory=org.apache.storm.serialization.DefaultKryoFactory, supervisor.scheduler.meta={instancetype=r4.2xlarge}, worker.heap.memory.mb=768, storm.network.topography.plugin=org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping, supervisor.slots.ports=[6700, 6701, 6702, 6703, 6704, 6705, 6706], resource.aware.scheduler.eviction.strategy=org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy, topology.stats.sample.rate=0.05, storm.local.dir=storm-local, pacemaker.host=localhost, storm.messaging.netty.max_retries=300, topology.testing.always.try.serialize=false, storm.principal.tolocal=org.apache.storm.security.auth.DefaultPrincipalToLocal, java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib, worker.gc.childopts=, storm.group.mapping.service.cache.duration.secs=120, zmq.linger.millis=5000, topology.multilang.serializer=org.apache.storm.multilang.JsonSerializer, drpc.request.timeout.secs=600, zmq.threads=1, nimbus.blobstore.class=org.apache.storm.blobstore.LocalFsBlobStore, topology.state.synchronization.timeout.secs=60, topology.worker.shared.thread.pool.size=4, topology.executor.receive.buffer.size=1024, supervisor.monitor.frequency.secs=3, storm.nimbus.retry.times=5, transactional.zookeeper.port=null, storm.auth.simple-white-list.users=[], topology.scheduler.strategy=org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy, storm.zookeeper.port=2181, storm.zookeeper.retry.intervalceiling.millis=30000, storm.cluster.state.store=org.apache.storm.cluster_state.zookeeper_state_factory, nimbus.thrift.port=6627, nimbus.thrift.threads=64, supervisor.supervisors=[], nimbus.seeds=[20.0.3.212], storm.cluster.metrics.consumer.publish.interval.secs=60, storm.auth.simple-acl.admins=[], topology.min.replication.count=1, ui.http.x-frame-options=DENY, nimbus.blobstore.expiration.secs=600, storm.group.mapping.service=org.apache.storm.security.auth.ShellBasedGroupsMapping, storm.nimbus.retry.interval.millis=2000, topology.max.task.parallelism=null, drpc.https.keystore.password=, supervisor.heartbeat.frequency.secs=5, nimbus.credential.renewers.freq.secs=600, storm.thrift.transport=org.apache.storm.security.auth.SimpleTransportPlugin, storm.auth.simple-acl.users=[], storm.zookeeper.auth.password=null, ui.port=8080, drpc.authorizer.acl.strict=false, topology.message.timeout.secs=30, topology.error.throttle.interval.secs=10, drpc.https.keystore.type=JKS, supervisor.memory.capacity.mb=3072.0, drpc.authorizer.acl.filename=drpc-auth-acl.yaml, topology.builtin.metrics.bucket.size.secs=60, storm.local.mode.zmq=false, ui.header.buffer.bytes=4096, topology.shellbolt.max.pending=100, drpc.max_buffer_size=1048576, storm.codedistributor.class=org.apache.storm.codedistributor.LocalFileSystemCodeDistributor, worker.profiler.childopts=-XX:+UnlockCommercialFeatures -XX:+FlightRecorder, nimbus.supervisor.timeout.secs=60, topology.worker.max.heap.size.mb=768.0, storm.zookeeper.root=/storm, topology.disable.loadaware.messaging=false, zmq.hwm=0, topology.sleep.spout.wait.strategy.time.ms=1, nimbus.topology.validator=org.apache.storm.nimbus.DefaultTopologyValidator, worker.heartbeat.frequency.secs=1, storm.messaging.netty.max_wait_ms=1000, topology.max.error.report.per.interval=5, nimbus.thrift.max_buffer_size=1048576, pacemaker.max.threads=50, supervisor.blobstore.download.max_retries=3, topology.enable.message.timeouts=true, storm.messaging.netty.transfer.batch.size=262144, supervisor.run.worker.as.user=false, storm.messaging.netty.client_worker_threads=1, topology.tasks=null, storm.group.mapping.service.params=null, drpc.http.port=3774, transactional.zookeeper.root=/transactional, supervisor.blobstore.download.thread.count=5, pacemaker.kerberos.users=[], topology.spout.wait.strategy=org.apache.storm.spout.SleepSpoutWaitStrategy, storm.blobstore.inputstream.buffer.size.bytes=65536, supervisor.worker.timeout.secs=30, drpc.servers=[20.0.3.212], topology.worker.receiver.thread.count=1, logviewer.max.sum.worker.logs.size.mb=4096, nimbus.file.copy.expiration.secs=600, pacemaker.port=6699, topology.worker.logwriter.childopts=-Xmx64m, drpc.http.creds.plugin=org.apache.storm.security.auth.DefaultHttpCredentialsPlugin, storm.blobstore.acl.validation.enabled=false, ui.filter.params=null, topology.workers=1, topology.environment=null, drpc.invocations.port=3773, topology.disruptor.batch.size=100, nimbus.cleanup.inbox.freq.secs=600, client.blobstore.class=org.apache.storm.blobstore.NimbusBlobStore, topology.fall.back.on.java.serialization=true, storm.nimbus.retry.intervalceiling.millis=60000, logviewer.appender.name=A1, ui.users=null, pacemaker.childopts=-Xmx1024m, storm.messaging.netty.server_worker_threads=1, scheduler.display.resource=false, storm.auth.simple-acl.users.commands=[], ui.actions.enabled=true, storm.thrift.socket.timeout.ms=600000, storm.zookeeper.connection.timeout=15000, topology.tick.tuple.freq.secs=null, nimbus.inbox.jar.expiration.secs=3600, topology.debug=false, storm.zookeeper.retry.interval=1000, worker.log.level.reset.poll.secs=30, storm.exhibitor.poll.uripath=/exhibitor/v1/cluster/list, storm.zookeeper.retry.times=5, nimbus.code.sync.freq.secs=120, topology.component.resources.offheap.memory.mb=0.0, topology.state.checkpoint.interval.ms=1000, topology.priority=29, supervisor.localizer.cleanup.interval.ms=600000, storm.health.check.dir=healthchecks, topology.disruptor.wait.timeout.millis=1000, supervisor.cpu.capacity=400.0, topology.classpath=null, backpressure.disruptor.high.watermark=0.9, supervisor.localizer.cache.target.size.mb=10240, topology.worker.childopts=null, drpc.https.port=-1, topology.max.replication.wait.time.sec=60, topology.acker.executors=null, supervisor.worker.start.timeout.secs=120, supervisor.worker.shutdown.sleep.secs=3, logviewer.max.per.worker.logs.size.mb=2048, topology.trident.batch.emit.interval.millis=500, task.heartbeat.frequency.secs=3, supervisor.enable=true, supervisor.blobstore.class=org.apache.storm.blobstore.NimbusBlobStore, topology.backpressure.enable=false, drpc.worker.threads=64, resource.aware.scheduler.priority.strategy=org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy, storm.messaging.netty.socket.backlog=500, nimbus.queue.size=100000, drpc.queue.size=128, topology.eventlogger.executors=0, pacemaker.base.threads=10, nimbus.childopts=-Xmx1024m, nimbus.monitor.freq.secs=10, topology.executor.send.buffer.size=1024, storm.disable.symlinks=false, transactional.zookeeper.servers=null, nimbus.task.timeout.secs=30, logs.users=null, ui.host=0.0.0.0, pacemaker.thread.timeout=10, storm.meta.serialization.delegate=org.apache.storm.serialization.GzipThriftSerializationDelegate, dev.zookeeper.path=/tmp/dev-storm-zookeeper, backpressure.disruptor.low.watermark=0.4, topology.skip.missing.kryo.registrations=false, drpc.invocations.threads=64, storm.zookeeper.session.timeout=20000, storm.workers.artifacts.dir=workers-artifacts, topology.component.resources.onheap.memory.mb=128.0, storm.log4j2.conf.dir=log4j2, task.backpressure.poll.secs=30, storm.cluster.mode=distributed, ui.childopts=-Xmx768m, task.refresh.poll.secs=10, supervisor.childopts=-Xmx256m, task.credentials.poll.secs=30, storm.health.check.timeout.ms=5000, storm.blobstore.replication.factor=3, worker.profiler.command=flight.
2017-08-03 00:04:07.679 o.a.s.d.s.BasicContainer main [INFO] Recovered Worker
2017-08-03 00:04:07.683 o.a.s.d.s.Slot main [WARN] SLOT ip-20-0-1-66:6700 Starting in state RUNNING - assignment LocalAssignment(topology_id:CMDTopology-prod-12-1501556835, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:31, task_end:31), ExecutorInfo(task_start:19, task_end:19), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:25, task_end:25)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.
2017-08-03 00:04:07.683 o.a.s.d.s.Slot main [WARN] SLOT ip-20-0-1-66:6701 Starting in state EMPTY - assignment
2017-08-03 00:04:07.684 o.a.s.d.s.Slot main [WARN] SLOT ip-20-0-1-66:6702 Starting in state EMPTY - assignment
2017-08-03 00:04:07.685 o.a.s.d.s.Slot main [WARN] SLOT ip-20-0-1-66:6703 Starting in state EMPTY - assignment
2017-08-03 00:04:07.688 o.a.s.d.s.BasicContainer main [INFO] Recovered Worker
2017-08-03 00:04:07.690 o.a.s.d.s.Slot main [WARN] SLOT ip-20-0-1-66:6704 Starting in state RUNNING - assignment LocalAssignment(topology_id:LearningEventTopology-prod-10-1501556817, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:31, task_end:31), ExecutorInfo(task_start:43, task_end:43), ExecutorInfo(task_start:19, task_end:19), ExecutorInfo(task_start:37, task_end:37), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:49, task_end:49), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:25, task_end:25)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.
2017-08-03 00:04:07.692 o.a.s.d.s.BasicContainer main [INFO] Recovered Worker
2017-08-03 00:04:07.694 o.a.s.d.s.Slot main [WARN] SLOT ip-20-0-1-66:6705 Starting in state RUNNING - assignment LocalAssignment(topology_id:LearningEventS3Topology-prod-9-1501556809, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:19, task_end:19)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.
2017-08-03 00:04:07.696 o.a.s.d.s.BasicContainer main [INFO] Recovered Worker
2017-08-03 00:04:07.698 o.a.s.d.s.Slot main [WARN] SLOT ip-20-0-1-66:6706 Starting in state RUNNING - assignment LocalAssignment(topology_id:CMDEventS3Topology-prod-11-1501556826, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:19, task_end:19)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.
2017-08-03 00:04:07.698 o.a.s.l.AsyncLocalizer main [INFO] Cleaning up unused topologies in
2017-08-03 00:04:07.714 o.a.s.d.s.Supervisor main [INFO] Starting supervisor with id 5a779879-2f03-4b7d-b474-5086510e479d at host
2017-08-03 00:04:07.716 o.a.s.d.m.MetricsUtils main [INFO] Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.
2017-08-03 00:04:07.717 o.a.s.d.m.r.JmxPreparableReporter main [INFO]
2017-08-03 00:04:07.722 o.a.s.m.StormMetricsRegistry main [INFO] Started statistics report
2017-08-03 00:04:10.706 o.a.s.d.s.Slot SLOT_6705 [WARN] SLOT 6705: Assignment Changed from LocalAssignment(topology_id:LearningEventS3Topology-prod-9-1501556809, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:19, task_end:19)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)) to
2017-08-03 00:04:10.706 o.a.s.d.s.Slot SLOT_6704 [WARN] SLOT 6704: Assignment Changed from LocalAssignment(topology_id:LearningEventTopology-prod-10-1501556817, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:31, task_end:31), ExecutorInfo(task_start:43, task_end:43), ExecutorInfo(task_start:19, task_end:19), ExecutorInfo(task_start:37, task_end:37), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:49, task_end:49), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:25, task_end:25)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)) to
2017-08-03 00:04:10.706 o.a.s.d.s.Slot SLOT_6700 [WARN] SLOT 6700: Assignment Changed from LocalAssignment(topology_id:CMDTopology-prod-12-1501556835, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:31, task_end:31), ExecutorInfo(task_start:19, task_end:19), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:25, task_end:25)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)) to
2017-08-03 00:04:10.706 o.a.s.d.s.Slot SLOT_6706 [WARN] SLOT 6706: Assignment Changed from LocalAssignment(topology_id:CMDEventS3Topology-prod-11-1501556826, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:19, task_end:19)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)) to
2017-08-03 00:04:10.707 o.a.s.d.s.Container SLOT_6706 [INFO] Killing
2017-08-03 00:04:10.707 o.a.s.d.s.Container SLOT_6705 [INFO] Killing
2017-08-03 00:04:10.707 o.a.s.d.s.Container SLOT_6700 [INFO] Killing
2017-08-03 00:04:10.707 o.a.s.d.s.Container SLOT_6704 [INFO] Killing
2017-08-03 00:04:13.728 o.a.s.d.s.Slot SLOT_6704 [INFO] STATE RUNNING msInState: 6040 topo:LearningEventTopology-prod-10-1501556817 worker:632212f8-39c3-4750-bef8-7eb3870e8107 -> KILL msInState: 0 topo:LearningEventTopology-prod-10-1501556817
2017-08-03 00:04:13.728 o.a.s.d.s.Container SLOT_6704 [INFO] GET worker-user for
2017-08-03 00:04:13.728 o.a.s.d.s.Slot SLOT_6705 [INFO] STATE RUNNING msInState: 6036 topo:LearningEventS3Topology-prod-9-1501556809 worker:c421c4a3-27f0-4ca5-bb15-e6728db81da5 -> KILL msInState: 0 topo:LearningEventS3Topology-prod-9-1501556809
2017-08-03 00:04:13.728 o.a.s.d.s.Container SLOT_6705 [INFO] GET worker-user for
2017-08-03 00:04:13.729 o.a.s.d.s.Slot SLOT_6706 [INFO] STATE RUNNING msInState: 6032 topo:CMDEventS3Topology-prod-11-1501556826 worker:f27c17cb-1e83-4d99-b0b5-b1534b8a4127 -> KILL msInState: 0 topo:CMDEventS3Topology-prod-11-1501556826
2017-08-03 00:04:13.729 o.a.s.d.s.Slot SLOT_6700 [INFO] STATE RUNNING msInState: 6050 topo:CMDTopology-prod-12-1501556835 worker:a2a5c83e-a004-4c21-8ed7-163c05fbdbd2 -> KILL msInState: 0 topo:CMDTopology-prod-12-1501556835
2017-08-03 00:04:13.729 o.a.s.d.s.Container SLOT_6706 [INFO] GET worker-user for
2017-08-03 00:04:13.729 o.a.s.d.s.Container SLOT_6700 [INFO] GET worker-user for
2017-08-03 00:04:13.734 o.a.s.d.s.Slot SLOT_6700 [WARN] SLOT 6700 all processes are
2017-08-03 00:04:13.734 o.a.s.d.s.Slot SLOT_6706 [WARN] SLOT 6706 all processes are
2017-08-03 00:04:13.734 o.a.s.d.s.Slot SLOT_6705 [WARN] SLOT 6705 all processes are
2017-08-03 00:04:13.734 o.a.s.d.s.Slot SLOT_6704 [WARN] SLOT 6704 all processes are
2017-08-03 00:04:13.734 o.a.s.d.s.Container SLOT_6700 [INFO] Cleaning up
2017-08-03 00:04:13.734 o.a.s.d.s.Container SLOT_6706 [INFO] Cleaning up
2017-08-03 00:04:13.734 o.a.s.d.s.Container SLOT_6704 [INFO] Cleaning up
2017-08-03 00:04:13.734 o.a.s.d.s.Container SLOT_6700 [INFO] GET worker-user for
2017-08-03 00:04:13.734 o.a.s.d.s.Container SLOT_6706 [INFO] GET worker-user for
2017-08-03 00:04:13.734 o.a.s.d.s.Container SLOT_6704 [INFO] GET worker-user for
2017-08-03 00:04:13.734 o.a.s.d.s.Container SLOT_6705 [INFO] Cleaning up
2017-08-03 00:04:13.735 o.a.s.d.s.Container SLOT_6705 [INFO] GET worker-user for
2017-08-03 00:04:13.735 o.a.s.d.s.AdvancedFSOps SLOT_6700 [INFO] Deleting path
2017-08-03 00:04:13.735 o.a.s.d.s.AdvancedFSOps SLOT_6705 [INFO] Deleting path
2017-08-03 00:04:13.735 o.a.s.d.s.AdvancedFSOps SLOT_6706 [INFO] Deleting path
2017-08-03 00:04:13.735 o.a.s.d.s.AdvancedFSOps SLOT_6704 [INFO] Deleting path
2017-08-03 00:04:13.735 o.a.s.d.s.AdvancedFSOps SLOT_6705 [INFO] Deleting path
2017-08-03 00:04:13.735 o.a.s.d.s.AdvancedFSOps SLOT_6704 [INFO] Deleting path
2017-08-03 00:04:13.735 o.a.s.d.s.AdvancedFSOps SLOT_6700 [INFO] Deleting path
2017-08-03 00:04:13.736 o.a.s.d.s.AdvancedFSOps SLOT_6706 [INFO] Deleting path
2017-08-03 00:04:14.078 o.a.s.d.s.AdvancedFSOps SLOT_6700 [INFO] Deleting path
2017-08-03 00:04:14.078 o.a.s.d.s.AdvancedFSOps SLOT_6704 [INFO] Deleting path
2017-08-03 00:04:14.078 o.a.s.d.s.AdvancedFSOps SLOT_6700 [INFO] Deleting path
2017-08-03 00:04:14.078 o.a.s.d.s.AdvancedFSOps SLOT_6704 [INFO] Deleting path
2017-08-03 00:04:14.078 o.a.s.d.s.AdvancedFSOps SLOT_6704 [INFO] Deleting path
2017-08-03 00:04:14.078 o.a.s.d.s.AdvancedFSOps SLOT_6700 [INFO] Deleting path
2017-08-03 00:04:14.079 o.a.s.d.s.Container SLOT_6704 [INFO] REMOVE worker-user
2017-08-03 00:04:14.079 o.a.s.d.s.Container SLOT_6700 [INFO] REMOVE worker-user
2017-08-03 00:04:14.079 o.a.s.d.s.AdvancedFSOps SLOT_6704 [INFO] Deleting path
2017-08-03 00:04:14.079 o.a.s.d.s.AdvancedFSOps SLOT_6700 [INFO] Deleting path
2017-08-03 00:04:14.080 o.a.s.d.s.BasicContainer SLOT_6704 [INFO] Removed Worker ID
2017-08-03 00:04:14.080 o.a.s.l.AsyncLocalizer SLOT_6704 [INFO] Released blob reference LearningEventTopology-prod-10-1501556817 6704 Cleaning up BLOB
2017-08-03 00:04:14.081 o.a.s.d.s.BasicContainer SLOT_6700 [INFO] Removed Worker ID
2017-08-03 00:04:14.081 o.a.s.l.AsyncLocalizer SLOT_6704 [INFO] Released blob reference LearningEventTopology-prod-10-1501556817 6704 Cleaning up basic
2017-08-03 00:04:14.081 o.a.s.d.s.AdvancedFSOps SLOT_6704 [INFO] Deleting path
2017-08-03 00:04:14.085 o.a.s.d.s.AdvancedFSOps SLOT_6706 [INFO] Deleting path
2017-08-03 00:04:14.085 o.a.s.d.s.AdvancedFSOps SLOT_6706 [INFO] Deleting path
2017-08-03 00:04:14.085 o.a.s.d.s.AdvancedFSOps SLOT_6706 [INFO] Deleting path
2017-08-03 00:04:14.086 o.a.s.d.s.Container SLOT_6706 [INFO] REMOVE worker-user
2017-08-03 00:04:14.086 o.a.s.d.s.AdvancedFSOps SLOT_6706 [INFO] Deleting path
2017-08-03 00:04:14.087 o.a.s.d.s.BasicContainer SLOT_6706 [INFO] Removed Worker ID
2017-08-03 00:04:14.096 o.a.s.d.s.Slot SLOT_6704 [INFO] STATE KILL msInState: 368 topo:LearningEventTopology-prod-10-1501556817 worker:null -> EMPTY msInState:
2017-08-03 00:04:14.096 o.a.s.l.AsyncLocalizer SLOT_6706 [INFO] Released blob reference CMDEventS3Topology-prod-11-1501556826 6706 Cleaning up BLOB
2017-08-03 00:04:14.096 o.a.s.d.s.Slot SLOT_6704 [INFO] SLOT 6704: Changing current assignment from LocalAssignment(topology_id:LearningEventTopology-prod-10-1501556817, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:31, task_end:31), ExecutorInfo(task_start:43, task_end:43), ExecutorInfo(task_start:19, task_end:19), ExecutorInfo(task_start:37, task_end:37), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:49, task_end:49), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:25, task_end:25)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)) to
2017-08-03 00:04:14.097 o.a.s.l.AsyncLocalizer SLOT_6706 [INFO] Released blob reference CMDEventS3Topology-prod-11-1501556826 6706 Cleaning up basic
2017-08-03 00:04:14.097 o.a.s.d.s.AdvancedFSOps SLOT_6706 [INFO] Deleting path
2017-08-03 00:04:14.109 o.a.s.d.s.AdvancedFSOps SLOT_6705 [INFO] Deleting path
2017-08-03 00:04:14.109 o.a.s.d.s.AdvancedFSOps SLOT_6705 [INFO] Deleting path
2017-08-03 00:04:14.109 o.a.s.d.s.AdvancedFSOps SLOT_6705 [INFO] Deleting path
2017-08-03 00:04:14.110 o.a.s.d.s.Container SLOT_6705 [INFO] REMOVE worker-user
2017-08-03 00:04:14.110 o.a.s.d.s.AdvancedFSOps SLOT_6705 [INFO] Deleting path
2017-08-03 00:04:14.111 o.a.s.d.s.BasicContainer SLOT_6705 [INFO] Removed Worker ID
2017-08-03 00:04:14.112 o.a.s.d.s.Slot SLOT_6706 [INFO] STATE KILL msInState: 383 topo:CMDEventS3Topology-prod-11-1501556826 worker:null -> EMPTY msInState:
2017-08-03 00:04:14.112 o.a.s.l.AsyncLocalizer SLOT_6700 [INFO] Released blob reference CMDTopology-prod-12-1501556835 6700 Cleaning up BLOB
2017-08-03 00:04:14.112 o.a.s.d.s.Slot SLOT_6706 [INFO] SLOT 6706: Changing current assignment from LocalAssignment(topology_id:CMDEventS3Topology-prod-11-1501556826, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:19, task_end:19)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)) to
2017-08-03 00:04:14.113 o.a.s.l.AsyncLocalizer SLOT_6700 [INFO] Released blob reference CMDTopology-prod-12-1501556835 6700 Cleaning up basic
2017-08-03 00:04:14.113 o.a.s.d.s.AdvancedFSOps SLOT_6700 [INFO] Deleting path
2017-08-03 00:04:14.124 o.a.s.d.s.Slot SLOT_6700 [INFO] STATE KILL msInState: 395 topo:CMDTopology-prod-12-1501556835 worker:null -> EMPTY msInState:
2017-08-03 00:04:14.124 o.a.s.l.AsyncLocalizer SLOT_6705 [INFO] Released blob reference LearningEventS3Topology-prod-9-1501556809 6705 Cleaning up BLOB
2017-08-03 00:04:14.124 o.a.s.d.s.Slot SLOT_6700 [INFO] SLOT 6700: Changing current assignment from LocalAssignment(topology_id:CMDTopology-prod-12-1501556835, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:31, task_end:31), ExecutorInfo(task_start:19, task_end:19), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:25, task_end:25)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)) to
2017-08-03 00:04:14.125 o.a.s.l.AsyncLocalizer SLOT_6705 [INFO] Released blob reference LearningEventS3Topology-prod-9-1501556809 6705 Cleaning up basic
2017-08-03 00:04:14.125 o.a.s.d.s.AdvancedFSOps SLOT_6705 [INFO] Deleting path
2017-08-03 00:04:14.138 o.a.s.d.s.Slot SLOT_6705 [INFO] STATE KILL msInState: 410 topo:LearningEventS3Topology-prod-9-1501556809 worker:null -> EMPTY msInState:
2017-08-03 00:04:14.138 o.a.s.d.s.Slot SLOT_6705 [INFO] SLOT 6705: Changing current assignment from LocalAssignment(topology_id:LearningEventS3Topology-prod-9-1501556809, executors:[ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:1, task_end:1), ExecutorInfo(task_start:13, task_end:13), ExecutorInfo(task_start:19, task_end:19)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)) to
2017-08-03 03:44:39.855 o.a.s.d.s.Slot SLOT_6704 [INFO] STATE EMPTY msInState: 13225759 -> WAITING_FOR_BASIC_LOCALIZATION msInState: