Cannot connect to the Zookeeper instance <connectionString>. Current state: CONNECTING #603
Description
We are getting this weird error at random times while using the SolrNet.Cloud library (v 1.0.19) on a Solr Cloud installation with a ZK enclave of 3 ZK nodes:
SolrNet.Cloud.Exceptions.SolrNetCloudConnectionException: Cannot connect to the Zookeeper instance zk1.local.net:2181,zk2.local.net:2181,zk3.local.net:2181/solr. Current state: CONNECTING
at SolrNet.Cloud.ZooKeeperClient.SolrCloudStateProvider.d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
I see it comes from the code added here (although, I haven't managed to find yet on which branch it is applied - it seemed missing from master and develop):
https://github.com/SolrNet/SolrNet/pull/388/files#diff-3fecf2ac78599febba813069d1306ed53522ca0f94416fd6115f964315758d78
Any idea why the connection goes into a CONNECTING state?
Is this normal?
Should the patch code in the link above do something special when the state is CONNECTING?
I have tried using the ZookeeperNetEx package to hit the ZK directly and it worked, at the same time that the SolrNet.Cloud library was failing. Although, I didn't check the state to be honest, I was just querying the Solr nodes status from the zookeeper object directly. Perhaps the CONNECTING state should not throw an exception at all and let the code proceed as normal?
Also, another thing:
Inside the method SolrNet.Cloud.Startup.EnsureRegistrationAsync perhaps this async call should be made to use .ConfigureAwait(false); ?
await cloudStateProvider.InitAsync();
It might help prevent some deadlocks when the Startup.InitAsync<T>() gets called from a web application.