Error connecting with client certificate from smart card / etoken to WCF Service

We have a WCF service configured to work with client certificates.

Our client is a .net 3.5 WPF application.

While testing, it works perfectly with certificates generated by Microsoft CA and others. While testing it with a physical token (Aladdin/Safenet eToken Pro 64k) it also works well.

(On first try connecting to server the “Token Logon” window pops up. After successful authentication to the token the server request works, and the next try to connect to server succeed without the token logon message showing)

Now, if we remove the token and re-insert it, when trying to connect with the same certificate we get an error “The request was aborted: Could not create SSL/TLS secure channel”. HResult 0x80131509

The only way to make it work again is restart the application.

Certificates are retrieved with System.Security.Cryptogragpy:

var store = new X509Store("My", StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
//Getting the right certificate and then:
store.Close();

The connection to server is with this code:

string uriStr = txtUrl.Text + "/rest/auth/strong/Ping";
var client = WebRequest.Create(new Uri(uriStr)) as HttpWebRequest;

client.Method = "POST";
client.ContentType = "application/text; charset=unicode;";
client.ContentLength = 0;

client.PreAuthenticate = false;
client.KeepAlive = false;

client.ClientCertificates.Add(certificate); //Cert attached to the request

using (var res = client.GetResponse() as HttpWebResponse)
{
    using (var responseStream = res.GetResponseStream())
    {
         using (var reader = new StreamReader(responseStream))
         {
             string s = reader.ReadToEnd();
             ShowAndWriteToFile(s);
         }
    }
}

I looked at Verbose logging and Wireshark captures, Couldn't find the problem from there.

Other stuff I tried:

1. Clearing pin code cache – when clearing it before removing token, then on next attempt to connect to server the “Token logon” window pops up. Clearing it after remove and re-insert didn’t change anything.
2. Restarting the WCF Service – after the token re-inserted, still same result.
3. Checking how other operations on the certificate behaves before and after token remove and re-insert. For instance, if I Sign and verify some message with the certificate after the removal and re-insert of the token then the “Token logon” window pops up again and it continues working.
4. Trying to connect with self-created certificate put in the store – Worked, then deleting it from store and re-adding it – still worked.