Iot Edge transparent gateway + downstream device: Connection error to Iot Hub:TLS authentication error

I'm using Iot Edge transparent gateway + downstream device and I got this error: Connection error to Iot Hub:TLS authentication error

I follow this tutorial: https://docs.microsoft.com/en-ca/azure/iot-edge/how-to-create-transparent-gateway

My edge and downstream devices are on the same computer, following this scenario:

I deployed a monitoring module that loop through downstream devices to save their sensors in IoT Hub is working (Connection string without GatewayHostName).

But my problem, after appending "GatewayHostName=ADNM-CONEKEO" to the connection string to extend offline operation, the datas didn't send to Iot Hub and I got this error: Connection error to Iot Hub:TLS authentication error

Here what I did: 1) I added the devices in Manage child devices of the Edge device. 2) I generated the certificates and edit my config.yaml

certificates:
  device_ca_cert: "C:\\ADNM\\IoTEdge\\ssl\\certs\\new-edge-device-full-chain.cert.pem"
  device_ca_pk: "C:\\ADNM\\IoTEdge\\ssl\\private\\new-edge-device.key.pem"
  trusted_ca_certs: "C:\\ADNM\\IoTEdge\\ssl\\certs\\azure-iot-test-only.root.ca.cert.pem"

3) I added in Manage computer certificates/Trusted Root Certification Authorities the azure-iot-test-only.root.ca.cert.pem.

Route in my edge device: "route": "FROM /* INTO $upstream"

Monitoring Module installed in Edge device looping through one downstream device code sample ( I got 10 devices to monitor):

string connectionString = "HostName=xxxx.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxx;DeviceId=xxx;GatewayHostName=ADNM-CONEKEO"

// Connect to the IoT hub using the MQTT protocol
_DeviceClient = DeviceClient.CreateFromConnectionString(connectionString, TransportType.Mqtt);
_DeviceClient.OperationTimeoutInMilliseconds = 5000;

var retryPolicy = new ExponentialBackoff(
               5,
               TimeSpan.FromSeconds(2),
               TimeSpan.FromSeconds(60),
               TimeSpan.FromSeconds(4));

_DeviceClient.SetRetryPolicy(retryPolicy);

 try
  {           
   string messageString = null;
   Microsoft.Azure.Devices.Client.Message message = null;

    messageString = JsonConvert.SerializeObject(telemetryDataPoint);
    message = new Microsoft.Azure.Devices.Client.Message(Encoding.ASCII.GetBytes(messageString));

    // Send the telemetry message                                 
   Console.WriteLine("\n*[{0}]{1} > Sending message: {2}", equipment.DeviceId, DateTime.Now, messageString);
   await _DeviceClient.SendEventAsync(message).ConfigureAwait(false);
   Console.WriteLine("Message sent!");

   }
  catch (Exception e)
   {
    Console.WriteLine("Message not sent. Connection error to Iot Hub:" + e.Message);
   }

In my logs:

3/12/2019 8:53:11 AM info: iotedged -- Finished configuring certificates.
3/12/2019 8:53:11 AM info: iotedged -- Initializing hsm...
3/12/2019 8:53:11 AM info: iotedged -- Configuring the trusted CA certificates using
                     "C:\\ADNM\\IoTEdge\\ssl\\certs\\azure-iot-test-only.root.ca.cert.pem".
3/12/2019 8:53:11 AM info: iotedged -- Configuring the Device CA certificate using
                     "C:\\ADNM\\IoTEdge\\ssl\\certs\\new-edge-device-full-chain.cert.pem".
3/12/2019 8:53:11 AM info: iotedged -- Configuring the Device private key using
                     "C:\\ADNM\\IoTEdge\\ssl\\private\\new-edge-device.key.pem".
3/12/2019 8:53:11 AM info: iotedged -- Detecting if configuration file has changed...

when I'm running this script openssl s_client -connect ADNM-CONEKEO:8883 -CAfile C:/ADNM/IoTEdge/ssl/certs/azure-iot-test-only.root.ca.cert.pem -showcerts

Here the result of my test certificate:

CONNECTED(00000048)
depth=4 CN = Azure IoT CA TestOnly Root CA
verify return:1
depth=3 CN = Azure IoT CA TestOnly Intermediate 1 CA
verify return:1
depth=2 CN = ADNM-CONEKEO.ca
verify return:1
depth=1 CN = iotedged workload ca
verify return:1
depth=0 CN = adnm-conekeo
verify return:1
16760:error:89070063:lib(137):CAPI_RSA_SIGN:cant create hash object:e_capi.c:858:Error code= 0x80090008
16760:error:14099006:SSL routines:ssl3_send_client_verify:EVP lib:s3_clnt.c:3271:
---
Certificate chain
 0 s:/CN=adnm-conekeo
   i:/CN=iotedged workload ca
-----BEGIN CERTIFICATE-----
MIIEazCCAlOgAwIBAgIBKTANBgkqhkiG9w0BAQsFADAfMR0wGwYDVQQDDBRpb3Rl
ZGdlZCB3b3JrbG9hZCBjYTAeFw0xOTAzMTIxMjUzMjRaFw0xOTA0MTExMjQ4MzBa
MBcxFTATBgNVBAMMDGFkbm0tY29uZWtlbzCCASIwDQYJKoZIhvcNAQEBBQADggEP
ADCCAQoCggEBAOHUmdNX9JDSDsxemcoSi49FhmalkBsVcG+O/L/pHk+9fTlpVM7U
FWRqSeMBlhFB5OJC4cV/thxD3hu9MakdA2ToSnVTYpnSYyBCaZdPCuMwvR+C71HW
9jB879Ta7hCAfBhwO+DGlcBQUdm27rM02lXoMiBOn2pwEdnwumYq7d5ZZUrgpY5i
gld9LDPlO8FZQRfKK6CiZdcSplpgNp4hWwBCRtdjjDz8utN0/noMUDHAshSSfrtH
+6b6sAYgRV2UGOUCXOz8MpAFHSgR7K2x6QN3bmHwBIsa2LyErIXzlg/MuFDVrMku
aifUAbtkX3nvxz5Ug41oAm03NARayRFvmD0CAwEAAaOBuTCBtjAJBgNVHRMEAjAA
MA4GA1UdDwEB/wQEAwID+DATBgNVHSUEDDAKBggrBgEFBQcDATAgBgNVHREEGTAX
ggxhZG5tLWNvbmVrZW+CB2VkZ2VodWIwHQYDVR0OBBYEFAc29idSjGRISaPFO/zk
mLs/RLLjMEMGA1UdIwQ8MDqAFIsxOxaRQcZYYd6J9ltp/zZPqx9yoR6kHDAaMRgw
FgYDVQQDDA9BRE5NLUNPTkVLRU8uY2GCAndRMA0GCSqGSIb3DQEBCwUAA4ICAQCI
uOsOzhHndpeI2cnRjTZXUtZ/3sB3E1NDO03TF9iyj93nk5VAT2tOTFV8PxCJAizd
TkWy8amPqsuOsq6/QwrnjUoMCoW1a64VRjcesiQXCG9MaM9CeyJHu/pVbGnnchtt
JBY0y1zf5ZF3DCtnk0syRj8/ER+TpqTGVGNdP9APNK5j3ld+Q080KyAa1h5q8Sq5
PevXxMjbRp3UG71dzjFlRaNHaTZRTtrfrzgzOjUYk23H6ODOXhRIHyId2OJkAZ/u
4W4BmT4s39vhfZiNnzAA4hNhWr8IzdpznUR/wvzoyBlwn/MSEFUlXd4hTFLt0vs0
rRoyKgDSj2jXhl/6xkLSQdlrIBDbdDz4en6pQEoNb5jTtmWu8HPKAOuuFUvXM7ZG
T4PMuC1ezS80Xg52HDzqxprqx9EDCSSbUdv+AhjtRvF1+/IcH3XXDQU17WHj4BVn
qeakUFyeDuq9kl71r/PfLefFaHE4qTa8ObbxSq/52x1S/RKO9scAi7/8+qlX9/GF
sGRP8KBovq6VEbCd1MHRYR7cER0CWpg3zBK3I8e/AanlCdetj1ZVW4lphXpZV7CD
kQN2vnv70Ni1jCZu7jlQWaUi/yR/GiVe/lpiXISweHmFFRoT3vCToBtxHfpeB0bx
pCEk3kzLqA7mLvyD2YpW5H2r+sVgFD9zwwY9/8SMDg==
-----END CERTIFICATE-----
 1 s:/CN=iotedged workload ca
   i:/CN=ADNM-CONEKEO.ca
-----BEGIN CERTIFICATE-----
MIIEZzCCA0+gAwIBAgICd1EwDQYJKoZIhvcNAQELBQAwGjEYMBYGA1UEAwwPQURO
TS1DT05FS0VPLmNhMB4XDTE5MDMxMjEyNTMxNFoXDTE5MDQxMTEyNDgzMFowHzEd
MBsGA1UEAwwUaW90ZWRnZWQgd29ya2xvYWQgY2EwggIiMA0GCSqGSIb3DQEBAQUA
A4ICDwAwggIKAoICAQCqoaoBLosYcrRr/gaR49IWnLwCB7r+6v+i1Zez4tt/MKNI
vF+3UPxp5v5KDG2Rz6PZveVT4SaLtRuWxaB4jOTw+0woiv/CQ2BRJkrCqp4E2bUy
QHIVjVJxZ2TzRpe967mjT5dkr9cip6aiNw13BZa3M0BIHq0E3gcAkx+ynyxJ28PZ
soqEM9+R9zlKMZiG5nAhIi4cPKGli+FzjZquwBkWC5hKm89SUI9W4y/jERwhPjJ/
foM6WkBUqQ6aQAXVaoDcZxMVzgzj3QfN3Ed4ERmlJeSoq82hkvJl9i0SDmdZMcK4
3jRhWOcpsLeacMJi7oWHGVq31rvR3S3CRxbPW2gFpX4Q9OfzQ7+c7I9F0hrMmNKA
i/12oZhJ9BIiwtHSz5hh1/JGU1XCn2ibZT+lDHWpGeHq7wYP8VH5t6HtDx5v1rnC
jwm8DSJTAgaoGXggorKRUzp8c7xG6XiPzhae9ODjuFkdbOgmse9h/yuCwyW1TnOo
Pz30gFSGUd/apPw2SMtJYt8fCd8c5ORRG7VqBF8pcmAKwQY7kElIq7TmFPxN9Y+9
RuapOp0p4hwK2c751cnxW4KdL9lT42JigwT8diUJ58fDjQIbwTusXW0aHv9WsIY1
+CM6Zsy27FIFrLpJR9orFX8P8mTtcJQy2syoQkiiPdsAPhhGiDyQkestOTfvmwID
AQABo4GxMIGuMBIGA1UdEwEB/wQIMAYBAQECAQAwDgYDVR0PAQH/BAQDAgKEMB0G
A1UdDgQWBBSLMTsWkUHGWGHeifZbaf82T6sfcjBpBgNVHSMEYjBggBSP3OqWR6Lx
9NhtQi2ttytPFiEAoqE2pDQwMjEwMC4GA1UEAwwnQXp1cmUgSW9UIENBIFRlc3RP
bmx5IEludGVybWVkaWF0ZSAxIENBghAYVJXxc+s6p0C+9bw2InZcMA0GCSqGSIb3
DQEBCwUAA4IBAQCU8EbYN5EE6BR3O16h/SsfAhyqbw+Rtu/8heRH2lGc6awuCU62
FjD/jFiuYfw4S6wc7Rqmoj+g7hi1urIIabqIy6Cc3zZiAbCKylhcFkeSSrspvdqH
KPEzDIc5ITM51llHQGPQ96AhohgCtWY/mqR/I8G1/sj5z3MARn5KfF6xibDsfrrx
hwx3eVrqdLVvUzA8KICu0nnAWlQEnhCxhDdOEQX1HeQdqoOugAWZ4X8AYg+ywxpX
ycKoujuA3HtJM7MHaQf/MzJIC4sPHE+Gp4kOQCRMdQCJI+rBdq4iOe0db8DuVKNP
pZEBz9casR3nKxxEuli8ddYjjKa+KIK0ygVw
-----END CERTIFICATE-----
 2 s:/CN=ADNM-CONEKEO.ca
   i:/CN=Azure IoT CA TestOnly Intermediate 1 CA
-----BEGIN CERTIFICATE-----
MIIDeTCCAmGgAwIBAgIQGFSV8XPrOqdAvvW8NiJ2XDANBgkqhkiG9w0BAQsFADAy
MTAwLgYDVQQDDCdBenVyZSBJb1QgQ0EgVGVzdE9ubHkgSW50ZXJtZWRpYXRlIDEg
Q0EwHhcNMTkwMzEyMTIzODMwWhcNMTkwNDExMTI0ODMwWjAaMRgwFgYDVQQDDA9B
RE5NLUNPTkVLRU8uY2EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDG
Gg7FnRlYha94ZUY8F/xBLch4TjWdq7nmvF1P4WrpzgPWbBbe2cHqoqxx6/w85xgK
AiTQxl+3dpJfmUnJnrchISb9IQFUu7SYtuhpgalnBnucjzL1shGfWVRPfucwBI7i
SzgN6KRf4a+w3RzW8rCseHAwyi9d1MPXHdepOhxSw/O5Xl0vj6Brgxh0JLtVvB9k
s56c3AANt/iJvH3Rvu/RHjQUxbGJ+tqBNBrWaCHibYjmX+zZsTIfLhKP/Bmj3B7+
Ea0xPweVXIjAiO6y2OVsli7Nx8bGLj+yWBr9MMaetHGfBYhKTC7qUfAC6x3RlT7/
o4D5igQN1teIC14AnK1NAgMBAAGjgaIwgZ8wDgYDVR0PAQH/BAQDAgIEMB0GA1Ud
JQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATAaBgNVHREEEzARgg9BRE5NLUNPTkVL
RU8uY2EwEgYDVR0TAQH/BAgwBgEB/wIBDDAfBgNVHSMEGDAWgBRjDxJuO4t/Ewur
ELJgAvI8Kq64wTAdBgNVHQ4EFgQUj9zqlkei8fTYbUItrbcrTxYhAKIwDQYJKoZI
hvcNAQELBQADggEBAKD1JRKnSr8erOFREuq65u9S5aHQlBIZzdVsmM6iAaFGEVDY
KwE/p0BprTWZaqwpurDXgYswkcNjJoeh2uZqJKrYn8RvFRdvDNTFNbTfeFWs+ENw
eIMXVZ4TO/KIc2uKVeD6WXa6muHbfLS423nzsj/ewx5aElp0ZnGKoj1uBdtG8YgH
0pJQQtRuEdqxZVMICqI/wBVI5v+q1o+Agwwz2dV6fv5AVuKu7M7r4hfCvLnfGmX/
TCkZifyvBuLjNw3yTQ/+hQWKvkHTmJWM2BhCs2PC6NvYhc/fjQbNP/bKNXmVUWvu
ecl9Pap65NJbXmdx/YQQA/oHqN4lKz1JBZolbTI=
-----END CERTIFICATE-----
 3 s:/CN=Azure IoT CA TestOnly Intermediate 1 CA
   i:/CN=Azure IoT CA TestOnly Root CA
-----BEGIN CERTIFICATE-----
MIIDnzCCAoegAwIBAgIQP4mhsKetPZhE1MU/bfW6kDANBgkqhkiG9w0BAQsFADAo
MSYwJAYDVQQDDB1BenVyZSBJb1QgQ0EgVGVzdE9ubHkgUm9vdCBDQTAeFw0xOTAz
MTIxMjM0MjdaFw0xOTA0MTExMjQ0MjdaMDIxMDAuBgNVBAMMJ0F6dXJlIElvVCBD
QSBUZXN0T25seSBJbnRlcm1lZGlhdGUgMSBDQTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBAMqWe5Pnpz/ffZN8zAJBoXa5n2ZmGwyXESfTOU2+aWaZUVTH
rrHt4onK4/iQM1esfG+W3WA30JXw8/wsBwkCtGSRspTThlk6tzofb0lqW7vx7bIz
+FldWH3rzwtiW3RHDnUWh1G97qgmUF2xXn1BLqTVz6DZGejF3kUnESoUmxXHrlgr
cNxjcutQxMToOjdJJLf7pjylVfBzKK7y/vYbMVzF6oWcNZJxrZxV6LQ503wbHcS9
TJhlRiDd5OhjmoIcdm2OdVGMjj1zLJZeSefPIrpR7YLLaMuid6UJ3AKXJKfJtgCH
jz8rnF6HDEylOS4NVTbbK9yzB4NonvhN8yauyWUCAwEAAaOBujCBtzAOBgNVHQ8B
Af8EBAMCAgQwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMDIGA1UdEQQr
MCmCJ0F6dXJlIElvVCBDQSBUZXN0T25seSBJbnRlcm1lZGlhdGUgMSBDQTASBgNV
HRMBAf8ECDAGAQH/AgEMMB8GA1UdIwQYMBaAFOcxTCVhPz33bQh3uye5Ha4xDjo7
MB0GA1UdDgQWBBRjDxJuO4t/EwurELJgAvI8Kq64wTANBgkqhkiG9w0BAQsFAAOC
AQEARNiFKGo6FXQB0Rj1wBOxFgToQCz56yTieXmCfTaXQfC0c2Ajda3cKU45wvRs
gp0p/yApyuqW/WxPyKWRp6ywC6KjNPa0RDSOTaEbpfzqoG2df6ULrwO6ENvSN1Lp
X0UFJo7IKUyVGOLYizIX4Jaz2muROu5MCkEeHo3JjO6gpGmaOgaazp6yLtp6SU3F
iYJFuLcIWxCWGGcHxcrEhDtgzyakha2W8FaX7hFG3IEH3PdC5zoEpyZZ/N1clHmV
wn5CUpEM5tRHSyRaUzo7lMlpuSEcJ/28jWPnL7vT2DaAZLW7KrQXkqVdOnMnP7gT
VuEaVbTcHAxpAQA2zUxoQglYHA==
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=adnm-conekeo
issuer=/CN=iotedged workload ca
---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA1:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA1:DSA+SHA1:RSA+SHA512:ECDSA+SHA512
Shared Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA1:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA1:DSA+SHA1:RSA+SHA512:ECDSA+SHA512
Peer signing digest: SHA256
Server Temp Key: ECDH, P-384, 384 bits
---
SSL handshake has read 4594 bytes and written 1051 bytes
---
New, (NONE), Cipher is (NONE)
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 51300000E0B5BF00301BC59D91312B22165CC2B8B66A1074C6A998146E344788
    Session-ID-ctx:
    Master-Key: 34C233CA277E6BE03E469C200F41F2E42580B97C56D0C231C4C1FE3B850E45218BA5EB634F0FDEECB2869D314224F226
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1552395656
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)

---

And when I test this sample code https://github.com/Azure/iotedge/tree/master/samples/dotnet/EdgeDownstreamDevice

I got these errors: Cbs message missing audience property.

Unhandled Exception: System.AggregateException: One or more errors occurred. (Put token failed. status-code: 400, status-description: Cbs message missing audience property.) ---> 
Microsoft.Azure.Devices.Client.Exceptions.IotHubException: Put token failed. status-code: 400, status-description: Cbs message missing audience property.

Any idea? Is it the right way to implement the offline mode for my use case?

I open a issue ticket here also: https://github.com/Azure/iotedge/issues/937

Device (Host) Operating System

Windows 10

Container Operating System

Windows containers

Runtime Versions

iotedged

iotedge 1.0.6.1 (3fa6cbef8b7fc3c55a49a622735eb1021b8a5963)

Edge Agent

1.0.6.19913336

Edge Hub

1.0.6.19913336

Docker

Community 18.09.2