SSLSocket via another SSLSocket

So I tried to find out what goes wrong in case of android but so far I didn't find anything wrong with your code. Also since the code works for JRE, it asserts the hypothesis.

From the tcpdump you provided, there is substantial information to conclude how Android behaves with the same set of APIs as JRE.

Let's have a look at JRE tcpdump:

• See the initial handshake messages (Client Hello, Server Hello, change cipher spec). This shows the handshake between JRE client and proxy server. This is successful.
• Now we don't see the second handshake between JRE client and www.google.com (the end server) as that's encrypted as we are doing SSL over SSL. The proxy server is copying them bit by bit to the end server. So this is a correct behavior.

Now let's look at the android tcpdump:

• See the initial handshake messages (Client Hello, Server Hello, change cipher spec). This shows the handshake between android client and proxy server. This is successful.
• Now ideally we shouldn't see the second handshake as it should be encrypted. But here we can see that android client is sending a "client hello" and it is sending it to "www.google.com" even though the packet is sent to the proxy server.
• The above is bound to fail as the packet was supposed to be written over SSL socket and not over the initial plain socket. I reviewed your code and I see that you are doing the second handshake over SSLSocket and not over plain socket.

Analysis from proxy/stunnel wireshark:

JRE Case:

In case of JRE, the client does the initial SSL handshake with the stunnel/proxy server. The same can be seen below:

The handshake is successful and connection is done

Then the client tries to connect to remote server (www.google.com) and starts the handshake. So the client hello sent by client is seen as encrypted message in packet #34 and when stunnel decrypts the same, it is seen at "Client hello" which is forwarded to proxy server by stunnel

Now let's look at android client case.

Initial SSL handshake from client to stunnel/proxy is successful as seen above.

Then when android client starts the handshake with remote (www.google.com), ideally it should use SSL socket for the same. If this was the case, we should see encrypted traffic from android to stunnel (similar to packet #34 from JRE case), stunnel should decrypt and send "client hello" to proxy. however as you can see below, android client is sending a "client hello" over plain socket.

If you compare the packet #24 with packet #34 from JRE, we can spot this difference.

Conclusion:

This is a bug with android SSL (factory.createsocket() with SSL socket) implementation and I feel there may not be a magical workaround for the same using the same set of APIs. In fact I found this issue in android bug list. See below link: https://code.google.com/p/android/issues/detail?id=204159

This issue is still unresolved and you can probably follow-up with android dev team to fix the same.

Possible Solutions:

If we conclude that the same set of APIs cannot work then you are left with only one option:

1. Write your own SSL wrapper over the SSL socket. You can manually do handshake or use a third party implementation. This could take a while but looks like the only way.

Since HTTPS make sure no middle man interrupt the communication between the two. thats why you are unable to do so. so tha second handshake fails.

Here is the link might help.

HTTPS connections over proxy servers

I don't know if this helps but:
I've built your repo test environment, and my error is slightly different:

I/SurfaceTextureClient(20733): [0x52851b98] frames:2, duration:1.005000, fps:1.989805
I/System.out(20733): [socket][2] connection /192.168.1.100:10443;LocalPort=35380(0)
I/System.out(20733): [CDS]connect[/192.168.1.100:10443] tm:90
I/System.out(20733): [socket][/192.168.1.123:35380] connected
I/System.out(20733): Doing SSL handshake with 192.168.1.100:10443
I/System.out(20733): Supported protocols are: [SSLv3, TLSv1, TLSv1.1, TLSv1.2]
E/NativeCrypto(20733): ssl=0x53c96268 cert_verify_callback x509_store_ctx=0x542e0a80 arg=0x0
E/NativeCrypto(20733): ssl=0x53c96268 cert_verify_callback calling verifyCertificateChain authMethod=RSA
I/System.out(20733): Doing SSL handshake with 192.168.1.100:443
I/System.out(20733): Supported protocols are: [SSLv3, TLSv1, TLSv1.1, TLSv1.2]
E/NativeCrypto(20733): Unknown error during handshake
I/System.out(20733): Shutdown rx/tx
I/System.out(20733): [CDS]close[35380]
I/System.out(20733): close [socket][/0.0.0.0:35380]
W/System.err(20733): javax.net.ssl.SSLHandshakeException: javax.net.ssl.SSLProtocolException:
                        SSL handshake aborted: ssl=0x53c9c1d8:
                        Failure in SSL library, usually a protocol error
W/System.err(20733): error:140770FC:SSL routines:
                        SSL23_GET_SERVER_HELLO:
                        unknown protocol (external/openssl/ssl/s23_clnt.c:766 0x4e7cb3ad:0x00000000)
W/System.err(20733):    at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:413)
W/System.err(20733):    at com.bugreport.sslviassl.SecureWebProxyThread.doSSLHandshake(SecureWebProxyThread.java:147)
W/System.err(20733):    at com.bugreport.sslviassl.SecureWebProxyThread.run(SecureWebProxyThread.java:216)
W/System.err(20733): Caused by: javax.net.ssl.SSLProtocolException:
                        SSL handshake aborted: ssl=0x53c9c1d8:
                        Failure in SSL library, usually a protocol error
W/System.err(20733): error:140770FC:SSL routines:
    SSL23_GET_SERVER_HELLO:unknown protocol (external/openssl/ssl/s23_clnt.c:766 0x4e7cb3ad:0x00000000)
W/System.err(20733):    at org.apache.harmony.xnet.provider.jsse.NativeCrypto.SSL_do_handshake(Native Method)
W/System.err(20733):    at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:372)
W/System.err(20733):    ... 2 more
I/SurfaceTextureClient(20733): [0x52851b98] frames:5, duration:1.010000, fps:4.946089

One thought on threads: what thread are you running your doSSLHandshake() method on ?

public void policy()
{
    int SDK_INT = android.os.Build.VERSION.SDK_INT;
    if (SDK_INT > 8)
    {
        StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
        StrictMode.setThreadPolicy(policy);
    }
}

This indeed turned out to be a bug in the default SSL providers on any Android device I tested (my newest device is a Nexus 9 running Android 7.1.1 though). Eventually, I found that the engine-based SSLSocket implementation (freshly released at the time) of the standalone version of the Conscrypt SSL provider worked on Android in exactly the way I expected it to work.

For further details, have a look at the discussion with Conscrypt maintainers on GitHub: https://github.com/google/conscrypt/issues/104