Java SSLSocket/ServerSocket: Handshake failed when forcing TLSv1.2 on Android

Basically I am using SSLSocket to connect from an Android device to a SSLServerSocket for transmitting application data. I got myself a cert from LetsEncrypt and combined fullchain.pem with privkey.pem to be able to follow this tutorial on how to convert to JKS.

After some tinkering I came across this Exception which I am not able to fix:

Android Log:

2020-07-24 17:10:00.807 2440-2512/at.ls.bikewithme I/System.out: [TLSv1, TLSv1.1, TLSv1.2, TLSv1.3]
2020-07-24 17:10:01.132 2440-2512/at.ls.bikewithme V/NativeCrypto: SSL handshake aborted: ssl=0x738d61bfc8: Failure in SSL library, usually a protocol error
    error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE (third_party/openssl/boringssl/src/ssl/tls_record.cc:592 0x738d656088:0x00000001)
2020-07-24 17:10:01.136 2440-2512/at.ls.bikewithme W/System.err: javax.net.ssl.SSLHandshakeException: Handshake failed
2020-07-24 17:10:01.137 2440-2512/at.ls.bikewithme W/System.err:     at com.google.android.gms.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(:com.google.android.gms@202414019@20.24.14 (040400-319035315):35)
2020-07-24 17:10:01.137 2440-2512/at.ls.bikewithme W/System.err:     at me.ls.client.SecureClient.run(SecureClient.java:129)
2020-07-24 17:10:01.137 2440-2512/at.ls.bikewithme W/System.err:     at me.ls.client.SecureClient.retryConnection(SecureClient.java:151)
2020-07-24 17:10:01.137 2440-2512/at.ls.bikewithme W/System.err:     at me.ls.client.SecureClient.run(SecureClient.java:80)
2020-07-24 17:10:01.137 2440-2512/at.ls.bikewithme W/System.err:     at me.ls.client.SecureClient.retryConnection(SecureClient.java:151)
2020-07-24 17:10:01.137 2440-2512/at.ls.bikewithme W/System.err:     at me.ls.client.SecureClient.run(SecureClient.java:80)
2020-07-24 17:10:01.137 2440-2512/at.ls.bikewithme W/System.err:     at me.ls.client.SecureClient.retryConnection(SecureClient.java:151)
2020-07-24 17:10:01.137 2440-2512/at.ls.bikewithme W/System.err:     at me.ls.client.SecureClient.run(SecureClient.java:80)
2020-07-24 17:10:01.138 2440-2512/at.ls.bikewithme W/System.err: Caused by: javax.net.ssl.SSLProtocolException: SSL handshake aborted: ssl=0x738d61bfc8: Failure in SSL library, usually a protocol error
2020-07-24 17:10:01.138 2440-2512/at.ls.bikewithme W/System.err: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE (third_party/openssl/boringssl/src/ssl/tls_record.cc:592 0x738d656088:0x00000001)
2020-07-24 17:10:01.138 2440-2512/at.ls.bikewithme W/System.err:     at com.google.android.gms.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
2020-07-24 17:10:01.138 2440-2512/at.ls.bikewithme W/System.err:     at com.google.android.gms.org.conscrypt.NativeSsl.doHandshake(:com.google.android.gms@202414019@20.24.14 (040400-319035315):6)
2020-07-24 17:10:01.138 2440-2512/at.ls.bikewithme W/System.err:     at com.google.android.gms.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(:com.google.android.gms@202414019@20.24.14 (040400-319035315):16)
2020-07-24 17:10:01.138 2440-2512/at.ls.bikewithme W/System.err:    ... 7 more

Server Log:

javax.net.ssl.SSLHandshakeException: Empty server certificate chain
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:311)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
    at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:381)
    at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:366)
    at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422)
    at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:181)
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:167)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1462)
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1370)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:437)
    at me.ls.server.SecureServer.run(SecureServer.java:89)

Client Code:

    SSLContext context = null;
    try {
        context = SSLContext.getInstance("TLSv1.2");
        context.init(null,null,null);
    } catch (NoSuchAlgorithmException | KeyManagementException e) {
        e.printStackTrace();
    }
    if(context == null) return;
    SSLSocketFactory factory = context.getSocketFactory();

    try {
        socket = (SSLSocket) factory.createSocket(address.getHostString(), address.getPort());
        System.out.println(Arrays.toString(socket.getSupportedProtocols()));
        socket.setUseClientMode(true);
    } catch (IOException ioException) {
        if (running) {
            retryConnection(2000);
        }
        return;
    }
    //Handshake

Server Code:

        KeyStore ks = KeyStore.getInstance("JKS");
        InputStream ksIs = new FileInputStream(keystoreFile);
        try {
            ks.load(ksIs, keystorePass.toCharArray());
        } finally {
            ksIs.close();
        }

        KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory
                .getDefaultAlgorithm());
        kmf.init(ks, keystorePass.toCharArray());

        SSLContext context = SSLContext.getInstance("TLSv1.2");
        context.init(kmf.getKeyManagers(),null,null);
        SSLServerSocketFactory factory = context.getServerSocketFactory();

        socket = (SSLServerSocket) factory.createServerSocket(address.getPort());
        socket.setNeedClientAuth(true);
        socket.setUseClientMode(false);
       
        //accept clients