Invalid Private Key for certificate stored on remote machine certificate store

On machine A I run a code that generates self signed certificate. When generated, I store created certificate in store on remote machine B ("\hostname\My" as a store name). Finally I try to bind this certificate to the specific port. When certificate generated on machine A, the correspond private key container file is created under C:\Users\All Users\Microsoft\Crypto\RSA\MachineKeys.

However, when certificate stored on machine B it appears in the store as expected with note "You have a private key that corresponds to this certificate" but NO correspond private key container file under C:\Users\All Users\Microsoft\Crypto\RSA\MachineKeys on machine B.

Moreover, trying to export this certificate, the option to export private key is grayed out. As a result, when trying to bind certificate I get the following error "(0x80004005) a specified logon session does not exist. it may already have been terminated" which is basically indicates that something wrong with private key.

When certificate generation run on the same machine (B) where certificate is stored everything works well. Both machines are in the same domain and all is done under domain administrator account.

Here is a code to generate self signed certificate using BouncyCastle

public static X509Certificate2 GenerateCertificate(string subject)
        {

            var random = new SecureRandom();
            var certificateGenerator = new X509V3CertificateGenerator();

            var serialNumber = BigIntegers.CreateRandomInRange(BigInteger.One, BigInteger.ValueOf(Int64.MaxValue), random);
            certificateGenerator.SetSerialNumber(serialNumber);

            certificateGenerator.SetIssuerDN(new X509Name($"C=NL, O=SomeCompany, CN={subject}"));
            certificateGenerator.SetSubjectDN(new X509Name($"C=NL, O=SomeCompany, CN={subject}"));
            certificateGenerator.SetNotBefore(DateTime.UtcNow.Date);
            certificateGenerator.SetNotAfter(DateTime.UtcNow.Date.AddYears(1));

            const int strength = 2048;
            var keyGenerationParameters = new KeyGenerationParameters(random, strength);
            var keyPairGenerator = new RsaKeyPairGenerator();
            keyPairGenerator.Init(keyGenerationParameters);

            var subjectKeyPair = keyPairGenerator.GenerateKeyPair();
            certificateGenerator.SetPublicKey(subjectKeyPair.Public);

            var issuerKeyPair = subjectKeyPair;
            const string signatureAlgorithm = "SHA256WithRSA";
            var signatureFactory = new Asn1SignatureFactory(signatureAlgorithm, issuerKeyPair.Private);
            var bouncyCert = certificateGenerator.Generate(signatureFactory);

            X509Certificate2 certificate;

            Pkcs12Store store = new Pkcs12StoreBuilder().Build();
            store.SetKeyEntry($"{subject}_key", new AsymmetricKeyEntry(subjectKeyPair.Private), new[] { new X509CertificateEntry(bouncyCert) });
            string exportpw = Guid.NewGuid().ToString("x");

            using (var ms = new System.IO.MemoryStream())
            {
                store.Save(ms, exportpw.ToCharArray(), random);
                certificate = new X509Certificate2(ms.ToArray(), exportpw, X509KeyStorageFlags.Exportable);
            }
            return certificate;
        }

And the just store it on remote machine:

var cert = GenerateCertificate("Test Cert");
string storename = $"\\\\{hostname}\\{StoreName.My.ToString()}";
var store = new X509Store(storename, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadWrite);
store.Add(cert);
store.Close();

Any ideas how to correctly generate certificate on machine A and then store it on remote machine B with valid private key container file will be very appreciated. Thanks