The system cannot find the file specified. (Exception from HRESULT: 0x80070002) while removing an SSL certificate in Powershell

I have a PowerShell function that changes the SSL certificate of an IIS website if the thumbprint isn't matched. The code that does it, looks like this:

Get-WebBinding | ForEach-Object {
 if ($_.certificateHash -ne $CertPrint) {
  Write-Host "Working on"  $_ 
  $_.RemoveSslCertificate()
  $_.AddSslCertificate($CertPrint, 'My')
  }
}

In the above code, $CertPrint contains the thumbprint of the desired certificate.

I have an application that generates IIS websites and assigns the bindings that are needed. The code above matches the certificate thumbprint and assigns that certificate to the website. Now, I have no problem with this if the websites already exist. For example, if I have a bunch of IIS websites, I can run this code successfully and it replaces the certificate to match the one that I need. However, when the application starts and when the IIS sites are being created, it comes up with the following error:

The system cannot find the file specified. (Exception from HRESULT: 0x80070002)
At some/path/to/the/function.ps1:234 char:21
+                     $_.RemoveSslCertificate()
+                     ~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], FileNotFoundException
    + FullyQualifiedErrorId : System.IO.FileNotFoundException

What's causing this error and how can I fix this?