GPG secret key regularly unknown to my system

For a couple of days now at certain times my notebook does not know about my GPG secret key anymore. I have it saved on a Yubikey and use it to decrypt my passwords, as well as signing and encrypting my emails.

So sometimes, when I try to decrypt my passwords, I get presented with this:

user@notebook [~]
-> % pass -c path/to/my/credentials
gpg: decryption failed: No secret key
user@notebook [~]
-> %

When this first happened, I thought I lost the key for real, so I re-imported it by going to the edit mode of the card and fetching it again:

user@notebook [~]
-> % gpg --card-edit
...
gpg/card> fetch
gpg: requesting key from '<REDACTED>'
gpg: key <REDACTED> not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

gpg/card> quit
user@notebook [~]
-> % pass -c path/to/my/credentials
Copied path/to/my/credentials to clipboard. Will clear in 45 seconds.
user@notebook [~]
-> %

What threw me off the first time already was the key ... not changed. But it worked and I was able to use my secret key again.

When this happened a couple of more times in the following days, I found out that doing the gpg --card-edit and immediately quitting again does the trick already!

This happens consistently after booting up the machine and in general after plugging the Yubikey in. But it also happens randomly throughout the day without me (actively) unplugging the key. I have not yet found any specific reason why it does that, especially as I'm not constantly using the key and only see it on usage. And most of the time I have no idea then, how long it was not working already.

One idea I had was, that for some reason the agent crashes and is then restarted when I interact with the gpg binary. But I have nothing to proof that yet.

I unplugged the key and plugged it back in and afterwards had a look at the logs from the gpg-agent:

• 16:38:11 - unplugged the key
• 16:38:17 - plugged in the key and unsuccessful pass -c path/to/my/credentials
• 16:38:20 - gpg --card-edit + quit
• 16:38:24 - successful pass -c path/to/my/credentials

Logs of that time from journalctl --user -u gpg-agent:

Dec 14 16:38:11 notebook gpg-agent[1871]: scdaemon[1871]: pcsc_get_status_change failed: unknown reader (0x80100009)
Dec 14 16:38:11 notebook gpg-agent[1871]: scdaemon[1871]: DBG: Removal of a card: 0
Dec 14 16:38:11 notebook gpg-agent[1871]: scdaemon[1871]: pcsc_disconnect failed: invalid value (0x80100011)
Dec 14 16:38:17 notebook gpg-agent[1871]: scdaemon[1871]: detected reader 'Yubico YubiKey <REDACTED>'
Dec 14 16:38:17 notebook gpg-agent[1869]: detected card with S/N <REDACTED>
Dec 14 16:38:17 notebook gpg-agent[1871]: scdaemon[1871]: sending signal 12 to client 1869
Dec 14 16:38:17 notebook gpg-agent[1871]: scdaemon[1871]: app_decipher failed: Invalid ID
Dec 14 16:38:17 notebook gpg-agent[1869]: smartcard decryption failed: Invalid ID
Dec 14 16:38:17 notebook gpg-agent[1869]: command 'PKDECRYPT' failed: Invalid ID <SCD>
Dec 14 16:38:20 notebook gpg-agent[1869]: card has S/N: <REDACTED>
Dec 14 16:38:24 notebook gpg-agent[1869]: detected card with S/N <REDACTED>
Dec 14 16:38:24 notebook gpg-agent[1871]: [103B blob data]

But, tbh, I have no idea what to make of this. On my last notebook I had the issue, that my key was not found at all and I had to restart the pcscd service every now and then (iirc because another service was claiming or blocking the key). Maybe this is somewhat related?

Thanks in advance!