CAPI AES: Why does CryptSetKeyParam returns NTE_BAD_DATA (0x80090005)

0

I am trying to encrypt/decrypt data using AES 128 bit with zero padding. I can encrypt/decrypt fine if I don't attempt to futz with the padding mode (defaults to PKCS5_PADDING), so the following snippet of code is to show where things are breaking down:

// sanity check -- is key configured how I expect?
CryptGetKeyParam(hKey, KP_KEYLEN, temp, &tempSz, 0);
CryptGetKeyParam(hKey, KP_ALGID, temp, &tempSz, 0);
CryptGetKeyParam(hKey, KP_MODE, temp, &tempSz, 0);
CryptGetKeyParam(hKey, KP_PADDING, temp, &tempSz, 0);
// force padding to zero padding
DWORD padding = ZERO_PADDING;
CryptSetKeyParam(hKey, KP_PADDING, (PBYTE)&padding, 0);

each of the queries operates without issue with values: 128, 0x0000660e (CALG_AES_128), 1 (CRYPT_MODE_CBC) and 1 (PKCS5_PADDING) -- that is the default state.

then I attempt to set the padding to ZERO_PADDING (3) and the function returns FALSE with a last error of NTE_BAD_DATA. I tried, for giggles, to set it to PKCS5_PADDING -- no error. RANDOM_PADDING: error.

searching the interwebs as I might, I cannot find any documentation to suggest why changing the padding should fail (not even a note saying only PKCS5_PADDING is allowed for certain key algorithms).

does anyone have any advice?

winapi
encryption
aes
padding
cryptoapi
asked on Stack Overflow Jan 14, 2016 by rguilbault • edited Jan 15, 2016 by rguilbault

1 Answer

2

The cryptographic service providers provided by Microsoft only support PKCS5_PADDING.

See https://msdn.microsoft.com/en-us/library/windows/desktop/aa380272(v=vs.85).aspx for reference; search for KP_PADDING.

answered on Stack Overflow Jan 15, 2016 by Collin Dauphinee

User contributions licensed under CC BY-SA 3.0