On Thu, 2021-07-08 at 14:25 +0200, Hans Leidekker wrote:
/* CNG_RSA_PUBLIC_KEY_BLOB always stores the exponent and modulus* in big-endian format, so we need to convert from the native* endianness*/CRYPT_memcpy_be((BYTE *)pvStructInfo + sizeof(BCRYPT_RSAKEY_BLOB),(BYTE *)&decodedKey->pubexp, sizeof(DWORD));CRYPT_memcpy_be((BYTE *)pvStructInfo + sizeof(BCRYPT_RSAKEY_BLOB) +sizeof(DWORD), decodedKey->modulus.pbData,decodedKey->modulus.cbData);Exponent and modulus are handled as a byte arrays so native endianess is not relevant there. They should be reversed unconditionally here as well as in the encoder and the tests.
Though in the test you use a DWORD for the exponent, so you would need to reverse that depending on host endianness, or you could store it as a byte array.