md: Add Whirlpool bug emulation feature.

* src/gcrypt.h.in (GCRY_MD_FLAG_BUGEMU1): New.
* src/cipher-proto.h (gcry_md_init_t): Add arg FLAGS.  Change all code
to implement that flag.
* cipher/md.c (gcry_md_context):  Replace SECURE and FINALIZED by bit
field FLAGS.  Add flag BUGEMU1.  Change all users.
(md_open): Replace args SECURE and HMAC by FLAGS.  Init flags.bugemu1.
(_gcry_md_open): Add for GCRY_MD_FLAG_BUGEMU1.
(md_enable): Pass bugemu1 flag to the hash init function.
(_gcry_md_reset): Ditto.
--

This problem is for example exhibited in the Linux cryptsetup tool.
See https://bbs.archlinux.org/viewtopic.php?id=175737 .  It has be
been tracked down by Milan Broz.

The suggested way of using the flag is:

  if (whirlpool_bug_assumed)
    {
#if GCRYPT_VERSION_NUMBER >= 0x010601
      err = gcry_md_open (&hd, GCRY_MD_WHIRLPOOL, GCRY_MD_FLAG_BUGEMU1)
      if (gpg_err_code (err) == GPG_ERR_INV_ARG)
         error ("Need at least Libggcrypt 1.6.1 for the fix");
      else
         {
            do_hash (hd);
            gcry_md_close (hd);
          }
#endif
    }

Signed-off-by: Werner Koch <wk@gnupg.org>

1 files changed, 11 insertions, 3 deletions

diff --git a/cipher/rmd160.c b/cipher/rmd160.c
index 224694f2..1a58ba61 100644
--- a/cipher/rmd160.c
+++ b/cipher/rmd160.c
@@ -143,11 +143,13 @@
 static unsigned int
 transform ( void *ctx, const unsigned char *data, size_t nblks );
 
-void
-_gcry_rmd160_init (void *context)
+static void
+rmd160_init (void *context, unsigned int flags)
 {
   RMD160_CONTEXT *hd = context;
 
+  (void)flags;
+
   hd->h0 = 0x67452301;
   hd->h1 = 0xEFCDAB89;
   hd->h2 = 0x98BADCFE;
@@ -162,6 +164,12 @@ _gcry_rmd160_init (void *context)
 }
 
 
+void
+_gcry_rmd160_init (void *context)
+{
+  rmd160_init (context, 0);
+}
+
 
 /****************
  * Transform the message X which consists of 16 32-bit-words
@@ -528,6 +536,6 @@ gcry_md_spec_t _gcry_digest_spec_rmd160 =
   {
     GCRY_MD_RMD160, {0, 0},
     "RIPEMD160", asn, DIM (asn), oid_spec_rmd160, 20,
-    _gcry_rmd160_init, _gcry_md_block_write, rmd160_final, rmd160_read,
+    rmd160_init, _gcry_md_block_write, rmd160_final, rmd160_read,
     sizeof (RMD160_CONTEXT)
   };