md: Add Whirlpool bug emulation feature.

* src/gcrypt.h.in (GCRY_MD_FLAG_BUGEMU1): New.
* src/cipher-proto.h (gcry_md_init_t): Add arg FLAGS.  Change all code
to implement that flag.
* cipher/md.c (gcry_md_context):  Replace SECURE and FINALIZED by bit
field FLAGS.  Add flag BUGEMU1.  Change all users.
(md_open): Replace args SECURE and HMAC by FLAGS.  Init flags.bugemu1.
(_gcry_md_open): Add for GCRY_MD_FLAG_BUGEMU1.
(md_enable): Pass bugemu1 flag to the hash init function.
(_gcry_md_reset): Ditto.
--

This problem is for example exhibited in the Linux cryptsetup tool.
See https://bbs.archlinux.org/viewtopic.php?id=175737 .  It has be
been tracked down by Milan Broz.

The suggested way of using the flag is:

  if (whirlpool_bug_assumed)
    {
#if GCRYPT_VERSION_NUMBER >= 0x010601
      err = gcry_md_open (&hd, GCRY_MD_WHIRLPOOL, GCRY_MD_FLAG_BUGEMU1)
      if (gpg_err_code (err) == GPG_ERR_INV_ARG)
         error ("Need at least Libggcrypt 1.6.1 for the fix");
      else
         {
            do_hash (hd);
            gcry_md_close (hd);
          }
#endif
    }

Signed-off-by: Werner Koch <wk@gnupg.org>

1 files changed, 16 insertions, 0 deletions

diff --git a/doc/gcrypt.texi b/doc/gcrypt.texi
index 7712b80a..4a917901 100644
--- a/doc/gcrypt.texi
+++ b/doc/gcrypt.texi
@@ -3111,6 +3111,22 @@ The size of the MAC is equal to the message digest of the underlying
 hash algorithm.  If you want CBC message authentication codes based on
 a cipher, see @xref{Working with cipher handles}.
 
+@item GCRY_MD_FLAG_BUGEMU1
+@cindex bug emulation
+Versions of Libgcrypt before 1.6.0 had a bug in the Whirlpool code
+which led to a wrong result for certain input sizes and write
+patterns.  Using this flag emulates that bug.  This may for example be
+useful for applications which use Whirlpool as part of their key
+generation.  It is strongly suggested to use this flag only if really
+needed and if possible to the data should be re-processed using the
+regular Whirlpool algorithm.
+
+Note that this flag works for the entire hash context.  If needed
+arises it may be used to enable bug emulation for other hash
+algorithms.  Thus you should not use this flag for a multi-algorithm
+hash context.
+
+
 @end table
 @c begin table of hash flags