Improve the speed of the cipher mode code

* cipher/bufhelp.h (buf_cpy): New.
(buf_xor, buf_xor_2dst): If buffers unaligned, always jump to per-byte
processing.
(buf_xor_n_copy_2): New.
(buf_xor_n_copy): Use 'buf_xor_n_copy_2'.
* cipher/blowfish.c (_gcry_blowfish_cbc_dec): Avoid extra memory copy
and use new 'buf_xor_n_copy_2'.
* cipher/camellia-glue.c (_gcry_camellia_cbc_dec): Ditto.
* cipher/cast5.c (_gcry_cast_cbc_dec): Ditto.
* cipher/serpent.c (_gcry_serpent_cbc_dec): Ditto.
* cipher/twofish.c (_gcry_twofish_cbc_dec): Ditto.
* cipher/rijndael.c (_gcry_aes_cbc_dec): Ditto.
(do_encrypt, do_decrypt): Use 'buf_cpy' instead of 'memcpy'.
(_gcry_aes_cbc_enc): Avoid copying IV, use 'last_iv' pointer instead.
* cipher/cipher-cbc.c (_gcry_cipher_cbc_encrypt): Avoid copying IV,
update pointer to IV instead.
(_gcry_cipher_cbc_decrypt): Avoid extra memory copy and use new
'buf_xor_n_copy_2'.
(_gcry_cipher_cbc_encrypt, _gcry_cipher_cbc_decrypt): Avoid extra
accesses to c->spec, use 'buf_cpy' instead of memcpy.
* cipher/cipher-ccm.c (do_cbc_mac): Ditto.
* cipher/cipher-cfb.c (_gcry_cipher_cfb_encrypt)
(_gcry_cipher_cfb_decrypt): Ditto.
* cipher/cipher-ctr.c (_gcry_cipher_ctr_encrypt): Ditto.
* cipher/cipher-ofb.c (_gcry_cipher_ofb_encrypt)
(_gcry_cipher_ofb_decrypt): Ditto.
* cipher/cipher.c (do_ecb_encrypt, do_ecb_decrypt): Ditto.
--

Patch improves the speed of the generic block cipher mode code. Especially on
targets without faster unaligned memory accesses, the generic code was slower
than the algorithm specific bulk versions. With this patch, this issue should
be solved.

Tests on Cortex-A8; compiled for ARMv4, without unaligned-accesses:

 Before:
                  ECB/Stream         CBC             CFB             OFB             CTR             CCM
               --------------- --------------- --------------- --------------- --------------- ---------------
  SEED           490ms   500ms   560ms   580ms   530ms   540ms   560ms   560ms   550ms   540ms  1080ms  1080ms
  TWOFISH        230ms   230ms   290ms   300ms   260ms   240ms   290ms   290ms   240ms   240ms   520ms   510ms
  DES            720ms   720ms   800ms   860ms   770ms   770ms   810ms   820ms   770ms   780ms       -       -
  CAST5          340ms   340ms   440ms   250ms   390ms   250ms   440ms   430ms   260ms   250ms       -       -

 After:
                  ECB/Stream         CBC             CFB             OFB             CTR             CCM
               --------------- --------------- --------------- --------------- --------------- ---------------
  SEED           500ms   490ms   520ms   520ms   530ms   520ms   530ms   540ms   500ms   520ms  1060ms  1070ms
  TWOFISH        230ms   220ms   250ms   230ms   260ms   230ms   260ms   260ms   230ms   230ms   500ms   490ms
  DES            720ms   720ms   750ms   760ms   740ms   750ms   770ms   770ms   760ms   760ms       -       -
  CAST5          340ms   340ms   370ms   250ms   370ms   250ms   380ms   390ms   250ms   250ms       -       -

Tests on Cortex-A8; compiled for ARMv7-A, with unaligned-accesses:

 Before:
                  ECB/Stream         CBC             CFB             OFB             CTR             CCM
               --------------- --------------- --------------- --------------- --------------- ---------------
  SEED           430ms   440ms   480ms   530ms   470ms   460ms   490ms   480ms   470ms   460ms   930ms   940ms
  TWOFISH        220ms   220ms   250ms   230ms   240ms   230ms   270ms   250ms   230ms   240ms   480ms   470ms
  DES            550ms   540ms   620ms   690ms   570ms   540ms   630ms   650ms   590ms   580ms       -       -
  CAST5          300ms   300ms   380ms   230ms   330ms   230ms   380ms   370ms   230ms   230ms       -       -

 After:
                  ECB/Stream         CBC             CFB             OFB             CTR             CCM
               --------------- --------------- --------------- --------------- --------------- ---------------
  SEED           430ms   430ms   460ms   450ms   460ms   450ms   470ms   470ms   460ms   470ms   900ms   930ms
  TWOFISH        220ms   210ms   240ms   230ms   230ms   230ms   250ms   250ms   230ms   230ms   470ms   470ms
  DES            540ms   540ms   580ms   570ms   570ms   570ms   560ms   620ms   580ms   570ms       -       -
  CAST5          300ms   290ms   310ms   230ms   320ms   230ms   350ms   350ms   230ms   230ms       -       -

Tests on Intel Atom N160 (i386):

 Before:
                  ECB/Stream         CBC             CFB             OFB             CTR             CCM
               --------------- --------------- --------------- --------------- --------------- ---------------
  SEED           380ms   380ms   410ms   420ms   400ms   400ms   410ms   410ms   390ms   400ms   820ms   800ms
  TWOFISH        340ms   340ms   370ms   350ms   360ms   340ms   370ms   370ms   330ms   340ms   710ms   700ms
  DES            660ms   650ms   710ms   740ms   680ms   700ms   700ms   710ms   680ms   680ms       -       -
  CAST5          340ms   340ms   380ms   330ms   360ms   330ms   390ms   390ms   320ms   330ms       -       -

 After:
                  ECB/Stream         CBC             CFB             OFB             CTR             CCM
               --------------- --------------- --------------- --------------- --------------- ---------------
  SEED           380ms   380ms   390ms   410ms   400ms   390ms   410ms   400ms   400ms   390ms   810ms   800ms
  TWOFISH        330ms   340ms   350ms   360ms   350ms   340ms   380ms   370ms   340ms   360ms   700ms   710ms
  DES            630ms   640ms   660ms   690ms   680ms   680ms   700ms   690ms   680ms   680ms       -       -
  CAST5          340ms   330ms   350ms   330ms   370ms   340ms   380ms   390ms   330ms   330ms       -       -

Tests in Intel i5-4570 (x86-64):

  Before:
                  ECB/Stream         CBC             CFB             OFB             CTR             CCM
               --------------- --------------- --------------- --------------- --------------- ---------------
  SEED           560ms   560ms   600ms   590ms   600ms   570ms   570ms   570ms   580ms   590ms  1200ms  1180ms
  TWOFISH        240ms   240ms   270ms   160ms   260ms   160ms   250ms   250ms   160ms   160ms   430ms   430ms
  DES            570ms   570ms   640ms   590ms   630ms   580ms   600ms   600ms   610ms   620ms       -       -
  CAST5          410ms   410ms   470ms   150ms   470ms   150ms   450ms   450ms   150ms   160ms       -       -

  After:
                  ECB/Stream         CBC             CFB             OFB             CTR             CCM
               --------------- --------------- --------------- --------------- --------------- ---------------
  SEED           560ms   560ms   590ms   570ms   580ms   570ms   570ms   570ms   590ms   590ms  1200ms  1200ms
  TWOFISH        240ms   240ms   260ms   160ms   250ms   170ms   250ms   250ms   160ms   160ms   430ms   430ms
  DES            570ms   570ms   620ms   580ms   630ms   570ms   600ms   590ms   620ms   620ms       -       -
  CAST5          410ms   410ms   460ms   150ms   460ms   160ms   450ms   450ms   150ms   150ms       -       -


Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>

1 files changed, 6 insertions, 4 deletions

diff --git a/cipher/cipher.c b/cipher/cipher.c
index 5214d26c..c0d1d0be 100644
--- a/cipher/cipher.c
+++ b/cipher/cipher.c
@@ -631,6 +631,7 @@ do_ecb_encrypt (gcry_cipher_hd_t c,
                 unsigned char *outbuf, unsigned int outbuflen,
                 const unsigned char *inbuf, unsigned int inbuflen)
 {
+  gcry_cipher_encrypt_t enc_fn = c->spec->encrypt;
   unsigned int blocksize = c->spec->blocksize;
   unsigned int n, nblocks;
   unsigned int burn, nburn;
@@ -640,12 +641,12 @@ do_ecb_encrypt (gcry_cipher_hd_t c,
   if ((inbuflen % blocksize))
     return GPG_ERR_INV_LENGTH;
 
-  nblocks = inbuflen / c->spec->blocksize;
+  nblocks = inbuflen / blocksize;
   burn = 0;
 
   for (n=0; n < nblocks; n++ )
     {
-      nburn = c->spec->encrypt (&c->context.c, outbuf, (byte*)/*arggg*/inbuf);
+      nburn = enc_fn (&c->context.c, outbuf, (byte*)/*arggg*/inbuf);
       burn = nburn > burn ? nburn : burn;
       inbuf  += blocksize;
       outbuf += blocksize;
@@ -662,6 +663,7 @@ do_ecb_decrypt (gcry_cipher_hd_t c,
                 unsigned char *outbuf, unsigned int outbuflen,
                 const unsigned char *inbuf, unsigned int inbuflen)
 {
+  gcry_cipher_decrypt_t dec_fn = c->spec->decrypt;
   unsigned int blocksize = c->spec->blocksize;
   unsigned int n, nblocks;
   unsigned int burn, nburn;
@@ -671,12 +673,12 @@ do_ecb_decrypt (gcry_cipher_hd_t c,
   if ((inbuflen % blocksize))
     return GPG_ERR_INV_LENGTH;
 
-  nblocks = inbuflen / c->spec->blocksize;
+  nblocks = inbuflen / blocksize;
   burn = 0;
 
   for (n=0; n < nblocks; n++ )
     {
-      nburn = c->spec->decrypt (&c->context.c, outbuf, (byte*)/*arggg*/inbuf);
+      nburn = dec_fn (&c->context.c, outbuf, (byte*)/*arggg*/inbuf);
       burn = nburn > burn ? nburn : burn;
       inbuf  += blocksize;
       outbuf += blocksize;