kernel-ark/crypto
Ard Biesheuvel db91af0fbe crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic
Instead of unconditionally forcing 4 byte alignment for all generic
chaining modes that rely on crypto_xor() or crypto_inc() (which may
result in unnecessary copying of data when the underlying hardware
can perform unaligned accesses efficiently), make those functions
deal with unaligned input explicitly, but only if the Kconfig symbol
HAVE_EFFICIENT_UNALIGNED_ACCESS is set. This will allow us to drop
the alignmasks from the CBC, CMAC, CTR, CTS, PCBC and SEQIV drivers.

For crypto_inc(), this simply involves making the 4-byte stride
conditional on HAVE_EFFICIENT_UNALIGNED_ACCESS being set, given that
it typically operates on 16 byte buffers.

For crypto_xor(), an algorithm is implemented that simply runs through
the input using the largest strides possible if unaligned accesses are
allowed. If they are not, an optimal sequence of memory accesses is
emitted that takes the relative alignment of the input buffers into
account, e.g., if the relative misalignment of dst and src is 4 bytes,
the entire xor operation will be completed using 4 byte loads and stores
(modulo unaligned bits at the start and end). Note that all expressions
involving misalign are simply eliminated by the compiler when
HAVE_EFFICIENT_UNALIGNED_ACCESS is defined.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-02-11 17:52:28 +08:00
..
asymmetric_keys Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2016-12-15 11:41:37 -08:00
async_tx async_pq_val: fix DMA memory leak 2016-10-05 06:18:09 +05:30
.gitignore crypto: rsa - add .gitignore for crypto/*.-asn1.[ch] files 2015-06-25 23:29:24 +08:00
842.c crypto: acomp - add support for 842 via scomp 2016-10-25 11:08:33 +08:00
ablk_helper.c crypto: ablk_helper - Fix cryptd reordering 2016-06-23 18:29:53 +08:00
ablkcipher.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
acompress.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
aead.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
aes_generic.c crypto: aes-generic - drop alignment requirement 2017-02-11 17:50:43 +08:00
aes_ti.c crypto: aes - add generic time invariant AES cipher 2017-02-11 17:50:43 +08:00
af_alg.c crypto: af_alg - Forbid bind(2) when nokey child sockets are present 2016-01-18 18:16:33 +08:00
ahash.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
akcipher.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
algapi.c crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic 2017-02-11 17:52:28 +08:00
algboss.c crypto: testmgr - Do not test internal algorithms 2016-11-28 21:23:20 +08:00
algif_aead.c crypto: algif_aead - Fix kernel panic on list_del 2017-02-03 17:45:48 +08:00
algif_hash.c crypto: algif_hash - avoid zero-sized array 2016-12-27 17:50:52 +08:00
algif_rng.c crypto: algif_rng - Remove obsolete const-removal cast 2015-04-22 09:30:21 +08:00
algif_skcipher.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2016-12-14 13:31:29 -08:00
ansi_cprng.c crypto: ansi_cprng - Convert to new rng interface 2015-04-22 09:30:18 +08:00
anubis.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
api.c crypto: api - Do not clear type bits in crypto_larval_lookup 2016-11-28 21:23:18 +08:00
arc4.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
authenc.c crypto: skcipher - Get rid of crypto_spawn_skcipher2() 2016-11-01 08:37:17 +08:00
authencesn.c crypto: skcipher - Get rid of crypto_spawn_skcipher2() 2016-11-01 08:37:17 +08:00
blkcipher.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
blowfish_common.c
blowfish_generic.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
camellia_generic.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
cast5_generic.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
cast6_generic.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
cast_common.c crypto: make tables used from assembler __visible 2013-08-14 20:42:03 +10:00
cbc.c crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic 2017-02-11 17:52:28 +08:00
ccm.c crypto: ccm - switch to separate cbcmac driver 2017-02-11 17:50:45 +08:00
chacha20_generic.c crypto: chacha20 - convert generic and x86 versions to skcipher 2016-12-27 17:47:31 +08:00
chacha20poly1305.c crypto: skcipher - Get rid of crypto_spawn_skcipher2() 2016-11-01 08:37:17 +08:00
cipher.c crypto: api - Remove no-op exit_ops code 2016-10-21 11:03:42 +08:00
cmac.c crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic 2017-02-11 17:52:28 +08:00
compress.c crypto: api - Remove no-op exit_ops code 2016-10-21 11:03:42 +08:00
crc32_generic.c crypto: crc32 - Rename generic implementation 2016-01-30 22:11:22 +08:00
crc32c_generic.c crypto: crc32c - Fix crc32c soft dependency 2016-01-19 15:52:10 +08:00
crct10dif_common.c crypto: crct10dif - Add fallback for broken initrds 2013-09-12 15:31:34 +10:00
crct10dif_generic.c crypto: squash lines for simple wrapper functions 2016-09-13 20:27:26 +08:00
cryptd.c crypto: cryptd - Add support for skcipher 2016-11-28 21:23:18 +08:00
crypto_engine.c crypto: engine - Handle the kthread worker using the new API 2016-10-25 11:08:25 +08:00
crypto_null.c crypto: null - Remove default null blkcipher 2016-07-18 17:35:44 +08:00
crypto_user.c crypto: acomp - add asynchronous compression api 2016-10-25 11:08:30 +08:00
crypto_wq.c crypto: crypto_wq - Fix late crypto work queue initialization 2014-03-21 21:54:28 +08:00
ctr.c crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic 2017-02-11 17:52:28 +08:00
cts.c crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic 2017-02-11 17:52:28 +08:00
deflate.c crypto: acomp - add support for deflate via scomp 2016-10-25 11:08:36 +08:00
des_generic.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
dh_helper.c crypto: dh - Add DH software implementation 2016-06-23 18:29:56 +08:00
dh.c crypto: dh - Consistenly return negative error codes 2016-11-13 17:45:04 +08:00
drbg.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2016-11-30 19:53:12 +08:00
ecb.c crypto: include crypto- module prefix in template 2014-11-26 20:06:30 +08:00
ecc_curve_defs.h crypto: ecdh - Add ECDH software support 2016-06-23 18:29:57 +08:00
ecc.c crypto: ecdh - make ecdh_shared_secret unique 2016-06-24 21:24:59 +08:00
ecc.h crypto: ecdh - make ecdh_shared_secret unique 2016-06-24 21:24:59 +08:00
ecdh_helper.c crypto: ecdh - Add ECDH software support 2016-06-23 18:29:57 +08:00
ecdh.c crypto: ecdh - make ecdh_shared_secret unique 2016-06-24 21:24:59 +08:00
echainiv.c crypto: echainiv - Replace chaining with multiplication 2016-09-13 18:44:57 +08:00
fcrypt.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
fips.c crypto: fips - Move fips_enabled sysctl into fips.c 2015-04-23 14:18:09 +08:00
gcm.c crypto: skcipher - Get rid of crypto_spawn_skcipher2() 2016-11-01 08:37:17 +08:00
gf128mul.c crypto: gf128mul - Zero memory when freeing multiplication table 2016-11-17 23:34:59 +08:00
ghash-generic.c crypto: ghash-generic - move common definitions to a new header file 2016-10-02 22:26:40 +08:00
hash_info.c keys, trusted: select hash algorithm for TPM2 chips 2015-12-20 15:27:12 +02:00
hmac.c crypto: include crypto- module prefix in template 2014-11-26 20:06:30 +08:00
internal.h crypto: api - Remove no-op exit_ops code 2016-10-21 11:03:42 +08:00
jitterentropy-kcapi.c crypto: jitterentropy - drop duplicate header module.h 2016-11-17 23:34:52 +08:00
jitterentropy.c crypto: jitterentropy - Delete unnecessary checks before the function call "kzfree" 2015-06-25 23:18:33 +08:00
Kconfig crypto: ccm - switch to separate cbcmac driver 2017-02-11 17:50:45 +08:00
keywrap.c crypto: keywrap - memzero the correct memory 2016-02-01 22:27:05 +08:00
khazad.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
kpp.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
lrw.c crypto: lrw - Convert to skcipher 2016-11-28 21:23:17 +08:00
lz4.c crypto: acomp - add support for lz4 via scomp 2016-10-25 11:08:32 +08:00
lz4hc.c crypto: acomp - add support for lz4hc via scomp 2016-10-25 11:08:32 +08:00
lzo.c crypto: acomp - add support for lzo via scomp 2016-10-25 11:08:31 +08:00
Makefile crypto: improve gcc optimization flags for serpent and wp512 2017-02-11 17:52:26 +08:00
mcryptd.c crypto: mcryptd - Check mcryptd algorithm compatibility 2016-12-07 19:55:37 +08:00
md4.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
md5.c crypto: hash - add zero length message hash for shax and md5 2015-12-22 20:43:35 +08:00
memneq.c crypto: memneq - fix for archs without efficient unaligned access 2013-12-09 20:09:12 +08:00
michael_mic.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
pcbc.c crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic 2017-02-11 17:52:28 +08:00
pcrypt.c crypto: aead - Remove CRYPTO_ALG_AEAD_NEW flag 2015-08-17 16:53:53 +08:00
poly1305_generic.c crypto: poly1305 - Use unaligned access where required 2016-11-13 17:45:03 +08:00
proc.c crypto: fips - Move fips_enabled sysctl into fips.c 2015-04-23 14:18:09 +08:00
ripemd.h
rmd128.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
rmd160.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
rmd256.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
rmd320.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
rng.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
rsa_helper.c crypto: rsa - allow keys >= 2048 bits in FIPS mode 2016-08-24 21:07:10 +08:00
rsa-pkcs1pad.c crypto: rsa-pkcs1pad - Handle leading zero for decryption 2016-09-22 17:42:08 +08:00
rsa.c crypto: rsa - Generate fixed-length output 2016-07-01 23:45:18 +08:00
rsaprivkey.asn1 crypto: rsa - Store rest of the private key components 2016-07-05 23:05:26 +08:00
rsapubkey.asn1 crypto: akcipher - Changes to asymmetric key API 2015-10-14 22:23:16 +08:00
salsa20_generic.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
scatterwalk.c crypto: scatterwalk - Remove unnecessary aliasing check in map_and_copy 2016-11-22 15:02:25 +08:00
scompress.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
seed.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
seqiv.c crypto: algapi - make crypto_xor() and crypto_inc() alignment agnostic 2017-02-11 17:52:28 +08:00
serpent_generic.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
sha1_generic.c crypto: hash - add zero length message hash for shax and md5 2015-12-22 20:43:35 +08:00
sha3_generic.c crypto: sha3 - Add missing ULL suffixes for 64-bit constants 2016-08-08 23:43:46 +08:00
sha256_generic.c crypto: hash - add zero length message hash for shax and md5 2015-12-22 20:43:35 +08:00
sha512_generic.c crypto: sha512-generic - move to generic glue implementation 2015-04-10 21:39:41 +08:00
shash.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
simd.c crypto: simd - Add simd skcipher helper 2016-11-28 21:23:18 +08:00
skcipher.c crypto: Replaced gcc specific attributes with macros from compiler.h 2017-01-13 00:24:39 +08:00
tcrypt.c crypto: tcrypt - Add debug prints 2017-01-23 22:50:24 +08:00
tcrypt.h crypto: tcrypt - Add ChaCha20/Poly1305 speed tests 2015-07-17 21:20:20 +08:00
tea.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
testmgr.c crypto: testmgr - add test cases for cbcmac(aes) 2017-02-11 17:50:44 +08:00
testmgr.h crypto: testmgr - add test cases for cbcmac(aes) 2017-02-11 17:50:44 +08:00
tgr192.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
twofish_common.c
twofish_generic.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
vmac.c crypto: include crypto- module prefix in template 2014-11-26 20:06:30 +08:00
wp512.c crypto: add missing crypto module aliases 2015-01-13 22:29:11 +11:00
xcbc.c crypto: include crypto- module prefix in template 2014-11-26 20:06:30 +08:00
xor.c crypto: xor - Fix warning when XOR_SELECT_TEMPLATE is unset 2016-08-31 23:00:48 +08:00
xts.c crypto: xts - Convert to skcipher 2016-11-28 21:23:18 +08:00