Olivier Bacon f716374c4e crypto: engine - fix crypto_queue backlog handling ... [ Upstream commit 4140aafcff ] CRYPTO_TFM_REQ_MAY_BACKLOG tells the crypto driver that it should internally backlog requests until the crypto hw's queue becomes full. At that point, crypto_engine backlogs the request and returns -EBUSY. Calling driver such as dm-crypt then waits until the complete() function is called with a status of -EINPROGRESS before sending a new request. The problem lies in the call to complete() with a value of -EINPROGRESS that is made when a backlog item is present on the queue. The call is done before the successful execution of the crypto request. In the case that do_one_request() returns < 0 and the retry support is available, the request is put back in the queue. This leads upper drivers to send a new request even if the queue is still full. The problem can be reproduced by doing a large dd into a crypto dm-crypt device. This is pretty easy to see when using Freescale CAAM crypto driver and SWIOTLB dma. Since the actual amount of requests that can be hold in the queue is unlimited we get IOs error and dma allocation. The fix is to call complete with a value of -EINPROGRESS only if the request is not enqueued back in crypto_queue. This is done by calling complete() later in the code. In order to delay the decision, crypto_queue is modified to correctly set the backlog pointer when a request is enqueued back. Fixes: 6a89f492f8 ("crypto: engine - support for parallel requests based on retry mechanism") Co-developed-by: Sylvain Ouellet <souellet@genetec.com> Signed-off-by: Sylvain Ouellet <souellet@genetec.com> Signed-off-by: Olivier Bacon <obacon@genetec.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Sasha Levin <sashal@kernel.org>		2023-05-17 11:50:20 +02:00
..
asymmetric_keys	asymmetric_keys: log on fatal failures in PE/pkcs7	2023-04-20 12:13:54 +02:00
async_tx	async_xor: check src_offs is not NULL before updating it	2021-06-10 19:40:14 -07:00
842.c	…
Kconfig	crypto: blake2s - remove shash module	2022-08-17 14:24:19 +02:00
Makefile	crypto: blake2s - remove shash module	2022-08-17 14:24:19 +02:00
acompress.c	…
adiantum.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
aead.c	…
aegis.h	crypto: aegis128 - Move simd prototypes into aegis.h	2021-03-19 21:59:45 +11:00
aegis128-core.c	crypto: aegis128 - Move simd prototypes into aegis.h	2021-03-19 21:59:45 +11:00
aegis128-neon-inner.c	…
aegis128-neon.c	crypto: aegis128 - Move simd prototypes into aegis.h	2021-03-19 21:59:45 +11:00
aes_generic.c	…
aes_ti.c	…
af_alg.c	crypto: af_alg - use DIV_ROUND_UP helper macro for calculations	2021-06-03 20:24:04 +08:00
ahash.c	…
akcipher.c	crypto: akcipher - default implementation for setting a private key	2022-10-26 12:35:25 +02:00
algapi.c	crypto: engine - fix crypto_queue backlog handling	2023-05-17 11:50:20 +02:00
algboss.c	crypto: api - remove CRYPTOA_U32 and related functions	2021-06-17 15:07:31 +08:00
algif_aead.c	…
algif_hash.c	…
algif_rng.c	…
algif_skcipher.c	…
ansi_cprng.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
anubis.c	…
api.c	crypto: api - Move cryptomgr soft dependency into algapi	2022-02-11 09:10:26 +01:00
arc4.c	…
authenc.c	crypto: authenc - Fix sleep in atomic context in decrypt_tail	2022-04-08 14:23:05 +02:00
authencesn.c	…
blake2b_generic.c	crypto: blake2b - update file comment	2021-01-03 08:41:39 +11:00
blowfish_common.c	…
blowfish_generic.c	crypto: blowfish - use unaligned accessors instead of alignmask	2021-02-10 17:55:56 +11:00
camellia_generic.c	crypto: camellia - use unaligned accessors instead of alignmask	2021-02-10 17:55:56 +11:00
cast5_generic.c	crypto: cast5 - use unaligned accessors instead of alignmask	2021-02-10 17:55:56 +11:00
cast6_generic.c	crypto: cast6 - use unaligned accessors instead of alignmask	2021-02-10 17:55:56 +11:00
cast_common.c	…
cbc.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
ccm.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
cfb.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
chacha20poly1305.c	…
chacha_generic.c	…
cipher.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
cmac.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
compress.c	…
crc32_generic.c	crypto: crc32-generic - Use SPDX-License-Identifier	2021-04-16 21:24:27 +10:00
crc32c_generic.c	…
crct10dif_common.c	…
crct10dif_generic.c	…
cryptd.c	crypto: cryptd - Use request context instead of stack for sub-request	2022-12-31 13:14:23 +01:00
crypto_engine.c	crypto: engine - fix crypto_queue backlog handling	2023-05-17 11:50:20 +02:00
crypto_null.c	…
crypto_user_base.c	…
crypto_user_stat.c	…
ctr.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
cts.c	…
curve25519-generic.c	…
deflate.c	…
des_generic.c	…
dh.c	…
dh_helper.c	…
drbg.c	crypto: drbg - Only fail when jent is unavailable in FIPS mode	2023-05-11 23:00:27 +09:00
ecb.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
ecc.c	crypto: ecc - Correct an error in the comments	2021-03-26 20:15:56 +11:00
ecc.h	crypto: ecc - handle unaligned input buffer in ecc_swap_digits	2021-07-30 10:58:36 +08:00
ecc_curve_defs.h	Merge branch 'ecc'	2021-03-26 19:55:55 +11:00
ecdh.c	crypto: ecdh - register NIST P384 tfm	2021-05-28 15:11:47 +08:00
ecdh_helper.c	crypto: ecdh - move curve_id of ECDH from the key to algorithm name	2021-03-13 00:04:03 +11:00
ecdsa.c	crypto: ecdsa - Register NIST P384 and extend test suite	2021-03-26 19:41:58 +11:00
ecdsasignature.asn1	crypto: ecdsa - Add support for ECDSA signature verification	2021-03-26 19:41:58 +11:00
echainiv.c	…
ecrdsa.c	crypto: ecrdsa - Fix incorrect use of vli_cmp	2022-06-06 08:43:39 +02:00
ecrdsa_defs.h	…
ecrdsa_params.asn1	…
ecrdsa_pub_key.asn1	…
essiv.c	crypto: essiv - Handle EBUSY correctly	2023-03-10 09:39:11 +01:00
fcrypt.c	crypto: fcrypt - Remove 'do while(0)' loop for single statement macro	2021-04-02 18:28:13 +11:00
fips.c	…
gcm.c	…
geniv.c	…
gf128mul.c	…
ghash-generic.c	…
hash_info.c	…
hmac.c	…
internal.h	crypto: api - Move crypto attr definitions out of crypto.h	2021-06-24 14:51:35 +08:00
jitterentropy-kcapi.c	…
jitterentropy.c	crypto: jitter - consider 32 LSB for APT	2022-01-27 11:04:31 +01:00
jitterentropy.h	…
keywrap.c	crypto: keywrap - Remove else after break statement	2021-04-02 18:28:13 +11:00
khazad.c	crypto: khazad,wp512 - remove leading spaces before tabs	2021-05-28 15:11:44 +08:00
kpp.c	…
lrw.c	…
lz4.c	…
lz4hc.c	…
lzo-rle.c	…
lzo.c	…
md4.c	…
md5.c	…
michael_mic.c	crypto: michael_mic - fix broken misalignment handling	2021-02-10 17:55:55 +11:00
nhpoly1305.c	…
ofb.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
pcbc.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
pcrypt.c	crypto: pcrypt - Delay write to padata->info	2021-11-18 19:16:44 +01:00
poly1305_generic.c	…
proc.c	…
ripemd.h	crypto: rmd320 - remove RIPE-MD 320 hash algorithm	2021-01-29 16:07:04 +11:00
rmd160.c	…
rng.c	crypto: rng - fix crypto_rng_reset() refcounting when !CRYPTO_STATS	2021-04-02 18:28:12 +11:00
rsa-pkcs1pad.c	crypto: rsa-pkcs1pad - Use akcipher_request_complete	2023-03-10 09:39:16 +01:00
rsa.c	…
rsa_helper.c	…
rsaprivkey.asn1	…
rsapubkey.asn1	…
scatterwalk.c	…
scompress.c	…
seed.c	crypto: seed - remove trailing semicolon in macro definition	2020-12-04 18:13:16 +11:00
seqiv.c	crypto: seqiv - Handle EBUSY correctly	2023-03-10 09:39:11 +01:00
serpent_generic.c	crypto: serpent - Fix sparse byte order warnings	2021-03-07 15:13:15 +11:00
sha1_generic.c	…
sha3_generic.c	…
sha256_generic.c	…
sha512_generic.c	crypto: sha512 - remove imaginary and mystifying clearing of variables	2021-08-27 16:30:19 +08:00
shash.c	crypto: shash - avoid comparing pointers to exported functions under CFI	2021-06-17 15:07:31 +08:00
simd.c	…
skcipher.c	crypto: skcipher - in_irq() cleanup	2021-08-21 15:44:58 +08:00
sm2.c	crypto: sm2 - fix a memory leak in sm2	2021-06-11 15:03:30 +08:00
sm2signature.asn1	…
sm3_generic.c	…
sm4_generic.c	crypto: arm64/sm4-ce - Make dependent on sm4 library instead of sm4-generic	2021-07-30 10:58:30 +08:00
streebog_generic.c	…
tcrypt.c	crypto: tcrypt - Fix multibuffer skcipher speed test mem leak	2022-12-31 13:14:24 +01:00
tcrypt.h	…
tea.c	…
testmgr.c	crypto: blake2s - remove shash module	2022-08-17 14:24:19 +02:00
testmgr.h	crypto: blake2s - remove shash module	2022-08-17 14:24:19 +02:00
twofish_common.c	…
twofish_generic.c	crypto: twofish - use unaligned accessors instead of alignmask	2021-02-10 17:55:57 +11:00
vmac.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
wp512.c	crypto: wp512 - correct a non-kernel-doc comment	2021-08-12 19:32:17 +08:00
xcbc.c	crypto: remove cipher routines from public crypto API	2021-01-03 08:41:35 +11:00
xor.c	Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6	2021-02-21 17:23:56 -08:00
xts.c	crypto: xts - Handle EBUSY correctly	2023-03-10 09:39:13 +01:00
xxhash_generic.c	…
zstd.c	…