2017-12-18 06:00:59 +03:00
|
|
|
// SPDX-License-Identifier: GPL-2.0
|
2006-10-11 12:20:50 +04:00
|
|
|
/*
|
2006-10-11 12:20:53 +04:00
|
|
|
* linux/fs/ext4/hash.c
|
2006-10-11 12:20:50 +04:00
|
|
|
*
|
|
|
|
* Copyright (C) 2002 by Theodore Ts'o
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <linux/fs.h>
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-25 21:12:08 +03:00
|
|
|
#include <linux/unicode.h>
|
2017-02-02 19:52:14 +03:00
|
|
|
#include <linux/compiler.h>
|
|
|
|
#include <linux/bitops.h>
|
2008-04-30 02:13:32 +04:00
|
|
|
#include "ext4.h"
|
2006-10-11 12:20:50 +04:00
|
|
|
|
|
|
|
#define DELTA 0x9E3779B9
|
|
|
|
|
|
|
|
static void TEA_transform(__u32 buf[4], __u32 const in[])
|
|
|
|
{
|
|
|
|
__u32 sum = 0;
|
|
|
|
__u32 b0 = buf[0], b1 = buf[1];
|
|
|
|
__u32 a = in[0], b = in[1], c = in[2], d = in[3];
|
|
|
|
int n = 16;
|
|
|
|
|
|
|
|
do {
|
|
|
|
sum += DELTA;
|
|
|
|
b0 += ((b1 << 4)+a) ^ (b1+sum) ^ ((b1 >> 5)+b);
|
|
|
|
b1 += ((b0 << 4)+c) ^ (b0+sum) ^ ((b0 >> 5)+d);
|
2008-09-09 06:25:24 +04:00
|
|
|
} while (--n);
|
2006-10-11 12:20:50 +04:00
|
|
|
|
|
|
|
buf[0] += b0;
|
|
|
|
buf[1] += b1;
|
|
|
|
}
|
|
|
|
|
2017-02-02 19:52:14 +03:00
|
|
|
/* F, G and H are basic MD4 functions: selection, majority, parity */
|
|
|
|
#define F(x, y, z) ((z) ^ ((x) & ((y) ^ (z))))
|
|
|
|
#define G(x, y, z) (((x) & (y)) + (((x) ^ (y)) & (z)))
|
|
|
|
#define H(x, y, z) ((x) ^ (y) ^ (z))
|
|
|
|
|
|
|
|
/*
|
|
|
|
* The generic round function. The application is so specific that
|
|
|
|
* we don't bother protecting all the arguments with parens, as is generally
|
|
|
|
* good macro practice, in favor of extra legibility.
|
|
|
|
* Rotation is separate from addition to prevent recomputation
|
|
|
|
*/
|
|
|
|
#define ROUND(f, a, b, c, d, x, s) \
|
|
|
|
(a += f(b, c, d) + x, a = rol32(a, s))
|
|
|
|
#define K1 0
|
|
|
|
#define K2 013240474631UL
|
|
|
|
#define K3 015666365641UL
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Basic cut-down MD4 transform. Returns only 32 bits of result.
|
|
|
|
*/
|
|
|
|
static __u32 half_md4_transform(__u32 buf[4], __u32 const in[8])
|
|
|
|
{
|
|
|
|
__u32 a = buf[0], b = buf[1], c = buf[2], d = buf[3];
|
|
|
|
|
|
|
|
/* Round 1 */
|
|
|
|
ROUND(F, a, b, c, d, in[0] + K1, 3);
|
|
|
|
ROUND(F, d, a, b, c, in[1] + K1, 7);
|
|
|
|
ROUND(F, c, d, a, b, in[2] + K1, 11);
|
|
|
|
ROUND(F, b, c, d, a, in[3] + K1, 19);
|
|
|
|
ROUND(F, a, b, c, d, in[4] + K1, 3);
|
|
|
|
ROUND(F, d, a, b, c, in[5] + K1, 7);
|
|
|
|
ROUND(F, c, d, a, b, in[6] + K1, 11);
|
|
|
|
ROUND(F, b, c, d, a, in[7] + K1, 19);
|
|
|
|
|
|
|
|
/* Round 2 */
|
|
|
|
ROUND(G, a, b, c, d, in[1] + K2, 3);
|
|
|
|
ROUND(G, d, a, b, c, in[3] + K2, 5);
|
|
|
|
ROUND(G, c, d, a, b, in[5] + K2, 9);
|
|
|
|
ROUND(G, b, c, d, a, in[7] + K2, 13);
|
|
|
|
ROUND(G, a, b, c, d, in[0] + K2, 3);
|
|
|
|
ROUND(G, d, a, b, c, in[2] + K2, 5);
|
|
|
|
ROUND(G, c, d, a, b, in[4] + K2, 9);
|
|
|
|
ROUND(G, b, c, d, a, in[6] + K2, 13);
|
|
|
|
|
|
|
|
/* Round 3 */
|
|
|
|
ROUND(H, a, b, c, d, in[3] + K3, 3);
|
|
|
|
ROUND(H, d, a, b, c, in[7] + K3, 9);
|
|
|
|
ROUND(H, c, d, a, b, in[2] + K3, 11);
|
|
|
|
ROUND(H, b, c, d, a, in[6] + K3, 15);
|
|
|
|
ROUND(H, a, b, c, d, in[1] + K3, 3);
|
|
|
|
ROUND(H, d, a, b, c, in[5] + K3, 9);
|
|
|
|
ROUND(H, c, d, a, b, in[0] + K3, 11);
|
|
|
|
ROUND(H, b, c, d, a, in[4] + K3, 15);
|
|
|
|
|
|
|
|
buf[0] += a;
|
|
|
|
buf[1] += b;
|
|
|
|
buf[2] += c;
|
|
|
|
buf[3] += d;
|
|
|
|
|
|
|
|
return buf[1]; /* "most hashed" word */
|
|
|
|
}
|
|
|
|
#undef ROUND
|
|
|
|
#undef K1
|
|
|
|
#undef K2
|
|
|
|
#undef K3
|
|
|
|
#undef F
|
|
|
|
#undef G
|
|
|
|
#undef H
|
2006-10-11 12:20:50 +04:00
|
|
|
|
|
|
|
/* The old legacy hash */
|
2008-10-28 20:21:44 +03:00
|
|
|
static __u32 dx_hack_hash_unsigned(const char *name, int len)
|
2006-10-11 12:20:50 +04:00
|
|
|
{
|
2008-10-28 20:21:44 +03:00
|
|
|
__u32 hash, hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9;
|
|
|
|
const unsigned char *ucp = (const unsigned char *) name;
|
|
|
|
|
|
|
|
while (len--) {
|
|
|
|
hash = hash1 + (hash0 ^ (((int) *ucp++) * 7152373));
|
|
|
|
|
|
|
|
if (hash & 0x80000000)
|
|
|
|
hash -= 0x7fffffff;
|
|
|
|
hash1 = hash0;
|
|
|
|
hash0 = hash;
|
|
|
|
}
|
|
|
|
return hash0 << 1;
|
|
|
|
}
|
|
|
|
|
|
|
|
static __u32 dx_hack_hash_signed(const char *name, int len)
|
|
|
|
{
|
|
|
|
__u32 hash, hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9;
|
|
|
|
const signed char *scp = (const signed char *) name;
|
|
|
|
|
2006-10-11 12:20:50 +04:00
|
|
|
while (len--) {
|
2008-10-28 20:21:44 +03:00
|
|
|
hash = hash1 + (hash0 ^ (((int) *scp++) * 7152373));
|
2006-10-11 12:20:50 +04:00
|
|
|
|
2008-10-28 20:21:44 +03:00
|
|
|
if (hash & 0x80000000)
|
|
|
|
hash -= 0x7fffffff;
|
2006-10-11 12:20:50 +04:00
|
|
|
hash1 = hash0;
|
|
|
|
hash0 = hash;
|
|
|
|
}
|
2008-10-28 20:21:44 +03:00
|
|
|
return hash0 << 1;
|
|
|
|
}
|
|
|
|
|
|
|
|
static void str2hashbuf_signed(const char *msg, int len, __u32 *buf, int num)
|
|
|
|
{
|
|
|
|
__u32 pad, val;
|
|
|
|
int i;
|
|
|
|
const signed char *scp = (const signed char *) msg;
|
|
|
|
|
|
|
|
pad = (__u32)len | ((__u32)len << 8);
|
|
|
|
pad |= pad << 16;
|
|
|
|
|
|
|
|
val = pad;
|
|
|
|
if (len > num*4)
|
|
|
|
len = num * 4;
|
|
|
|
for (i = 0; i < len; i++) {
|
|
|
|
val = ((int) scp[i]) + (val << 8);
|
|
|
|
if ((i % 4) == 3) {
|
|
|
|
*buf++ = val;
|
|
|
|
val = pad;
|
|
|
|
num--;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if (--num >= 0)
|
|
|
|
*buf++ = val;
|
|
|
|
while (--num >= 0)
|
|
|
|
*buf++ = pad;
|
2006-10-11 12:20:50 +04:00
|
|
|
}
|
|
|
|
|
2008-10-28 20:21:44 +03:00
|
|
|
static void str2hashbuf_unsigned(const char *msg, int len, __u32 *buf, int num)
|
2006-10-11 12:20:50 +04:00
|
|
|
{
|
|
|
|
__u32 pad, val;
|
|
|
|
int i;
|
2008-10-28 20:21:44 +03:00
|
|
|
const unsigned char *ucp = (const unsigned char *) msg;
|
2006-10-11 12:20:50 +04:00
|
|
|
|
|
|
|
pad = (__u32)len | ((__u32)len << 8);
|
|
|
|
pad |= pad << 16;
|
|
|
|
|
|
|
|
val = pad;
|
|
|
|
if (len > num*4)
|
|
|
|
len = num * 4;
|
2008-09-09 06:25:24 +04:00
|
|
|
for (i = 0; i < len; i++) {
|
2008-10-28 20:21:44 +03:00
|
|
|
val = ((int) ucp[i]) + (val << 8);
|
2006-10-11 12:20:50 +04:00
|
|
|
if ((i % 4) == 3) {
|
|
|
|
*buf++ = val;
|
|
|
|
val = pad;
|
|
|
|
num--;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if (--num >= 0)
|
|
|
|
*buf++ = val;
|
|
|
|
while (--num >= 0)
|
|
|
|
*buf++ = pad;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Returns the hash of a filename. If len is 0 and name is NULL, then
|
|
|
|
* this function can be used to test whether or not a hash version is
|
|
|
|
* supported.
|
|
|
|
*
|
|
|
|
* The seed is an 4 longword (32 bits) "secret" which can be used to
|
|
|
|
* uniquify a hash. If the seed is all zero's, then some default seed
|
|
|
|
* may be used.
|
|
|
|
*
|
|
|
|
* A particular hash version specifies whether or not the seed is
|
|
|
|
* represented, and whether or not the returned hash is 32 bits or 64
|
|
|
|
* bits. 32 bit hashes will return 0 for the minor hash.
|
|
|
|
*/
|
2021-03-19 10:34:13 +03:00
|
|
|
static int __ext4fs_dirhash(const struct inode *dir, const char *name, int len,
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-25 21:12:08 +03:00
|
|
|
struct dx_hash_info *hinfo)
|
2006-10-11 12:20:50 +04:00
|
|
|
{
|
|
|
|
__u32 hash;
|
|
|
|
__u32 minor_hash = 0;
|
|
|
|
const char *p;
|
|
|
|
int i;
|
|
|
|
__u32 in[8], buf[4];
|
2008-10-28 20:21:44 +03:00
|
|
|
void (*str2hashbuf)(const char *, int, __u32 *, int) =
|
|
|
|
str2hashbuf_signed;
|
2006-10-11 12:20:50 +04:00
|
|
|
|
|
|
|
/* Initialize the default seed for the hash checksum functions */
|
|
|
|
buf[0] = 0x67452301;
|
|
|
|
buf[1] = 0xefcdab89;
|
|
|
|
buf[2] = 0x98badcfe;
|
|
|
|
buf[3] = 0x10325476;
|
|
|
|
|
|
|
|
/* Check to see if the seed is all zero's */
|
|
|
|
if (hinfo->seed) {
|
2008-09-09 06:25:24 +04:00
|
|
|
for (i = 0; i < 4; i++) {
|
2013-02-02 07:33:21 +04:00
|
|
|
if (hinfo->seed[i]) {
|
|
|
|
memcpy(buf, hinfo->seed, sizeof(buf));
|
2006-10-11 12:20:50 +04:00
|
|
|
break;
|
2013-02-02 07:33:21 +04:00
|
|
|
}
|
2006-10-11 12:20:50 +04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
switch (hinfo->hash_version) {
|
2008-10-28 20:21:44 +03:00
|
|
|
case DX_HASH_LEGACY_UNSIGNED:
|
|
|
|
hash = dx_hack_hash_unsigned(name, len);
|
|
|
|
break;
|
2006-10-11 12:20:50 +04:00
|
|
|
case DX_HASH_LEGACY:
|
2008-10-28 20:21:44 +03:00
|
|
|
hash = dx_hack_hash_signed(name, len);
|
2006-10-11 12:20:50 +04:00
|
|
|
break;
|
2008-10-28 20:21:44 +03:00
|
|
|
case DX_HASH_HALF_MD4_UNSIGNED:
|
|
|
|
str2hashbuf = str2hashbuf_unsigned;
|
2020-08-10 14:44:35 +03:00
|
|
|
fallthrough;
|
2006-10-11 12:20:50 +04:00
|
|
|
case DX_HASH_HALF_MD4:
|
|
|
|
p = name;
|
|
|
|
while (len > 0) {
|
2008-10-28 20:21:44 +03:00
|
|
|
(*str2hashbuf)(p, len, in, 8);
|
2006-10-11 12:20:50 +04:00
|
|
|
half_md4_transform(buf, in);
|
|
|
|
len -= 32;
|
|
|
|
p += 32;
|
|
|
|
}
|
|
|
|
minor_hash = buf[2];
|
|
|
|
hash = buf[1];
|
|
|
|
break;
|
2008-10-28 20:21:44 +03:00
|
|
|
case DX_HASH_TEA_UNSIGNED:
|
|
|
|
str2hashbuf = str2hashbuf_unsigned;
|
2020-08-10 14:44:35 +03:00
|
|
|
fallthrough;
|
2006-10-11 12:20:50 +04:00
|
|
|
case DX_HASH_TEA:
|
|
|
|
p = name;
|
|
|
|
while (len > 0) {
|
2008-10-28 20:21:44 +03:00
|
|
|
(*str2hashbuf)(p, len, in, 4);
|
2006-10-11 12:20:50 +04:00
|
|
|
TEA_transform(buf, in);
|
|
|
|
len -= 16;
|
|
|
|
p += 16;
|
|
|
|
}
|
|
|
|
hash = buf[0];
|
|
|
|
minor_hash = buf[1];
|
|
|
|
break;
|
2021-03-19 10:34:13 +03:00
|
|
|
case DX_HASH_SIPHASH:
|
|
|
|
{
|
|
|
|
struct qstr qname = QSTR_INIT(name, len);
|
|
|
|
__u64 combined_hash;
|
|
|
|
|
|
|
|
if (fscrypt_has_encryption_key(dir)) {
|
|
|
|
combined_hash = fscrypt_fname_siphash(dir, &qname);
|
|
|
|
} else {
|
|
|
|
ext4_warning_inode(dir, "Siphash requires key");
|
|
|
|
return -1;
|
|
|
|
}
|
|
|
|
|
|
|
|
hash = (__u32)(combined_hash >> 32);
|
|
|
|
minor_hash = (__u32)combined_hash;
|
|
|
|
break;
|
|
|
|
}
|
2006-10-11 12:20:50 +04:00
|
|
|
default:
|
|
|
|
hinfo->hash = 0;
|
|
|
|
return -1;
|
|
|
|
}
|
|
|
|
hash = hash & ~1;
|
2012-03-19 06:44:40 +04:00
|
|
|
if (hash == (EXT4_HTREE_EOF_32BIT << 1))
|
|
|
|
hash = (EXT4_HTREE_EOF_32BIT - 1) << 1;
|
2006-10-11 12:20:50 +04:00
|
|
|
hinfo->hash = hash;
|
|
|
|
hinfo->minor_hash = minor_hash;
|
|
|
|
return 0;
|
|
|
|
}
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-25 21:12:08 +03:00
|
|
|
|
|
|
|
int ext4fs_dirhash(const struct inode *dir, const char *name, int len,
|
|
|
|
struct dx_hash_info *hinfo)
|
|
|
|
{
|
|
|
|
#ifdef CONFIG_UNICODE
|
2020-10-28 08:08:20 +03:00
|
|
|
const struct unicode_map *um = dir->i_sb->s_encoding;
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-25 21:12:08 +03:00
|
|
|
int r, dlen;
|
|
|
|
unsigned char *buff;
|
|
|
|
struct qstr qstr = {.name = name, .len = len };
|
|
|
|
|
2021-03-19 10:34:13 +03:00
|
|
|
if (len && IS_CASEFOLDED(dir) && um &&
|
|
|
|
(!IS_ENCRYPTED(dir) || fscrypt_has_encryption_key(dir))) {
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-25 21:12:08 +03:00
|
|
|
buff = kzalloc(sizeof(char) * PATH_MAX, GFP_KERNEL);
|
|
|
|
if (!buff)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
dlen = utf8_casefold(um, &qstr, buff, PATH_MAX);
|
|
|
|
if (dlen < 0) {
|
|
|
|
kfree(buff);
|
|
|
|
goto opaque_seq;
|
|
|
|
}
|
|
|
|
|
2021-03-19 10:34:13 +03:00
|
|
|
r = __ext4fs_dirhash(dir, buff, dlen, hinfo);
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-25 21:12:08 +03:00
|
|
|
|
|
|
|
kfree(buff);
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
opaque_seq:
|
|
|
|
#endif
|
2021-03-19 10:34:13 +03:00
|
|
|
return __ext4fs_dirhash(dir, name, len, hinfo);
|
ext4: Support case-insensitive file name lookups
This patch implements the actual support for case-insensitive file name
lookups in ext4, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (EXT4_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, Ext4 only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in ext4 is implemented
lives in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for EXT4 because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-25 21:12:08 +03:00
|
|
|
}
|