git on Mac OS and precomposed unicode
Mac OS X mangles file names containing unicode on file systems HFS+,
VFAT or SAMBA. When a file using unicode code points outside ASCII
is created on a HFS+ drive, the file name is converted into
decomposed unicode and written to disk. No conversion is done if
the file name is already decomposed unicode.
Calling open("\xc3\x84", ...) with a precomposed "Ä" yields the same
result as open("\x41\xcc\x88",...) with a decomposed "Ä".
As a consequence, readdir() returns the file names in decomposed
unicode, even if the user expects precomposed unicode. Unlike on
HFS+, Mac OS X stores files on a VFAT drive (e.g. an USB drive) in
precomposed unicode, but readdir() still returns file names in
decomposed unicode. When a git repository is stored on a network
share using SAMBA, file names are send over the wire and written to
disk on the remote system in precomposed unicode, but Mac OS X
readdir() returns decomposed unicode to be compatible with its
behaviour on HFS+ and VFAT.
The unicode decomposition causes many problems:
- The names "git add" and other commands get from the end user may
often be precomposed form (the decomposed form is not easily input
from the keyboard), but when the commands read from the filesystem
to see what it is going to update the index with already is on the
filesystem, readdir() will give decomposed form, which is different.
- Similarly "git log", "git mv" and all other commands that need to
compare pathnames found on the command line (often but not always
precomposed form; a command line input resulting from globbing may
be in decomposed) with pathnames found in the tree objects (should
be precomposed form to be compatible with other systems and for
consistency in general).
- The same for names stored in the index, which should be
precomposed, that may need to be compared with the names read from
readdir().
NFS mounted from Linux is fully transparent and does not suffer from
the above.
As Mac OS X treats precomposed and decomposed file names as equal,
we can
- wrap readdir() on Mac OS X to return the precomposed form, and
- normalize decomposed form given from the command line also to the
precomposed form,
to ensure that all pathnames used in Git are always in the
precomposed form. This behaviour can be requested by setting
"core.precomposedunicode" configuration variable to true.
The code in compat/precomposed_utf8.c implements basically 4 new
functions: precomposed_utf8_opendir(), precomposed_utf8_readdir(),
precomposed_utf8_closedir() and precompose_argv(). The first three
are to wrap opendir(3), readdir(3), and closedir(3) functions.
The argv[] conversion allows to use the TAB filename completion done
by the shell on command line. It tolerates other tools which use
readdir() to feed decomposed file names into git.
When creating a new git repository with "git init" or "git clone",
"core.precomposedunicode" will be set "false".
The user needs to activate this feature manually. She typically
sets core.precomposedunicode to "true" on HFS and VFAT, or file
systems mounted via SAMBA.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-08 17:50:25 +04:00
|
|
|
#!/bin/sh
|
|
|
|
|
#
|
|
|
|
|
# Copyright (c) 2012 Torsten Bögershausen
|
|
|
|
|
#
|
|
|
|
|
|
|
|
|
|
test_description='utf-8 decomposed (nfd) converted to precomposed (nfc)'
|
|
|
|
|
|
tests: mark tests relying on the current default for `init.defaultBranch`
In addition to the manual adjustment to let the `linux-gcc` CI job run
the test suite with `master` and then with `main`, this patch makes sure
that GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME is set in all test scripts
that currently rely on the initial branch name being `master by default.
To determine which test scripts to mark up, the first step was to
force-set the default branch name to `master` in
- all test scripts that contain the keyword `master`,
- t4211, which expects `t/t4211/history.export` with a hard-coded ref to
initialize the default branch,
- t5560 because it sources `t/t556x_common` which uses `master`,
- t8002 and t8012 because both source `t/annotate-tests.sh` which also
uses `master`)
This trick was performed by this command:
$ sed -i '/^ *\. \.\/\(test-lib\|lib-\(bash\|cvs\|git-svn\)\|gitweb-lib\)\.sh$/i\
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\
' $(git grep -l master t/t[0-9]*.sh) \
t/t4211*.sh t/t5560*.sh t/t8002*.sh t/t8012*.sh
After that, careful, manual inspection revealed that some of the test
scripts containing the needle `master` do not actually rely on a
specific default branch name: either they mention `master` only in a
comment, or they initialize that branch specificially, or they do not
actually refer to the current default branch. Therefore, the
aforementioned modification was undone in those test scripts thusly:
$ git checkout HEAD -- \
t/t0027-auto-crlf.sh t/t0060-path-utils.sh \
t/t1011-read-tree-sparse-checkout.sh \
t/t1305-config-include.sh t/t1309-early-config.sh \
t/t1402-check-ref-format.sh t/t1450-fsck.sh \
t/t2024-checkout-dwim.sh \
t/t2106-update-index-assume-unchanged.sh \
t/t3040-subprojects-basic.sh t/t3301-notes.sh \
t/t3308-notes-merge.sh t/t3423-rebase-reword.sh \
t/t3436-rebase-more-options.sh \
t/t4015-diff-whitespace.sh t/t4257-am-interactive.sh \
t/t5323-pack-redundant.sh t/t5401-update-hooks.sh \
t/t5511-refspec.sh t/t5526-fetch-submodules.sh \
t/t5529-push-errors.sh t/t5530-upload-pack-error.sh \
t/t5548-push-porcelain.sh \
t/t5552-skipping-fetch-negotiator.sh \
t/t5572-pull-submodule.sh t/t5608-clone-2gb.sh \
t/t5614-clone-submodules-shallow.sh \
t/t7508-status.sh t/t7606-merge-custom.sh \
t/t9302-fast-import-unpack-limit.sh
We excluded one set of test scripts in these commands, though: the range
of `git p4` tests. The reason? `git p4` stores the (foreign) remote
branch in the branch called `p4/master`, which is obviously not the
default branch. Manual analysis revealed that only five of these tests
actually require a specific default branch name to pass; They were
modified thusly:
$ sed -i '/^ *\. \.\/lib-git-p4\.sh$/i\
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\
' t/t980[0167]*.sh t/t9811*.sh
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-19 02:44:19 +03:00
|
|
|
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
|
|
|
|
|
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
|
|
|
|
|
|
git on Mac OS and precomposed unicode
Mac OS X mangles file names containing unicode on file systems HFS+,
VFAT or SAMBA. When a file using unicode code points outside ASCII
is created on a HFS+ drive, the file name is converted into
decomposed unicode and written to disk. No conversion is done if
the file name is already decomposed unicode.
Calling open("\xc3\x84", ...) with a precomposed "Ä" yields the same
result as open("\x41\xcc\x88",...) with a decomposed "Ä".
As a consequence, readdir() returns the file names in decomposed
unicode, even if the user expects precomposed unicode. Unlike on
HFS+, Mac OS X stores files on a VFAT drive (e.g. an USB drive) in
precomposed unicode, but readdir() still returns file names in
decomposed unicode. When a git repository is stored on a network
share using SAMBA, file names are send over the wire and written to
disk on the remote system in precomposed unicode, but Mac OS X
readdir() returns decomposed unicode to be compatible with its
behaviour on HFS+ and VFAT.
The unicode decomposition causes many problems:
- The names "git add" and other commands get from the end user may
often be precomposed form (the decomposed form is not easily input
from the keyboard), but when the commands read from the filesystem
to see what it is going to update the index with already is on the
filesystem, readdir() will give decomposed form, which is different.
- Similarly "git log", "git mv" and all other commands that need to
compare pathnames found on the command line (often but not always
precomposed form; a command line input resulting from globbing may
be in decomposed) with pathnames found in the tree objects (should
be precomposed form to be compatible with other systems and for
consistency in general).
- The same for names stored in the index, which should be
precomposed, that may need to be compared with the names read from
readdir().
NFS mounted from Linux is fully transparent and does not suffer from
the above.
As Mac OS X treats precomposed and decomposed file names as equal,
we can
- wrap readdir() on Mac OS X to return the precomposed form, and
- normalize decomposed form given from the command line also to the
precomposed form,
to ensure that all pathnames used in Git are always in the
precomposed form. This behaviour can be requested by setting
"core.precomposedunicode" configuration variable to true.
The code in compat/precomposed_utf8.c implements basically 4 new
functions: precomposed_utf8_opendir(), precomposed_utf8_readdir(),
precomposed_utf8_closedir() and precompose_argv(). The first three
are to wrap opendir(3), readdir(3), and closedir(3) functions.
The argv[] conversion allows to use the TAB filename completion done
by the shell on command line. It tolerates other tools which use
readdir() to feed decomposed file names into git.
When creating a new git repository with "git init" or "git clone",
"core.precomposedunicode" will be set "false".
The user needs to activate this feature manually. She typically
sets core.precomposedunicode to "true" on HFS and VFAT, or file
systems mounted via SAMBA.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-08 17:50:25 +04:00
|
|
|
. ./test-lib.sh
|
|
|
|
|
|
2012-07-30 13:57:18 +04:00
|
|
|
if ! test_have_prereq UTF8_NFD_TO_NFC
|
|
|
|
|
then
|
|
|
|
|
skip_all="filesystem does not corrupt utf-8"
|
|
|
|
|
test_done
|
|
|
|
|
fi
|
|
|
|
|
|
|
|
|
|
# create utf-8 variables
|
2014-04-30 20:22:55 +04:00
|
|
|
Adiarnfc=$(printf '\303\204')
|
|
|
|
|
Adiarnfd=$(printf 'A\314\210')
|
git on Mac OS and precomposed unicode
Mac OS X mangles file names containing unicode on file systems HFS+,
VFAT or SAMBA. When a file using unicode code points outside ASCII
is created on a HFS+ drive, the file name is converted into
decomposed unicode and written to disk. No conversion is done if
the file name is already decomposed unicode.
Calling open("\xc3\x84", ...) with a precomposed "Ä" yields the same
result as open("\x41\xcc\x88",...) with a decomposed "Ä".
As a consequence, readdir() returns the file names in decomposed
unicode, even if the user expects precomposed unicode. Unlike on
HFS+, Mac OS X stores files on a VFAT drive (e.g. an USB drive) in
precomposed unicode, but readdir() still returns file names in
decomposed unicode. When a git repository is stored on a network
share using SAMBA, file names are send over the wire and written to
disk on the remote system in precomposed unicode, but Mac OS X
readdir() returns decomposed unicode to be compatible with its
behaviour on HFS+ and VFAT.
The unicode decomposition causes many problems:
- The names "git add" and other commands get from the end user may
often be precomposed form (the decomposed form is not easily input
from the keyboard), but when the commands read from the filesystem
to see what it is going to update the index with already is on the
filesystem, readdir() will give decomposed form, which is different.
- Similarly "git log", "git mv" and all other commands that need to
compare pathnames found on the command line (often but not always
precomposed form; a command line input resulting from globbing may
be in decomposed) with pathnames found in the tree objects (should
be precomposed form to be compatible with other systems and for
consistency in general).
- The same for names stored in the index, which should be
precomposed, that may need to be compared with the names read from
readdir().
NFS mounted from Linux is fully transparent and does not suffer from
the above.
As Mac OS X treats precomposed and decomposed file names as equal,
we can
- wrap readdir() on Mac OS X to return the precomposed form, and
- normalize decomposed form given from the command line also to the
precomposed form,
to ensure that all pathnames used in Git are always in the
precomposed form. This behaviour can be requested by setting
"core.precomposedunicode" configuration variable to true.
The code in compat/precomposed_utf8.c implements basically 4 new
functions: precomposed_utf8_opendir(), precomposed_utf8_readdir(),
precomposed_utf8_closedir() and precompose_argv(). The first three
are to wrap opendir(3), readdir(3), and closedir(3) functions.
The argv[] conversion allows to use the TAB filename completion done
by the shell on command line. It tolerates other tools which use
readdir() to feed decomposed file names into git.
When creating a new git repository with "git init" or "git clone",
"core.precomposedunicode" will be set "false".
The user needs to activate this feature manually. She typically
sets core.precomposedunicode to "true" on HFS and VFAT, or file
systems mounted via SAMBA.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-08 17:50:25 +04:00
|
|
|
|
2014-04-30 20:22:55 +04:00
|
|
|
Odiarnfc=$(printf '\303\226')
|
|
|
|
|
Odiarnfd=$(printf 'O\314\210')
|
|
|
|
|
AEligatu=$(printf '\303\206')
|
|
|
|
|
Invalidu=$(printf '\303\377')
|
git on Mac OS and precomposed unicode
Mac OS X mangles file names containing unicode on file systems HFS+,
VFAT or SAMBA. When a file using unicode code points outside ASCII
is created on a HFS+ drive, the file name is converted into
decomposed unicode and written to disk. No conversion is done if
the file name is already decomposed unicode.
Calling open("\xc3\x84", ...) with a precomposed "Ä" yields the same
result as open("\x41\xcc\x88",...) with a decomposed "Ä".
As a consequence, readdir() returns the file names in decomposed
unicode, even if the user expects precomposed unicode. Unlike on
HFS+, Mac OS X stores files on a VFAT drive (e.g. an USB drive) in
precomposed unicode, but readdir() still returns file names in
decomposed unicode. When a git repository is stored on a network
share using SAMBA, file names are send over the wire and written to
disk on the remote system in precomposed unicode, but Mac OS X
readdir() returns decomposed unicode to be compatible with its
behaviour on HFS+ and VFAT.
The unicode decomposition causes many problems:
- The names "git add" and other commands get from the end user may
often be precomposed form (the decomposed form is not easily input
from the keyboard), but when the commands read from the filesystem
to see what it is going to update the index with already is on the
filesystem, readdir() will give decomposed form, which is different.
- Similarly "git log", "git mv" and all other commands that need to
compare pathnames found on the command line (often but not always
precomposed form; a command line input resulting from globbing may
be in decomposed) with pathnames found in the tree objects (should
be precomposed form to be compatible with other systems and for
consistency in general).
- The same for names stored in the index, which should be
precomposed, that may need to be compared with the names read from
readdir().
NFS mounted from Linux is fully transparent and does not suffer from
the above.
As Mac OS X treats precomposed and decomposed file names as equal,
we can
- wrap readdir() on Mac OS X to return the precomposed form, and
- normalize decomposed form given from the command line also to the
precomposed form,
to ensure that all pathnames used in Git are always in the
precomposed form. This behaviour can be requested by setting
"core.precomposedunicode" configuration variable to true.
The code in compat/precomposed_utf8.c implements basically 4 new
functions: precomposed_utf8_opendir(), precomposed_utf8_readdir(),
precomposed_utf8_closedir() and precompose_argv(). The first three
are to wrap opendir(3), readdir(3), and closedir(3) functions.
The argv[] conversion allows to use the TAB filename completion done
by the shell on command line. It tolerates other tools which use
readdir() to feed decomposed file names into git.
When creating a new git repository with "git init" or "git clone",
"core.precomposedunicode" will be set "false".
The user needs to activate this feature manually. She typically
sets core.precomposedunicode to "true" on HFS and VFAT, or file
systems mounted via SAMBA.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-08 17:50:25 +04:00
|
|
|
|
|
|
|
|
|
2012-07-30 13:57:18 +04:00
|
|
|
#Create a string with 255 bytes (decomposed)
|
|
|
|
|
Alongd=$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd #21 Byte
|
|
|
|
|
Alongd=$Alongd$Alongd$Alongd #63 Byte
|
|
|
|
|
Alongd=$Alongd$Alongd$Alongd$Alongd$Adiarnfd #255 Byte
|
git on Mac OS and precomposed unicode
Mac OS X mangles file names containing unicode on file systems HFS+,
VFAT or SAMBA. When a file using unicode code points outside ASCII
is created on a HFS+ drive, the file name is converted into
decomposed unicode and written to disk. No conversion is done if
the file name is already decomposed unicode.
Calling open("\xc3\x84", ...) with a precomposed "Ä" yields the same
result as open("\x41\xcc\x88",...) with a decomposed "Ä".
As a consequence, readdir() returns the file names in decomposed
unicode, even if the user expects precomposed unicode. Unlike on
HFS+, Mac OS X stores files on a VFAT drive (e.g. an USB drive) in
precomposed unicode, but readdir() still returns file names in
decomposed unicode. When a git repository is stored on a network
share using SAMBA, file names are send over the wire and written to
disk on the remote system in precomposed unicode, but Mac OS X
readdir() returns decomposed unicode to be compatible with its
behaviour on HFS+ and VFAT.
The unicode decomposition causes many problems:
- The names "git add" and other commands get from the end user may
often be precomposed form (the decomposed form is not easily input
from the keyboard), but when the commands read from the filesystem
to see what it is going to update the index with already is on the
filesystem, readdir() will give decomposed form, which is different.
- Similarly "git log", "git mv" and all other commands that need to
compare pathnames found on the command line (often but not always
precomposed form; a command line input resulting from globbing may
be in decomposed) with pathnames found in the tree objects (should
be precomposed form to be compatible with other systems and for
consistency in general).
- The same for names stored in the index, which should be
precomposed, that may need to be compared with the names read from
readdir().
NFS mounted from Linux is fully transparent and does not suffer from
the above.
As Mac OS X treats precomposed and decomposed file names as equal,
we can
- wrap readdir() on Mac OS X to return the precomposed form, and
- normalize decomposed form given from the command line also to the
precomposed form,
to ensure that all pathnames used in Git are always in the
precomposed form. This behaviour can be requested by setting
"core.precomposedunicode" configuration variable to true.
The code in compat/precomposed_utf8.c implements basically 4 new
functions: precomposed_utf8_opendir(), precomposed_utf8_readdir(),
precomposed_utf8_closedir() and precompose_argv(). The first three
are to wrap opendir(3), readdir(3), and closedir(3) functions.
The argv[] conversion allows to use the TAB filename completion done
by the shell on command line. It tolerates other tools which use
readdir() to feed decomposed file names into git.
When creating a new git repository with "git init" or "git clone",
"core.precomposedunicode" will be set "false".
The user needs to activate this feature manually. She typically
sets core.precomposedunicode to "true" on HFS and VFAT, or file
systems mounted via SAMBA.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-08 17:50:25 +04:00
|
|
|
|
2012-07-30 13:57:18 +04:00
|
|
|
#Create a string with 254 bytes (precomposed)
|
|
|
|
|
Alongc=$AEligatu$AEligatu$AEligatu$AEligatu$AEligatu #10 Byte
|
|
|
|
|
Alongc=$Alongc$Alongc$Alongc$Alongc$Alongc #50 Byte
|
|
|
|
|
Alongc=$Alongc$Alongc$Alongc$Alongc$Alongc #250 Byte
|
|
|
|
|
Alongc=$Alongc$AEligatu$AEligatu #254 Byte
|
git on Mac OS and precomposed unicode
Mac OS X mangles file names containing unicode on file systems HFS+,
VFAT or SAMBA. When a file using unicode code points outside ASCII
is created on a HFS+ drive, the file name is converted into
decomposed unicode and written to disk. No conversion is done if
the file name is already decomposed unicode.
Calling open("\xc3\x84", ...) with a precomposed "Ä" yields the same
result as open("\x41\xcc\x88",...) with a decomposed "Ä".
As a consequence, readdir() returns the file names in decomposed
unicode, even if the user expects precomposed unicode. Unlike on
HFS+, Mac OS X stores files on a VFAT drive (e.g. an USB drive) in
precomposed unicode, but readdir() still returns file names in
decomposed unicode. When a git repository is stored on a network
share using SAMBA, file names are send over the wire and written to
disk on the remote system in precomposed unicode, but Mac OS X
readdir() returns decomposed unicode to be compatible with its
behaviour on HFS+ and VFAT.
The unicode decomposition causes many problems:
- The names "git add" and other commands get from the end user may
often be precomposed form (the decomposed form is not easily input
from the keyboard), but when the commands read from the filesystem
to see what it is going to update the index with already is on the
filesystem, readdir() will give decomposed form, which is different.
- Similarly "git log", "git mv" and all other commands that need to
compare pathnames found on the command line (often but not always
precomposed form; a command line input resulting from globbing may
be in decomposed) with pathnames found in the tree objects (should
be precomposed form to be compatible with other systems and for
consistency in general).
- The same for names stored in the index, which should be
precomposed, that may need to be compared with the names read from
readdir().
NFS mounted from Linux is fully transparent and does not suffer from
the above.
As Mac OS X treats precomposed and decomposed file names as equal,
we can
- wrap readdir() on Mac OS X to return the precomposed form, and
- normalize decomposed form given from the command line also to the
precomposed form,
to ensure that all pathnames used in Git are always in the
precomposed form. This behaviour can be requested by setting
"core.precomposedunicode" configuration variable to true.
The code in compat/precomposed_utf8.c implements basically 4 new
functions: precomposed_utf8_opendir(), precomposed_utf8_readdir(),
precomposed_utf8_closedir() and precompose_argv(). The first three
are to wrap opendir(3), readdir(3), and closedir(3) functions.
The argv[] conversion allows to use the TAB filename completion done
by the shell on command line. It tolerates other tools which use
readdir() to feed decomposed file names into git.
When creating a new git repository with "git init" or "git clone",
"core.precomposedunicode" will be set "false".
The user needs to activate this feature manually. She typically
sets core.precomposedunicode to "true" on HFS and VFAT, or file
systems mounted via SAMBA.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-08 17:50:25 +04:00
|
|
|
|
2012-07-30 13:57:18 +04:00
|
|
|
test_expect_success "detect if nfd needed" '
|
2014-04-30 20:22:55 +04:00
|
|
|
precomposeunicode=$(git config core.precomposeunicode) &&
|
2013-08-27 17:50:40 +04:00
|
|
|
test "$precomposeunicode" = true &&
|
2012-07-30 13:57:18 +04:00
|
|
|
git config core.precomposeunicode true
|
|
|
|
|
'
|
|
|
|
|
test_expect_success "setup" '
|
|
|
|
|
>x &&
|
|
|
|
|
git add x &&
|
|
|
|
|
git commit -m "1st commit" &&
|
|
|
|
|
git rm x &&
|
|
|
|
|
git commit -m "rm x"
|
|
|
|
|
'
|
|
|
|
|
test_expect_success "setup case mac" '
|
|
|
|
|
git checkout -b mac_os
|
|
|
|
|
'
|
2016-05-13 23:41:02 +03:00
|
|
|
# This will test nfd2nfc in git diff
|
|
|
|
|
test_expect_success "git diff f.Adiar" '
|
|
|
|
|
touch f.$Adiarnfc &&
|
|
|
|
|
git add f.$Adiarnfc &&
|
|
|
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
|
|
|
git diff f.$Adiarnfd >expect &&
|
|
|
|
|
git diff f.$Adiarnfc >actual &&
|
|
|
|
|
test_cmp expect actual &&
|
|
|
|
|
git reset HEAD f.Adiarnfc &&
|
|
|
|
|
rm f.$Adiarnfc expect actual
|
|
|
|
|
'
|
|
|
|
|
# This will test nfd2nfc in git diff-files
|
|
|
|
|
test_expect_success "git diff-files f.Adiar" '
|
|
|
|
|
touch f.$Adiarnfc &&
|
|
|
|
|
git add f.$Adiarnfc &&
|
|
|
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
|
|
|
git diff-files f.$Adiarnfd >expect &&
|
|
|
|
|
git diff-files f.$Adiarnfc >actual &&
|
|
|
|
|
test_cmp expect actual &&
|
|
|
|
|
git reset HEAD f.Adiarnfc &&
|
|
|
|
|
rm f.$Adiarnfc expect actual
|
|
|
|
|
'
|
|
|
|
|
# This will test nfd2nfc in git diff-index
|
|
|
|
|
test_expect_success "git diff-index f.Adiar" '
|
|
|
|
|
touch f.$Adiarnfc &&
|
|
|
|
|
git add f.$Adiarnfc &&
|
|
|
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
|
|
|
git diff-index HEAD f.$Adiarnfd >expect &&
|
|
|
|
|
git diff-index HEAD f.$Adiarnfc >actual &&
|
|
|
|
|
test_cmp expect actual &&
|
|
|
|
|
git reset HEAD f.Adiarnfc &&
|
|
|
|
|
rm f.$Adiarnfc expect actual
|
|
|
|
|
'
|
2012-07-30 13:57:18 +04:00
|
|
|
# This will test nfd2nfc in readdir()
|
|
|
|
|
test_expect_success "add file Adiarnfc" '
|
|
|
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
|
|
|
git add f.$Adiarnfc &&
|
|
|
|
|
git commit -m "add f.$Adiarnfc"
|
|
|
|
|
'
|
2016-05-13 23:41:02 +03:00
|
|
|
# This will test nfd2nfc in git diff-tree
|
|
|
|
|
test_expect_success "git diff-tree f.Adiar" '
|
|
|
|
|
echo f.Adiarnfc >>f.$Adiarnfc &&
|
|
|
|
|
git diff-tree HEAD f.$Adiarnfd >expect &&
|
|
|
|
|
git diff-tree HEAD f.$Adiarnfc >actual &&
|
|
|
|
|
test_cmp expect actual &&
|
|
|
|
|
git checkout f.$Adiarnfc &&
|
|
|
|
|
rm expect actual
|
|
|
|
|
'
|
2012-07-30 13:57:18 +04:00
|
|
|
# This will test nfd2nfc in git stage()
|
|
|
|
|
test_expect_success "stage file d.Adiarnfd/f.Adiarnfd" '
|
|
|
|
|
mkdir d.$Adiarnfd &&
|
|
|
|
|
echo d.$Adiarnfd/f.$Adiarnfd >d.$Adiarnfd/f.$Adiarnfd &&
|
|
|
|
|
git stage d.$Adiarnfd/f.$Adiarnfd &&
|
|
|
|
|
git commit -m "add d.$Adiarnfd/f.$Adiarnfd"
|
|
|
|
|
'
|
|
|
|
|
test_expect_success "add link Adiarnfc" '
|
|
|
|
|
ln -s d.$Adiarnfd/f.$Adiarnfd l.$Adiarnfc &&
|
|
|
|
|
git add l.$Adiarnfc &&
|
|
|
|
|
git commit -m "add l.Adiarnfc"
|
|
|
|
|
'
|
|
|
|
|
# This will test git log
|
|
|
|
|
test_expect_success "git log f.Adiar" '
|
|
|
|
|
git log f.$Adiarnfc > f.Adiarnfc.log &&
|
|
|
|
|
git log f.$Adiarnfd > f.Adiarnfd.log &&
|
|
|
|
|
test -s f.Adiarnfc.log &&
|
|
|
|
|
test -s f.Adiarnfd.log &&
|
|
|
|
|
test_cmp f.Adiarnfc.log f.Adiarnfd.log &&
|
|
|
|
|
rm f.Adiarnfc.log f.Adiarnfd.log
|
|
|
|
|
'
|
|
|
|
|
# This will test git ls-files
|
|
|
|
|
test_expect_success "git lsfiles f.Adiar" '
|
|
|
|
|
git ls-files f.$Adiarnfc > f.Adiarnfc.log &&
|
|
|
|
|
git ls-files f.$Adiarnfd > f.Adiarnfd.log &&
|
|
|
|
|
test -s f.Adiarnfc.log &&
|
|
|
|
|
test -s f.Adiarnfd.log &&
|
|
|
|
|
test_cmp f.Adiarnfc.log f.Adiarnfd.log &&
|
|
|
|
|
rm f.Adiarnfc.log f.Adiarnfd.log
|
|
|
|
|
'
|
|
|
|
|
# This will test git mv
|
|
|
|
|
test_expect_success "git mv" '
|
|
|
|
|
git mv f.$Adiarnfd f.$Odiarnfc &&
|
|
|
|
|
git mv d.$Adiarnfd d.$Odiarnfc &&
|
|
|
|
|
git mv l.$Adiarnfd l.$Odiarnfc &&
|
|
|
|
|
git commit -m "mv Adiarnfd Odiarnfc"
|
|
|
|
|
'
|
|
|
|
|
# Files can be checked out as nfc
|
|
|
|
|
# And the link has been corrected from nfd to nfc
|
|
|
|
|
test_expect_success "git checkout nfc" '
|
|
|
|
|
rm f.$Odiarnfc &&
|
|
|
|
|
git checkout f.$Odiarnfc
|
|
|
|
|
'
|
|
|
|
|
# Make it possible to checkout files with their NFD names
|
|
|
|
|
test_expect_success "git checkout file nfd" '
|
|
|
|
|
rm -f f.* &&
|
|
|
|
|
git checkout f.$Odiarnfd
|
|
|
|
|
'
|
|
|
|
|
# Make it possible to checkout links with their NFD names
|
|
|
|
|
test_expect_success "git checkout link nfd" '
|
|
|
|
|
rm l.* &&
|
|
|
|
|
git checkout l.$Odiarnfd
|
|
|
|
|
'
|
|
|
|
|
test_expect_success "setup case mac2" '
|
|
|
|
|
git checkout master &&
|
|
|
|
|
git reset --hard &&
|
|
|
|
|
git checkout -b mac_os_2
|
|
|
|
|
'
|
|
|
|
|
# This will test nfd2nfc in git commit
|
|
|
|
|
test_expect_success "commit file d2.Adiarnfd/f.Adiarnfd" '
|
|
|
|
|
mkdir d2.$Adiarnfd &&
|
|
|
|
|
echo d2.$Adiarnfd/f.$Adiarnfd >d2.$Adiarnfd/f.$Adiarnfd &&
|
|
|
|
|
git add d2.$Adiarnfd/f.$Adiarnfd &&
|
|
|
|
|
git commit -m "add d2.$Adiarnfd/f.$Adiarnfd" -- d2.$Adiarnfd/f.$Adiarnfd
|
|
|
|
|
'
|
|
|
|
|
test_expect_success "setup for long decomposed filename" '
|
|
|
|
|
git checkout master &&
|
|
|
|
|
git reset --hard &&
|
|
|
|
|
git checkout -b mac_os_long_nfd_fn
|
|
|
|
|
'
|
|
|
|
|
test_expect_success "Add long decomposed filename" '
|
|
|
|
|
echo longd >$Alongd &&
|
|
|
|
|
git add * &&
|
|
|
|
|
git commit -m "Long filename"
|
|
|
|
|
'
|
|
|
|
|
test_expect_success "setup for long precomposed filename" '
|
|
|
|
|
git checkout master &&
|
|
|
|
|
git reset --hard &&
|
|
|
|
|
git checkout -b mac_os_long_nfc_fn
|
|
|
|
|
'
|
|
|
|
|
test_expect_success "Add long precomposed filename" '
|
|
|
|
|
echo longc >$Alongc &&
|
|
|
|
|
git add * &&
|
|
|
|
|
git commit -m "Long filename"
|
|
|
|
|
'
|
2014-04-28 20:16:31 +04:00
|
|
|
|
|
|
|
|
test_expect_failure 'handle existing decomposed filenames' '
|
|
|
|
|
echo content >"verbatim.$Adiarnfd" &&
|
|
|
|
|
git -c core.precomposeunicode=false add "verbatim.$Adiarnfd" &&
|
|
|
|
|
git commit -m "existing decomposed file" &&
|
|
|
|
|
git ls-files --exclude-standard -o "verbatim*" >untracked &&
|
2018-07-27 20:48:11 +03:00
|
|
|
test_must_be_empty untracked
|
2014-04-28 20:16:31 +04:00
|
|
|
'
|
|
|
|
|
|
2012-07-30 13:57:18 +04:00
|
|
|
# Test if the global core.precomposeunicode stops autosensing
|
|
|
|
|
# Must be the last test case
|
|
|
|
|
test_expect_success "respect git config --global core.precomposeunicode" '
|
|
|
|
|
git config --global core.precomposeunicode true &&
|
|
|
|
|
rm -rf .git &&
|
|
|
|
|
git init &&
|
2014-04-30 20:22:55 +04:00
|
|
|
precomposeunicode=$(git config core.precomposeunicode) &&
|
2012-07-30 13:57:18 +04:00
|
|
|
test "$precomposeunicode" = "true"
|
|
|
|
|
'
|
git on Mac OS and precomposed unicode
Mac OS X mangles file names containing unicode on file systems HFS+,
VFAT or SAMBA. When a file using unicode code points outside ASCII
is created on a HFS+ drive, the file name is converted into
decomposed unicode and written to disk. No conversion is done if
the file name is already decomposed unicode.
Calling open("\xc3\x84", ...) with a precomposed "Ä" yields the same
result as open("\x41\xcc\x88",...) with a decomposed "Ä".
As a consequence, readdir() returns the file names in decomposed
unicode, even if the user expects precomposed unicode. Unlike on
HFS+, Mac OS X stores files on a VFAT drive (e.g. an USB drive) in
precomposed unicode, but readdir() still returns file names in
decomposed unicode. When a git repository is stored on a network
share using SAMBA, file names are send over the wire and written to
disk on the remote system in precomposed unicode, but Mac OS X
readdir() returns decomposed unicode to be compatible with its
behaviour on HFS+ and VFAT.
The unicode decomposition causes many problems:
- The names "git add" and other commands get from the end user may
often be precomposed form (the decomposed form is not easily input
from the keyboard), but when the commands read from the filesystem
to see what it is going to update the index with already is on the
filesystem, readdir() will give decomposed form, which is different.
- Similarly "git log", "git mv" and all other commands that need to
compare pathnames found on the command line (often but not always
precomposed form; a command line input resulting from globbing may
be in decomposed) with pathnames found in the tree objects (should
be precomposed form to be compatible with other systems and for
consistency in general).
- The same for names stored in the index, which should be
precomposed, that may need to be compared with the names read from
readdir().
NFS mounted from Linux is fully transparent and does not suffer from
the above.
As Mac OS X treats precomposed and decomposed file names as equal,
we can
- wrap readdir() on Mac OS X to return the precomposed form, and
- normalize decomposed form given from the command line also to the
precomposed form,
to ensure that all pathnames used in Git are always in the
precomposed form. This behaviour can be requested by setting
"core.precomposedunicode" configuration variable to true.
The code in compat/precomposed_utf8.c implements basically 4 new
functions: precomposed_utf8_opendir(), precomposed_utf8_readdir(),
precomposed_utf8_closedir() and precompose_argv(). The first three
are to wrap opendir(3), readdir(3), and closedir(3) functions.
The argv[] conversion allows to use the TAB filename completion done
by the shell on command line. It tolerates other tools which use
readdir() to feed decomposed file names into git.
When creating a new git repository with "git init" or "git clone",
"core.precomposedunicode" will be set "false".
The user needs to activate this feature manually. She typically
sets core.precomposedunicode to "true" on HFS and VFAT, or file
systems mounted via SAMBA.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-08 17:50:25 +04:00
|
|
|
|
|
|
|
|
test_done
|
