Some feature names used in tests had minor differences compared to
the well-known ones from `curl -V`. This patch syncs them to make test
results easier to grok.
Closes#14183
When CRLF line end conversion was enabled (--crlf), input after the last
newline in the upload buffer was not sent, if the buffer contained a
newline.
Reported-by: vuonganh1993 on github
Fixes#14165Closes#14169
- disbable this test on WIN32 platforms. It uses the file describtor '1'
as valid socket without events. Not portable.
- reduce trace output somewhat on other runs
Fixes#14177
Reported-by: Viktor Szakats
Closes#14191
- make sure the TLS handshake after a successful STARTTLS command is
fully done before further sending/receiving on the connection.
Reported-by: tomy2105 on github
Fixes#14166Closes#14190
The man pages for curl_easy_getinfo, curl_easy_setopt and
curl_multi_setopt now feature the lists of options alphabetically
sorted. Test 1139 verify that they are.
The curl_multi_setopt page also got brief explanations of the listed
options.
Closes#14156
Add flags UIDNA_NONTRANSITIONAL_TO_ASCII and
UIDNA_NONTRANSITIONAL_TO_UNICODE to encode ß correctly.
It fixes test 165.
Reported-by: Viktor Szakats
Bug: #14176Closes#14179
It caused IDN to appear in `curl-config`, `libidn2` referenced from
`libcurl.pc`, fail to fallback to `pkg-config` detection. But libidn2
not actually used.
It came up in macOS CI builds after enabling cmake build tests. It
remained hidden for a while due to setting `-DUSE_APPLE_IDN=ON`.
(The half-detection of Homebrew libidn2 was the result of configuring
with `-DCMAKE_EXE_LINKER_FLAGS=-L$(brew --prefix)/lib`, to fix
linking GnuTLS that needs the `nettle` lib from the brew prefix.)
```
FAIL 1014: [Compare curl --version with curl-config --features] curl-config
```
Ref: https://github.com/curl/curl/actions/runs/9919357748/job/27405080722
Cherry-picked from #14097Closes#14175
In specific builds configs, cmake failed to build test `unit1600`,
due missing an OpenSSL (or wolfSSL) header.
The test code relies on `lib/curl_ntlm_core.h`, which in turn included
TLS library headers. But, dependency header directories are not setup
in cmake for tests, because they should not normally be needed.
The issue was hidden in most builds because TLS headers are usually
found under the system prefix. One counterexample is macOS + Homebrew
LibreSSL builds, where OpenSSL is purposefully unlinked from there to
avoid a mixup with LibreSSL that resides under its own prefix. It was
also hidden in autotools, possibly because it sets up header directories
globally, tests included.
The actual bug however is that `lib/curl_ntlm_core.h` should not include
TLS headers. None of its internal users need it, and `curl_ntlm_core.c`
included them already directly.
Fix it by deleting the TLS header includes from this internal header.
Fixes:
```
In file included from curl/tests/unit/unit1600.c:27:
curl/lib/curl_ntlm_core.h:32:12: fatal error: 'openssl/ssl.h' file not found
# include <openssl/ssl.h>
^~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9912684737/job/27388041520#step:12:1694
Follow-up to 48eb71ade4#10322
Cherry-picked from #14097Closes#14172
Fix `-Wpointer-bool-conversion` warnings with the method suggested by
both Apple clang and mainline llvm. This was already tried and dropped
in #1705 (in year 2017), but the issue reported there no longer
replicates.
Verified with Apple clang 14, llvm 15, llvm 18 and gcc 11, 14 that the
generated objects are bit by bit identical before and after this patch.
Also:
- stop silencing `-Wtautological-pointer-compare`. This warning don't
seem to be appearing anymore (with or without this patch), at least
with the tested compilers and SDKs (clang 13.1.6-16.0.0beta, llvm 15,
18, gcc 11, 14) and minimum macOS target of 10.8. Older targets fail
to build curl with SecureTransport.
- silence `-Wunreachable-code` for clang only. Previously I applied it
also to GCC, by mistake.
Ref: 8d7172d20a
Apple clang `-Wpointer-bool-conversion`:
```
curl/lib/vtls/sectransp.c:1103:6: error: address of function 'SSLCreateContext' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
if(SSLCreateContext) { /* use the newer API if available */
~~ ^~~~~~~~~~~~~~~~
curl/lib/vtls/sectransp.c:1103:6: note: prefix with the address-of operator to silence this warning
if(SSLCreateContext) { /* use the newer API if available */
^
&
```
Ref: https://github.com/curl/curl/actions/runs/9819538439/job/27113201384#step:8:382
llvm `-Wpointer-bool-conversion`:
```
curl/lib/vtls/sectransp.c:2663:8: error: address of function 'SSLCreateContext' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
if(SSLCreateContext)
~~ ^~~~~~~~~~~~~~~~
curl/lib/vtls/sectransp.c:2663:8: note: prefix with the address-of operator to silence this warning
if(SSLCreateContext)
^
&
```
Ref: https://github.com/curl/curl/actions/runs/9819538439/job/27113200291#step:8:417
gcc still needs `-Waddress` suppressed to avoid these:
```
curl/lib/vtls/n/sectransp.c: In function 'getsubject':
curl/lib/vtls/n/sectransp.c:379:6: warning: the address of 'SecCertificateCopyLongDescription' will always evaluate as 'true' [-Waddress]
379 | if(&SecCertificateCopyLongDescription)
| ^
[...]
```
Follow-up to 59cadacfcc#14128
Follow-up to af271ce9b9#1722
Follow-up to 2b7ce3f56d#1706
Cherry-picked from #14097Closes#14162
- bump parallel test for Linux jobs.
Credit-to: Dan Fandrich
Cherry-picked from #11510
- bump parallel test for macOS jobs.
- drop no longer necessary `-Wno-vla` option.
- fold long lines.
- drop `--enable-maintainer-mode` `./configure` option.
- replace a hard-coded prefix with `brew --prefix`.
- update documentation link.
- move `--enable-debug` in front.
- tidy up quotes.
Closes#14171
This PR fixes a leak and a crash that can happen when curl encounters
bad HTTPS RR values in DNS. We're starting to do better testing of that
kind of thing and e.g. have published bad HTTPS RR values at
dodgy.test.defo.ie.
Closes#14151
Fixup faulty target macro initialization in macOS SDK since v14.4 (as of
15.0 beta). The SDK target detection in `TargetConditionals.h` correctly
detects macOS, but fails to set the macro's old name `TARGET_OS_OSX`,
then continues to set it to a default value of 0. Other parts of the SDK
still rely on the old name, and with this inconsistency our builds fail
due to missing declarations. It happens when using mainline llvm older
than v18. Later versions fixed it by predefining these target macros,
avoiding the faulty dynamic detection. gcc is not affected (for now)
because it lacks the necessary dynamic detection features, so the SDK
falls back to a codepath that sets both the old and new macro to 1.
Also move the `TargetConditionals.h` include to the top of to make sure
including it also for c-ares builds, combined with SecureTransport or
other curl features that may call use an Apple SDK.
Before this patch, affected build combinations (e.g. in GHA runners,
llvm@15 + Xcode 15.3, 15.4, 16.0 with their default SDKs +
SecureTransport) fail with:
```
error: use of undeclared identifier 'noErr'
or 'SecCertificateCopyLongDescription'
or 'SecItemImportExportKeyParameters'
or 'SecExternalFormat'
or 'SecExternalItemType'
or 'SEC_KEY_IMPORT_EXPORT_PARAMS_VERSION'
```
Example:
```
curl/lib/vtls/sectransp.c:311:18: error: use of undeclared identifier 'noErr'
OSStatus rtn = noErr;
^
curl/lib/vtls/sectransp.c:379:7: error: use of undeclared identifier 'SecCertificateCopyLongDescription'
if(&SecCertificateCopyLongDescription)
^
curl/lib/vtls/sectransp.c:381:7: error: call to undeclared function 'SecCertificateCopyLongDescription'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
SecCertificateCopyLongDescription(NULL, cert, NULL);
^
curl/lib/vtls/sectransp.c:380:25: error: incompatible integer to pointer conversion assigning to 'CFStringRef' (aka 'const struct __CFString *') from 'int' [-Wint-conversion]
server_cert_summary =
^
[...]
```
Ref: https://github.com/curl/curl/actions/runs/9893867519/job/27330135969#step:10:22
llvm v18 patches implementing the predefined macros:
https://github.com/llvm/llvm-project/pull/746766e1f19168bhttps://github.com/llvm/llvm-project/pull/82833e5ed7b6e2f
Cherry-picked from #14097Closes#14159
Homebrew gcc builds starting with 12.4.0, 13.3.0 and 14.1.0 enabled
the `availability` attribute.
This broke builds because the way the Apple SDK uses attributes (when
available) are incompatible with how gcc accepts them. Causing these
errors:
```
error: attributes should be specified before the declarator in a function definition
error: expected ',' or '}' before
```
Upstream commits implementing the `availability` macro:
gcc-12: fd5530b7cb
gcc-13: cb7e4eca68
gcc-14: ff62a10886
The project above is a Darwin gcc compatibility pack, that is applied
to Homebrew gcc builds.
This patch works by redefining the `availability` macro to an invalid
value, making `__has_attribute(availability)` checks fail, stopping
Apple SDK from inserting the incompatible attributes.
It also replaces the previous, local workaround for `lib/macos.c`.
Example with gcc 12.4.0 with macOS SDK 14.0 (Xcode 15.0.1):
```
In file included from <path-to-sdk>/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:54,
from <path-to-sdk>/MacOSX14.0.sdk/System/Library/Frameworks/SystemConfiguration.framework/Headers/SCDynamicStoreCopySpecific.h:30,
from /Users/runner/work/curl/curl/lib/macos.c:33,
from /Users/runner/work/curl/curl/build/lib/CMakeFiles/libcurl_shared.dir/Unity/unity_0_c.c:244:
<path-to-sdk>/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFUserNotification.h:126:1: error: attributes should be specified before the declarator in a function definition
126 | CF_INLINE CFOptionFlags CFUserNotificationCheckBoxChecked(CFIndex i) API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOptionFlags)(1UL << (8 + i)));}
| ^~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9787982387/job/27025351601?pr=14096#step:7:18
The gcc vs. llvm/clang incompatibility possibly tracked here upstream:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108796
More info:
https://github.com/llvm/llvm-project/issues/817678433baadechttps://discourse.llvm.org/t/changing-attribute-ast-printing-location-for-gcc-compatibility/73215https://reviews.llvm.org/D159362
Follow-up to db135f8d72#14119
Ref: https://github.com/curl/curl/pull/14091#issuecomment-2222703468Fixes#13700
Cherry-picked from #14097Closes#14155
Also:
- GHA/non-native: install `pkg-config` to detect libidn2 with cmake
on NetBSD and FreeBSD.
- GHA/non-native: tidy-up `curl --version` command if here.
Cherry-picked from #14097Closes#14137
Xcode v15 (2023) or newer requires the built-in macro
`__ENVIRONMENT_OS_VERSION_MIN_REQUIRED__`. This macro is missing from
mainline llvm versions released earlier. llvm v17 introduced it here:
c8e2dd8c6f
This patch defines the missing macro when the necessary conditions
align, by using the value via the macro's old name.
The issue affected SecureTransport builds: The SecureTransport code,
`lib/md4.c` and `lib/md5.c`.
Existing gcc versions (as of v14) also don't define this macro, so apply
the patch to it as well. Even though gcc is incompatible in other ways,
so this isn't fixing an actual curl build case that I could find yet.
GHA macOS runner images have llvm v15 pre-installed, which broke builds
when building with an affected Xcode:
```
curl/lib/md4.c:80:14: error: '__ENVIRONMENT_OS_VERSION_MIN_REQUIRED__' is not defined, evaluates to 0 [-Werror,-Wundef]
(__MAC_OS_X_VERSION_MIN_REQUIRED < 101500)) || \
^
/Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/AvailabilityInternal.h:40:53: note: expanded from macro '__MAC_OS_X_VERSION_MIN_REQUIRED'
#define __MAC_OS_X_VERSION_MIN_REQUIRED __ENVIRONMENT_OS_VERSION_MIN_REQUIRED__
^
In file included from curl/build/lib/CMakeFiles/libcurl_shared.dir/Unity/unity_0_c.c:250:
curl/lib/md5.c:75:14: error: '__ENVIRONMENT_OS_VERSION_MIN_REQUIRED__' is not defined, evaluates to 0 [-Werror,-Wundef]
(__MAC_OS_X_VERSION_MIN_REQUIRED < 101500)) || \
^
/Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/AvailabilityInternal.h:40:53: note: expanded from macro '__MAC_OS_X_VERSION_MIN_REQUIRED'
#define __MAC_OS_X_VERSION_MIN_REQUIRED __ENVIRONMENT_OS_VERSION_MIN_REQUIRED__
^
2 errors generated.
```
Ref: https://github.com/curl/curl/actions/runs/9811974634/job/27095218578#step:4:20
Cherry-picked from #14097Closes#14134
- sync detection snippet between autotools and cmake
It wasn't causing issues, but it's useful to avoid unnecessary
differences while debugging.
- cmake: limit check to `APPLE`.
Ref: b05dc7eb35#14122
Cherry-picked from #14097Closes#14127
The framework this check detects is necessary for the function
`SCDynamicStoreCopyProxies()` used in `lib/macos.c`. Non-c-ares,
IPv6-enabled builds touch this codepath.
Limit the feature check for builds that actually need it.
It brings this in sync with CMake which already worked this way.
Cherry-picked from #14097Closes#14126
Before this patch, `SystemConfiguration` detection failed due to this
error when compiling the detection snippet:
```
/Applications/Xcode_15.3.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/TargetConditionals.h:140:50: error: missing binary operator before token "("
140 | #if !defined(__has_extension) || !__has_extension(define_target_os_macros)
| ^
```
Ref: https://github.com/curl/curl/actions/runs/9821817534/job/27117929218#step:6:1079
It occured with gcc-11 when combined with macOS SDK 14.4 and 14.5
(default SDKs in Xcode 15.3 and 15.4 respectively). It did not happen
with earlier releases.
Despite the failure in `./configure`, `lib/macos.c` compiled with
Apple's `TargetConditionals.h` just fine.
Turns out that including the `sys/types.h` header before the SDK
header fixes the error and makes the detection snippet compile.
Cherry-picked from #14097Closes#14130
- cmake: enable Apple-specific `-Werror=partial-availability` to match
autotools.
- autotools: enable `-pedantic-errors` with llvm/clang to match gcc and
CMake.
- autotools: enable `-Werror-implicit-function-declaration` for
llvm/clang to match gcc.
- cmake: enable `-Werror-implicit-function-declaration` to match
autotools.
- move `-Wpointer-bool-conversion` from autotools to the local file
(`sectransp.c`) it was meant to apply. This way it applies to all
build methods.
- autotoos: show `CURL_CFLAG_EXTRAS` in the `./configure` summary.
(it may contain `-Werror` and/or `-pedentic-errors`.)
Cherry-picked from #14097Closes#14128
Currently we're using WOLFSSL_MAX_ERROR_SZ to define the error buffer
size, this value is user defined which means it can be overwritten with
-DWOLFSSL_MAX_ERROR_SZ=512 when building wolfssl and this overwrite is
not exported to the users of wolfssl.
Instead of relying on WOLFSSL_MAX_ERROR_SZ we'll just use a 256 bytes
error buffer and use wolfSSL_ERR_error_string_n to fill it thus dropping
the dependency on WOLFSSL_MAX_ERROR_SZ altogether.
Closes#14114
- deduplicate the code in many tls backends that check
for an existing id and delete it before adding the new one
- rename ssl_primary_config's `sessionid` bool to `cache_session`
Closes#14121
Necessary to find generated files in the out-of-tree build directory.
E.g. `tests/configurehelp.pm`, for tests 1119 and 1167.
Before this patch macOS autotools builds were failing these two tests
due to falling back to the default preprocessor (`cpp`) instead of
the actual one configured. Then `cpp` failing to compile Apple SDK
headers referenced by curl headers.
Cherry-picked from #14097Closes#14124
- sort features case-insensitively to match `curl -V` and cmake.
`sort -f` is POSIX, but check if it's available anyway.
- make protocols lowercase to match `curl -V` and cmake.
- replace two outlier `$()` with backticks.
Closes#14117
`HAVE_BUILTIN_AVAILABLE` is a curl macro set via autotools and cmake.
Like other `HAVE_`s it signals availability if defined.
SecureTransport code was specifically looking for the value 1, which
triggered compiler warnings when the feature was not present.
Replace the existing workaround of locally suppressing the compiler
warning with using `defined()`.
autotools:
```
767 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
| ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c: In function 'sectransp_connect_step1':
../../lib/vtls/sectransp.c:1140:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
1140 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
| ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c:1240:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
1240 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
| ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c: In function 'sectransp_connect_step2':
```
Ref: https://github.com/curl/curl/actions/runs/9815428701/job/27104448045#step:6:499
cmake gcc:
```
1140 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
| ^~~~~~~~~~~~~~~~~~~~~~
/Users/runner/work/curl/curl/lib/vtls/sectransp.c:1240:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
1240 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
| ^~~~~~~~~~~~~~~~~~~~~~
/Users/runner/work/curl/curl/lib/vtls/sectransp.c: In function 'sectransp_connect_step2':
/Users/runner/work/curl/curl/lib/vtls/sectransp.c:2231:51: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
2231 | #if(CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
| ^~~~~~~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9815428701/job/27104445425#step:8:355
Cherry-picked from #14097Closes#14122
Viktor Szakats
Stefan Eissing
Daniel Stenberg
MonkeybreadSoftware
Stephen Farrell
Dan Fandrich
Martin Peck
renovate[bot]
Dominik Piątkowski
CMD
Alex Snast
Yedaya Katsman
Andy Reitz