Some feature names used in tests had minor differences compared to
the well-known ones from `curl -V`. This patch syncs them to make test
results easier to grok.
Closes#14183
When CRLF line end conversion was enabled (--crlf), input after the last
newline in the upload buffer was not sent, if the buffer contained a
newline.
Reported-by: vuonganh1993 on github
Fixes#14165Closes#14169
- disbable this test on WIN32 platforms. It uses the file describtor '1'
as valid socket without events. Not portable.
- reduce trace output somewhat on other runs
Fixes#14177
Reported-by: Viktor Szakats
Closes#14191
The man pages for curl_easy_getinfo, curl_easy_setopt and
curl_multi_setopt now feature the lists of options alphabetically
sorted. Test 1139 verify that they are.
The curl_multi_setopt page also got brief explanations of the listed
options.
Closes#14156
This PR fixes a leak and a crash that can happen when curl encounters
bad HTTPS RR values in DNS. We're starting to do better testing of that
kind of thing and e.g. have published bad HTTPS RR values at
dodgy.test.defo.ie.
Closes#14151
Necessary to find generated files in the out-of-tree build directory.
E.g. `tests/configurehelp.pm`, for tests 1119 and 1167.
Before this patch macOS autotools builds were failing these two tests
due to falling back to the default preprocessor (`cpp`) instead of
the actual one configured. Then `cpp` failing to compile Apple SDK
headers referenced by curl headers.
Cherry-picked from #14097Closes#14124
This was missed during some refactoring more than a year ago and is
causing a warning "Use of uninitialized value $path in pattern match".
Follow-up to 70d2fca2
Ref: #10818Closes#14113
Also:
- remove stray `ECH` and `HTTPSRR` from cmake protocol list.
- stop excluding `Debug` and `TrackMemory` in `test1013.pl`.
- configure: delete `CURL_CHECK_CURLDEBUG` check.
Ref: 065047dc62
This check was effectively doing nothing, except disabling
`--enable-curldebug` in `curl-config` for
Cygwin/MSYS/cegcc/OS2/AIX targets with c-ares enabled.
Closes#14096
The commit 6483813b was missing changes necessitated by 2abfc75 that
causes a crash. Also, use ARRAYSIZE() for cleaner code.
Follow-up to 6483813b
Ref #14055
This eliminates the need to run an extra help subcommand to get the
possible categories, reducing the friction in getting relevant help. The
help wording was also slightly tweaked for grammatical accuracy.
Closes#14055
Option cleanups:
--get is not upload
--form* are post
- added several options into ldap, smtp, imap and pop3
- shortened the category descriptions in the list
category curl fixes:
--create-dirs removed from 'curl'
--ftp-create-dirs removed from 'curl'
--netrc moved to 'auth' from 'curl'
--netrc-file moved to 'auth' from 'curl'
--netrc-optional moved to 'auth' from 'curl'
--no-buffer moved to 'output' from 'curl'
--no-clobber removed from 'curl'
--output removed from 'curl'
--output-dir removed from 'curl'
--remove-on-error removed from 'curl'
Add a "global" category:
- Made all "global" options set this category
Add a "deprecated" category:
- Moved the deprecated options to it (maybe they should not be in any
category long term)
Add a 'timeout' category
- Put a number of appropriate options in it
Add an 'ldap' category
- Put the LDAP related option in there
Remove categories "ECH" and "ipfs"
- They should not be categories. Had only one single option each.
Remove category "misc"
- It should not be a category as it is impossible to know when to browse
it.
--use-ascii moved to ftp and output
--xattr moved to output
--service-name moved to auth
Managen fixes:
- errors if an option is given a category name that is not already setup
for in code
- verifies that options set `scope: global` also is put in category
`global´
Closes#14101
- add a DEBUGASSERT for when a transfer's pollset should not be empty.
- move write unpausing from transfer loop into curl_easy_pause. This
make sure that the url_updatesocket() finds the correct state when
updating socket events.
- fix HTTP/2 proxy during connect phase to set sockets correctly
- fix test2600 to simulate a socket set
- move write unpausing from transfer loop into curl_easy_pause. This
make sure that the url_updatesocket() finds the correct state when
updating socket events.
- waiting for the resolver to deliver might not involve any sockets to
wait for. Do not generate a warning.
Fixes#14047Closes#14074
Based on the standards and guidelines we use for our documentation.
- expand contractions (they're => they are etc)
- host name = > hostname
- file name => filename
- user name = username
- man page => manpage
- run-time => runtime
- set-up => setup
- back-end => backend
- a HTTP => an HTTP
- Two spaces after a period => one space after period
Closes#14073
When an individual file ended with a quote (typically an example), the
render function would return without ending the quote correctly with a
".fi" (fill in) in the manpage output.
This made the additional text provided below to render wrongly.
Closes#14048
Fix issues detected.
Also:
- One of the `.vc` files used LF EOLs, while the other didn't.
Make that one also use LF EOLs, as this is apparently supported by
`nmake`.
- Drop `.dsw` and `.btn` types from `.gitattributes`.
The repository doesn't use them.
- Sync section order with the rest of files in
`tests/certs/EdelCurlRoot-ca.prm`.
- Indent/align `.prm` and `.pem` files.
- Delete dummy `[something]` section from `.prm` and `.pem` files.
Mental note:
MSVC `.sln` files seem to accept spaces for indentation and also support
LF line-endings. I cannot test this and I don't know what's more
convenient when updating them, so left them as-is, with specific
exclusions.
Closes#14031
It needs to be set to the leading digits and dots only, so that the
`-[date]` suffix strings are not included, as those used in the daily
snapshots.
Fixes#14035
Reported-by: Marcel Raad
Closes#14036
When libcurl discards a connection there are two phases this may go
through: "shutdown" and "closing". If a connection is aborted, the
shutdown phase is skipped and it is closed right away.
The connection filters attached to the connection implement the phases
in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a
`shutdown` flags next to `connected` to keep track of the shutdown
operation.
Filters are shut down from top to bottom. If a filter is not connected,
its shutdown is skipped. Notable filters that *do* something during
shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends
its close notify and expects to receive a close notify from the server.
As sends and receives may EAGAIN on the network, a shutdown is often not
successful right away and needs to poll the connection's socket(s). To
facilitate this, such connections are placed on a new shutdown list
inside the connection cache.
Since managing this list requires the cooperation of a multi handle,
only the connection cache belonging to a multi handle is used. If a
connection was in another cache when being discarded, it is removed
there and added to the multi's cache. If no multi handle is available at
that time, the connection is shutdown and closed in a one-time,
best-effort attempt.
When a multi handle is destroyed, all connection still on the shutdown
list are discarded with a final shutdown attempt and close. In curl
debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be
set to make this graceful with a timeout in milliseconds given by the
variable.
The shutdown list is limited to the max number of connections configured
for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the
limit is reached, the oldest connection on the shutdown list is
discarded.
- In multi_wait() and multi_waitfds(), collect all connection caches
involved (each transfer might carry its own) into a temporary list.
Let each connection cache on the list contribute sockets and
POLLIN/OUT events it's connections are waiting for.
- in multi_perform() collect the connection caches the same way and let
them peform their maintenance. This will make another non-blocking
attempt to shutdown all connections on its shutdown list.
- for event based multis (multi->socket_cb set), add the sockets and
their poll events via the callback. When `multi_socket()` is invoked
for a socket not known by an active transfer, forward this to the
multi's cache for processing. On closing a connection, remove its
socket(s) via the callback.
TLS connection filters MUST NOT send close nofity messages in their
`do_close()` implementation. The reason is that a TLS close notify
signals a success. When a connection is aborted and skips its shutdown
phase, the server needs to see a missing close notify to detect
something has gone wrong.
A graceful shutdown of FTP's data connection is performed implicitly
before regarding the upload/download as complete and continuing on the
control connection. For FTP without TLS, there is just the socket close
happening. But with TLS, the sent/received close notify signals that the
transfer is complete and healthy. Servers like `vsftpd` verify that and
reject uploads without a TLS close notify.
- added test_19_* for shutdown related tests
- test_19_01 and test_19_02 test for TCP RST packets
which happen without a graceful shutdown and should
no longer appear otherwise.
- add test_19_03 for handling shutdowns by the server
- add test_19_04 for handling shutdowns by curl
- add test_19_05 for event based shutdowny by server
- add test_30_06/07 and test_31_06/07 for shutdown checks
on FTP up- and downloads.
Closes#13976
Since the framework is already returning that variable by default.
Avoids a warning for unreachable code.
Reported-by: Tal Regev
Fixes#13967Closes#13973
The function we use is called 'gnutls_x509_crt_check_hostname()' but if
we pass in the hostname with a trailing dot, the check fails. If we pass
in the SNI name, which cannot have a trailing dot, it succeeds for
https://pyropus.ca./
I consider this as a flaw in GnuTLS and have submitted this issue
upstream:
https://gitlab.com/gnutls/gnutls/-/issues/1548
In order to work with old and existing GnuTLS versions, we still need
this change no matter how they view the issue or might change it in the
future.
Fixes#13428
Reported-by: Ryan Carsten Schmidt
Closes#13949
- Parse etag and content-disposition headers for 3xx replies.
For example, a server may send a content-disposition filename header
with a redirect reply (3xx) but not with the final response (2xx).
Without this change curl would ignore the server's specified filename
and continue to use the filename extracted from the user-specified URL.
Prior to this change, 75d79a4 had limited etag and content-disposition
to 2xx replies only.
Tests-by: Daniel Stenberg
Reported-by: Morgan Willcock
Fixes https://github.com/curl/curl/issues/13302Closes#13484
As the list of variable names grows, doing a simple loop to find the
name get increasingly worse. This switches to a bsearch.
Also: do a case sensitive check for the variable name. The names have
not been documented to be case insensitive and there is no point in
having them so.
Closes#13914
This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.
- add cfilter `Curl_cft_shutdown` callback
- keep a shutdown start timestamp and timeout at connectdata
- provide shutdown timeout default and member in
`data->set.shutdowntimeout`.
- provide methods for starting, interrogating and clearing
shutdown timers
- provide `Curl_conn_shutdown_blocking()` to shutdown the
`sockindex` filter chain in a blocking way. Use that in FTP.
- add `Curl_conn_cf_poll()` to wait for socket events during
shutdown of a connection filter chain.
This gets the monitoring sockets and events via the filters
"adjust_pollset()" methods. This gives correct behaviour when
shutting down a TLS connection through a HTTP/2 proxy.
- Implement shutdown for all socket filters
- for HTTP/2 and h2 proxying to send GOAWAY
- for TLS backends to the best of their capabilities
- for tcp socket filter to make a final, nonblocking
receive to avoid unwanted RST states
- add shutdown forwarding to happy eyeballers and
https connect ballers when applicable.
Closes#13904
Add new job to test building for UWP (aka `CURL_WINDOWS_APP`).
Fix fallouts when building for UWP:
- rand: do not use `BCryptGenRandom()`.
- cmake: disable using win32 LDAP.
- cmake: disable telnet.
- version_win32: fix code before declaration.
- schannel: disable `HAS_MANUAL_VERIFY_API`.
- schannel: disable `SSLSUPP_PINNEDPUBKEY`
and make `schannel_checksum()` a stub.
Ref: e178fbd40a#1429
- schannel: make `cert_get_name_string()` a failing stub.
- system_win32: make `Curl_win32_impersonating()` a failing stub.
- system_win32: try to fix `Curl_win32_init()` (untested).
- threads: fix to use `CreateThread()`.
- src: disable searching `PATH` for the CA bundle.
- src: disable bold text support and capability detection.
- src: disable `getfiletime()`/`setfiletime()`.
- tests: make `win32_load_system_library()` a failing stub.
- tests/server/util: make it compile.
- tests/server/sockfilt: make it compile.
- tests/lib3026: fix to use `CreateThread()`.
See individual commits for build error details.
Some of these fixes may have better solutions, and some may not work
as expected. The goal of this patch is to make curl build for UWP.
Closes#13870
Introduce new notation for CURLOPT_INTERFACE / --interface:
ifhost!<interface>!<host>
Binding to an interface doesn't set the address, and an interface can
have multiple addresses.
When binding to an address (without interface), the kernel is free to
choose the route, and it can route through any device that can access
the target address, not necessarily the one with the chosen address.
Moreover, it is possible for different interfaces to have the same IP
address, on which case we need to provide a way to be more specific.
Factor out the parsing part of interface option, and add unit tests:
1663.
Closes#13719
- add special sauce to disable unwanted peer verification by mbedtls
when negotiating TLS v1.3
- add special sauce for MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET
return code on *writing* TLS data. We assume the data had not been
written and EAGAIN.
- return correct Curl error code when peer verification failed.
- disable test_08_05 with 50 HTTP/1.1 connections, as mbedtls reports a
memory allocation failed during handshake.
- bump CI mbedtls version to 3.6.0
Fixes#13653Closes#13838
Viktor Szakats
Stefan Eissing
Daniel Stenberg
Stephen Farrell
Dan Fandrich
Sergey Markelov
Jay Satiro
Andy Pan
Guilherme Puida
Orgad Shaneh