FAQ: Update FTP directory listing section for MLSD command

Explain how some FTP servers support the machine readable listing
format MLSD from RFC 3659 and compare it to LIST.

Ref: https://github.com/curl/curl/issues/906
This commit is contained in:
Jay Satiro 2016-07-09 03:05:55 -04:00
Родитель 7c9cfd6c51
Коммит f9eed596a3
1 изменённых файлов: 20 добавлений и 10 удалений

Просмотреть файл

@ -1402,17 +1402,27 @@ FAQ
CURLOPT_CUSTOMREQUEST to alter what exact listing command libcurl would use
to list the files.
The follow-up question that tend to follow the previous one, is how a
program is supposed to parse the directory listing. How does it know what's
a file and what's a dir and what's a symlink etc. The harsh reality is that
FTP provides no such fine and easy-to-parse output. The output format FTP
servers respond to LIST commands are entirely at the server's own liking and
the NLST output doesn't reveal any types and in many cases don't even
include all the directory entries. Also, both LIST and NLST tend to hide
unix-style hidden files (those that start with a dot) by default so you need
to do "LIST -a" or similar to see them.
The follow-up question tends to be how is a program supposed to parse the
directory listing. How does it know what's a file and what's a dir and what's
a symlink etc. If the FTP server supports the MLSD command then it will
return data in a machine-readable format that can be parsed for type. The
types are specified by RFC3659 section 7.5.1. If MLSD is not supported then
you have to work with what you're given. The LIST output format is entirely
at the server's own liking and the NLST output doesn't reveal any types and
in many cases doesn't even include all the directory entries. Also, both LIST
and NLST tend to hide unix-style hidden files (those that start with a dot)
by default so you need to do "LIST -a" or similar to see them.
The application thus needs to parse the LIST output. One such existing
Example - List only directories.
ftp.funet.fi supports MLSD and ftp.kernel.org does not:
curl -s ftp.funet.fi/pub/ -X MLSD | \
perl -lne 'print if s/(?:^|;)type=dir;[^ ]+ (.+)$/$1/'
curl -s ftp.kernel.org/pub/linux/kernel/ | \
perl -lne 'print if s/^d[-rwx]{9}(?: +[^ ]+){7} (.+)$/$1/'
If you need to parse LIST output in libcurl one such existing
list parser is available at https://cr.yp.to/ftpparse.html Versions of
libcurl since 7.21.0 also provide the ability to specify a wildcard to
download multiple files from one FTP directory.