Граф коммитов

21 Коммитов

Автор SHA1 Сообщение Дата
Junio C Hamano ef49a7a012 zlib: zlib can only process 4GB at a time
The size of objects we read from the repository and data we try to put
into the repository are represented in "unsigned long", so that on larger
architectures we can handle objects that weigh more than 4GB.

But the interface defined in zlib.h to communicate with inflate/deflate
limits avail_in (how many bytes of input are we calling zlib with) and
avail_out (how many bytes of output from zlib are we ready to accept)
fields effectively to 4GB by defining their type to be uInt.

In many places in our code, we allocate a large buffer (e.g. mmap'ing a
large loose object file) and tell zlib its size by assigning the size to
avail_in field of the stream, but that will truncate the high octets of
the real size. The worst part of this story is that we often pass around
z_stream (the state object used by zlib) to keep track of the number of
used bytes in input/output buffer by inspecting these two fields, which
practically limits our callchain to the same 4GB limit.

Wrap z_stream in another structure git_zstream that can express avail_in
and avail_out in unsigned long. For now, just die() when the caller gives
a size that cannot be given to a single zlib call. In later patches in the
series, we would make git_inflate() and git_deflate() internally loop to
give callers an illusion that our "improved" version of zlib interface can
operate on a buffer larger than 4GB in one go.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-10 11:52:15 -07:00
Junio C Hamano 5e86c1fb86 zlib: wrap inflateInit2 used to accept only for gzip format
http-backend.c uses inflateInit2() to tell the library that it wants to
accept only gzip format. Wrap it in a helper function so that readers do
not have to wonder what the magic numbers 15 and 16 are for.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-10 10:51:02 -07:00
Junio C Hamano 9e7e5ca372 zlib: wrap remaining calls to direct inflate/inflateEnd
Two callsites in http-backend.c to inflate() and inflateEnd()
were not using git_ prefixed versions.  After this, running

    $ find all objects -print | xargs nm -ugo | grep inflate

shows only zlib.c makes direct calls to zlib for inflate operation,
except for a singlecall to inflateInit2 in http-backend.c

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-10 10:39:27 -07:00
Tay Ray Chuan cf688cc273 http-backend: use end_url_with_slash()
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-26 14:50:45 -08:00
Junio C Hamano a53deac89e Merge branch 'jp/string-list-api-cleanup'
* jp/string-list-api-cleanup:
  string_list: Fix argument order for string_list_append
  string_list: Fix argument order for string_list_lookup
  string_list: Fix argument order for string_list_insert_at_index
  string_list: Fix argument order for string_list_insert
  string_list: Fix argument order for for_each_string_list
  string_list: Fix argument order for print_string_list
2010-06-30 11:55:38 -07:00
Julian Phillips e8c8b7139c string_list: Fix argument order for string_list_lookup
Update the definition and callers of string_list_lookup to use the
string_list as the first argument.  This helps make the string_list
API easier to use by being more consistent.

Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-27 10:06:51 -07:00
Julian Phillips 78a395d371 string_list: Fix argument order for string_list_insert
Update the definition and callers of string_list_insert to use the
string_list as the first argument.  This helps make the string_list
API easier to use by being more consistent.

Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-27 10:06:51 -07:00
Junio C Hamano 762655010d Merge branch 'js/async-thread'
* js/async-thread:
  fast-import: die_nicely() back to vsnprintf (reverts part of ebaa79f)
  Enable threaded async procedures whenever pthreads is available
  Dying in an async procedure should only exit the thread, not the process.
  Reimplement async procedures using pthreads
  Windows: more pthreads functions
  Fix signature of fcntl() compatibility dummy
  Make report() from usage.c public as vreportf() and use it.
  Modernize t5530-upload-pack-error.

Conflicts:
	http-backend.c
2010-06-21 06:02:45 -07:00
Jeff King 638794cde0 make url-related functions reusable
The is_url function and url percent-decoding functions were
static, but are generally useful. Let's make them available
to other parts of the code.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-24 16:48:32 -07:00
Shawn O. Pearce 5856b5f568 http-backend: Don't infinite loop during die()
If stdout has already been closed by the CGI and die() gets called,
the CGI will fail to write the "Status: 500 Internal Server Error" to
the pipe, which results in die() being called again (via safe_write).
This goes on in an infinite loop until the stack overflows and the
process is killed by SIGSEGV.

Instead set a flag on the first die() invocation and if we came back to
the handler, just die silently, as it only means we failed to report the
failure---we cannot report anything anyway in such a case.  This way
failures to write the error messages to the stdout pipe do not result in
an infinite loop.

We also now report on the death to stderr before we report to stdout,
to increase the chances that the cause of the die() invocation will
appear in the server's error log.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fixup! http-backend.c: Don't infinite loop

Now die_webcgi() actually can return during a recursive call into it,
causing

    http-backend.c:554: error: 'noreturn' function does return

The only reason we would come back to the die handler is because we
failed during it, so we cannot report anything anyway.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-03-24 14:40:56 -07:00
Johannes Sixt ebaa79f462 Make report() from usage.c public as vreportf() and use it.
There exist already a number of static functions named 'report', therefore,
the function name was changed.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-03-07 00:37:35 -08:00
Tarmigan Casebolt 8b2bd7cdac Smart-http: check if repository is OK to export before serving it
Similar to how git-daemon checks whether a repository is OK to be
exported, smart-http should also check.  This check can be satisfied
in two different ways: the environmental variable GIT_HTTP_EXPORT_ALL
may be set to export all repositories, or the individual repository
may have the file git-daemon-export-ok.

Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Tarmigan Casebolt <tarmigan+git@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-01-06 01:16:50 -08:00
Tarmigan Casebolt 354870171b http-backend: Let gcc check the format of more printf-type functions.
We already have these checks in many printf-type functions that have
prototypes which are in header files.  Add these same checks to
static functions in http-backend.c

Signed-off-by: Tarmigan Casebolt <tarmigan+git@gmail.com>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-15 22:15:01 -08:00
Tarmigan Casebolt 48aec1b1f1 http-backend: Fix access beyond end of string.
Found with valgrind while looking for Content-Length corruption in
smart http.

Signed-off-by: Tarmigan Casebolt <tarmigan+git@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-15 22:14:51 -08:00
Shawn O. Pearce 4a5328d644 http-backend: Fix bad treatment of uintmax_t in Content-Length
Our Content-Length needs to report an off_t, which could be larger
precision than size_t on this system (e.g. 32 bit binary built with
64 bit large file support).

We also shouldn't be passing a size_t parameter to printf when
we've used PRIuMAX as the format specifier.

Fix both issues by using uintmax_t for the hdr_int() routine,
allowing strbuf's size_t to automatically upcast, and off_t to
always fit.

Also fixed the copy loop we use inside of send_local_file(), we never
actually updated the size variable so we might as well not use it.

Reported-by: Tarmigan <tarmigan+git@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-13 14:40:05 -08:00
Shawn O. Pearce 34b6cb8bb0 http-backend: Protect GIT_PROJECT_ROOT from /../ requests
Eons ago HPA taught git-daemon how to protect itself from /../
attacks, which Junio brought back into service in d79374c7b5
("daemon.c and path.enter_repo(): revamp path validation").

I did not carry this into git-http-backend as originally we relied
only upon PATH_TRANSLATED, and assumed the HTTP server had done
its access control checks to validate the resolved path was within
a directory permitting access from the remote client.  This would
usually be sufficient to protect a server from requests for its
/etc/passwd file by http://host/smart/../etc/passwd sorts of URLs.

However in 917adc0360 Mark Lodato added GIT_PROJECT_ROOT as an
additional method of configuring the CGI.  When this environment
variable is used the web server does not generate the final access
path and therefore may blindly pass through "/../etc/passwd"
in PATH_INFO under the assumption that "/../" might have special
meaning to the invoked CGI.

Instead of permitting these sorts of malformed path requests, we
now reject them back at the client, with an error message for the
server log.  This matches git-daemon behavior.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-09 16:37:33 -08:00
Shawn O. Pearce 92815b3363 Git-aware CGI to provide dumb HTTP transport
http-backend: Fix symbol clash on AIX 5.3

Mike says:

 > > +static void send_file(const char *the_type, const char *name)
 > > +{
 >
 > I think a symbol clash here is responsible for a build breakage in
 > next on AIX 5.3:
 >
 > CC http-backend.o
 > http-backend.c:213: error: conflicting types for `send_file'
 > /usr/include/sys/socket.h:676: error: previous declaration of `send_file'
 > gmake: *** [http-backend.o] Error 1

So we rename the function send_local_file().

Reported-by: Mike Ralphson <mike.ralphson@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-09 09:30:01 -08:00
Shawn O. Pearce 5abb013b3d http-backend: Use http.getanyfile to disable dumb HTTP serving
Some repository owners may wish to enable smart HTTP, but disallow
dumb content serving.  Disallowing dumb serving might be because
the owners want to rely upon reachability to control which objects
clients may access from the repository, or they just want to
encourage clients to use the more bandwidth efficient transport.

If http.getanyfile is set to false the backend CGI will return with
'403 Forbidden' when an object file is accessed by a dumb client.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-04 17:58:16 -08:00
Mark Lodato 917adc0360 http-backend: add GIT_PROJECT_ROOT environment var
Add a new environment variable, GIT_PROJECT_ROOT, to override the
method of using PATH_TRANSLATED to find the git repository on disk.
This makes it much easier to configure the web server, especially when
the web server's DocumentRoot does not contain the git repositories,
which is the usual case.

Signed-off-by: Mark Lodato <lodatom@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-04 17:58:15 -08:00
Shawn O. Pearce 556cfa3b6d Smart fetch and push over HTTP: server side
Requests for $GIT_URL/git-receive-pack and $GIT_URL/git-upload-pack
are forwarded to the corresponding backend process by directly
executing it and leaving stdin and stdout connected to the invoking
web server.  Prior to starting the backend process the HTTP response
headers are sent, thereby freeing the backend from needing to know
about the HTTP protocol.

Requests that are encoded with Content-Encoding: gzip are
automatically inflated before being streamed into the backend.
This is primarily useful for the git-upload-pack backend, which
receives highly repetitive text data from clients that easily
compresses to 50% of its original size.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-04 17:58:15 -08:00
Shawn O. Pearce 2f4038ab33 Git-aware CGI to provide dumb HTTP transport
The git-http-backend CGI can be configured into any Apache server
using ScriptAlias, such as with the following configuration:

  LoadModule cgi_module /usr/libexec/apache2/mod_cgi.so
  LoadModule alias_module /usr/libexec/apache2/mod_alias.so
  ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/

Repositories are accessed via the translated PATH_INFO.

The CGI is backwards compatible with the dumb client, allowing all
older HTTP clients to continue to download repositories which are
managed by the CGI.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-04 17:58:04 -08:00