2022-08-04 19:28:36 +03:00
|
|
|
gitprotocol-v2(5)
|
|
|
|
=================
|
|
|
|
|
|
|
|
NAME
|
|
|
|
----
|
|
|
|
gitprotocol-v2 - Git Wire Protocol, Version 2
|
|
|
|
|
|
|
|
SYNOPSIS
|
|
|
|
--------
|
|
|
|
[verse]
|
|
|
|
<over-the-wire-protocol>
|
|
|
|
|
|
|
|
DESCRIPTION
|
|
|
|
-----------
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
This document presents a specification for a version 2 of Git's wire
|
|
|
|
protocol. Protocol v2 will improve upon v1 in the following ways:
|
|
|
|
|
|
|
|
* Instead of multiple service names, multiple commands will be
|
|
|
|
supported by a single service
|
|
|
|
* Easily extendable as capabilities are moved into their own section
|
|
|
|
of the protocol, no longer being hidden behind a NUL byte and
|
|
|
|
limited by the size of a pkt-line
|
|
|
|
* Separate out other information hidden behind NUL bytes (e.g. agent
|
|
|
|
string as a capability and symrefs can be requested using 'ls-refs')
|
|
|
|
* Reference advertisement will be omitted unless explicitly requested
|
|
|
|
* ls-refs command to explicitly request some refs
|
|
|
|
* Designed with http and stateless-rpc in mind. With clear flush
|
|
|
|
semantics the http remote helper can simply act as a proxy
|
|
|
|
|
|
|
|
In protocol v2 communication is command oriented. When first contacting a
|
|
|
|
server a list of capabilities will advertised. Some of these capabilities
|
|
|
|
will be commands which a client can request be executed. Once a command
|
|
|
|
has completed, a client can reuse the connection and request that other
|
|
|
|
commands be executed.
|
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
Packet-Line Framing
|
|
|
|
-------------------
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
All communication is done using packet-line framing, just as in v1. See
|
2022-08-04 19:28:36 +03:00
|
|
|
linkgit:gitprotocol-pack[5] and linkgit:gitprotocol-common[5] for more information.
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
In protocol v2 these special packets will have the following semantics:
|
|
|
|
|
|
|
|
* '0000' Flush Packet (flush-pkt) - indicates the end of a message
|
|
|
|
* '0001' Delimiter Packet (delim-pkt) - separates sections of a message
|
2021-02-18 03:11:22 +03:00
|
|
|
* '0002' Response End Packet (response-end-pkt) - indicates the end of a
|
|
|
|
response for stateless connections
|
2018-03-15 20:31:19 +03:00
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
Initial Client Request
|
|
|
|
----------------------
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
In general a client can request to speak protocol v2 by sending
|
|
|
|
`version=2` through the respective side-channel for the transport being
|
|
|
|
used which inevitably sets `GIT_PROTOCOL`. More information can be
|
2022-08-04 19:28:41 +03:00
|
|
|
found in linkgit:gitprotocol-pack[5] and linkgit:gitprotocol-http[5], as well as the
|
2021-09-10 17:10:19 +03:00
|
|
|
`GIT_PROTOCOL` definition in `git.txt`. In all cases the
|
2018-03-15 20:31:19 +03:00
|
|
|
response from the server is the capability advertisement.
|
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
Git Transport
|
|
|
|
~~~~~~~~~~~~~
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
When using the git:// transport, you can request to use protocol v2 by
|
|
|
|
sending "version=2" as an extra parameter:
|
|
|
|
|
|
|
|
003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
|
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
SSH and File Transport
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
When using either the ssh:// or file:// transport, the GIT_PROTOCOL
|
|
|
|
environment variable must be set explicitly to include "version=2".
|
2021-09-10 17:10:19 +03:00
|
|
|
The server may need to be configured to allow this environment variable
|
|
|
|
to pass.
|
2018-03-15 20:31:19 +03:00
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
HTTP Transport
|
|
|
|
~~~~~~~~~~~~~~
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
When using the http:// or https:// transport a client makes a "smart"
|
2022-08-04 19:28:41 +03:00
|
|
|
info/refs request as described in linkgit:gitprotocol-http[5] and requests that
|
2018-03-15 20:31:19 +03:00
|
|
|
v2 be used by supplying "version=2" in the `Git-Protocol` header.
|
|
|
|
|
|
|
|
C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0
|
2018-06-22 22:01:12 +03:00
|
|
|
C: Git-Protocol: version=2
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
A v2 server would reply:
|
|
|
|
|
|
|
|
S: 200 OK
|
|
|
|
S: <Some headers>
|
|
|
|
S: ...
|
|
|
|
S:
|
|
|
|
S: 000eversion 2\n
|
|
|
|
S: <capability-advertisement>
|
|
|
|
|
|
|
|
Subsequent requests are then made directly to the service
|
|
|
|
`$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
|
|
|
|
|
2021-08-05 04:25:43 +03:00
|
|
|
Uses the `--http-backend-info-refs` option to
|
|
|
|
linkgit:git-upload-pack[1].
|
|
|
|
|
2021-09-10 17:10:19 +03:00
|
|
|
The server may need to be configured to pass this header's contents via
|
|
|
|
the `GIT_PROTOCOL` variable. See the discussion in `git-http-backend.txt`.
|
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
Capability Advertisement
|
|
|
|
------------------------
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
A server which decides to communicate (based on a request from a client)
|
|
|
|
using protocol version 2, notifies the client by sending a version string
|
|
|
|
in its initial response followed by an advertisement of its capabilities.
|
|
|
|
Each capability is a key with an optional value. Clients must ignore all
|
|
|
|
unknown keys. Semantics of unknown values are left to the definition of
|
|
|
|
each key. Some capabilities will describe commands which can be requested
|
|
|
|
to be executed by the client.
|
|
|
|
|
|
|
|
capability-advertisement = protocol-version
|
|
|
|
capability-list
|
|
|
|
flush-pkt
|
|
|
|
|
|
|
|
protocol-version = PKT-LINE("version 2" LF)
|
|
|
|
capability-list = *capability
|
|
|
|
capability = PKT-LINE(key[=value] LF)
|
|
|
|
|
|
|
|
key = 1*(ALPHA | DIGIT | "-_")
|
|
|
|
value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
|
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
Command Request
|
|
|
|
---------------
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
After receiving the capability advertisement, a client can then issue a
|
|
|
|
request to select the command it wants with any particular capabilities
|
|
|
|
or arguments. There is then an optional section where the client can
|
|
|
|
provide any command specific parameters or queries. Only a single
|
|
|
|
command can be requested at a time.
|
|
|
|
|
|
|
|
request = empty-request | command-request
|
|
|
|
empty-request = flush-pkt
|
|
|
|
command-request = command
|
|
|
|
capability-list
|
2021-11-12 01:00:48 +03:00
|
|
|
delim-pkt
|
|
|
|
command-args
|
2018-03-15 20:31:19 +03:00
|
|
|
flush-pkt
|
|
|
|
command = PKT-LINE("command=" key LF)
|
2021-11-12 01:00:48 +03:00
|
|
|
command-args = *command-specific-arg
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
command-specific-args are packet line framed arguments defined by
|
|
|
|
each individual command.
|
|
|
|
|
|
|
|
The server will then check to ensure that the client's request is
|
|
|
|
comprised of a valid command as well as valid capabilities which were
|
|
|
|
advertised. If the request is valid the server will then execute the
|
|
|
|
command. A server MUST wait till it has received the client's entire
|
|
|
|
request before issuing a response. The format of the response is
|
|
|
|
determined by the command being executed, but in all cases a flush-pkt
|
|
|
|
indicates the end of the response.
|
|
|
|
|
|
|
|
When a command has finished, and the client has received the entire
|
|
|
|
response from the server, a client can either request that another
|
|
|
|
command be executed or can terminate the connection. A client may
|
|
|
|
optionally send an empty request consisting of just a flush-pkt to
|
|
|
|
indicate that no more requests will be made.
|
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
Capabilities
|
|
|
|
------------
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
There are two different types of capabilities: normal capabilities,
|
2019-08-10 08:59:14 +03:00
|
|
|
which can be used to convey information or alter the behavior of a
|
2018-03-15 20:31:19 +03:00
|
|
|
request, and commands, which are the core actions that a client wants to
|
|
|
|
perform (fetch, push, etc).
|
|
|
|
|
|
|
|
Protocol version 2 is stateless by default. This means that all commands
|
|
|
|
must only last a single round and be stateless from the perspective of the
|
|
|
|
server side, unless the client has requested a capability indicating that
|
|
|
|
state should be maintained by the server. Clients MUST NOT require state
|
|
|
|
management on the server side in order to function correctly. This
|
|
|
|
permits simple round-robin load-balancing on the server side, without
|
|
|
|
needing to worry about state management.
|
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
agent
|
|
|
|
~~~~~
|
2018-03-15 20:31:19 +03:00
|
|
|
|
|
|
|
The server can advertise the `agent` capability with a value `X` (in the
|
|
|
|
form `agent=X`) to notify the client that the server is running version
|
|
|
|
`X`. The client may optionally send its own agent string by including
|
|
|
|
the `agent` capability with a value `Y` (in the form `agent=Y`) in its
|
|
|
|
request to the server (but it MUST NOT do so if the server did not
|
|
|
|
advertise the agent capability). The `X` and `Y` strings may contain any
|
|
|
|
printable ASCII characters except space (i.e., the byte range 32 < x <
|
|
|
|
127), and are typically of the form "package/version" (e.g.,
|
|
|
|
"git/1.8.3.1"). The agent strings are purely informative for statistics
|
|
|
|
and debugging purposes, and MUST NOT be used to programmatically assume
|
|
|
|
the presence or absence of particular features.
|
2018-03-15 20:31:20 +03:00
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
ls-refs
|
|
|
|
~~~~~~~
|
2018-03-15 20:31:20 +03:00
|
|
|
|
|
|
|
`ls-refs` is the command used to request a reference advertisement in v2.
|
|
|
|
Unlike the current reference advertisement, ls-refs takes in arguments
|
|
|
|
which can be used to limit the refs sent from the server.
|
|
|
|
|
|
|
|
Additional features not supported in the base command will be advertised
|
|
|
|
as the value of the command in the capability advertisement in the form
|
|
|
|
of a space separated list of features: "<command>=<feature 1> <feature 2>"
|
|
|
|
|
|
|
|
ls-refs takes in the following arguments:
|
|
|
|
|
|
|
|
symrefs
|
|
|
|
In addition to the object pointed by it, show the underlying ref
|
|
|
|
pointed by it when showing a symbolic ref.
|
|
|
|
peel
|
|
|
|
Show peeled tags.
|
|
|
|
ref-prefix <prefix>
|
|
|
|
When specified, only references having a prefix matching one of
|
2021-09-15 21:35:34 +03:00
|
|
|
the provided prefixes are displayed. Multiple instances may be
|
|
|
|
given, in which case references matching any prefix will be
|
|
|
|
shown. Note that this is purely for optimization; a server MAY
|
|
|
|
show refs not matching the prefix if it chooses, and clients
|
|
|
|
should filter the result themselves.
|
2018-03-15 20:31:20 +03:00
|
|
|
|
ls-refs: report unborn targets of symrefs
When cloning, we choose the default branch based on the remote HEAD.
But if there is no remote HEAD reported (which could happen if the
target of the remote HEAD is unborn), we'll fall back to using our local
init.defaultBranch. Traditionally this hasn't been a big deal, because
most repos used "master" as the default. But these days it is likely to
cause confusion if the server and client implementations choose
different values (e.g., if the remote started with "main", we may choose
"master" locally, create commits there, and then the user is surprised
when they push to "master" and not "main").
To solve this, the remote needs to communicate the target of the HEAD
symref, even if it is unborn, and "git clone" needs to use this
information.
Currently, symrefs that have unborn targets (such as in this case) are
not communicated by the protocol. Teach Git to advertise and support the
"unborn" feature in "ls-refs" (by default, this is advertised, but
server administrators may turn this off through the lsrefs.unborn
config). This feature indicates that "ls-refs" supports the "unborn"
argument; when it is specified, "ls-refs" will send the HEAD symref with
the name of its unborn target.
This change is only for protocol v2. A similar change for protocol v0
would require independent protocol design (there being no analogous
position to signal support for "unborn") and client-side plumbing of the
data required, so the scope of this patch set is limited to protocol v2.
The client side will be updated to use this in a subsequent commit.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-05 23:48:47 +03:00
|
|
|
If the 'unborn' feature is advertised the following argument can be
|
|
|
|
included in the client's request.
|
|
|
|
|
|
|
|
unborn
|
|
|
|
The server will send information about HEAD even if it is a symref
|
|
|
|
pointing to an unborn branch in the form "unborn HEAD
|
|
|
|
symref-target:<target>".
|
|
|
|
|
2018-03-15 20:31:20 +03:00
|
|
|
The output of ls-refs is as follows:
|
|
|
|
|
|
|
|
output = *ref
|
|
|
|
flush-pkt
|
ls-refs: report unborn targets of symrefs
When cloning, we choose the default branch based on the remote HEAD.
But if there is no remote HEAD reported (which could happen if the
target of the remote HEAD is unborn), we'll fall back to using our local
init.defaultBranch. Traditionally this hasn't been a big deal, because
most repos used "master" as the default. But these days it is likely to
cause confusion if the server and client implementations choose
different values (e.g., if the remote started with "main", we may choose
"master" locally, create commits there, and then the user is surprised
when they push to "master" and not "main").
To solve this, the remote needs to communicate the target of the HEAD
symref, even if it is unborn, and "git clone" needs to use this
information.
Currently, symrefs that have unborn targets (such as in this case) are
not communicated by the protocol. Teach Git to advertise and support the
"unborn" feature in "ls-refs" (by default, this is advertised, but
server administrators may turn this off through the lsrefs.unborn
config). This feature indicates that "ls-refs" supports the "unborn"
argument; when it is specified, "ls-refs" will send the HEAD symref with
the name of its unborn target.
This change is only for protocol v2. A similar change for protocol v0
would require independent protocol design (there being no analogous
position to signal support for "unborn") and client-side plumbing of the
data required, so the scope of this patch set is limited to protocol v2.
The client side will be updated to use this in a subsequent commit.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-05 23:48:47 +03:00
|
|
|
obj-id-or-unborn = (obj-id | "unborn")
|
|
|
|
ref = PKT-LINE(obj-id-or-unborn SP refname *(SP ref-attribute) LF)
|
2018-03-15 20:31:20 +03:00
|
|
|
ref-attribute = (symref | peeled)
|
|
|
|
symref = "symref-target:" symref-target
|
|
|
|
peeled = "peeled:" obj-id
|
2018-03-15 20:31:27 +03:00
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
fetch
|
|
|
|
~~~~~
|
2018-03-15 20:31:27 +03:00
|
|
|
|
|
|
|
`fetch` is the command used to fetch a packfile in v2. It can be looked
|
|
|
|
at as a modified version of the v1 fetch where the ref-advertisement is
|
|
|
|
stripped out (since the `ls-refs` command fills that role) and the
|
|
|
|
message format is tweaked to eliminate redundancies and permit easy
|
|
|
|
addition of future extensions.
|
|
|
|
|
|
|
|
Additional features not supported in the base command will be advertised
|
|
|
|
as the value of the command in the capability advertisement in the form
|
|
|
|
of a space separated list of features: "<command>=<feature 1> <feature 2>"
|
|
|
|
|
|
|
|
A `fetch` request can take the following arguments:
|
|
|
|
|
|
|
|
want <oid>
|
|
|
|
Indicates to the server an object which the client wants to
|
|
|
|
retrieve. Wants can be anything and are not limited to
|
|
|
|
advertised objects.
|
|
|
|
|
|
|
|
have <oid>
|
|
|
|
Indicates to the server an object which the client has locally.
|
|
|
|
This allows the server to make a packfile which only contains
|
|
|
|
the objects that the client needs. Multiple 'have' lines can be
|
|
|
|
supplied.
|
|
|
|
|
|
|
|
done
|
|
|
|
Indicates to the server that negotiation should terminate (or
|
|
|
|
not even begin if performing a clone) and that the server should
|
|
|
|
use the information supplied in the request to construct the
|
|
|
|
packfile.
|
|
|
|
|
|
|
|
thin-pack
|
|
|
|
Request that a thin pack be sent, which is a pack with deltas
|
|
|
|
which reference base objects not contained within the pack (but
|
|
|
|
are known to exist at the receiving end). This can reduce the
|
|
|
|
network traffic significantly, but it requires the receiving end
|
|
|
|
to know how to "thicken" these packs by adding the missing bases
|
|
|
|
to the pack.
|
|
|
|
|
|
|
|
no-progress
|
|
|
|
Request that progress information that would normally be sent on
|
|
|
|
side-band channel 2, during the packfile transfer, should not be
|
|
|
|
sent. However, the side-band channel 3 is still used for error
|
|
|
|
responses.
|
|
|
|
|
|
|
|
include-tag
|
|
|
|
Request that annotated tags should be sent if the objects they
|
|
|
|
point to are being sent.
|
|
|
|
|
|
|
|
ofs-delta
|
|
|
|
Indicate that the client understands PACKv2 with delta referring
|
|
|
|
to its base by position in pack rather than by an oid. That is,
|
2019-11-05 20:07:20 +03:00
|
|
|
they can read OBJ_OFS_DELTA (aka type 6) in a packfile.
|
2018-03-15 20:31:27 +03:00
|
|
|
|
2018-03-15 20:31:29 +03:00
|
|
|
If the 'shallow' feature is advertised the following arguments can be
|
|
|
|
included in the clients request as well as the potential addition of the
|
|
|
|
'shallow-info' section in the server's response as explained below.
|
|
|
|
|
2018-03-15 20:31:28 +03:00
|
|
|
shallow <oid>
|
|
|
|
A client must notify the server of all commits for which it only
|
|
|
|
has shallow copies (meaning that it doesn't have the parents of
|
|
|
|
a commit) by supplying a 'shallow <oid>' line for each such
|
|
|
|
object so that the server is aware of the limitations of the
|
|
|
|
client's history. This is so that the server is aware that the
|
|
|
|
client may not have all objects reachable from such commits.
|
|
|
|
|
|
|
|
deepen <depth>
|
|
|
|
Requests that the fetch/clone should be shallow having a commit
|
|
|
|
depth of <depth> relative to the remote side.
|
|
|
|
|
|
|
|
deepen-relative
|
|
|
|
Requests that the semantics of the "deepen" command be changed
|
|
|
|
to indicate that the depth requested is relative to the client's
|
|
|
|
current shallow boundary, instead of relative to the requested
|
|
|
|
commits.
|
|
|
|
|
|
|
|
deepen-since <timestamp>
|
|
|
|
Requests that the shallow clone/fetch should be cut at a
|
|
|
|
specific time, instead of depth. Internally it's equivalent to
|
|
|
|
doing "git rev-list --max-age=<timestamp>". Cannot be used with
|
|
|
|
"deepen".
|
|
|
|
|
|
|
|
deepen-not <rev>
|
|
|
|
Requests that the shallow clone/fetch should be cut at a
|
|
|
|
specific revision specified by '<rev>', instead of a depth.
|
|
|
|
Internally it's equivalent of doing "git rev-list --not <rev>".
|
|
|
|
Cannot be used with "deepen", but can be used with
|
|
|
|
"deepen-since".
|
|
|
|
|
2018-05-04 02:46:56 +03:00
|
|
|
If the 'filter' feature is advertised, the following argument can be
|
|
|
|
included in the client's request:
|
|
|
|
|
|
|
|
filter <filter-spec>
|
|
|
|
Request that various objects from the packfile be omitted
|
|
|
|
using one of several filtering techniques. These are intended
|
|
|
|
for use with partial clone and partial fetch operations. See
|
2019-01-08 03:17:09 +03:00
|
|
|
`rev-list` for possible "filter-spec" values. When communicating
|
|
|
|
with other processes, senders SHOULD translate scaled integers
|
|
|
|
(e.g. "1k") into a fully-expanded form (e.g. "1024") to aid
|
|
|
|
interoperability with older receivers that may not understand
|
|
|
|
newly-invented scaling suffixes. However, receivers SHOULD
|
|
|
|
accept the following suffixes: 'k', 'm', and 'g' for 1024,
|
|
|
|
1048576, and 1073741824, respectively.
|
2018-05-04 02:46:56 +03:00
|
|
|
|
2018-06-28 01:30:17 +03:00
|
|
|
If the 'ref-in-want' feature is advertised, the following argument can
|
|
|
|
be included in the client's request as well as the potential addition of
|
|
|
|
the 'wanted-refs' section in the server's response as explained below.
|
|
|
|
|
|
|
|
want-ref <ref>
|
|
|
|
Indicates to the server that the client wants to retrieve a
|
|
|
|
particular ref, where <ref> is the full name of a ref on the
|
|
|
|
server.
|
|
|
|
|
2019-01-16 22:28:14 +03:00
|
|
|
If the 'sideband-all' feature is advertised, the following argument can be
|
|
|
|
included in the client's request:
|
|
|
|
|
|
|
|
sideband-all
|
|
|
|
Instruct the server to send the whole response multiplexed, not just
|
|
|
|
the packfile section. All non-flush and non-delim PKT-LINE in the
|
|
|
|
response (not only in the packfile section) will then start with a byte
|
|
|
|
indicating its sideband (1, 2, or 3), and the server may send "0005\2"
|
|
|
|
(a PKT-LINE of sideband 2 with no payload) as a keepalive packet.
|
|
|
|
|
2020-06-10 23:57:20 +03:00
|
|
|
If the 'packfile-uris' feature is advertised, the following argument
|
|
|
|
can be included in the client's request as well as the potential
|
|
|
|
addition of the 'packfile-uris' section in the server's response as
|
|
|
|
explained below.
|
|
|
|
|
|
|
|
packfile-uris <comma-separated list of protocols>
|
|
|
|
Indicates to the server that the client is willing to receive
|
|
|
|
URIs of any of the given protocols in place of objects in the
|
|
|
|
sent packfile. Before performing the connectivity check, the
|
|
|
|
client should download from all given URIs. Currently, the
|
|
|
|
protocols supported are "http" and "https".
|
|
|
|
|
fetch: teach independent negotiation (no packfile)
Currently, the packfile negotiation step within a Git fetch cannot be
done independent of sending the packfile, even though there is at least
one application wherein this is useful. Therefore, make it possible for
this negotiation step to be done independently. A subsequent commit will
use this for one such application - push negotiation.
This feature is for protocol v2 only. (An implementation for protocol v0
would require a separate implementation in the fetch, transport, and
transport helper code.)
In the protocol, the main hindrance towards independent negotiation is
that the server can unilaterally decide to send the packfile. This is
solved by a "wait-for-done" argument: the server will then wait for the
client to say "done". In practice, the client will never say it; instead
it will cease requests once it is satisfied.
In the client, the main change lies in the transport and transport
helper code. fetch_refs_via_pack() performs everything needed - protocol
version and capability checks, and the negotiation itself.
There are 2 code paths that do not go through fetch_refs_via_pack() that
needed to be individually excluded: the bundle transport (excluded
through requiring smart_options, which the bundle transport doesn't
support) and transport helpers that do not support takeover. If or when
we support independent negotiation for protocol v0, we will need to
modify these 2 code paths to support it. But for now, report failure if
independent negotiation is requested in these cases.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05 00:16:01 +03:00
|
|
|
If the 'wait-for-done' feature is advertised, the following argument
|
|
|
|
can be included in the client's request.
|
|
|
|
|
|
|
|
wait-for-done
|
|
|
|
Indicates to the server that it should never send "ready", but
|
|
|
|
should wait for the client to say "done" before sending the
|
|
|
|
packfile.
|
|
|
|
|
2018-03-15 20:31:27 +03:00
|
|
|
The response of `fetch` is broken into a number of sections separated by
|
|
|
|
delimiter packets (0001), with each section beginning with its section
|
2020-06-10 23:57:19 +03:00
|
|
|
header. Most sections are sent only when the packfile is sent.
|
2018-03-15 20:31:27 +03:00
|
|
|
|
2020-06-10 23:57:19 +03:00
|
|
|
output = acknowledgements flush-pkt |
|
|
|
|
[acknowledgments delim-pkt] [shallow-info delim-pkt]
|
2020-06-10 23:57:20 +03:00
|
|
|
[wanted-refs delim-pkt] [packfile-uris delim-pkt]
|
|
|
|
packfile flush-pkt
|
2018-03-15 20:31:27 +03:00
|
|
|
|
|
|
|
acknowledgments = PKT-LINE("acknowledgments" LF)
|
|
|
|
(nak | *ack)
|
|
|
|
(ready)
|
|
|
|
ready = PKT-LINE("ready" LF)
|
|
|
|
nak = PKT-LINE("NAK" LF)
|
|
|
|
ack = PKT-LINE("ACK" SP obj-id LF)
|
|
|
|
|
2018-03-15 20:31:28 +03:00
|
|
|
shallow-info = PKT-LINE("shallow-info" LF)
|
|
|
|
*PKT-LINE((shallow | unshallow) LF)
|
|
|
|
shallow = "shallow" SP obj-id
|
|
|
|
unshallow = "unshallow" SP obj-id
|
|
|
|
|
2018-06-28 01:30:17 +03:00
|
|
|
wanted-refs = PKT-LINE("wanted-refs" LF)
|
|
|
|
*PKT-LINE(wanted-ref LF)
|
|
|
|
wanted-ref = obj-id SP refname
|
|
|
|
|
2020-06-10 23:57:20 +03:00
|
|
|
packfile-uris = PKT-LINE("packfile-uris" LF) *packfile-uri
|
|
|
|
packfile-uri = PKT-LINE(40*(HEXDIGIT) SP *%x20-ff LF)
|
|
|
|
|
2018-03-15 20:31:27 +03:00
|
|
|
packfile = PKT-LINE("packfile" LF)
|
|
|
|
*PKT-LINE(%x01-03 *%x00-ff)
|
|
|
|
|
|
|
|
acknowledgments section
|
2020-06-10 23:57:19 +03:00
|
|
|
* If the client determines that it is finished with negotiations by
|
|
|
|
sending a "done" line (thus requiring the server to send a packfile),
|
|
|
|
the acknowledgments sections MUST be omitted from the server's
|
|
|
|
response.
|
2018-03-15 20:31:27 +03:00
|
|
|
|
|
|
|
* Always begins with the section header "acknowledgments"
|
|
|
|
|
|
|
|
* The server will respond with "NAK" if none of the object ids sent
|
|
|
|
as have lines were common.
|
|
|
|
|
|
|
|
* The server will respond with "ACK obj-id" for all of the
|
|
|
|
object ids sent as have lines which are common.
|
|
|
|
|
|
|
|
* A response cannot have both "ACK" lines as well as a "NAK"
|
|
|
|
line.
|
|
|
|
|
|
|
|
* The server will respond with a "ready" line indicating that
|
|
|
|
the server has found an acceptable common base and is ready to
|
|
|
|
make and send a packfile (which will be found in the packfile
|
|
|
|
section of the same response)
|
|
|
|
|
|
|
|
* If the server has found a suitable cut point and has decided
|
|
|
|
to send a "ready" line, then the server can decide to (as an
|
|
|
|
optimization) omit any "ACK" lines it would have sent during
|
|
|
|
its response. This is because the server will have already
|
|
|
|
determined the objects it plans to send to the client and no
|
|
|
|
further negotiation is needed.
|
|
|
|
|
2018-03-15 20:31:28 +03:00
|
|
|
shallow-info section
|
2018-03-15 20:31:29 +03:00
|
|
|
* If the client has requested a shallow fetch/clone, a shallow
|
|
|
|
client requests a fetch or the server is shallow then the
|
|
|
|
server's response may include a shallow-info section. The
|
|
|
|
shallow-info section will be included if (due to one of the
|
|
|
|
above conditions) the server needs to inform the client of any
|
|
|
|
shallow boundaries or adjustments to the clients already
|
|
|
|
existing shallow boundaries.
|
2018-03-15 20:31:28 +03:00
|
|
|
|
|
|
|
* Always begins with the section header "shallow-info"
|
|
|
|
|
|
|
|
* If a positive depth is requested, the server will compute the
|
|
|
|
set of commits which are no deeper than the desired depth.
|
|
|
|
|
|
|
|
* The server sends a "shallow obj-id" line for each commit whose
|
|
|
|
parents will not be sent in the following packfile.
|
|
|
|
|
|
|
|
* The server sends an "unshallow obj-id" line for each commit
|
|
|
|
which the client has indicated is shallow, but is no longer
|
|
|
|
shallow as a result of the fetch (due to its parents being
|
|
|
|
sent in the following packfile).
|
|
|
|
|
|
|
|
* The server MUST NOT send any "unshallow" lines for anything
|
|
|
|
which the client has not indicated was shallow as a part of
|
|
|
|
its request.
|
|
|
|
|
2018-06-28 01:30:17 +03:00
|
|
|
wanted-refs section
|
|
|
|
* This section is only included if the client has requested a
|
|
|
|
ref using a 'want-ref' line and if a packfile section is also
|
|
|
|
included in the response.
|
|
|
|
|
|
|
|
* Always begins with the section header "wanted-refs".
|
|
|
|
|
|
|
|
* The server will send a ref listing ("<oid> <refname>") for
|
|
|
|
each reference requested using 'want-ref' lines.
|
|
|
|
|
|
|
|
* The server MUST NOT send any refs which were not requested
|
|
|
|
using 'want-ref' lines.
|
|
|
|
|
2020-06-10 23:57:20 +03:00
|
|
|
packfile-uris section
|
|
|
|
* This section is only included if the client sent
|
|
|
|
'packfile-uris' and the server has at least one such URI to
|
|
|
|
send.
|
|
|
|
|
|
|
|
* Always begins with the section header "packfile-uris".
|
|
|
|
|
|
|
|
* For each URI the server sends, it sends a hash of the pack's
|
|
|
|
contents (as output by git index-pack) followed by the URI.
|
|
|
|
|
|
|
|
* The hashes are 40 hex characters long. When Git upgrades to a new
|
|
|
|
hash algorithm, this might need to be updated. (It should match
|
|
|
|
whatever index-pack outputs after "pack\t" or "keep\t".
|
|
|
|
|
2018-03-15 20:31:27 +03:00
|
|
|
packfile section
|
|
|
|
* This section is only included if the client has sent 'want'
|
|
|
|
lines in its request and either requested that no more
|
|
|
|
negotiation be done by sending 'done' or if the server has
|
|
|
|
decided it has found a sufficient cut point to produce a
|
|
|
|
packfile.
|
|
|
|
|
|
|
|
* Always begins with the section header "packfile"
|
|
|
|
|
|
|
|
* The transmission of the packfile begins immediately after the
|
|
|
|
section header
|
|
|
|
|
|
|
|
* The data transfer of the packfile is always multiplexed, using
|
|
|
|
the same semantics of the 'side-band-64k' capability from
|
|
|
|
protocol version 1. This means that each packet, during the
|
|
|
|
packfile data stream, is made up of a leading 4-byte pkt-line
|
|
|
|
length (typical of the pkt-line format), followed by a 1-byte
|
|
|
|
stream code, followed by the actual data.
|
|
|
|
|
|
|
|
The stream code can be one of:
|
|
|
|
1 - pack data
|
|
|
|
2 - progress messages
|
|
|
|
3 - fatal error message just before stream aborts
|
2018-04-24 01:46:22 +03:00
|
|
|
|
2019-03-29 15:35:17 +03:00
|
|
|
server-option
|
|
|
|
~~~~~~~~~~~~~
|
2018-04-24 01:46:22 +03:00
|
|
|
|
|
|
|
If advertised, indicates that any number of server specific options can be
|
|
|
|
included in a request. This is done by sending each option as a
|
|
|
|
"server-option=<option>" capability line in the capability-list section of
|
|
|
|
a request.
|
|
|
|
|
|
|
|
The provided options must not contain a NUL or LF character.
|
2020-05-25 22:59:14 +03:00
|
|
|
|
|
|
|
object-format
|
|
|
|
~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
The server can advertise the `object-format` capability with a value `X` (in the
|
|
|
|
form `object-format=X`) to notify the client that the server is able to deal
|
|
|
|
with objects using hash algorithm X. If not specified, the server is assumed to
|
|
|
|
only handle SHA-1. If the client would like to use a hash algorithm other than
|
|
|
|
SHA-1, it should specify its object-format string.
|
2020-11-12 02:29:24 +03:00
|
|
|
|
|
|
|
session-id=<session id>
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
The server may advertise a session ID that can be used to identify this process
|
|
|
|
across multiple requests. The client may advertise its own session ID back to
|
|
|
|
the server as well.
|
|
|
|
|
|
|
|
Session IDs should be unique to a given process. They must fit within a
|
|
|
|
packet-line, and must not contain non-printable or whitespace characters. The
|
|
|
|
current implementation uses trace2 session IDs (see
|
2022-09-16 09:23:02 +03:00
|
|
|
link:technical/api-trace2.html[api-trace2] for details), but this may change
|
|
|
|
and users of the session ID should not rely on this fact.
|
2021-04-21 02:38:31 +03:00
|
|
|
|
|
|
|
object-info
|
|
|
|
~~~~~~~~~~~
|
|
|
|
|
|
|
|
`object-info` is the command to retrieve information about one or more objects.
|
|
|
|
Its main purpose is to allow a client to make decisions based on this
|
|
|
|
information without having to fully fetch objects. Object size is the only
|
|
|
|
information that is currently supported.
|
|
|
|
|
|
|
|
An `object-info` request takes the following arguments:
|
|
|
|
|
|
|
|
size
|
|
|
|
Requests size information to be returned for each listed object id.
|
|
|
|
|
|
|
|
oid <oid>
|
|
|
|
Indicates to the server an object which the client wants to obtain
|
|
|
|
information for.
|
|
|
|
|
2021-06-11 14:18:50 +03:00
|
|
|
The response of `object-info` is a list of the requested object ids
|
2021-04-21 02:38:31 +03:00
|
|
|
and associated requested information, each separated by a single space.
|
|
|
|
|
|
|
|
output = info flush-pkt
|
|
|
|
|
|
|
|
info = PKT-LINE(attrs) LF)
|
|
|
|
*PKT-LINE(obj-info LF)
|
|
|
|
|
|
|
|
attrs = attr | attrs SP attrs
|
|
|
|
|
|
|
|
attr = "size"
|
|
|
|
|
|
|
|
obj-info = obj-id SP obj-size
|
2022-08-04 19:28:36 +03:00
|
|
|
|
2022-12-22 18:14:07 +03:00
|
|
|
bundle-uri
|
|
|
|
~~~~~~~~~~
|
|
|
|
|
|
|
|
If the 'bundle-uri' capability is advertised, the server supports the
|
|
|
|
`bundle-uri' command.
|
|
|
|
|
|
|
|
The capability is currently advertised with no value (i.e. not
|
|
|
|
"bundle-uri=somevalue"), a value may be added in the future for
|
|
|
|
supporting command-wide extensions. Clients MUST ignore any unknown
|
|
|
|
capability values and proceed with the 'bundle-uri` dialog they
|
|
|
|
support.
|
|
|
|
|
|
|
|
The 'bundle-uri' command is intended to be issued before `fetch` to
|
|
|
|
get URIs to bundle files (see linkgit:git-bundle[1]) to "seed" and
|
|
|
|
inform the subsequent `fetch` command.
|
|
|
|
|
|
|
|
The client CAN issue `bundle-uri` before or after any other valid
|
|
|
|
command. To be useful to clients it's expected that it'll be issued
|
|
|
|
after an `ls-refs` and before `fetch`, but CAN be issued at any time
|
|
|
|
in the dialog.
|
|
|
|
|
|
|
|
DISCUSSION of bundle-uri
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
The intent of the feature is optimize for server resource consumption
|
|
|
|
in the common case by changing the common case of fetching a very
|
|
|
|
large PACK during linkgit:git-clone[1] into a smaller incremental
|
|
|
|
fetch.
|
|
|
|
|
|
|
|
It also allows servers to achieve better caching in combination with
|
|
|
|
an `uploadpack.packObjectsHook` (see linkgit:git-config[1]).
|
|
|
|
|
|
|
|
By having new clones or fetches be a more predictable and common
|
|
|
|
negotiation against the tips of recently produces *.bundle file(s).
|
|
|
|
Servers might even pre-generate the results of such negotiations for
|
|
|
|
the `uploadpack.packObjectsHook` as new pushes come in.
|
|
|
|
|
|
|
|
One way that servers could take advantage of these bundles is that the
|
|
|
|
server would anticipate that fresh clones will download a known bundle,
|
|
|
|
followed by catching up to the current state of the repository using ref
|
|
|
|
tips found in that bundle (or bundles).
|
|
|
|
|
|
|
|
PROTOCOL for bundle-uri
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
A `bundle-uri` request takes no arguments, and as noted above does not
|
|
|
|
currently advertise a capability value. Both may be added in the
|
|
|
|
future.
|
|
|
|
|
|
|
|
When the client issues a `command=bundle-uri` request, the response is a
|
|
|
|
list of key-value pairs provided as packet lines with value
|
|
|
|
`<key>=<value>`. Each `<key>` should be interpreted as a config key from
|
|
|
|
the `bundle.*` namespace to construct a list of bundles. These keys are
|
|
|
|
grouped by a `bundle.<id>.` subsection, where each key corresponding to a
|
|
|
|
given `<id>` contributes attributes to the bundle defined by that `<id>`.
|
|
|
|
See linkgit:git-config[1] for the specific details of these keys and how
|
|
|
|
the Git client will interpret their values.
|
|
|
|
|
|
|
|
Clients MUST parse the line according to the above format, lines that do
|
|
|
|
not conform to the format SHOULD be discarded. The user MAY be warned in
|
|
|
|
such a case.
|
|
|
|
|
|
|
|
bundle-uri CLIENT AND SERVER EXPECTATIONS
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
URI CONTENTS::
|
|
|
|
The content at the advertised URIs MUST be one of two types.
|
|
|
|
+
|
|
|
|
The advertised URI may contain a bundle file that `git bundle verify`
|
|
|
|
would accept. I.e. they MUST contain one or more reference tips for
|
|
|
|
use by the client, MUST indicate prerequisites (in any) with standard
|
|
|
|
"-" prefixes, and MUST indicate their "object-format", if
|
|
|
|
applicable.
|
|
|
|
+
|
|
|
|
The advertised URI may alternatively contain a plaintext file that `git
|
|
|
|
config --list` would accept (with the `--file` option). The key-value
|
|
|
|
pairs in this list are in the `bundle.*` namespace (see
|
|
|
|
linkgit:git-config[1]).
|
|
|
|
|
|
|
|
bundle-uri CLIENT ERROR RECOVERY::
|
|
|
|
A client MUST above all gracefully degrade on errors, whether that
|
|
|
|
error is because of bad missing/data in the bundle URI(s), because
|
|
|
|
that client is too dumb to e.g. understand and fully parse out bundle
|
|
|
|
headers and their prerequisite relationships, or something else.
|
|
|
|
+
|
|
|
|
Server operators should feel confident in turning on "bundle-uri" and
|
|
|
|
not worry if e.g. their CDN goes down that clones or fetches will run
|
2023-01-07 16:56:55 +03:00
|
|
|
into hard failures. Even if the server bundle(s) are
|
2022-12-22 18:14:07 +03:00
|
|
|
incomplete, or bad in some way the client should still end up with a
|
|
|
|
functioning repository, just as if it had chosen not to use this
|
|
|
|
protocol extension.
|
|
|
|
+
|
|
|
|
All subsequent discussion on client and server interaction MUST keep
|
|
|
|
this in mind.
|
|
|
|
|
|
|
|
bundle-uri SERVER TO CLIENT::
|
|
|
|
The ordering of the returned bundle uris is not significant. Clients
|
|
|
|
MUST parse their headers to discover their contained OIDS and
|
|
|
|
prerequisites. A client MUST consider the content of the bundle(s)
|
|
|
|
themselves and their header as the ultimate source of truth.
|
|
|
|
+
|
|
|
|
A server MAY even return bundle(s) that don't have any direct
|
|
|
|
relationship to the repository being cloned (either through accident,
|
|
|
|
or intentional "clever" configuration), and expect a client to sort
|
|
|
|
out what data they'd like from the bundle(s), if any.
|
|
|
|
|
|
|
|
bundle-uri CLIENT TO SERVER::
|
|
|
|
The client SHOULD provide reference tips found in the bundle header(s)
|
|
|
|
as 'have' lines in any subsequent `fetch` request. A client MAY also
|
|
|
|
ignore the bundle(s) entirely if doing so is deemed worse for some
|
|
|
|
reason, e.g. if the bundles can't be downloaded, it doesn't like the
|
|
|
|
tips it finds etc.
|
|
|
|
|
|
|
|
WHEN ADVERTISED BUNDLE(S) REQUIRE NO FURTHER NEGOTIATION::
|
|
|
|
If after issuing `bundle-uri` and `ls-refs`, and getting the header(s)
|
|
|
|
of the bundle(s) the client finds that the ref tips it wants can be
|
|
|
|
retrieved entirely from advertised bundle(s), the client MAY disconnect
|
|
|
|
from the Git server. The results of such a 'clone' or 'fetch' should be
|
|
|
|
indistinguishable from the state attained without using bundle-uri.
|
|
|
|
|
|
|
|
EARLY CLIENT DISCONNECTIONS AND ERROR RECOVERY::
|
|
|
|
A client MAY perform an early disconnect while still downloading the
|
|
|
|
bundle(s) (having streamed and parsed their headers). In such a case
|
|
|
|
the client MUST gracefully recover from any errors related to
|
|
|
|
finishing the download and validation of the bundle(s).
|
|
|
|
+
|
|
|
|
I.e. a client might need to re-connect and issue a 'fetch' command,
|
|
|
|
and possibly fall back to not making use of 'bundle-uri' at all.
|
|
|
|
+
|
|
|
|
This "MAY" behavior is specified as such (and not a "SHOULD") on the
|
|
|
|
assumption that a server advertising bundle uris is more likely than
|
|
|
|
not to be serving up a relatively large repository, and to be pointing
|
|
|
|
to URIs that have a good chance of being in working order. A client
|
|
|
|
MAY e.g. look at the payload size of the bundles as a heuristic to see
|
|
|
|
if an early disconnect is worth it, should falling back on a full
|
|
|
|
"fetch" dialog be necessary.
|
|
|
|
|
|
|
|
WHEN ADVERTISED BUNDLE(S) REQUIRE FURTHER NEGOTIATION::
|
|
|
|
A client SHOULD commence a negotiation of a PACK from the server via
|
|
|
|
the "fetch" command using the OID tips found in advertised bundles,
|
|
|
|
even if's still in the process of downloading those bundle(s).
|
|
|
|
+
|
|
|
|
This allows for aggressive early disconnects from any interactive
|
|
|
|
server dialog. The client blindly trusts that the advertised OID tips
|
|
|
|
are relevant, and issues them as 'have' lines, it then requests any
|
|
|
|
tips it would like (usually from the "ls-refs" advertisement) via
|
|
|
|
'want' lines. The server will then compute a (hopefully small) PACK
|
|
|
|
with the expected difference between the tips from the bundle(s) and
|
|
|
|
the data requested.
|
|
|
|
+
|
|
|
|
The only connection the client then needs to keep active is to the
|
|
|
|
concurrently downloading static bundle(s), when those and the
|
|
|
|
incremental PACK are retrieved they should be inflated and
|
|
|
|
validated. Any errors at this point should be gracefully recovered
|
|
|
|
from, see above.
|
|
|
|
|
|
|
|
bundle-uri PROTOCOL FEATURES
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
The client constructs a bundle list from the `<key>=<value>` pairs
|
|
|
|
provided by the server. These pairs are part of the `bundle.*` namespace
|
|
|
|
as documented in linkgit:git-config[1]. In this section, we discuss some
|
|
|
|
of these keys and describe the actions the client will do in response to
|
|
|
|
this information.
|
|
|
|
|
|
|
|
In particular, the `bundle.version` key specifies an integer value. The
|
|
|
|
only accepted value at the moment is `1`, but if the client sees an
|
|
|
|
unexpected value here then the client MUST ignore the bundle list.
|
|
|
|
|
|
|
|
As long as `bundle.version` is understood, all other unknown keys MAY be
|
|
|
|
ignored by the client. The server will guarantee compatibility with older
|
|
|
|
clients, though newer clients may be better able to use the extra keys to
|
|
|
|
minimize downloads.
|
|
|
|
|
|
|
|
Any backwards-incompatible addition of pre-URI key-value will be
|
|
|
|
guarded by a new `bundle.version` value or values in 'bundle-uri'
|
|
|
|
capability advertisement itself, and/or by new future `bundle-uri`
|
|
|
|
request arguments.
|
|
|
|
|
|
|
|
Some example key-value pairs that are not currently implemented but could
|
|
|
|
be implemented in the future include:
|
|
|
|
|
|
|
|
* Add a "hash=<val>" or "size=<bytes>" advertise the expected hash or
|
|
|
|
size of the bundle file.
|
|
|
|
|
|
|
|
* Advertise that one or more bundle files are the same (to e.g. have
|
|
|
|
clients round-robin or otherwise choose one of N possible files).
|
|
|
|
|
|
|
|
* A "oid=<OID>" shortcut and "prerequisite=<OID>" shortcut. For
|
|
|
|
expressing the common case of a bundle with one tip and no
|
|
|
|
prerequisites, or one tip and one prerequisite.
|
|
|
|
+
|
|
|
|
This would allow for optimizing the common case of servers who'd like
|
|
|
|
to provide one "big bundle" containing only their "main" branch,
|
|
|
|
and/or incremental updates thereof.
|
|
|
|
+
|
|
|
|
A client receiving such a a response MAY assume that they can skip
|
|
|
|
retrieving the header from a bundle at the indicated URI, and thus
|
|
|
|
save themselves and the server(s) the request(s) needed to inspect the
|
|
|
|
headers of that bundle or bundles.
|
|
|
|
|
2022-08-04 19:28:36 +03:00
|
|
|
GIT
|
|
|
|
---
|
|
|
|
Part of the linkgit:git[1] suite
|