зеркало из https://github.com/microsoft/git.git
Merge branch 'ls/filter-process'
The smudge/clean filter API expect an external process is spawned to filter the contents for each path that has a filter defined. A new type of "process" filter API has been added to allow the first request to run the filter for a path to spawn a single process, and all filtering need is served by this single process for multiple paths, reducing the process creation overhead. * ls/filter-process: contrib/long-running-filter: add long running filter example convert: add filter.<driver>.process option convert: prepare filter.<driver>.process option convert: make apply_filter() adhere to standard Git error handling pkt-line: add functions to read/write flush terminated packet streams pkt-line: add packet_write_gently() pkt-line: add packet_flush_gently() pkt-line: add packet_write_fmt_gently() pkt-line: extract set_packet_header() pkt-line: rename packet_write() to packet_write_fmt() run-command: add clean_on_exit_handler run-command: move check_pipe() from write_or_die to run_command convert: modernize tests convert: quote filter names in error messages
This commit is contained in:
Коммит
dbaa6bdce2
|
@ -293,7 +293,15 @@ checkout, when the `smudge` command is specified, the command is
|
|||
fed the blob object from its standard input, and its standard
|
||||
output is used to update the worktree file. Similarly, the
|
||||
`clean` command is used to convert the contents of worktree file
|
||||
upon checkin.
|
||||
upon checkin. By default these commands process only a single
|
||||
blob and terminate. If a long running `process` filter is used
|
||||
in place of `clean` and/or `smudge` filters, then Git can process
|
||||
all blobs with a single filter command invocation for the entire
|
||||
life of a single Git command, for example `git add --all`. If a
|
||||
long running `process` filter is configured then it always takes
|
||||
precedence over a configured single blob filter. See section
|
||||
below for the description of the protocol used to communicate with
|
||||
a `process` filter.
|
||||
|
||||
One use of the content filtering is to massage the content into a shape
|
||||
that is more convenient for the platform, filesystem, and the user to use.
|
||||
|
@ -373,6 +381,153 @@ not exist, or may have different contents. So, smudge and clean commands
|
|||
should not try to access the file on disk, but only act as filters on the
|
||||
content provided to them on standard input.
|
||||
|
||||
Long Running Filter Process
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If the filter command (a string value) is defined via
|
||||
`filter.<driver>.process` then Git can process all blobs with a
|
||||
single filter invocation for the entire life of a single Git
|
||||
command. This is achieved by using a packet format (pkt-line,
|
||||
see technical/protocol-common.txt) based protocol over standard
|
||||
input and standard output as follows. All packets, except for the
|
||||
"*CONTENT" packets and the "0000" flush packet, are considered
|
||||
text and therefore are terminated by a LF.
|
||||
|
||||
Git starts the filter when it encounters the first file
|
||||
that needs to be cleaned or smudged. After the filter started
|
||||
Git sends a welcome message ("git-filter-client"), a list of supported
|
||||
protocol version numbers, and a flush packet. Git expects to read a welcome
|
||||
response message ("git-filter-server"), exactly one protocol version number
|
||||
from the previously sent list, and a flush packet. All further
|
||||
communication will be based on the selected version. The remaining
|
||||
protocol description below documents "version=2". Please note that
|
||||
"version=42" in the example below does not exist and is only there
|
||||
to illustrate how the protocol would look like with more than one
|
||||
version.
|
||||
|
||||
After the version negotiation Git sends a list of all capabilities that
|
||||
it supports and a flush packet. Git expects to read a list of desired
|
||||
capabilities, which must be a subset of the supported capabilities list,
|
||||
and a flush packet as response:
|
||||
------------------------
|
||||
packet: git> git-filter-client
|
||||
packet: git> version=2
|
||||
packet: git> version=42
|
||||
packet: git> 0000
|
||||
packet: git< git-filter-server
|
||||
packet: git< version=2
|
||||
packet: git< 0000
|
||||
packet: git> capability=clean
|
||||
packet: git> capability=smudge
|
||||
packet: git> capability=not-yet-invented
|
||||
packet: git> 0000
|
||||
packet: git< capability=clean
|
||||
packet: git< capability=smudge
|
||||
packet: git< 0000
|
||||
------------------------
|
||||
Supported filter capabilities in version 2 are "clean" and
|
||||
"smudge".
|
||||
|
||||
Afterwards Git sends a list of "key=value" pairs terminated with
|
||||
a flush packet. The list will contain at least the filter command
|
||||
(based on the supported capabilities) and the pathname of the file
|
||||
to filter relative to the repository root. Right after the flush packet
|
||||
Git sends the content split in zero or more pkt-line packets and a
|
||||
flush packet to terminate content. Please note, that the filter
|
||||
must not send any response before it received the content and the
|
||||
final flush packet.
|
||||
------------------------
|
||||
packet: git> command=smudge
|
||||
packet: git> pathname=path/testfile.dat
|
||||
packet: git> 0000
|
||||
packet: git> CONTENT
|
||||
packet: git> 0000
|
||||
------------------------
|
||||
|
||||
The filter is expected to respond with a list of "key=value" pairs
|
||||
terminated with a flush packet. If the filter does not experience
|
||||
problems then the list must contain a "success" status. Right after
|
||||
these packets the filter is expected to send the content in zero
|
||||
or more pkt-line packets and a flush packet at the end. Finally, a
|
||||
second list of "key=value" pairs terminated with a flush packet
|
||||
is expected. The filter can change the status in the second list
|
||||
or keep the status as is with an empty list. Please note that the
|
||||
empty list must be terminated with a flush packet regardless.
|
||||
|
||||
------------------------
|
||||
packet: git< status=success
|
||||
packet: git< 0000
|
||||
packet: git< SMUDGED_CONTENT
|
||||
packet: git< 0000
|
||||
packet: git< 0000 # empty list, keep "status=success" unchanged!
|
||||
------------------------
|
||||
|
||||
If the result content is empty then the filter is expected to respond
|
||||
with a "success" status and a flush packet to signal the empty content.
|
||||
------------------------
|
||||
packet: git< status=success
|
||||
packet: git< 0000
|
||||
packet: git< 0000 # empty content!
|
||||
packet: git< 0000 # empty list, keep "status=success" unchanged!
|
||||
------------------------
|
||||
|
||||
In case the filter cannot or does not want to process the content,
|
||||
it is expected to respond with an "error" status.
|
||||
------------------------
|
||||
packet: git< status=error
|
||||
packet: git< 0000
|
||||
------------------------
|
||||
|
||||
If the filter experiences an error during processing, then it can
|
||||
send the status "error" after the content was (partially or
|
||||
completely) sent.
|
||||
------------------------
|
||||
packet: git< status=success
|
||||
packet: git< 0000
|
||||
packet: git< HALF_WRITTEN_ERRONEOUS_CONTENT
|
||||
packet: git< 0000
|
||||
packet: git< status=error
|
||||
packet: git< 0000
|
||||
------------------------
|
||||
|
||||
In case the filter cannot or does not want to process the content
|
||||
as well as any future content for the lifetime of the Git process,
|
||||
then it is expected to respond with an "abort" status at any point
|
||||
in the protocol.
|
||||
------------------------
|
||||
packet: git< status=abort
|
||||
packet: git< 0000
|
||||
------------------------
|
||||
|
||||
Git neither stops nor restarts the filter process in case the
|
||||
"error"/"abort" status is set. However, Git sets its exit code
|
||||
according to the `filter.<driver>.required` flag, mimicking the
|
||||
behavior of the `filter.<driver>.clean` / `filter.<driver>.smudge`
|
||||
mechanism.
|
||||
|
||||
If the filter dies during the communication or does not adhere to
|
||||
the protocol then Git will stop the filter process and restart it
|
||||
with the next file that needs to be processed. Depending on the
|
||||
`filter.<driver>.required` flag Git will interpret that as error.
|
||||
|
||||
After the filter has processed a blob it is expected to wait for
|
||||
the next "key=value" list containing a command. Git will close
|
||||
the command pipe on exit. The filter is expected to detect EOF
|
||||
and exit gracefully on its own. Git will wait until the filter
|
||||
process has stopped.
|
||||
|
||||
A long running filter demo implementation can be found in
|
||||
`contrib/long-running-filter/example.pl` located in the Git
|
||||
core repository. If you develop your own long running filter
|
||||
process then the `GIT_TRACE_PACKET` environment variables can be
|
||||
very helpful for debugging (see linkgit:git[1]).
|
||||
|
||||
Please note that you cannot use an existing `filter.<driver>.clean`
|
||||
or `filter.<driver>.smudge` command with `filter.<driver>.process`
|
||||
because the former two use a different inter process communication
|
||||
protocol than the latter one.
|
||||
|
||||
|
||||
Interaction between checkin/checkout attributes
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
|
|
|
@ -47,10 +47,10 @@ static int run_remote_archiver(int argc, const char **argv,
|
|||
if (name_hint) {
|
||||
const char *format = archive_format_from_filename(name_hint);
|
||||
if (format)
|
||||
packet_write(fd[1], "argument --format=%s\n", format);
|
||||
packet_write_fmt(fd[1], "argument --format=%s\n", format);
|
||||
}
|
||||
for (i = 1; i < argc; i++)
|
||||
packet_write(fd[1], "argument %s\n", argv[i]);
|
||||
packet_write_fmt(fd[1], "argument %s\n", argv[i]);
|
||||
packet_flush(fd[1]);
|
||||
|
||||
buf = packet_read_line(fd[0], NULL);
|
||||
|
|
|
@ -227,7 +227,7 @@ static int receive_pack_config(const char *var, const char *value, void *cb)
|
|||
static void show_ref(const char *path, const unsigned char *sha1)
|
||||
{
|
||||
if (sent_capabilities) {
|
||||
packet_write(1, "%s %s\n", sha1_to_hex(sha1), path);
|
||||
packet_write_fmt(1, "%s %s\n", sha1_to_hex(sha1), path);
|
||||
} else {
|
||||
struct strbuf cap = STRBUF_INIT;
|
||||
|
||||
|
@ -242,7 +242,7 @@ static void show_ref(const char *path, const unsigned char *sha1)
|
|||
if (advertise_push_options)
|
||||
strbuf_addstr(&cap, " push-options");
|
||||
strbuf_addf(&cap, " agent=%s", git_user_agent_sanitized());
|
||||
packet_write(1, "%s %s%c%s\n",
|
||||
packet_write_fmt(1, "%s %s%c%s\n",
|
||||
sha1_to_hex(sha1), path, 0, cap.buf);
|
||||
strbuf_release(&cap);
|
||||
sent_capabilities = 1;
|
||||
|
|
|
@ -128,9 +128,9 @@ static void send_git_request(int stdin_fd, const char *serv, const char *repo,
|
|||
const char *vhost)
|
||||
{
|
||||
if (!vhost)
|
||||
packet_write(stdin_fd, "%s %s%c", serv, repo, 0);
|
||||
packet_write_fmt(stdin_fd, "%s %s%c", serv, repo, 0);
|
||||
else
|
||||
packet_write(stdin_fd, "%s %s%chost=%s%c", serv, repo, 0,
|
||||
packet_write_fmt(stdin_fd, "%s %s%chost=%s%c", serv, repo, 0,
|
||||
vhost, 0);
|
||||
}
|
||||
|
||||
|
|
|
@ -88,11 +88,11 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix)
|
|||
writer.git_cmd = 1;
|
||||
if (start_command(&writer)) {
|
||||
int err = errno;
|
||||
packet_write(1, "NACK unable to spawn subprocess\n");
|
||||
packet_write_fmt(1, "NACK unable to spawn subprocess\n");
|
||||
die("upload-archive: %s", strerror(err));
|
||||
}
|
||||
|
||||
packet_write(1, "ACK\n");
|
||||
packet_write_fmt(1, "ACK\n");
|
||||
packet_flush(1);
|
||||
|
||||
while (1) {
|
||||
|
|
|
@ -750,7 +750,7 @@ struct child_process *git_connect(int fd[2], const char *url,
|
|||
* Note: Do not add any other headers here! Doing so
|
||||
* will cause older git-daemon servers to crash.
|
||||
*/
|
||||
packet_write(fd[1],
|
||||
packet_write_fmt(fd[1],
|
||||
"%s %s%chost=%s%c",
|
||||
prog, path, 0,
|
||||
target_host, 0);
|
||||
|
|
|
@ -0,0 +1,128 @@
|
|||
#!/usr/bin/perl
|
||||
#
|
||||
# Example implementation for the Git filter protocol version 2
|
||||
# See Documentation/gitattributes.txt, section "Filter Protocol"
|
||||
#
|
||||
# Please note, this pass-thru filter is a minimal skeleton. No proper
|
||||
# error handling was implemented.
|
||||
#
|
||||
|
||||
use strict;
|
||||
use warnings;
|
||||
|
||||
my $MAX_PACKET_CONTENT_SIZE = 65516;
|
||||
|
||||
sub packet_bin_read {
|
||||
my $buffer;
|
||||
my $bytes_read = read STDIN, $buffer, 4;
|
||||
if ( $bytes_read == 0 ) {
|
||||
|
||||
# EOF - Git stopped talking to us!
|
||||
exit();
|
||||
}
|
||||
elsif ( $bytes_read != 4 ) {
|
||||
die "invalid packet: '$buffer'";
|
||||
}
|
||||
my $pkt_size = hex($buffer);
|
||||
if ( $pkt_size == 0 ) {
|
||||
return ( 1, "" );
|
||||
}
|
||||
elsif ( $pkt_size > 4 ) {
|
||||
my $content_size = $pkt_size - 4;
|
||||
$bytes_read = read STDIN, $buffer, $content_size;
|
||||
if ( $bytes_read != $content_size ) {
|
||||
die "invalid packet ($content_size bytes expected; $bytes_read bytes read)";
|
||||
}
|
||||
return ( 0, $buffer );
|
||||
}
|
||||
else {
|
||||
die "invalid packet size: $pkt_size";
|
||||
}
|
||||
}
|
||||
|
||||
sub packet_txt_read {
|
||||
my ( $res, $buf ) = packet_bin_read();
|
||||
unless ( $buf =~ s/\n$// ) {
|
||||
die "A non-binary line MUST be terminated by an LF.";
|
||||
}
|
||||
return ( $res, $buf );
|
||||
}
|
||||
|
||||
sub packet_bin_write {
|
||||
my $buf = shift;
|
||||
print STDOUT sprintf( "%04x", length($buf) + 4 );
|
||||
print STDOUT $buf;
|
||||
STDOUT->flush();
|
||||
}
|
||||
|
||||
sub packet_txt_write {
|
||||
packet_bin_write( $_[0] . "\n" );
|
||||
}
|
||||
|
||||
sub packet_flush {
|
||||
print STDOUT sprintf( "%04x", 0 );
|
||||
STDOUT->flush();
|
||||
}
|
||||
|
||||
( packet_txt_read() eq ( 0, "git-filter-client" ) ) || die "bad initialize";
|
||||
( packet_txt_read() eq ( 0, "version=2" ) ) || die "bad version";
|
||||
( packet_bin_read() eq ( 1, "" ) ) || die "bad version end";
|
||||
|
||||
packet_txt_write("git-filter-server");
|
||||
packet_txt_write("version=2");
|
||||
packet_flush();
|
||||
|
||||
( packet_txt_read() eq ( 0, "capability=clean" ) ) || die "bad capability";
|
||||
( packet_txt_read() eq ( 0, "capability=smudge" ) ) || die "bad capability";
|
||||
( packet_bin_read() eq ( 1, "" ) ) || die "bad capability end";
|
||||
|
||||
packet_txt_write("capability=clean");
|
||||
packet_txt_write("capability=smudge");
|
||||
packet_flush();
|
||||
|
||||
while (1) {
|
||||
my ($command) = packet_txt_read() =~ /^command=([^=]+)$/;
|
||||
my ($pathname) = packet_txt_read() =~ /^pathname=([^=]+)$/;
|
||||
|
||||
packet_bin_read();
|
||||
|
||||
my $input = "";
|
||||
{
|
||||
binmode(STDIN);
|
||||
my $buffer;
|
||||
my $done = 0;
|
||||
while ( !$done ) {
|
||||
( $done, $buffer ) = packet_bin_read();
|
||||
$input .= $buffer;
|
||||
}
|
||||
}
|
||||
|
||||
my $output;
|
||||
if ( $command eq "clean" ) {
|
||||
### Perform clean here ###
|
||||
$output = $input;
|
||||
}
|
||||
elsif ( $command eq "smudge" ) {
|
||||
### Perform smudge here ###
|
||||
$output = $input;
|
||||
}
|
||||
else {
|
||||
die "bad command '$command'";
|
||||
}
|
||||
|
||||
packet_txt_write("status=success");
|
||||
packet_flush();
|
||||
while ( length($output) > 0 ) {
|
||||
my $packet = substr( $output, 0, $MAX_PACKET_CONTENT_SIZE );
|
||||
packet_bin_write($packet);
|
||||
if ( length($output) > $MAX_PACKET_CONTENT_SIZE ) {
|
||||
$output = substr( $output, $MAX_PACKET_CONTENT_SIZE );
|
||||
}
|
||||
else {
|
||||
$output = "";
|
||||
}
|
||||
}
|
||||
packet_flush(); # flush content!
|
||||
packet_flush(); # empty list, keep "status=success" unchanged!
|
||||
|
||||
}
|
376
convert.c
376
convert.c
|
@ -3,6 +3,7 @@
|
|||
#include "run-command.h"
|
||||
#include "quote.h"
|
||||
#include "sigchain.h"
|
||||
#include "pkt-line.h"
|
||||
|
||||
/*
|
||||
* convert.c - convert a file when checking it out and checking it in.
|
||||
|
@ -416,7 +417,7 @@ static int filter_buffer_or_fd(int in, int out, void *data)
|
|||
child_process.out = out;
|
||||
|
||||
if (start_command(&child_process))
|
||||
return error("cannot fork to run external filter %s", params->cmd);
|
||||
return error("cannot fork to run external filter '%s'", params->cmd);
|
||||
|
||||
sigchain_push(SIGPIPE, SIG_IGN);
|
||||
|
||||
|
@ -434,19 +435,19 @@ static int filter_buffer_or_fd(int in, int out, void *data)
|
|||
if (close(child_process.in))
|
||||
write_err = 1;
|
||||
if (write_err)
|
||||
error("cannot feed the input to external filter %s", params->cmd);
|
||||
error("cannot feed the input to external filter '%s'", params->cmd);
|
||||
|
||||
sigchain_pop(SIGPIPE);
|
||||
|
||||
status = finish_command(&child_process);
|
||||
if (status)
|
||||
error("external filter %s failed %d", params->cmd, status);
|
||||
error("external filter '%s' failed %d", params->cmd, status);
|
||||
|
||||
strbuf_release(&cmd);
|
||||
return (write_err || status);
|
||||
}
|
||||
|
||||
static int apply_filter(const char *path, const char *src, size_t len, int fd,
|
||||
static int apply_single_file_filter(const char *path, const char *src, size_t len, int fd,
|
||||
struct strbuf *dst, const char *cmd)
|
||||
{
|
||||
/*
|
||||
|
@ -455,17 +456,11 @@ static int apply_filter(const char *path, const char *src, size_t len, int fd,
|
|||
*
|
||||
* (child --> cmd) --> us
|
||||
*/
|
||||
int ret = 1;
|
||||
int err = 0;
|
||||
struct strbuf nbuf = STRBUF_INIT;
|
||||
struct async async;
|
||||
struct filter_params params;
|
||||
|
||||
if (!cmd || !*cmd)
|
||||
return 0;
|
||||
|
||||
if (!dst)
|
||||
return 1;
|
||||
|
||||
memset(&async, 0, sizeof(async));
|
||||
async.proc = filter_buffer_or_fd;
|
||||
async.data = ¶ms;
|
||||
|
@ -481,23 +476,304 @@ static int apply_filter(const char *path, const char *src, size_t len, int fd,
|
|||
return 0; /* error was already reported */
|
||||
|
||||
if (strbuf_read(&nbuf, async.out, len) < 0) {
|
||||
error("read from external filter %s failed", cmd);
|
||||
ret = 0;
|
||||
err = error("read from external filter '%s' failed", cmd);
|
||||
}
|
||||
if (close(async.out)) {
|
||||
error("read from external filter %s failed", cmd);
|
||||
ret = 0;
|
||||
err = error("read from external filter '%s' failed", cmd);
|
||||
}
|
||||
if (finish_async(&async)) {
|
||||
error("external filter %s failed", cmd);
|
||||
ret = 0;
|
||||
err = error("external filter '%s' failed", cmd);
|
||||
}
|
||||
|
||||
if (ret) {
|
||||
if (!err) {
|
||||
strbuf_swap(dst, &nbuf);
|
||||
}
|
||||
strbuf_release(&nbuf);
|
||||
return ret;
|
||||
return !err;
|
||||
}
|
||||
|
||||
#define CAP_CLEAN (1u<<0)
|
||||
#define CAP_SMUDGE (1u<<1)
|
||||
|
||||
struct cmd2process {
|
||||
struct hashmap_entry ent; /* must be the first member! */
|
||||
unsigned int supported_capabilities;
|
||||
const char *cmd;
|
||||
struct child_process process;
|
||||
};
|
||||
|
||||
static int cmd_process_map_initialized;
|
||||
static struct hashmap cmd_process_map;
|
||||
|
||||
static int cmd2process_cmp(const struct cmd2process *e1,
|
||||
const struct cmd2process *e2,
|
||||
const void *unused)
|
||||
{
|
||||
return strcmp(e1->cmd, e2->cmd);
|
||||
}
|
||||
|
||||
static struct cmd2process *find_multi_file_filter_entry(struct hashmap *hashmap, const char *cmd)
|
||||
{
|
||||
struct cmd2process key;
|
||||
hashmap_entry_init(&key, strhash(cmd));
|
||||
key.cmd = cmd;
|
||||
return hashmap_get(hashmap, &key, NULL);
|
||||
}
|
||||
|
||||
static int packet_write_list(int fd, const char *line, ...)
|
||||
{
|
||||
va_list args;
|
||||
int err;
|
||||
va_start(args, line);
|
||||
for (;;) {
|
||||
if (!line)
|
||||
break;
|
||||
if (strlen(line) > LARGE_PACKET_DATA_MAX)
|
||||
return -1;
|
||||
err = packet_write_fmt_gently(fd, "%s\n", line);
|
||||
if (err)
|
||||
return err;
|
||||
line = va_arg(args, const char*);
|
||||
}
|
||||
va_end(args);
|
||||
return packet_flush_gently(fd);
|
||||
}
|
||||
|
||||
static void read_multi_file_filter_status(int fd, struct strbuf *status)
|
||||
{
|
||||
struct strbuf **pair;
|
||||
char *line;
|
||||
for (;;) {
|
||||
line = packet_read_line(fd, NULL);
|
||||
if (!line)
|
||||
break;
|
||||
pair = strbuf_split_str(line, '=', 2);
|
||||
if (pair[0] && pair[0]->len && pair[1]) {
|
||||
/* the last "status=<foo>" line wins */
|
||||
if (!strcmp(pair[0]->buf, "status=")) {
|
||||
strbuf_reset(status);
|
||||
strbuf_addbuf(status, pair[1]);
|
||||
}
|
||||
}
|
||||
strbuf_list_free(pair);
|
||||
}
|
||||
}
|
||||
|
||||
static void kill_multi_file_filter(struct hashmap *hashmap, struct cmd2process *entry)
|
||||
{
|
||||
if (!entry)
|
||||
return;
|
||||
|
||||
entry->process.clean_on_exit = 0;
|
||||
kill(entry->process.pid, SIGTERM);
|
||||
finish_command(&entry->process);
|
||||
|
||||
hashmap_remove(hashmap, entry, NULL);
|
||||
free(entry);
|
||||
}
|
||||
|
||||
static void stop_multi_file_filter(struct child_process *process)
|
||||
{
|
||||
sigchain_push(SIGPIPE, SIG_IGN);
|
||||
/* Closing the pipe signals the filter to initiate a shutdown. */
|
||||
close(process->in);
|
||||
close(process->out);
|
||||
sigchain_pop(SIGPIPE);
|
||||
/* Finish command will wait until the shutdown is complete. */
|
||||
finish_command(process);
|
||||
}
|
||||
|
||||
static struct cmd2process *start_multi_file_filter(struct hashmap *hashmap, const char *cmd)
|
||||
{
|
||||
int err;
|
||||
struct cmd2process *entry;
|
||||
struct child_process *process;
|
||||
const char *argv[] = { cmd, NULL };
|
||||
struct string_list cap_list = STRING_LIST_INIT_NODUP;
|
||||
char *cap_buf;
|
||||
const char *cap_name;
|
||||
|
||||
entry = xmalloc(sizeof(*entry));
|
||||
entry->cmd = cmd;
|
||||
entry->supported_capabilities = 0;
|
||||
process = &entry->process;
|
||||
|
||||
child_process_init(process);
|
||||
process->argv = argv;
|
||||
process->use_shell = 1;
|
||||
process->in = -1;
|
||||
process->out = -1;
|
||||
process->clean_on_exit = 1;
|
||||
process->clean_on_exit_handler = stop_multi_file_filter;
|
||||
|
||||
if (start_command(process)) {
|
||||
error("cannot fork to run external filter '%s'", cmd);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
hashmap_entry_init(entry, strhash(cmd));
|
||||
|
||||
sigchain_push(SIGPIPE, SIG_IGN);
|
||||
|
||||
err = packet_write_list(process->in, "git-filter-client", "version=2", NULL);
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
err = strcmp(packet_read_line(process->out, NULL), "git-filter-server");
|
||||
if (err) {
|
||||
error("external filter '%s' does not support filter protocol version 2", cmd);
|
||||
goto done;
|
||||
}
|
||||
err = strcmp(packet_read_line(process->out, NULL), "version=2");
|
||||
if (err)
|
||||
goto done;
|
||||
err = packet_read_line(process->out, NULL) != NULL;
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
err = packet_write_list(process->in, "capability=clean", "capability=smudge", NULL);
|
||||
|
||||
for (;;) {
|
||||
cap_buf = packet_read_line(process->out, NULL);
|
||||
if (!cap_buf)
|
||||
break;
|
||||
string_list_split_in_place(&cap_list, cap_buf, '=', 1);
|
||||
|
||||
if (cap_list.nr != 2 || strcmp(cap_list.items[0].string, "capability"))
|
||||
continue;
|
||||
|
||||
cap_name = cap_list.items[1].string;
|
||||
if (!strcmp(cap_name, "clean")) {
|
||||
entry->supported_capabilities |= CAP_CLEAN;
|
||||
} else if (!strcmp(cap_name, "smudge")) {
|
||||
entry->supported_capabilities |= CAP_SMUDGE;
|
||||
} else {
|
||||
warning(
|
||||
"external filter '%s' requested unsupported filter capability '%s'",
|
||||
cmd, cap_name
|
||||
);
|
||||
}
|
||||
|
||||
string_list_clear(&cap_list, 0);
|
||||
}
|
||||
|
||||
done:
|
||||
sigchain_pop(SIGPIPE);
|
||||
|
||||
if (err || errno == EPIPE) {
|
||||
error("initialization for external filter '%s' failed", cmd);
|
||||
kill_multi_file_filter(hashmap, entry);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
hashmap_add(hashmap, entry);
|
||||
return entry;
|
||||
}
|
||||
|
||||
static int apply_multi_file_filter(const char *path, const char *src, size_t len,
|
||||
int fd, struct strbuf *dst, const char *cmd,
|
||||
const unsigned int wanted_capability)
|
||||
{
|
||||
int err;
|
||||
struct cmd2process *entry;
|
||||
struct child_process *process;
|
||||
struct strbuf nbuf = STRBUF_INIT;
|
||||
struct strbuf filter_status = STRBUF_INIT;
|
||||
const char *filter_type;
|
||||
|
||||
if (!cmd_process_map_initialized) {
|
||||
cmd_process_map_initialized = 1;
|
||||
hashmap_init(&cmd_process_map, (hashmap_cmp_fn) cmd2process_cmp, 0);
|
||||
entry = NULL;
|
||||
} else {
|
||||
entry = find_multi_file_filter_entry(&cmd_process_map, cmd);
|
||||
}
|
||||
|
||||
fflush(NULL);
|
||||
|
||||
if (!entry) {
|
||||
entry = start_multi_file_filter(&cmd_process_map, cmd);
|
||||
if (!entry)
|
||||
return 0;
|
||||
}
|
||||
process = &entry->process;
|
||||
|
||||
if (!(wanted_capability & entry->supported_capabilities))
|
||||
return 0;
|
||||
|
||||
if (CAP_CLEAN & wanted_capability)
|
||||
filter_type = "clean";
|
||||
else if (CAP_SMUDGE & wanted_capability)
|
||||
filter_type = "smudge";
|
||||
else
|
||||
die("unexpected filter type");
|
||||
|
||||
sigchain_push(SIGPIPE, SIG_IGN);
|
||||
|
||||
assert(strlen(filter_type) < LARGE_PACKET_DATA_MAX - strlen("command=\n"));
|
||||
err = packet_write_fmt_gently(process->in, "command=%s\n", filter_type);
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
err = strlen(path) > LARGE_PACKET_DATA_MAX - strlen("pathname=\n");
|
||||
if (err) {
|
||||
error("path name too long for external filter");
|
||||
goto done;
|
||||
}
|
||||
|
||||
err = packet_write_fmt_gently(process->in, "pathname=%s\n", path);
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
err = packet_flush_gently(process->in);
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
if (fd >= 0)
|
||||
err = write_packetized_from_fd(fd, process->in);
|
||||
else
|
||||
err = write_packetized_from_buf(src, len, process->in);
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
read_multi_file_filter_status(process->out, &filter_status);
|
||||
err = strcmp(filter_status.buf, "success");
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
err = read_packetized_to_strbuf(process->out, &nbuf) < 0;
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
read_multi_file_filter_status(process->out, &filter_status);
|
||||
err = strcmp(filter_status.buf, "success");
|
||||
|
||||
done:
|
||||
sigchain_pop(SIGPIPE);
|
||||
|
||||
if (err || errno == EPIPE) {
|
||||
if (!strcmp(filter_status.buf, "error")) {
|
||||
/* The filter signaled a problem with the file. */
|
||||
} else if (!strcmp(filter_status.buf, "abort")) {
|
||||
/*
|
||||
* The filter signaled a permanent problem. Don't try to filter
|
||||
* files with the same command for the lifetime of the current
|
||||
* Git process.
|
||||
*/
|
||||
entry->supported_capabilities &= ~wanted_capability;
|
||||
} else {
|
||||
/*
|
||||
* Something went wrong with the protocol filter.
|
||||
* Force shutdown and restart if another blob requires filtering.
|
||||
*/
|
||||
error("external filter '%s' failed", cmd);
|
||||
kill_multi_file_filter(&cmd_process_map, entry);
|
||||
}
|
||||
} else {
|
||||
strbuf_swap(dst, &nbuf);
|
||||
}
|
||||
strbuf_release(&nbuf);
|
||||
return !err;
|
||||
}
|
||||
|
||||
static struct convert_driver {
|
||||
|
@ -505,9 +781,35 @@ static struct convert_driver {
|
|||
struct convert_driver *next;
|
||||
const char *smudge;
|
||||
const char *clean;
|
||||
const char *process;
|
||||
int required;
|
||||
} *user_convert, **user_convert_tail;
|
||||
|
||||
static int apply_filter(const char *path, const char *src, size_t len,
|
||||
int fd, struct strbuf *dst, struct convert_driver *drv,
|
||||
const unsigned int wanted_capability)
|
||||
{
|
||||
const char *cmd = NULL;
|
||||
|
||||
if (!drv)
|
||||
return 0;
|
||||
|
||||
if (!dst)
|
||||
return 1;
|
||||
|
||||
if ((CAP_CLEAN & wanted_capability) && !drv->process && drv->clean)
|
||||
cmd = drv->clean;
|
||||
else if ((CAP_SMUDGE & wanted_capability) && !drv->process && drv->smudge)
|
||||
cmd = drv->smudge;
|
||||
|
||||
if (cmd && *cmd)
|
||||
return apply_single_file_filter(path, src, len, fd, dst, cmd);
|
||||
else if (drv->process && *drv->process)
|
||||
return apply_multi_file_filter(path, src, len, fd, dst, drv->process, wanted_capability);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int read_convert_config(const char *var, const char *value, void *cb)
|
||||
{
|
||||
const char *key, *name;
|
||||
|
@ -545,6 +847,9 @@ static int read_convert_config(const char *var, const char *value, void *cb)
|
|||
if (!strcmp("clean", key))
|
||||
return git_config_string(&drv->clean, var, value);
|
||||
|
||||
if (!strcmp("process", key))
|
||||
return git_config_string(&drv->process, var, value);
|
||||
|
||||
if (!strcmp("required", key)) {
|
||||
drv->required = git_config_bool(var, value);
|
||||
return 0;
|
||||
|
@ -846,7 +1151,7 @@ int would_convert_to_git_filter_fd(const char *path)
|
|||
if (!ca.drv->required)
|
||||
return 0;
|
||||
|
||||
return apply_filter(path, NULL, 0, -1, NULL, ca.drv->clean);
|
||||
return apply_filter(path, NULL, 0, -1, NULL, ca.drv, CAP_CLEAN);
|
||||
}
|
||||
|
||||
const char *get_convert_attr_ascii(const char *path)
|
||||
|
@ -879,18 +1184,12 @@ int convert_to_git(const char *path, const char *src, size_t len,
|
|||
struct strbuf *dst, enum safe_crlf checksafe)
|
||||
{
|
||||
int ret = 0;
|
||||
const char *filter = NULL;
|
||||
int required = 0;
|
||||
struct conv_attrs ca;
|
||||
|
||||
convert_attrs(&ca, path);
|
||||
if (ca.drv) {
|
||||
filter = ca.drv->clean;
|
||||
required = ca.drv->required;
|
||||
}
|
||||
|
||||
ret |= apply_filter(path, src, len, -1, dst, filter);
|
||||
if (!ret && required)
|
||||
ret |= apply_filter(path, src, len, -1, dst, ca.drv, CAP_CLEAN);
|
||||
if (!ret && ca.drv && ca.drv->required)
|
||||
die("%s: clean filter '%s' failed", path, ca.drv->name);
|
||||
|
||||
if (ret && dst) {
|
||||
|
@ -912,9 +1211,9 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,
|
|||
convert_attrs(&ca, path);
|
||||
|
||||
assert(ca.drv);
|
||||
assert(ca.drv->clean);
|
||||
assert(ca.drv->clean || ca.drv->process);
|
||||
|
||||
if (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))
|
||||
if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN))
|
||||
die("%s: clean filter '%s' failed", path, ca.drv->name);
|
||||
|
||||
crlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
|
||||
|
@ -926,15 +1225,9 @@ static int convert_to_working_tree_internal(const char *path, const char *src,
|
|||
int normalizing)
|
||||
{
|
||||
int ret = 0, ret_filter = 0;
|
||||
const char *filter = NULL;
|
||||
int required = 0;
|
||||
struct conv_attrs ca;
|
||||
|
||||
convert_attrs(&ca, path);
|
||||
if (ca.drv) {
|
||||
filter = ca.drv->smudge;
|
||||
required = ca.drv->required;
|
||||
}
|
||||
|
||||
ret |= ident_to_worktree(path, src, len, dst, ca.ident);
|
||||
if (ret) {
|
||||
|
@ -943,9 +1236,10 @@ static int convert_to_working_tree_internal(const char *path, const char *src,
|
|||
}
|
||||
/*
|
||||
* CRLF conversion can be skipped if normalizing, unless there
|
||||
* is a smudge filter. The filter might expect CRLFs.
|
||||
* is a smudge or process filter (even if the process filter doesn't
|
||||
* support smudge). The filters might expect CRLFs.
|
||||
*/
|
||||
if (filter || !normalizing) {
|
||||
if ((ca.drv && (ca.drv->smudge || ca.drv->process)) || !normalizing) {
|
||||
ret |= crlf_to_worktree(path, src, len, dst, ca.crlf_action);
|
||||
if (ret) {
|
||||
src = dst->buf;
|
||||
|
@ -953,8 +1247,8 @@ static int convert_to_working_tree_internal(const char *path, const char *src,
|
|||
}
|
||||
}
|
||||
|
||||
ret_filter = apply_filter(path, src, len, -1, dst, filter);
|
||||
if (!ret_filter && required)
|
||||
ret_filter = apply_filter(path, src, len, -1, dst, ca.drv, CAP_SMUDGE);
|
||||
if (!ret_filter && ca.drv && ca.drv->required)
|
||||
die("%s: smudge filter %s failed", path, ca.drv->name);
|
||||
|
||||
return ret | ret_filter;
|
||||
|
@ -1406,7 +1700,7 @@ struct stream_filter *get_stream_filter(const char *path, const unsigned char *s
|
|||
struct stream_filter *filter = NULL;
|
||||
|
||||
convert_attrs(&ca, path);
|
||||
if (ca.drv && (ca.drv->smudge || ca.drv->clean))
|
||||
if (ca.drv && (ca.drv->process || ca.drv->smudge || ca.drv->clean))
|
||||
return NULL;
|
||||
|
||||
if (ca.crlf_action == CRLF_AUTO || ca.crlf_action == CRLF_AUTO_CRLF)
|
||||
|
|
2
daemon.c
2
daemon.c
|
@ -298,7 +298,7 @@ static int daemon_error(const char *dir, const char *msg)
|
|||
{
|
||||
if (!informative_errors)
|
||||
msg = "access denied or repository not exported";
|
||||
packet_write(1, "ERR %s: %s", msg, dir);
|
||||
packet_write_fmt(1, "ERR %s: %s", msg, dir);
|
||||
return -1;
|
||||
}
|
||||
|
||||
|
|
|
@ -464,7 +464,7 @@ static void get_info_refs(struct strbuf *hdr, char *arg)
|
|||
hdr_str(hdr, content_type, buf.buf);
|
||||
end_headers(hdr);
|
||||
|
||||
packet_write(1, "# service=git-%s\n", svc->name);
|
||||
packet_write_fmt(1, "# service=git-%s\n", svc->name);
|
||||
packet_flush(1);
|
||||
|
||||
argv[0] = svc->name;
|
||||
|
|
152
pkt-line.c
152
pkt-line.c
|
@ -91,16 +91,34 @@ void packet_flush(int fd)
|
|||
write_or_die(fd, "0000", 4);
|
||||
}
|
||||
|
||||
int packet_flush_gently(int fd)
|
||||
{
|
||||
packet_trace("0000", 4, 1);
|
||||
if (write_in_full(fd, "0000", 4) == 4)
|
||||
return 0;
|
||||
return error("flush packet write failed");
|
||||
}
|
||||
|
||||
void packet_buf_flush(struct strbuf *buf)
|
||||
{
|
||||
packet_trace("0000", 4, 1);
|
||||
strbuf_add(buf, "0000", 4);
|
||||
}
|
||||
|
||||
#define hex(a) (hexchar[(a) & 15])
|
||||
static void format_packet(struct strbuf *out, const char *fmt, va_list args)
|
||||
static void set_packet_header(char *buf, const int size)
|
||||
{
|
||||
static char hexchar[] = "0123456789abcdef";
|
||||
|
||||
#define hex(a) (hexchar[(a) & 15])
|
||||
buf[0] = hex(size >> 12);
|
||||
buf[1] = hex(size >> 8);
|
||||
buf[2] = hex(size >> 4);
|
||||
buf[3] = hex(size);
|
||||
#undef hex
|
||||
}
|
||||
|
||||
static void format_packet(struct strbuf *out, const char *fmt, va_list args)
|
||||
{
|
||||
size_t orig_len, n;
|
||||
|
||||
orig_len = out->len;
|
||||
|
@ -111,23 +129,63 @@ static void format_packet(struct strbuf *out, const char *fmt, va_list args)
|
|||
if (n > LARGE_PACKET_MAX)
|
||||
die("protocol error: impossibly long line");
|
||||
|
||||
out->buf[orig_len + 0] = hex(n >> 12);
|
||||
out->buf[orig_len + 1] = hex(n >> 8);
|
||||
out->buf[orig_len + 2] = hex(n >> 4);
|
||||
out->buf[orig_len + 3] = hex(n);
|
||||
set_packet_header(&out->buf[orig_len], n);
|
||||
packet_trace(out->buf + orig_len + 4, n - 4, 1);
|
||||
}
|
||||
|
||||
void packet_write(int fd, const char *fmt, ...)
|
||||
static int packet_write_fmt_1(int fd, int gently,
|
||||
const char *fmt, va_list args)
|
||||
{
|
||||
struct strbuf buf = STRBUF_INIT;
|
||||
ssize_t count;
|
||||
|
||||
format_packet(&buf, fmt, args);
|
||||
count = write_in_full(fd, buf.buf, buf.len);
|
||||
if (count == buf.len)
|
||||
return 0;
|
||||
|
||||
if (!gently) {
|
||||
check_pipe(errno);
|
||||
die_errno("packet write with format failed");
|
||||
}
|
||||
return error("packet write with format failed");
|
||||
}
|
||||
|
||||
void packet_write_fmt(int fd, const char *fmt, ...)
|
||||
{
|
||||
static struct strbuf buf = STRBUF_INIT;
|
||||
va_list args;
|
||||
|
||||
strbuf_reset(&buf);
|
||||
va_start(args, fmt);
|
||||
format_packet(&buf, fmt, args);
|
||||
packet_write_fmt_1(fd, 0, fmt, args);
|
||||
va_end(args);
|
||||
write_or_die(fd, buf.buf, buf.len);
|
||||
}
|
||||
|
||||
int packet_write_fmt_gently(int fd, const char *fmt, ...)
|
||||
{
|
||||
int status;
|
||||
va_list args;
|
||||
|
||||
va_start(args, fmt);
|
||||
status = packet_write_fmt_1(fd, 1, fmt, args);
|
||||
va_end(args);
|
||||
return status;
|
||||
}
|
||||
|
||||
static int packet_write_gently(const int fd_out, const char *buf, size_t size)
|
||||
{
|
||||
static char packet_write_buffer[LARGE_PACKET_MAX];
|
||||
size_t packet_size;
|
||||
|
||||
if (size > sizeof(packet_write_buffer) - 4)
|
||||
return error("packet write failed - data exceeds max packet size");
|
||||
|
||||
packet_trace(buf, size, 1);
|
||||
packet_size = size + 4;
|
||||
set_packet_header(packet_write_buffer, packet_size);
|
||||
memcpy(packet_write_buffer + 4, buf, size);
|
||||
if (write_in_full(fd_out, packet_write_buffer, packet_size) == packet_size)
|
||||
return 0;
|
||||
return error("packet write failed");
|
||||
}
|
||||
|
||||
void packet_buf_write(struct strbuf *buf, const char *fmt, ...)
|
||||
|
@ -139,6 +197,46 @@ void packet_buf_write(struct strbuf *buf, const char *fmt, ...)
|
|||
va_end(args);
|
||||
}
|
||||
|
||||
int write_packetized_from_fd(int fd_in, int fd_out)
|
||||
{
|
||||
static char buf[LARGE_PACKET_DATA_MAX];
|
||||
int err = 0;
|
||||
ssize_t bytes_to_write;
|
||||
|
||||
while (!err) {
|
||||
bytes_to_write = xread(fd_in, buf, sizeof(buf));
|
||||
if (bytes_to_write < 0)
|
||||
return COPY_READ_ERROR;
|
||||
if (bytes_to_write == 0)
|
||||
break;
|
||||
err = packet_write_gently(fd_out, buf, bytes_to_write);
|
||||
}
|
||||
if (!err)
|
||||
err = packet_flush_gently(fd_out);
|
||||
return err;
|
||||
}
|
||||
|
||||
int write_packetized_from_buf(const char *src_in, size_t len, int fd_out)
|
||||
{
|
||||
int err = 0;
|
||||
size_t bytes_written = 0;
|
||||
size_t bytes_to_write;
|
||||
|
||||
while (!err) {
|
||||
if ((len - bytes_written) > LARGE_PACKET_DATA_MAX)
|
||||
bytes_to_write = LARGE_PACKET_DATA_MAX;
|
||||
else
|
||||
bytes_to_write = len - bytes_written;
|
||||
if (bytes_to_write == 0)
|
||||
break;
|
||||
err = packet_write_gently(fd_out, src_in + bytes_written, bytes_to_write);
|
||||
bytes_written += bytes_to_write;
|
||||
}
|
||||
if (!err)
|
||||
err = packet_flush_gently(fd_out);
|
||||
return err;
|
||||
}
|
||||
|
||||
static int get_packet_data(int fd, char **src_buf, size_t *src_size,
|
||||
void *dst, unsigned size, int options)
|
||||
{
|
||||
|
@ -229,3 +327,35 @@ char *packet_read_line_buf(char **src, size_t *src_len, int *dst_len)
|
|||
{
|
||||
return packet_read_line_generic(-1, src, src_len, dst_len);
|
||||
}
|
||||
|
||||
ssize_t read_packetized_to_strbuf(int fd_in, struct strbuf *sb_out)
|
||||
{
|
||||
int packet_len;
|
||||
|
||||
size_t orig_len = sb_out->len;
|
||||
size_t orig_alloc = sb_out->alloc;
|
||||
|
||||
for (;;) {
|
||||
strbuf_grow(sb_out, LARGE_PACKET_DATA_MAX);
|
||||
packet_len = packet_read(fd_in, NULL, NULL,
|
||||
/* strbuf_grow() above always allocates one extra byte to
|
||||
* store a '\0' at the end of the string. packet_read()
|
||||
* writes a '\0' extra byte at the end, too. Let it know
|
||||
* that there is already room for the extra byte.
|
||||
*/
|
||||
sb_out->buf + sb_out->len, LARGE_PACKET_DATA_MAX+1,
|
||||
PACKET_READ_GENTLE_ON_EOF);
|
||||
if (packet_len <= 0)
|
||||
break;
|
||||
sb_out->len += packet_len;
|
||||
}
|
||||
|
||||
if (packet_len < 0) {
|
||||
if (orig_alloc == 0)
|
||||
strbuf_release(sb_out);
|
||||
else
|
||||
strbuf_setlen(sb_out, orig_len);
|
||||
return packet_len;
|
||||
}
|
||||
return sb_out->len - orig_len;
|
||||
}
|
||||
|
|
12
pkt-line.h
12
pkt-line.h
|
@ -20,9 +20,13 @@
|
|||
* side can't, we stay with pure read/write interfaces.
|
||||
*/
|
||||
void packet_flush(int fd);
|
||||
void packet_write(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
|
||||
void packet_write_fmt(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
|
||||
void packet_buf_flush(struct strbuf *buf);
|
||||
void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
|
||||
int packet_flush_gently(int fd);
|
||||
int packet_write_fmt_gently(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
|
||||
int write_packetized_from_fd(int fd_in, int fd_out);
|
||||
int write_packetized_from_buf(const char *src_in, size_t len, int fd_out);
|
||||
|
||||
/*
|
||||
* Read a packetized line into the buffer, which must be at least size bytes
|
||||
|
@ -75,8 +79,14 @@ char *packet_read_line(int fd, int *size);
|
|||
*/
|
||||
char *packet_read_line_buf(char **src_buf, size_t *src_len, int *size);
|
||||
|
||||
/*
|
||||
* Reads a stream of variable sized packets until a flush packet is detected.
|
||||
*/
|
||||
ssize_t read_packetized_to_strbuf(int fd_in, struct strbuf *sb_out);
|
||||
|
||||
#define DEFAULT_PACKET_MAX 1000
|
||||
#define LARGE_PACKET_MAX 65520
|
||||
#define LARGE_PACKET_DATA_MAX (LARGE_PACKET_MAX - 4)
|
||||
extern char packet_buffer[LARGE_PACKET_MAX];
|
||||
|
||||
#endif
|
||||
|
|
|
@ -21,6 +21,7 @@ void child_process_clear(struct child_process *child)
|
|||
|
||||
struct child_to_clean {
|
||||
pid_t pid;
|
||||
struct child_process *process;
|
||||
struct child_to_clean *next;
|
||||
};
|
||||
static struct child_to_clean *children_to_clean;
|
||||
|
@ -31,6 +32,18 @@ static void cleanup_children(int sig, int in_signal)
|
|||
while (children_to_clean) {
|
||||
struct child_to_clean *p = children_to_clean;
|
||||
children_to_clean = p->next;
|
||||
|
||||
if (p->process && !in_signal) {
|
||||
struct child_process *process = p->process;
|
||||
if (process->clean_on_exit_handler) {
|
||||
trace_printf(
|
||||
"trace: run_command: running exit handler for pid %"
|
||||
PRIuMAX, (uintmax_t)p->pid
|
||||
);
|
||||
process->clean_on_exit_handler(process);
|
||||
}
|
||||
}
|
||||
|
||||
kill(p->pid, sig);
|
||||
if (!in_signal)
|
||||
free(p);
|
||||
|
@ -49,10 +62,11 @@ static void cleanup_children_on_exit(void)
|
|||
cleanup_children(SIGTERM, 0);
|
||||
}
|
||||
|
||||
static void mark_child_for_cleanup(pid_t pid)
|
||||
static void mark_child_for_cleanup(pid_t pid, struct child_process *process)
|
||||
{
|
||||
struct child_to_clean *p = xmalloc(sizeof(*p));
|
||||
p->pid = pid;
|
||||
p->process = process;
|
||||
p->next = children_to_clean;
|
||||
children_to_clean = p;
|
||||
|
||||
|
@ -422,7 +436,7 @@ fail_pipe:
|
|||
if (cmd->pid < 0)
|
||||
error_errno("cannot fork() for %s", cmd->argv[0]);
|
||||
else if (cmd->clean_on_exit)
|
||||
mark_child_for_cleanup(cmd->pid);
|
||||
mark_child_for_cleanup(cmd->pid, cmd);
|
||||
|
||||
/*
|
||||
* Wait for child's execvp. If the execvp succeeds (or if fork()
|
||||
|
@ -483,7 +497,7 @@ fail_pipe:
|
|||
if (cmd->pid < 0 && (!cmd->silent_exec_failure || errno != ENOENT))
|
||||
error_errno("cannot spawn %s", cmd->argv[0]);
|
||||
if (cmd->clean_on_exit && cmd->pid >= 0)
|
||||
mark_child_for_cleanup(cmd->pid);
|
||||
mark_child_for_cleanup(cmd->pid, cmd);
|
||||
|
||||
argv_array_clear(&nargv);
|
||||
cmd->argv = sargv;
|
||||
|
@ -634,7 +648,7 @@ int in_async(void)
|
|||
return !pthread_equal(main_thread, pthread_self());
|
||||
}
|
||||
|
||||
void NORETURN async_exit(int code)
|
||||
static void NORETURN async_exit(int code)
|
||||
{
|
||||
pthread_exit((void *)(intptr_t)code);
|
||||
}
|
||||
|
@ -684,13 +698,26 @@ int in_async(void)
|
|||
return process_is_async;
|
||||
}
|
||||
|
||||
void NORETURN async_exit(int code)
|
||||
static void NORETURN async_exit(int code)
|
||||
{
|
||||
exit(code);
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
void check_pipe(int err)
|
||||
{
|
||||
if (err == EPIPE) {
|
||||
if (in_async())
|
||||
async_exit(141);
|
||||
|
||||
signal(SIGPIPE, SIG_DFL);
|
||||
raise(SIGPIPE);
|
||||
/* Should never happen, but just in case... */
|
||||
exit(141);
|
||||
}
|
||||
}
|
||||
|
||||
int start_async(struct async *async)
|
||||
{
|
||||
int need_in, need_out;
|
||||
|
@ -752,7 +779,7 @@ int start_async(struct async *async)
|
|||
exit(!!async->proc(proc_in, proc_out, async->data));
|
||||
}
|
||||
|
||||
mark_child_for_cleanup(async->pid);
|
||||
mark_child_for_cleanup(async->pid, NULL);
|
||||
|
||||
if (need_in)
|
||||
close(fdin[0]);
|
||||
|
|
|
@ -43,6 +43,8 @@ struct child_process {
|
|||
unsigned stdout_to_stderr:1;
|
||||
unsigned use_shell:1;
|
||||
unsigned clean_on_exit:1;
|
||||
void (*clean_on_exit_handler)(struct child_process *process);
|
||||
void *clean_on_exit_handler_cbdata;
|
||||
};
|
||||
|
||||
#define CHILD_PROCESS_INIT { NULL, ARGV_ARRAY_INIT, ARGV_ARRAY_INIT }
|
||||
|
@ -139,7 +141,7 @@ struct async {
|
|||
int start_async(struct async *async);
|
||||
int finish_async(struct async *async);
|
||||
int in_async(void);
|
||||
void NORETURN async_exit(int code);
|
||||
void check_pipe(int err);
|
||||
|
||||
/**
|
||||
* This callback should initialize the child process and preload the
|
||||
|
|
|
@ -338,7 +338,7 @@ static int advertise_shallow_grafts_cb(const struct commit_graft *graft, void *c
|
|||
{
|
||||
int fd = *(int *)cb;
|
||||
if (graft->nr_parent == -1)
|
||||
packet_write(fd, "shallow %s\n", oid_to_hex(&graft->oid));
|
||||
packet_write_fmt(fd, "shallow %s\n", oid_to_hex(&graft->oid));
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
|
@ -4,13 +4,72 @@ test_description='blob conversion via gitattributes'
|
|||
|
||||
. ./test-lib.sh
|
||||
|
||||
cat <<EOF >rot13.sh
|
||||
TEST_ROOT="$(pwd)"
|
||||
|
||||
cat <<EOF >"$TEST_ROOT/rot13.sh"
|
||||
#!$SHELL_PATH
|
||||
tr \
|
||||
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ' \
|
||||
'nopqrstuvwxyzabcdefghijklmNOPQRSTUVWXYZABCDEFGHIJKLM'
|
||||
EOF
|
||||
chmod +x rot13.sh
|
||||
chmod +x "$TEST_ROOT/rot13.sh"
|
||||
|
||||
generate_random_characters () {
|
||||
LEN=$1
|
||||
NAME=$2
|
||||
test-genrandom some-seed $LEN |
|
||||
perl -pe "s/./chr((ord($&) % 26) + ord('a'))/sge" >"$TEST_ROOT/$NAME"
|
||||
}
|
||||
|
||||
file_size () {
|
||||
cat "$1" | wc -c | sed "s/^[ ]*//"
|
||||
}
|
||||
|
||||
filter_git () {
|
||||
rm -f rot13-filter.log &&
|
||||
git "$@" 2>git-stderr.log &&
|
||||
rm -f git-stderr.log
|
||||
}
|
||||
|
||||
# Compare two files and ensure that `clean` and `smudge` respectively are
|
||||
# called at least once if specified in the `expect` file. The actual
|
||||
# invocation count is not relevant because their number can vary.
|
||||
# c.f. http://public-inbox.org/git/xmqqshv18i8i.fsf@gitster.mtv.corp.google.com/
|
||||
test_cmp_count () {
|
||||
expect=$1
|
||||
actual=$2
|
||||
for FILE in "$expect" "$actual"
|
||||
do
|
||||
sort "$FILE" | uniq -c | sed "s/^[ ]*//" |
|
||||
sed "s/^\([0-9]\) IN: clean/x IN: clean/" |
|
||||
sed "s/^\([0-9]\) IN: smudge/x IN: smudge/" >"$FILE.tmp" &&
|
||||
mv "$FILE.tmp" "$FILE"
|
||||
done &&
|
||||
test_cmp "$expect" "$actual"
|
||||
}
|
||||
|
||||
# Compare two files but exclude all `clean` invocations because Git can
|
||||
# call `clean` zero or more times.
|
||||
# c.f. http://public-inbox.org/git/xmqqshv18i8i.fsf@gitster.mtv.corp.google.com/
|
||||
test_cmp_exclude_clean () {
|
||||
expect=$1
|
||||
actual=$2
|
||||
for FILE in "$expect" "$actual"
|
||||
do
|
||||
grep -v "IN: clean" "$FILE" >"$FILE.tmp" &&
|
||||
mv "$FILE.tmp" "$FILE"
|
||||
done &&
|
||||
test_cmp "$expect" "$actual"
|
||||
}
|
||||
|
||||
# Check that the contents of two files are equal and that their rot13 version
|
||||
# is equal to the committed content.
|
||||
test_cmp_committed_rot13 () {
|
||||
test_cmp "$1" "$2" &&
|
||||
"$TEST_ROOT/rot13.sh" <"$1" >expected &&
|
||||
git cat-file blob :"$2" >actual &&
|
||||
test_cmp expected actual
|
||||
}
|
||||
|
||||
test_expect_success setup '
|
||||
git config filter.rot13.smudge ./rot13.sh &&
|
||||
|
@ -31,15 +90,18 @@ test_expect_success setup '
|
|||
cat test >test.i &&
|
||||
git add test test.t test.i &&
|
||||
rm -f test test.t test.i &&
|
||||
git checkout -- test test.t test.i
|
||||
git checkout -- test test.t test.i &&
|
||||
|
||||
echo "content-test2" >test2.o &&
|
||||
echo "content-test3 - filename with special characters" >"test3 '\''sq'\'',\$x.o"
|
||||
'
|
||||
|
||||
script='s/^\$Id: \([0-9a-f]*\) \$/\1/p'
|
||||
|
||||
test_expect_success check '
|
||||
|
||||
cmp test.o test &&
|
||||
cmp test.o test.t &&
|
||||
test_cmp test.o test &&
|
||||
test_cmp test.o test.t &&
|
||||
|
||||
# ident should be stripped in the repository
|
||||
git diff --raw --exit-code :test :test.i &&
|
||||
|
@ -47,10 +109,10 @@ test_expect_success check '
|
|||
embedded=$(sed -ne "$script" test.i) &&
|
||||
test "z$id" = "z$embedded" &&
|
||||
|
||||
git cat-file blob :test.t > test.r &&
|
||||
git cat-file blob :test.t >test.r &&
|
||||
|
||||
./rot13.sh < test.o > test.t &&
|
||||
cmp test.r test.t
|
||||
./rot13.sh <test.o >test.t &&
|
||||
test_cmp test.r test.t
|
||||
'
|
||||
|
||||
# If an expanded ident ever gets into the repository, we want to make sure that
|
||||
|
@ -130,7 +192,7 @@ test_expect_success 'filter shell-escaped filenames' '
|
|||
|
||||
# delete the files and check them out again, using a smudge filter
|
||||
# that will count the args and echo the command-line back to us
|
||||
git config filter.argc.smudge "sh ./argc.sh %f" &&
|
||||
test_config filter.argc.smudge "sh ./argc.sh %f" &&
|
||||
rm "$normal" "$special" &&
|
||||
git checkout -- "$normal" "$special" &&
|
||||
|
||||
|
@ -141,7 +203,7 @@ test_expect_success 'filter shell-escaped filenames' '
|
|||
test_cmp expect "$special" &&
|
||||
|
||||
# do the same thing, but with more args in the filter expression
|
||||
git config filter.argc.smudge "sh ./argc.sh %f --my-extra-arg" &&
|
||||
test_config filter.argc.smudge "sh ./argc.sh %f --my-extra-arg" &&
|
||||
rm "$normal" "$special" &&
|
||||
git checkout -- "$normal" "$special" &&
|
||||
|
||||
|
@ -154,9 +216,9 @@ test_expect_success 'filter shell-escaped filenames' '
|
|||
'
|
||||
|
||||
test_expect_success 'required filter should filter data' '
|
||||
git config filter.required.smudge ./rot13.sh &&
|
||||
git config filter.required.clean ./rot13.sh &&
|
||||
git config filter.required.required true &&
|
||||
test_config filter.required.smudge ./rot13.sh &&
|
||||
test_config filter.required.clean ./rot13.sh &&
|
||||
test_config filter.required.required true &&
|
||||
|
||||
echo "*.r filter=required" >.gitattributes &&
|
||||
|
||||
|
@ -165,17 +227,17 @@ test_expect_success 'required filter should filter data' '
|
|||
|
||||
rm -f test.r &&
|
||||
git checkout -- test.r &&
|
||||
cmp test.o test.r &&
|
||||
test_cmp test.o test.r &&
|
||||
|
||||
./rot13.sh <test.o >expected &&
|
||||
git cat-file blob :test.r >actual &&
|
||||
cmp expected actual
|
||||
test_cmp expected actual
|
||||
'
|
||||
|
||||
test_expect_success 'required filter smudge failure' '
|
||||
git config filter.failsmudge.smudge false &&
|
||||
git config filter.failsmudge.clean cat &&
|
||||
git config filter.failsmudge.required true &&
|
||||
test_config filter.failsmudge.smudge false &&
|
||||
test_config filter.failsmudge.clean cat &&
|
||||
test_config filter.failsmudge.required true &&
|
||||
|
||||
echo "*.fs filter=failsmudge" >.gitattributes &&
|
||||
|
||||
|
@ -186,9 +248,9 @@ test_expect_success 'required filter smudge failure' '
|
|||
'
|
||||
|
||||
test_expect_success 'required filter clean failure' '
|
||||
git config filter.failclean.smudge cat &&
|
||||
git config filter.failclean.clean false &&
|
||||
git config filter.failclean.required true &&
|
||||
test_config filter.failclean.smudge cat &&
|
||||
test_config filter.failclean.clean false &&
|
||||
test_config filter.failclean.required true &&
|
||||
|
||||
echo "*.fc filter=failclean" >.gitattributes &&
|
||||
|
||||
|
@ -197,8 +259,8 @@ test_expect_success 'required filter clean failure' '
|
|||
'
|
||||
|
||||
test_expect_success 'filtering large input to small output should use little memory' '
|
||||
git config filter.devnull.clean "cat >/dev/null" &&
|
||||
git config filter.devnull.required true &&
|
||||
test_config filter.devnull.clean "cat >/dev/null" &&
|
||||
test_config filter.devnull.required true &&
|
||||
for i in $(test_seq 1 30); do printf "%1048576d" 1; done >30MB &&
|
||||
echo "30MB filter=devnull" >.gitattributes &&
|
||||
GIT_MMAP_LIMIT=1m GIT_ALLOC_LIMIT=1m git add 30MB
|
||||
|
@ -207,7 +269,7 @@ test_expect_success 'filtering large input to small output should use little mem
|
|||
test_expect_success 'filter that does not read is fine' '
|
||||
test-genrandom foo $((128 * 1024 + 1)) >big &&
|
||||
echo "big filter=epipe" >.gitattributes &&
|
||||
git config filter.epipe.clean "echo xyzzy" &&
|
||||
test_config filter.epipe.clean "echo xyzzy" &&
|
||||
git add big &&
|
||||
git cat-file blob :big >actual &&
|
||||
echo xyzzy >expect &&
|
||||
|
@ -215,20 +277,20 @@ test_expect_success 'filter that does not read is fine' '
|
|||
'
|
||||
|
||||
test_expect_success EXPENSIVE 'filter large file' '
|
||||
git config filter.largefile.smudge cat &&
|
||||
git config filter.largefile.clean cat &&
|
||||
test_config filter.largefile.smudge cat &&
|
||||
test_config filter.largefile.clean cat &&
|
||||
for i in $(test_seq 1 2048); do printf "%1048576d" 1; done >2GB &&
|
||||
echo "2GB filter=largefile" >.gitattributes &&
|
||||
git add 2GB 2>err &&
|
||||
! test -s err &&
|
||||
test_must_be_empty err &&
|
||||
rm -f 2GB &&
|
||||
git checkout -- 2GB 2>err &&
|
||||
! test -s err
|
||||
test_must_be_empty err
|
||||
'
|
||||
|
||||
test_expect_success "filter: clean empty file" '
|
||||
git config filter.in-repo-header.clean "echo cleaned && cat" &&
|
||||
git config filter.in-repo-header.smudge "sed 1d" &&
|
||||
test_config filter.in-repo-header.clean "echo cleaned && cat" &&
|
||||
test_config filter.in-repo-header.smudge "sed 1d" &&
|
||||
|
||||
echo "empty-in-worktree filter=in-repo-header" >>.gitattributes &&
|
||||
>empty-in-worktree &&
|
||||
|
@ -240,8 +302,8 @@ test_expect_success "filter: clean empty file" '
|
|||
'
|
||||
|
||||
test_expect_success "filter: smudge empty file" '
|
||||
git config filter.empty-in-repo.clean "cat >/dev/null" &&
|
||||
git config filter.empty-in-repo.smudge "echo smudged && cat" &&
|
||||
test_config filter.empty-in-repo.clean "cat >/dev/null" &&
|
||||
test_config filter.empty-in-repo.smudge "echo smudged && cat" &&
|
||||
|
||||
echo "empty-in-repo filter=empty-in-repo" >>.gitattributes &&
|
||||
echo dead data walking >empty-in-repo &&
|
||||
|
@ -279,4 +341,380 @@ test_expect_success 'diff does not reuse worktree files that need cleaning' '
|
|||
test_line_count = 0 count
|
||||
'
|
||||
|
||||
test_expect_success PERL 'required process filter should filter data' '
|
||||
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
|
||||
test_config_global filter.protocol.required true &&
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
echo "git-stderr.log" >.gitignore &&
|
||||
echo "*.r filter=protocol" >.gitattributes &&
|
||||
git add . &&
|
||||
git commit . -m "test commit 1" &&
|
||||
git branch empty-branch &&
|
||||
|
||||
cp "$TEST_ROOT/test.o" test.r &&
|
||||
cp "$TEST_ROOT/test2.o" test2.r &&
|
||||
mkdir testsubdir &&
|
||||
cp "$TEST_ROOT/test3 '\''sq'\'',\$x.o" "testsubdir/test3 '\''sq'\'',\$x.r" &&
|
||||
>test4-empty.r &&
|
||||
|
||||
S=$(file_size test.r) &&
|
||||
S2=$(file_size test2.r) &&
|
||||
S3=$(file_size "testsubdir/test3 '\''sq'\'',\$x.r") &&
|
||||
|
||||
filter_git add . &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: clean test.r $S [OK] -- OUT: $S . [OK]
|
||||
IN: clean test2.r $S2 [OK] -- OUT: $S2 . [OK]
|
||||
IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
|
||||
IN: clean testsubdir/test3 '\''sq'\'',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_count expected.log rot13-filter.log &&
|
||||
|
||||
filter_git commit . -m "test commit 2" &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: clean test.r $S [OK] -- OUT: $S . [OK]
|
||||
IN: clean test2.r $S2 [OK] -- OUT: $S2 . [OK]
|
||||
IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
|
||||
IN: clean testsubdir/test3 '\''sq'\'',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
|
||||
IN: clean test.r $S [OK] -- OUT: $S . [OK]
|
||||
IN: clean test2.r $S2 [OK] -- OUT: $S2 . [OK]
|
||||
IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
|
||||
IN: clean testsubdir/test3 '\''sq'\'',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_count expected.log rot13-filter.log &&
|
||||
|
||||
rm -f test2.r "testsubdir/test3 '\''sq'\'',\$x.r" &&
|
||||
|
||||
filter_git checkout --quiet --no-progress . &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]
|
||||
IN: smudge testsubdir/test3 '\''sq'\'',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_exclude_clean expected.log rot13-filter.log &&
|
||||
|
||||
filter_git checkout --quiet --no-progress empty-branch &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: clean test.r $S [OK] -- OUT: $S . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_exclude_clean expected.log rot13-filter.log &&
|
||||
|
||||
filter_git checkout --quiet --no-progress master &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: smudge test.r $S [OK] -- OUT: $S . [OK]
|
||||
IN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]
|
||||
IN: smudge test4-empty.r 0 [OK] -- OUT: 0 [OK]
|
||||
IN: smudge testsubdir/test3 '\''sq'\'',\$x.r $S3 [OK] -- OUT: $S3 . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_exclude_clean expected.log rot13-filter.log &&
|
||||
|
||||
test_cmp_committed_rot13 "$TEST_ROOT/test.o" test.r &&
|
||||
test_cmp_committed_rot13 "$TEST_ROOT/test2.o" test2.r &&
|
||||
test_cmp_committed_rot13 "$TEST_ROOT/test3 '\''sq'\'',\$x.o" "testsubdir/test3 '\''sq'\'',\$x.r"
|
||||
)
|
||||
'
|
||||
|
||||
test_expect_success PERL 'required process filter takes precedence' '
|
||||
test_config_global filter.protocol.clean false &&
|
||||
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean" &&
|
||||
test_config_global filter.protocol.required true &&
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
echo "*.r filter=protocol" >.gitattributes &&
|
||||
cp "$TEST_ROOT/test.o" test.r &&
|
||||
S=$(file_size test.r) &&
|
||||
|
||||
# Check that the process filter is invoked here
|
||||
filter_git add . &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: clean test.r $S [OK] -- OUT: $S . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_count expected.log rot13-filter.log
|
||||
)
|
||||
'
|
||||
|
||||
test_expect_success PERL 'required process filter should be used only for "clean" operation only' '
|
||||
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean" &&
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
echo "*.r filter=protocol" >.gitattributes &&
|
||||
cp "$TEST_ROOT/test.o" test.r &&
|
||||
S=$(file_size test.r) &&
|
||||
|
||||
filter_git add . &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: clean test.r $S [OK] -- OUT: $S . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_count expected.log rot13-filter.log &&
|
||||
|
||||
rm test.r &&
|
||||
|
||||
filter_git checkout --quiet --no-progress . &&
|
||||
# If the filter would be used for "smudge", too, we would see
|
||||
# "IN: smudge test.r 57 [OK] -- OUT: 57 . [OK]" here
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_exclude_clean expected.log rot13-filter.log
|
||||
)
|
||||
'
|
||||
|
||||
test_expect_success PERL 'required process filter should process multiple packets' '
|
||||
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
|
||||
test_config_global filter.protocol.required true &&
|
||||
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
# Generate data requiring 1, 2, 3 packets
|
||||
S=65516 && # PKTLINE_DATA_MAXLEN -> Maximal size of a packet
|
||||
generate_random_characters $(($S )) 1pkt_1__.file &&
|
||||
generate_random_characters $(($S +1)) 2pkt_1+1.file &&
|
||||
generate_random_characters $(($S*2-1)) 2pkt_2-1.file &&
|
||||
generate_random_characters $(($S*2 )) 2pkt_2__.file &&
|
||||
generate_random_characters $(($S*2+1)) 3pkt_2+1.file &&
|
||||
|
||||
for FILE in "$TEST_ROOT"/*.file
|
||||
do
|
||||
cp "$FILE" . &&
|
||||
"$TEST_ROOT/rot13.sh" <"$FILE" >"$FILE.rot13"
|
||||
done &&
|
||||
|
||||
echo "*.file filter=protocol" >.gitattributes &&
|
||||
filter_git add *.file .gitattributes &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: clean 1pkt_1__.file $(($S )) [OK] -- OUT: $(($S )) . [OK]
|
||||
IN: clean 2pkt_1+1.file $(($S +1)) [OK] -- OUT: $(($S +1)) .. [OK]
|
||||
IN: clean 2pkt_2-1.file $(($S*2-1)) [OK] -- OUT: $(($S*2-1)) .. [OK]
|
||||
IN: clean 2pkt_2__.file $(($S*2 )) [OK] -- OUT: $(($S*2 )) .. [OK]
|
||||
IN: clean 3pkt_2+1.file $(($S*2+1)) [OK] -- OUT: $(($S*2+1)) ... [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_count expected.log rot13-filter.log &&
|
||||
|
||||
rm -f *.file &&
|
||||
|
||||
filter_git checkout --quiet --no-progress -- *.file &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: smudge 1pkt_1__.file $(($S )) [OK] -- OUT: $(($S )) . [OK]
|
||||
IN: smudge 2pkt_1+1.file $(($S +1)) [OK] -- OUT: $(($S +1)) .. [OK]
|
||||
IN: smudge 2pkt_2-1.file $(($S*2-1)) [OK] -- OUT: $(($S*2-1)) .. [OK]
|
||||
IN: smudge 2pkt_2__.file $(($S*2 )) [OK] -- OUT: $(($S*2 )) .. [OK]
|
||||
IN: smudge 3pkt_2+1.file $(($S*2+1)) [OK] -- OUT: $(($S*2+1)) ... [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_exclude_clean expected.log rot13-filter.log &&
|
||||
|
||||
for FILE in *.file
|
||||
do
|
||||
test_cmp_committed_rot13 "$TEST_ROOT/$FILE" $FILE
|
||||
done
|
||||
)
|
||||
'
|
||||
|
||||
test_expect_success PERL 'required process filter with clean error should fail' '
|
||||
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
|
||||
test_config_global filter.protocol.required true &&
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
echo "*.r filter=protocol" >.gitattributes &&
|
||||
|
||||
cp "$TEST_ROOT/test.o" test.r &&
|
||||
echo "this is going to fail" >clean-write-fail.r &&
|
||||
echo "content-test3-subdir" >test3.r &&
|
||||
|
||||
test_must_fail git add .
|
||||
)
|
||||
'
|
||||
|
||||
test_expect_success PERL 'process filter should restart after unexpected write failure' '
|
||||
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
echo "*.r filter=protocol" >.gitattributes &&
|
||||
|
||||
cp "$TEST_ROOT/test.o" test.r &&
|
||||
cp "$TEST_ROOT/test2.o" test2.r &&
|
||||
echo "this is going to fail" >smudge-write-fail.o &&
|
||||
cp smudge-write-fail.o smudge-write-fail.r &&
|
||||
|
||||
S=$(file_size test.r) &&
|
||||
S2=$(file_size test2.r) &&
|
||||
SF=$(file_size smudge-write-fail.r) &&
|
||||
|
||||
git add . &&
|
||||
rm -f *.r &&
|
||||
|
||||
rm -f rot13-filter.log &&
|
||||
git checkout --quiet --no-progress . 2>git-stderr.log &&
|
||||
|
||||
grep "smudge write error at" git-stderr.log &&
|
||||
grep "error: external filter" git-stderr.log &&
|
||||
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: smudge smudge-write-fail.r $SF [OK] -- OUT: $SF [WRITE FAIL]
|
||||
START
|
||||
init handshake complete
|
||||
IN: smudge test.r $S [OK] -- OUT: $S . [OK]
|
||||
IN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_exclude_clean expected.log rot13-filter.log &&
|
||||
|
||||
test_cmp_committed_rot13 "$TEST_ROOT/test.o" test.r &&
|
||||
test_cmp_committed_rot13 "$TEST_ROOT/test2.o" test2.r &&
|
||||
|
||||
# Smudge failed
|
||||
! test_cmp smudge-write-fail.o smudge-write-fail.r &&
|
||||
"$TEST_ROOT/rot13.sh" <smudge-write-fail.o >expected &&
|
||||
git cat-file blob :smudge-write-fail.r >actual &&
|
||||
test_cmp expected actual
|
||||
)
|
||||
'
|
||||
|
||||
test_expect_success PERL 'process filter should not be restarted if it signals an error' '
|
||||
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
echo "*.r filter=protocol" >.gitattributes &&
|
||||
|
||||
cp "$TEST_ROOT/test.o" test.r &&
|
||||
cp "$TEST_ROOT/test2.o" test2.r &&
|
||||
echo "this will cause an error" >error.o &&
|
||||
cp error.o error.r &&
|
||||
|
||||
S=$(file_size test.r) &&
|
||||
S2=$(file_size test2.r) &&
|
||||
SE=$(file_size error.r) &&
|
||||
|
||||
git add . &&
|
||||
rm -f *.r &&
|
||||
|
||||
filter_git checkout --quiet --no-progress . &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: smudge error.r $SE [OK] -- OUT: 0 [ERROR]
|
||||
IN: smudge test.r $S [OK] -- OUT: $S . [OK]
|
||||
IN: smudge test2.r $S2 [OK] -- OUT: $S2 . [OK]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_exclude_clean expected.log rot13-filter.log &&
|
||||
|
||||
test_cmp_committed_rot13 "$TEST_ROOT/test.o" test.r &&
|
||||
test_cmp_committed_rot13 "$TEST_ROOT/test2.o" test2.r &&
|
||||
test_cmp error.o error.r
|
||||
)
|
||||
'
|
||||
|
||||
test_expect_success PERL 'process filter abort stops processing of all further files' '
|
||||
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
echo "*.r filter=protocol" >.gitattributes &&
|
||||
|
||||
cp "$TEST_ROOT/test.o" test.r &&
|
||||
cp "$TEST_ROOT/test2.o" test2.r &&
|
||||
echo "error this blob and all future blobs" >abort.o &&
|
||||
cp abort.o abort.r &&
|
||||
|
||||
SA=$(file_size abort.r) &&
|
||||
|
||||
git add . &&
|
||||
rm -f *.r &&
|
||||
|
||||
# Note: This test assumes that Git filters files in alphabetical
|
||||
# order ("abort.r" before "test.r").
|
||||
filter_git checkout --quiet --no-progress . &&
|
||||
cat >expected.log <<-EOF &&
|
||||
START
|
||||
init handshake complete
|
||||
IN: smudge abort.r $SA [OK] -- OUT: 0 [ABORT]
|
||||
STOP
|
||||
EOF
|
||||
test_cmp_exclude_clean expected.log rot13-filter.log &&
|
||||
|
||||
test_cmp "$TEST_ROOT/test.o" test.r &&
|
||||
test_cmp "$TEST_ROOT/test2.o" test2.r &&
|
||||
test_cmp abort.o abort.r
|
||||
)
|
||||
'
|
||||
|
||||
test_expect_success PERL 'invalid process filter must fail (and not hang!)' '
|
||||
test_config_global filter.protocol.process cat &&
|
||||
test_config_global filter.protocol.required true &&
|
||||
rm -rf repo &&
|
||||
mkdir repo &&
|
||||
(
|
||||
cd repo &&
|
||||
git init &&
|
||||
|
||||
echo "*.r filter=protocol" >.gitattributes &&
|
||||
|
||||
cp "$TEST_ROOT/test.o" test.r &&
|
||||
test_must_fail git add . 2>git-stderr.log &&
|
||||
grep "does not support filter protocol version" git-stderr.log
|
||||
)
|
||||
'
|
||||
|
||||
test_done
|
||||
|
|
|
@ -0,0 +1,192 @@
|
|||
#!/usr/bin/perl
|
||||
#
|
||||
# Example implementation for the Git filter protocol version 2
|
||||
# See Documentation/gitattributes.txt, section "Filter Protocol"
|
||||
#
|
||||
# The script takes the list of supported protocol capabilities as
|
||||
# arguments ("clean", "smudge", etc).
|
||||
#
|
||||
# This implementation supports special test cases:
|
||||
# (1) If data with the pathname "clean-write-fail.r" is processed with
|
||||
# a "clean" operation then the write operation will die.
|
||||
# (2) If data with the pathname "smudge-write-fail.r" is processed with
|
||||
# a "smudge" operation then the write operation will die.
|
||||
# (3) If data with the pathname "error.r" is processed with any
|
||||
# operation then the filter signals that it cannot or does not want
|
||||
# to process the file.
|
||||
# (4) If data with the pathname "abort.r" is processed with any
|
||||
# operation then the filter signals that it cannot or does not want
|
||||
# to process the file and any file after that is processed with the
|
||||
# same command.
|
||||
#
|
||||
|
||||
use strict;
|
||||
use warnings;
|
||||
|
||||
my $MAX_PACKET_CONTENT_SIZE = 65516;
|
||||
my @capabilities = @ARGV;
|
||||
|
||||
open my $debug, ">>", "rot13-filter.log" or die "cannot open log file: $!";
|
||||
|
||||
sub rot13 {
|
||||
my $str = shift;
|
||||
$str =~ y/A-Za-z/N-ZA-Mn-za-m/;
|
||||
return $str;
|
||||
}
|
||||
|
||||
sub packet_bin_read {
|
||||
my $buffer;
|
||||
my $bytes_read = read STDIN, $buffer, 4;
|
||||
if ( $bytes_read == 0 ) {
|
||||
# EOF - Git stopped talking to us!
|
||||
print $debug "STOP\n";
|
||||
exit();
|
||||
}
|
||||
elsif ( $bytes_read != 4 ) {
|
||||
die "invalid packet: '$buffer'";
|
||||
}
|
||||
my $pkt_size = hex($buffer);
|
||||
if ( $pkt_size == 0 ) {
|
||||
return ( 1, "" );
|
||||
}
|
||||
elsif ( $pkt_size > 4 ) {
|
||||
my $content_size = $pkt_size - 4;
|
||||
$bytes_read = read STDIN, $buffer, $content_size;
|
||||
if ( $bytes_read != $content_size ) {
|
||||
die "invalid packet ($content_size bytes expected; $bytes_read bytes read)";
|
||||
}
|
||||
return ( 0, $buffer );
|
||||
}
|
||||
else {
|
||||
die "invalid packet size: $pkt_size";
|
||||
}
|
||||
}
|
||||
|
||||
sub packet_txt_read {
|
||||
my ( $res, $buf ) = packet_bin_read();
|
||||
unless ( $buf =~ s/\n$// ) {
|
||||
die "A non-binary line MUST be terminated by an LF.";
|
||||
}
|
||||
return ( $res, $buf );
|
||||
}
|
||||
|
||||
sub packet_bin_write {
|
||||
my $buf = shift;
|
||||
print STDOUT sprintf( "%04x", length($buf) + 4 );
|
||||
print STDOUT $buf;
|
||||
STDOUT->flush();
|
||||
}
|
||||
|
||||
sub packet_txt_write {
|
||||
packet_bin_write( $_[0] . "\n" );
|
||||
}
|
||||
|
||||
sub packet_flush {
|
||||
print STDOUT sprintf( "%04x", 0 );
|
||||
STDOUT->flush();
|
||||
}
|
||||
|
||||
print $debug "START\n";
|
||||
$debug->flush();
|
||||
|
||||
( packet_txt_read() eq ( 0, "git-filter-client" ) ) || die "bad initialize";
|
||||
( packet_txt_read() eq ( 0, "version=2" ) ) || die "bad version";
|
||||
( packet_bin_read() eq ( 1, "" ) ) || die "bad version end";
|
||||
|
||||
packet_txt_write("git-filter-server");
|
||||
packet_txt_write("version=2");
|
||||
packet_flush();
|
||||
|
||||
( packet_txt_read() eq ( 0, "capability=clean" ) ) || die "bad capability";
|
||||
( packet_txt_read() eq ( 0, "capability=smudge" ) ) || die "bad capability";
|
||||
( packet_bin_read() eq ( 1, "" ) ) || die "bad capability end";
|
||||
|
||||
foreach (@capabilities) {
|
||||
packet_txt_write( "capability=" . $_ );
|
||||
}
|
||||
packet_flush();
|
||||
print $debug "init handshake complete\n";
|
||||
$debug->flush();
|
||||
|
||||
while (1) {
|
||||
my ($command) = packet_txt_read() =~ /^command=([^=]+)$/;
|
||||
print $debug "IN: $command";
|
||||
$debug->flush();
|
||||
|
||||
my ($pathname) = packet_txt_read() =~ /^pathname=([^=]+)$/;
|
||||
print $debug " $pathname";
|
||||
$debug->flush();
|
||||
|
||||
# Flush
|
||||
packet_bin_read();
|
||||
|
||||
my $input = "";
|
||||
{
|
||||
binmode(STDIN);
|
||||
my $buffer;
|
||||
my $done = 0;
|
||||
while ( !$done ) {
|
||||
( $done, $buffer ) = packet_bin_read();
|
||||
$input .= $buffer;
|
||||
}
|
||||
print $debug " " . length($input) . " [OK] -- ";
|
||||
$debug->flush();
|
||||
}
|
||||
|
||||
my $output;
|
||||
if ( $pathname eq "error.r" or $pathname eq "abort.r" ) {
|
||||
$output = "";
|
||||
}
|
||||
elsif ( $command eq "clean" and grep( /^clean$/, @capabilities ) ) {
|
||||
$output = rot13($input);
|
||||
}
|
||||
elsif ( $command eq "smudge" and grep( /^smudge$/, @capabilities ) ) {
|
||||
$output = rot13($input);
|
||||
}
|
||||
else {
|
||||
die "bad command '$command'";
|
||||
}
|
||||
|
||||
print $debug "OUT: " . length($output) . " ";
|
||||
$debug->flush();
|
||||
|
||||
if ( $pathname eq "error.r" ) {
|
||||
print $debug "[ERROR]\n";
|
||||
$debug->flush();
|
||||
packet_txt_write("status=error");
|
||||
packet_flush();
|
||||
}
|
||||
elsif ( $pathname eq "abort.r" ) {
|
||||
print $debug "[ABORT]\n";
|
||||
$debug->flush();
|
||||
packet_txt_write("status=abort");
|
||||
packet_flush();
|
||||
}
|
||||
else {
|
||||
packet_txt_write("status=success");
|
||||
packet_flush();
|
||||
|
||||
if ( $pathname eq "${command}-write-fail.r" ) {
|
||||
print $debug "[WRITE FAIL]\n";
|
||||
$debug->flush();
|
||||
die "${command} write error";
|
||||
}
|
||||
|
||||
while ( length($output) > 0 ) {
|
||||
my $packet = substr( $output, 0, $MAX_PACKET_CONTENT_SIZE );
|
||||
packet_bin_write($packet);
|
||||
# dots represent the number of packets
|
||||
print $debug ".";
|
||||
if ( length($output) > $MAX_PACKET_CONTENT_SIZE ) {
|
||||
$output = substr( $output, $MAX_PACKET_CONTENT_SIZE );
|
||||
}
|
||||
else {
|
||||
$output = "";
|
||||
}
|
||||
}
|
||||
packet_flush();
|
||||
print $debug " [OK]\n";
|
||||
$debug->flush();
|
||||
packet_flush();
|
||||
}
|
||||
}
|
|
@ -398,13 +398,13 @@ static int get_common_commits(void)
|
|||
if (multi_ack == 2 && got_common
|
||||
&& !got_other && ok_to_give_up()) {
|
||||
sent_ready = 1;
|
||||
packet_write(1, "ACK %s ready\n", last_hex);
|
||||
packet_write_fmt(1, "ACK %s ready\n", last_hex);
|
||||
}
|
||||
if (have_obj.nr == 0 || multi_ack)
|
||||
packet_write(1, "NAK\n");
|
||||
packet_write_fmt(1, "NAK\n");
|
||||
|
||||
if (no_done && sent_ready) {
|
||||
packet_write(1, "ACK %s\n", last_hex);
|
||||
packet_write_fmt(1, "ACK %s\n", last_hex);
|
||||
return 0;
|
||||
}
|
||||
if (stateless_rpc)
|
||||
|
@ -421,20 +421,20 @@ static int get_common_commits(void)
|
|||
const char *hex = sha1_to_hex(sha1);
|
||||
if (multi_ack == 2) {
|
||||
sent_ready = 1;
|
||||
packet_write(1, "ACK %s ready\n", hex);
|
||||
packet_write_fmt(1, "ACK %s ready\n", hex);
|
||||
} else
|
||||
packet_write(1, "ACK %s continue\n", hex);
|
||||
packet_write_fmt(1, "ACK %s continue\n", hex);
|
||||
}
|
||||
break;
|
||||
default:
|
||||
got_common = 1;
|
||||
memcpy(last_hex, sha1_to_hex(sha1), 41);
|
||||
if (multi_ack == 2)
|
||||
packet_write(1, "ACK %s common\n", last_hex);
|
||||
packet_write_fmt(1, "ACK %s common\n", last_hex);
|
||||
else if (multi_ack)
|
||||
packet_write(1, "ACK %s continue\n", last_hex);
|
||||
packet_write_fmt(1, "ACK %s continue\n", last_hex);
|
||||
else if (have_obj.nr == 1)
|
||||
packet_write(1, "ACK %s\n", last_hex);
|
||||
packet_write_fmt(1, "ACK %s\n", last_hex);
|
||||
break;
|
||||
}
|
||||
continue;
|
||||
|
@ -442,10 +442,10 @@ static int get_common_commits(void)
|
|||
if (!strcmp(line, "done")) {
|
||||
if (have_obj.nr > 0) {
|
||||
if (multi_ack)
|
||||
packet_write(1, "ACK %s\n", last_hex);
|
||||
packet_write_fmt(1, "ACK %s\n", last_hex);
|
||||
return 0;
|
||||
}
|
||||
packet_write(1, "NAK\n");
|
||||
packet_write_fmt(1, "NAK\n");
|
||||
return -1;
|
||||
}
|
||||
die("git upload-pack: expected SHA1 list, got '%s'", line);
|
||||
|
@ -638,8 +638,8 @@ static void send_shallow(struct commit_list *result)
|
|||
while (result) {
|
||||
struct object *object = &result->item->object;
|
||||
if (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {
|
||||
packet_write(1, "shallow %s",
|
||||
oid_to_hex(&object->oid));
|
||||
packet_write_fmt(1, "shallow %s",
|
||||
oid_to_hex(&object->oid));
|
||||
register_shallow(object->oid.hash);
|
||||
shallow_nr++;
|
||||
}
|
||||
|
@ -655,8 +655,8 @@ static void send_unshallow(const struct object_array *shallows)
|
|||
struct object *object = shallows->objects[i].item;
|
||||
if (object->flags & NOT_SHALLOW) {
|
||||
struct commit_list *parents;
|
||||
packet_write(1, "unshallow %s",
|
||||
oid_to_hex(&object->oid));
|
||||
packet_write_fmt(1, "unshallow %s",
|
||||
oid_to_hex(&object->oid));
|
||||
object->flags &= ~CLIENT_SHALLOW;
|
||||
/*
|
||||
* We want to _register_ "object" as shallow, but we
|
||||
|
@ -932,7 +932,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
|
|||
struct strbuf symref_info = STRBUF_INIT;
|
||||
|
||||
format_symref_info(&symref_info, cb_data);
|
||||
packet_write(1, "%s %s%c%s%s%s%s%s agent=%s\n",
|
||||
packet_write_fmt(1, "%s %s%c%s%s%s%s%s agent=%s\n",
|
||||
oid_to_hex(oid), refname_nons,
|
||||
0, capabilities,
|
||||
(allow_unadvertised_object_request & ALLOW_TIP_SHA1) ?
|
||||
|
@ -944,11 +944,11 @@ static int send_ref(const char *refname, const struct object_id *oid,
|
|||
git_user_agent_sanitized());
|
||||
strbuf_release(&symref_info);
|
||||
} else {
|
||||
packet_write(1, "%s %s\n", oid_to_hex(oid), refname_nons);
|
||||
packet_write_fmt(1, "%s %s\n", oid_to_hex(oid), refname_nons);
|
||||
}
|
||||
capabilities = NULL;
|
||||
if (!peel_ref(refname, peeled.hash))
|
||||
packet_write(1, "%s %s^{}\n", oid_to_hex(&peeled), refname_nons);
|
||||
packet_write_fmt(1, "%s %s^{}\n", oid_to_hex(&peeled), refname_nons);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
|
@ -1,19 +1,6 @@
|
|||
#include "cache.h"
|
||||
#include "run-command.h"
|
||||
|
||||
static void check_pipe(int err)
|
||||
{
|
||||
if (err == EPIPE) {
|
||||
if (in_async())
|
||||
async_exit(141);
|
||||
|
||||
signal(SIGPIPE, SIG_DFL);
|
||||
raise(SIGPIPE);
|
||||
/* Should never happen, but just in case... */
|
||||
exit(141);
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* Some cases use stdio, but want to flush after the write
|
||||
* to get error handling (and to get better interactive
|
||||
|
|
Загрузка…
Ссылка в новой задаче