When SMTP servers receive 8-bit email messages, possibly with only
LF as line ending, some of them decide to change said LF to CRLF.
Some mailing list softwares, when receive 8-bit email messages,
decide to encode those messages in base64 or quoted-printable.
If an email is transfered through above mail servers, then distributed
by such mailing list softwares, the recipients will receive an email
contains a patch mungled with CRLF encoded inside another encoding.
Thus, such CR (in CRLF) couldn't be dropped by "mailsplit".
Hence, the mailed patch couldn't be applied cleanly.
Such accidents have been observed in the wild [1].
Instead of silently rejecting those messages, let's give our users
some warnings if such CR (as part of CRLF) is found.
[1]: https://nmbug.notmuchmail.org/nmweb/show/m2lf9ejegj.fsf%40guru.guru-group.fi
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
An author name, that spelled a backslash-quoted double quote in the
human readable part "My \"double quoted\" name", was not unquoted
correctly while applying a patch from a piece of e-mail.
* kd/mailinfo-quoted-string:
mailinfo: unescape quoted-pair in header fields
t5100-mailinfo: replace common path prefix with variable
rfc2822 has provisions for quoted strings in structured header fields,
but also allows for escaping these with so-called quoted-pairs.
The only thing git currently does is removing exterior quotes, but
quotes within are left alone.
Remove exterior quotes and remove escape characters so that they don't
show up in the author field.
Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Mailinfo currently handles multi-line headers, but it does not handle
multi-line in-body headers. Teach it to handle such headers, for
example, for this input:
From: author <author@example.com>
Date: Fri, 9 Jun 2006 00:44:16 -0700
Subject: a very long
broken line
Subject: another very long
broken line
interpret the in-body subject to be "another very long broken line"
instead of "another very long".
An existing test (t/t5100/msg0015) has an indented line immediately
after an in-body header - it has been modified to reflect the new
functionality.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This will allow us to parse the output of --pretty=mboxrd
and the output of other mboxrd generators.
Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This option adds the content of the Message-Id header at the end of the
commit message prepared by git-mailinfo. This is useful in order to
associate commit messages automatically with mailing list discussions.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since commit 81c5cf7 (mailinfo: skip bogus UNIX From line inside
body, 2006-05-21), we have treated lines like ">From" in the body as
headers. This makes "git am" work for people who erroneously paste
the whole output from format-patch:
From 12345abcd...fedcba543210 Mon Sep 17 00:00:00 2001
From: them
Subject: [PATCH] whatever
into their email body (assuming that an mbox writer then quotes
"From" as ">From", as otherwise we would actually mailsplit on the
in-body line).
However, this has false positives if somebody actually has a commit
body that starts with "From "; in this case we erroneously remove
the line entirely from the commit message. We can make this check
more robust by making sure the line actually looks like a real mbox
"From" line.
Inspect the line that begins with ">From " a more carefully to only
skip lines that match the expected pattern (note that the datestamp
part of the format-patch output is designed to be kept constant to
help those who write magic(5) entries).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When "git am" is fed an input that has multiple "Content-type: ..."
header, it did not grok charset= attribute correctly.
* jc/maint-mailinfo-mime-attr:
mailinfo: do not concatenate charset= attribute values from mime headers
"Content-type: text/plain; charset=UTF-8" header should not appear
twice in the input, but it is always better to gracefully deal with
such a case. The current code concatenates the value to the values
we have seen previously, producing nonsense such as "utf8UTF-8".
Instead of concatenating, forget the previous value and use the last
value we see.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Number of columns required for change counts is now computed based on
the maximum number of changed lines instead of being fixed. This means
that usually a few more columns will be available for the filenames
and the graph.
The graph width logic is also modified to include enough space for
"Bin XXX -> YYY bytes".
If changes to binary files are mixed with changes to text files,
change counts are padded to take at least three columns. And the other
way around, if change counts require more than three columns, then
"Bin"s are padded to align with the change count. This way, the +-
part starts in the same column as "XXX -> YYY" part for binary files.
This makes the graph easier to parse visually thanks to the empty
column. This mimics the layout of diff --stat before this change.
Tests and the tutorial are updated to reflect the new --stat output.
This means either the removal of extra padding and/or the addition of
up to three extra characters to truncated filenames. One test is added
to check the graph alignment when a binary file change and text file
change of more than 999 lines are committed together.
Signed-off-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we are rebasing we know that the header lines in the
patch are good and that we don't need to pick up any headers
from the body of the patch.
This makes it possible to rebase commits whose commit message
start with "From" or "Date".
Test vectors by Jeff King.
Signed-off-by: Lukas Sandström <luksan@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This enables gitk to show the patch text with correct glyphs if the locale
is not UTF-8.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This mbox file must have been added by accident in e9fe804 (git-mailinfo:
Fix getting the subject from the in-body [PATCH] line, 2008-07-14).
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This teaches mailinfo the scissors -- >8 -- mark; the command ignores
everything before it in the message body.
For lefties among us, we also support -- 8< -- ;-)
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 650d30d8a1.
Some mailing lists are configured add prefix "[listname] " to all their
messages, and also people hand-edit subject lines, be it an output from
format-patch or a patch generated by some other means.
We cannot stop people from mucking with the subject line, and with the
change, there always will be need for hand editing the subject when that
happens. People have depended on the leading [bracketed string] removal.
git-format-patch prepends patches with a [PATCH x/n] prefix, but
mailinfo used to remove any number of square-bracket pairs and
the content between them. This prevents one from using a commit
subject like this:
[ and ] must be allowed as input
Removing the square bracket pair from this rather clumsily
constructed subject line loses important information, so we must
take care not to.
This patch causes the subject stripping to stop after it has
encountered one pair of square brackets.
One possible downside of this patch is that the patch-handling
programs will now fail at removing author-added square-brackets
to be removed, such as
[RFC][PATCH x/n]
However, since format-patch only adds one set of square brackets,
this behaviour is quite easily undesrstood and defended while the
previous behaviour is not.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some ancient platforms do not have an extensive list of alternate names for
character encodings. For example, Solaris 7 does not know that ISO-8859-1
is the same as ISO8859-1. Modern platforms do know this, so use the older
names.
The following conversions were performed:
ISO-8859-1 --> ISO8859-1
ISO-8859-2 --> ISO8859-2
ISO-8859-8 --> ISO8859-8
iso-2022-jp --> ISO-2022-JP
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
currently for cases like
From: A U Thor <a.u.thor@example.com> (Comment)
mailinfo extracts the following 'Author:' field:
Author: A U Thor (Comment)
^^
which has two extra spaces left in there after removed email part.
I think this is wrong so here is a fix.
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Also as suggested by Junio, in order to try to catch other MIME
problems, test cases from the "8. Examples" section of RFC2047 are added
to t5100 testsuite as well.
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
At present we do headers unfolding (see RFC822 3.1.1. LONG HEADER FIELDS) for
all fields except 'From' (always) and 'Subject' (when keep_subject is set)
Not unfolding 'From' is a bug -- see above-mentioned RFC link.
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When native language (RU) is in use, subject header usually contains several
parts, e.g.
Subject: [Navy-patches] [PATCH]
=?utf-8?b?0JjQt9C80LXQvdGR0L0g0YHQv9C40YHQvtC6INC/0LA=?=
=?utf-8?b?0LrQtdGC0L7QsiDQvdC10L7QsdGF0L7QtNC40LzRi9GFINC00LvRjyA=?=
=?utf-8?b?0YHQsdC+0YDQutC4?=
This exposes several bugs in builtin-mailinfo.c:
1. decode_b_segment: do not append explicit NUL -- explicit NUL was preventing
correct header construction on parts concatenation via strbuf_addbuf in
decode_header_bq. Fixes:
-Subject: Изменён список пакетов необходимых для сборки
+Subject: Изменён список па
Then
2. Do not emit '\n' between "encoded-word" where RFC2046 says that linear
white space between them are ignored when displaying. Fixes:
-Subject: Изменён список пакетов необходимых для сборки
+Subject: Изменён список па кетов необходимых для сборки
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In handle_from, we calculate the end boundary of a section
to remove from a strbuf using strcspn like this:
el = strcspn(buf, set_of_end_boundaries);
strbuf_remove(&sb, start, el + 1);
This works fine if "el" is the offset of the boundary
character, meaning we remove up to and including that
character. But if the end boundary didn't match (that is, we
hit the end of the string as the boundary instead) then we
want just "el". Asking for "el+1" caught an out-of-bounds
assertion in the strbuf library.
This manifested itself when we got a 'From' header that had
just an email address with nothing else in it (the end of
the string was the end of the address, rather than, e.g., a
trailing '>' character), causing git-mailinfo to barf.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Recent changes to is_multipart_boundary() caused git-mailinfo to segfault.
The reason was after handling the end of the boundary the code tried to look
for another boundary. Because the boundary list was empty, dereferencing
the pointer to the top of the boundary caused the program to go boom.
The fix is to check to see if the list is empty and if so go on its merry
way instead of looking for another boundary.
I also fixed a couple of increments and decrements that didn't look correct
relating to content_top.
The boundary test case was updated to catch future problems like this again.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After finding a MIME multi-part message boundary line, the handle_body()
function is supposed to first flush any accumulated contents from the
previous part to the output stream. However, the code mistakenly output
the boundary line it found.
The old code that used one global, fixed-length buffer line[] used an
alternate static buffer newline[] for keeping track of this accumulated
contents and flushed newline[] upon seeing the boundary; when 3b6121f
(git-mailinfo: use strbuf's instead of fixed buffers, 2008-07-13)
converted a fixed-length buffer in this program to use strbuf,these two
buffers were converted to "line" and "prev" (the latter of which now has a
much more sensible name) strbufs, but the code mistakenly flushed "line"
(which contains the boundary we have just found), instead of "prev".
This resulted in the first boundary to be output in front of the first
line of the message.
The rewritten implementation of handle_boundary() lost the terminating
newline; this would then result in the second line of the message to be
stuck with the first line.
The is_multipart_boundary() was designed to catch both the internal
boundary and the terminating one (the one with trailing "--"); this also
was broken with the rewrite, and the code in the handle_boundary() to
handle the terminating boundary was never triggered.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"Subject: " isn't in the static array "header", and thus
memcmp("Subject:", header[i], 7) will never match.
Even if it did so, hdr_data[] may not have been allocated if there weren't
a "Subject: " in-body when we process "[PATCH]" in the affected codepath.
Signed-off-by: Lukas Sandström <lukass@etek.chalmers.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are broken filesystems that cannot have a file whose name is "nul"
anywhere on it. Rename the test file to make ourselves more portable.
Noticed by Mark Levedahl.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function fgets() has a big problem with NUL characters: it reads
them, but nobody will know if the NUL comes from the file stream, or
was appended at the end of the line.
So implement a custom read_line_with_nul() function.
Noticed by Tommy Thorn.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function is intended to be fed one logical line at a time to
inspect, but a QP encoded raw input line can have more than one
lines, just like BASE64 encoded one.
Quoting LF as =0A may be unusual but RFC2045 allows it.
The issue was noticed and fixed by Jay Soffian. JC added a test
to protect the fix from regressing later.
Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Don't translate the patch to UTF-8, instead preserve the data as
is. This also reverts a test case that was included in the
original patch series.
Also allow overwriting the authorship and title information we
gather from RFC2822 mail headers with additional in-body
headers, which was pointed out by Linus.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I am working on a project that required parsing through regular
mboxes that didn't necessarily have patches embedded in them. I
started by creating my own modified copy of git-am and working
from there. Very quickly, I noticed git-mailinfo wasn't able to
handle a big chunk of my email.
After hacking up numerous solutions and running into more
limitations, I decided it was just easier to rewrite a big chunk
of it. The following patch has a bunch of fixes and features
that I needed in order for me do what I wanted.
Note: I'm didn't follow any email rfc papers but I don't think
any of the changes I did required much knowledge (besides the
boundary stuff).
List of major changes/fixes:
- can't create empty patch files fix
- empty patch files don't fail, this failure will come inside git-am
- multipart boundaries are now handled
- only output inbody headers if a patch exists otherwise assume those
headers are part of the reply and instead output the original headers
- decode and filter base64 patches correctly
- various other accidental fixes
I believe I didn't break any existing functionality or
compatibility (other than what I describe above, which is really
only the empty patch file).
I tested this through various mailing list archives and
everything seemed to parse correctly (a couple thousand emails).
[jc: squashed in another patch from Don's five patch series to
fix the test case, as this patch exposes the bug in the test.]
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
It basically considers all the continuation lines to be lines of their
own, and if the total line is bigger than what we can fit in it, we just
truncate the result rather than stop in the middle and then get confused
when we try to parse the "next" line (which is just the remainder of the
first line).
[jc: added test, and tightened boundary a bit per list discussion.]
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Currently the test passes with 1.3.3 but not with the tip of
"master". This is to verify the fixes from Eric W Biedermann.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Đoàn Trần Công Danh
Junio C Hamano
Kevin Daudt
Jonathan Tan
Eric Wong
Paolo Bonzini
Jeff King
Zbigniew Jędrzejewski-Szmek
Lukas Sandström
Johannes Sixt
Andreas Ericsson
Brandon Casey
Kirill Smelkov
Kirill Smelkov
Don Zickus
Lukas Sandström
Johannes Schindelin
Jay Soffian
Simon Sasburg
Linus Torvalds
Junio C Hamano