Bug 1713225 - Fix diffoscope error reporting after bug 1694785. r=firefox-build-system-reviewers,andi

Bug 1694785 upgraded diffoscope, and the new version had differences in
its output format that just made our error reporting miss everything. So
we fix the error reporting to work with the new version, as well as add
a test case to ensure that we don't upgrade diffoscope to a version that
would break out reporting again without noticing.

Differential Revision: https://phabricator.services.mozilla.com/D118137

taskcluster/docker/diffoscope/Dockerfile

@@ -27,5 +27,7 @@ RUN apt-get install \
 COPY get_and_diffoscope /builds/worker/bin/get_and_diffoscope
 COPY readelf /builds/worker/bin/readelf
 COPY report_error /builds/worker/bin/report_error
+COPY test_diffoscope /builds/worker/bin/test_diffoscope
 
-RUN chown -R worker:worker /builds/worker/bin && chmod 755 /builds/worker/bin/*
+RUN chown -R worker:worker /builds/worker/bin && chmod 755 /builds/worker/bin/* && \
+    /builds/worker/bin/test_diffoscope

taskcluster/docker/diffoscope/report_error

@@ -4,30 +4,29 @@ import sys
 stem = sys.argv[1]
 
 # We "parse" the diff output, so we look at the lines that contain a "tee", like:
-# ├── firefox
-# │ ├── libxul.so
+# ├── +++ b/firefox
+# │ ├── +++ b/firefox/libxul.so
 # │ │ ├── readelf --wide --notes {}
 # We ignore lines like the last one, to only report file names. And we ignore
-# lines for directories such as the first one, but still look at them to report
-# full paths.
-TEE = '├──'
+# lines for directories such as the first one.
+TEE = "├── "
+VERTICAL_LINE = "│"
 paths = set()
-path = []
 with open(f"{stem}.txt") as fh:
     for l in fh:
-        if TEE not in l:
+        l = l.rstrip()
+        before, tee, after = l.partition(TEE)
+        if not tee:
             continue
-        fields = l.split()
-        # We rely on the number of │ to figure out at what level the file
-        # name applies.
-        if fields[-2:-1] == [TEE]:
-            path[len(fields) - 2:] = [fields[-1]]
-        else:
-            # Align path length to match the number of │
-            path.append(None)
-        path_ = [p for p in path if p]
-        full_path = '/'.join(path_)
-        parent_path = '/'.join(path_[:-1])
+        before = before.split()
+        assert all(x == VERTICAL_LINE for x in before)
+        depth = len(before)
+        _, plus, after = after.partition("+++ ")
+        if not plus:
+            continue
+        _, b, full_path = after.partition("b/")
+        assert b == "b/"
+        parent_path = "/".join(full_path.split("/")[:-1])
         if parent_path in paths:
             paths.remove(parent_path)
         if full_path:

taskcluster/docker/diffoscope/test_diffoscope

@@ -0,0 +1,47 @@
+#!/bin/bash
+
+set -e
+
+WORKDIR=$(mktemp -d)
+cd $WORKDIR
+
+mkdir -p a/foo/bar/bar
+mkdir -p a/foo/bar/baz
+mkdir -p b/foo/bar/bar
+mkdir -p b/foo/bar/baz
+
+# A file that is modified
+echo qux > a/foo/bar/qux
+echo quz > b/foo/bar/qux
+
+# A binary file that is modified
+cp $(which ls) a/foo/bin
+cp $(which cat) b/foo/bin
+
+# A file that is removed
+echo hoge > a/foo/bar/bar/hoge
+
+# A file that is created
+echo fuga > b/foo/bar/baz/fuga
+
+# Also add a zip file with the same contents
+(cd a/foo; zip -r bar.zip bar)
+(cd b/foo; zip -r bar.zip bar)
+
+if TERM=linux diffoscope --no-progress --text diff.txt a b; then
+	echo "diffoscope didn't find differences?"
+	exit 1
+fi
+
+cat > expected.txt <<EOF
+TEST-UNEXPECTED-FAIL | foo/bar.zip differs. See the diff.html or diff.txt artifact
+TEST-UNEXPECTED-FAIL | foo/bar/bar differs. See the diff.html or diff.txt artifact
+TEST-UNEXPECTED-FAIL | foo/bar/baz differs. See the diff.html or diff.txt artifact
+TEST-UNEXPECTED-FAIL | foo/bar/qux differs. See the diff.html or diff.txt artifact
+TEST-UNEXPECTED-FAIL | foo/bin differs. See the diff.html or diff.txt artifact
+EOF
+
+$(dirname $0)/report_error diff | diff -u - expected.txt || exit 1
+
+cd $OLDPWD
+rm -rf $WORKDIR