This commit is contained in:
Eli Barzilay 2017-03-28 01:43:33 -04:00
Родитель 810ea77fe3
Коммит f5fc99be98
18 изменённых файлов: 1110 добавлений и 0 удалений

5
.gitignore поставляемый Normal file
Просмотреть файл

@ -0,0 +1,5 @@
*~
\#*
.#*
.DS_Store
*.bak

3
CONTRIBUTING.md Normal file
Просмотреть файл

@ -0,0 +1,3 @@
# Contributing
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

22
LICENSE Normal file
Просмотреть файл

@ -0,0 +1,22 @@
MIT License
Copyright (c) Microsoft Corporation. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

149
README.md Normal file
Просмотреть файл

@ -0,0 +1,149 @@
# `docker-buildx`
### Usage:
docker-buildx [options...] <path>
`docker-buildx` is a tool to run a docker build in multiple steps, based on
annotations in the dockerfile, allowing you to selectively squash
parts of the build and more. This is done by chopping the dockerfile
into multiple files based on “meta-annotations”, and each fragment
is built with a tag that gets used by the following fragment's `FROM`
line (which gets added to all fragments except the first).
The following annotations are recognized:
* `#BUILD# [options...]` \
Run a build from the last `#BUILD#` (or the beginning of the file) to
this point, using the given options. The specified options (if any)
are used for this run, combined with options set on the command-line
(see below). Note that `--tag`, `--file`, and `<path>` are also added to
orchestrate the build, specifying the tag to use by the next step,
the created dockerfile, and the path.
* `#SQUASH#` \
This is a convenient shorthand for `#BUILD# --squash`, the main
use-case of `docker-buildx`. Remember that the build starts from the last
`#BUILD#` (or `#SQUASH#`). Squashing can be used to resolve
docker's inability to create files belonging to a non-root user, or
eliminate files holding temporary secrets -- with docker-buildx you can do
the following:
#BUILD#
USER root:root
ADD ["stuff", "secret", "some/where"]
RUN chown -R user:user some/where
USER user:user
RUN do-something-using some/where/secret
RUN rm some/where/secret
#SQUASH#
Note, however, that squashing preserves layer information, only the
contents is combined into a single layer. Specifically, the
descriptions of all steps (e.g., as seen by `docker history`) is
retained. You should therefore still avoid commands that include
explicit secrets. For example, copy file containing a secret as in
the above example, rather than some `RUN something p455w0rd`.
* `#INCLUDE# <file/glob>` \
Include the specified file(s) at this point. The arguments are as
in bash: you can include multiple files with a wildcard, use
variables, etc. Use quotes or a backslash for spaces. Paths are
taken as relative to the including file; includes can be nested.
* `#META# command...` \
Run the specified command(s), and include the resulting output in
the dockerfile. The output must contain plain dockerfile code. For
example, you can include a fragment with a `#META# cat some-file`
(this will be simple inclusion, no meta annotations so no nested
includes). The META code gets evaluated in a context that has some
environment variables set:
- `$DOCKERDIR`: the directory we're working in (the `<path>` argument)
- `$DOCKERFILE`: the original dockerfile name (`-f`)
- `$BUILDXFILE`: the temp fragment dockerfile name (`-F`)
- `$BUILDXTAG`: the temp tagname referring to the last build (`-T`)
- `DOCKERBUILD`: a function that works like `docker build` but is
displayed during the generation process
These can be useful when composing dockerfile code. For example,
say that you install some package that extends `.bashrc` with some
environment variables which you want to add to the dockerfile
(`.bashrc` is used in interactive bash runs only) -- you can add a
`#BUILD#` step after the installation, then add:
#META# R() { docker run --rm -h foo $BUILDXTAG bash -c "$*"; }
#META# comm -13 <(R "env"|sort) <(R ". .bashrc; env"|sort) |\
sed -e 's/^\([^=]*\)=/ENV \1 /'
* `#METAMETA# command...` \
`docker-buildx` works by generating bash code and running the result,
where META lines are commands that are inserted in the generated
code as fragments are running, and must produce plain dockerfile
code. METAMETA lines are similar to META lines, except that they
are evaluated when the code is generated (at “compile-time”).
They cannot be used to examine the built image since there is none,
but whatever they output is re-processed by `docker-buildx`, so they can
produce annotated `docker-buildx` code (e.g., implement a proper
“include”). They have access to the same variables, but note that
the `$BUILDX` variables refer to a file and a tag that do not exist,
yet.
The parsing of meta-annotations respects line-continuations: they're
ignored when following a backslashed-line, and they can contain
multiple lines if the annotated line ends with a backslash. Only the
annotations listed above are recognized (matched in any case), others
are left untouched (i.e., as comments) but this might change to throw
an error in the future.
In addition to a few general docker-build-like options that are
consumed by `docker-buildx` itself, you can specify additional flags that
are added to various build steps. These options are specified by meta
flags that look like *`--when:`* (see below for the actual names).
Options that follow such a flag are all collected for the specified
step(s), up to the next meta flag or up to a meta-closing flag of
`:--`. The collected options are added in the order of appearance
on the command line. See below for a list of these. Note: no
checking of arguments are done, neither in the meta-flags nor in
`#BUILD#` lines, they can even contain `;` and other such
characters.
### Docker-like Basic Options:
* `-h`, `--help`:
get more help
* `-f`, `--file <file>`:
dockerfile name (default: `<path>/Dockerfile`)
* `-t`, `--tag <tag>`:
shorthand for `--tag` in the `--last:` section
### Additional Options:
* `-F`, `--buildx-file <file>`:
temp dockerfile (default: `<dockerfile>x`) \
This file is created with dockerfile fragments
for each build step, the default is the same as
the docker file with an appended `x`.
* `-T`, `--buildx-tag <tag>`:
temp tag used in intermediate builds \
This tag is deleted at the end of the build,
defaults to `0buildx-temp`.
* `-X`, `--x-force`:
ignore existing buildx-file or buildx-tag
* `-S`, `--script`:
dump the script that does the actual work \
You can use this flag to save the code to
run yourself later, or to debug it.
### Meta-options for:
* `--all:`
all builds
* `--first:`
first build
* `--last:`
last build (note: a single build step is considered last)
* `--middle:`
non-first-or-last builds
* `:--`
back to docker-buildx options

361
docker-buildx Executable file
Просмотреть файл

@ -0,0 +1,361 @@
#!/usr/bin/env bash
NAME="$(basename $0)"
DIR="." FILE="" # dockerfile path & file
FILEx="" TAGx="" # dockerfile for fragments, tag for intermediate steps
SCRIPT_MODE=0
FORCE_MODE=0
fail() { # [-h] message...
local help=0; if [[ "x$1" == "x-h" ]]; then help=1; shift; fi
echo "$NAME: $*" 1>&2
if ((help)); then echo ""; usage; fi
exit 1
}
echos() { # [-pfx str] line...
local pfx=""
if [[ "x$1" == "x-pfx" ]]; then pfx="$2"; shift 2; fi
for line; do printf "%s%s\n" "${line:+$pfx}" "$line"; done
}
trim() { # [-l|-r] str
local mode="x"; if [[ "x$1" == "x-"[lr] ]]; then mode="${1:1}"; shift; fi
str="$1"
if [[ "$mode" != "r" ]]; then str="${str#"${str%%[![:space:]]*}"}"; fi
if [[ "$mode" != "l" ]]; then str="${str%"${str##[![:space:]]*}"}"; fi
printf "%s" "$str"
}
qstr() { # str...; quotes the input as a shell string, using $HOME
local replace='\ " ` $' str="$*" ch
for ch in $replace; do str="${str//"$ch"/\\$ch}"; done
echo "\"${str//$HOME/\$HOME}\""
}
is_cont() { # str
if [[ "$1" =~ ([^\\]|^)(\\\\)*\\$ ]]; then return 0; else return 1; fi
}
usage() { # show_all
local all=0; if [[ "$1" == [Yy]* ]]; then all=1; fi; shift
echo "Usage: $NAME [options...] <path>"
if ((all)); then
echos -pfx " " "" \
"$NAME is a tool to run a docker build in multiple steps, based on" \
"annotations in the dockerfile, allowing you to selectively squash" \
"parts of the build and more. This is done by chopping the dockerfile" \
"into multiple files based on \"meta-annotations\", and each fragment" \
"is built with a tag that gets used by the following fragment's FROM" \
"line (which gets added to all fragments except the first)." "" \
"The following annotations are recognized:" "" \
"* #BUILD# [options...]" \
" Run a build from the last #BUILD# (or the beginning of the file) to" \
" this point, using the given options. The specified options (if any)" \
" are used for this run, combined with options set on the command-line" \
" (see below). Note that --tag, --file, and <path> are also added to" \
" orchestrate the build, specifying the tag to use by the next step," \
" the created dockerfile, and the path." "" \
"* #SQUASH#" \
" This is a convenient shorthand for \"#BUILD# --squash\", the main" \
" use-case of $NAME. Remember that the build starts from the last" \
" \"#BUILD#\" (or \"#SQUASH#\"). Squashing can be used to resolve" \
" docker's inability to create files belonging to a non-root user, or" \
" eliminate files holding temporary secrets -- with $NAME you can do" \
" the following:" "" \
" #BUILD#" \
" USER root:root" \
" ADD [\"stuff\", \"secret\", \"some/where\"]" \
" RUN chown -R user:user some/where" \
" USER user:user" \
" RUN do-something-using some/where/secret" \
" RUN rm some/where/secret" \
" #SQUASH#" "" \
" Note, however, that squashing preserves layer information, only the" \
" contents is combined into a single layer. Specifically, the" \
" descriptions of all steps (e.g., as seen by \"docker history\") is" \
" retained. You should therefore still avoid commands that include" \
" explicit secrets. For example, copy file containing a secret as in" \
" the above example, rather than some \"RUN something p455w0rd\"." "" \
"* #INCLUDE# <file/glob>" \
" Include the specified file(s) at this point. The arguments are as" \
" in bash: you can include multiple files with a wildcard, use" \
" variables, etc. Use quotes or a backslash for spaces. Paths are" \
" taken as relative to the including file; includes can be nested." "" \
"* #META# command..." \
" Run the specified command(s), and include the resulting output in" \
" the dockerfile. The output must contain plain dockerfile code. For" \
" example, you can include a fragment with a \"#META# cat some-file\"" \
" (this will be simple inclusion, no meta annotations so no nested" \
" includes). The META code gets evaluated in a context that has some" \
" environment variables set:" \
" - \$DOCKERDIR: the directory we're working in (the <path> argument)" \
" - \$DOCKERFILE: the original dockerfile name (-f)" \
" - \$BUILDXFILE: the temp fragment dockerfile name (-F)" \
" - \$BUILDXTAG: the temp tagname referring to the last build (-T)" \
" - DOCKERBUILD: a function that works like \"docker build\" but is" \
" displayed during the generation process" \
" These can be useful when composing dockerfile code. For example," \
" say that you install some package that extends \".bashrc\" with some" \
" environment variables which you want to add to the dockerfile" \
" (.bashrc is used in interactive bash runs only) -- you can add a" \
" #BUILD# step after the installation, then add:" "" \
" #META# R() { docker run --rm -h foo \$BUILDXTAG bash -c \"\$*\"; }" \
" #META# comm -13 <(R \"env\"|sort) <(R \". .bashrc; env\"|sort) |\\" \
" sed -e 's/^\([^=]*\)=/ENV \1 /'" "" \
"* #METAMETA# command..." \
" $NAME works by generating bash code and running the result," \
" where META lines are commands that are inserted in the generated" \
" code as fragments are running, and must produce plain dockerfile" \
" code. METAMETA lines are similar to META lines, except that they" \
" are evaluated when the code is generated (at \"compile-time\")." \
" They cannot be used to examine the built image since there is none," \
" but whatever they output is re-processed by $NAME, so they can" \
" produce annotated $NAME code (e.g., implement a proper" \
" \"include\"). They have access to the same variables, but note that" \
" the \$BUILDX variables refer to a file and a tag that do not exist," \
" yet." "" \
"The parsing of meta-annotations respects line-continuations: they're" \
"ignored when following a backslashed-line, and they can contain" \
"multiple lines if the annotated line ends with a backslash. Only the" \
"annotations listed above are recognized (matched in any case), others" \
"are left untouched (i.e., as comments) but this might change to throw" \
"an error in the future." \
"" \
"In addition to a few general docker-build-like options that are" \
"consumed by $NAME itself, you can specify additional flags that" \
"are added to various build steps. These options are specified by meta" \
"flags that look like \"--when:\" (see below for the actual names)." \
"Options that follow such a flag are all collected for the specified" \
"step(s), up to the next meta flag or up to a meta-closing flag of" \
"\":--\". The collected options are added in the order of appearance" \
"on the command line. See below for a list of these. Note: no" \
"checking of arguments are done, neither in the meta-flags nor in" \
"\"#BUILD#\" lines, they can even contain \";\" and other such" \
"characters." ""
fi
echo "Docker-like Basic Options:"
echo " -h, --help get more help"
echo " -f, --file <file> dockerfile name (default: \"<path>/Dockerfile\")"
echo " -t, --tag <tag> shorthand for --tag in the --last: section"
echo "Additional Options:"
echo " -F, --buildx-file <file>"
echo " temp dockerfile (default: \"<dockerfile>x\")"
if ((all)); then echos -pfx " " \
"This file is created with dockerfile fragments" \
"for each build step, the default is the same as" \
"the docker file with an appended \"x\"."
fi
echo " -T, --buildx-tag <tag>"
echo " temp tag used in intermediate builds"
if ((all)); then echos -pfx " " \
"This tag is deleted at the end of the build," \
"defaults to \"0buildx-temp\"."
fi
echo " -X, --x-force ignore existing buildx-file or buildx-tag"
echo " -S, --script dump the script that does the actual work"
if ((all)); then echos -pfx " " \
"You can use this flag to save the code to" \
"run yourself later, or to debug it."
fi
echo -n "Meta-options for"
if ((all)); then echo ":"; else echo " (--help for more info):"; fi
echo " --all: all builds"
echo " --first: first build"
echo " --last: last build (note: a single build step is considered last)"
echo " --middle: non-first-or-last builds"
echo " :-- back to $NAME options"
if ((!all)); then echo "Use --help for full details."; fi
exit 0
}
meta_args=() rest_args=()
parse_args() { # options...
local mode="" a="" next=""
while [[ "$#" -gt 0 || "$next" != "" ]]; do
if [[ "$next" == "" ]]; then a="$1"; shift; else a="$next"; next=""; fi
case "$a" in
( ":--" ) mode="" ;;
( "--all:" ) mode="111" ;;
( "--first:" ) mode="100" ;;
( "--last:" ) mode="001" ;;
( "--middle:" ) mode="010" ;;
( "--"*":" ) fail -h "unknown meta-flag \"$a\"" ;;
( * ) if [[ "$mode" != "" ]]; then meta_args+=("$mode:$a"); else
case "$a" in
( "--help" | "-h"* ) usage y ;;
( "--file" | "-f" ) FILE="$1"; shift ;;
( "--file="* ) FILE="${a#*=}" ;;
( "-f"?* ) FILE="${a:2}" ;;
( "--buildx-file" | "-F" ) FILEx="$1"; shift ;;
( "--buildx-file="* ) FILEx="${a#*=}" ;;
( "-F"?* ) FILEx="${a:2}" ;;
( "--tag" | "-t" ) meta_args+=("001:--tag" "001:$1"); shift ;;
( "--tag="* ) meta_args+=("001:--tag" "001:${a#*=}") ;;
( "-t"?* ) meta_args+=("001:--tag" "001:${a:2}") ;;
( "--buildx-tag" | "-T" ) TAGx="$1"; shift ;;
( "--buildx-tag="* ) TAGx="${a#*=}" ;;
( "-T"?* ) TAGx="${a:2}" ;;
( "--x-force" | "-X" ) FORCE_MODE=1 ;;
( "--x-force="* ) fail -h "--x-force takes no arguments" ;;
( "-X"?* ) FORCE_MODE=1; next="-${a:2}" ;;
( "--script" | "-S" ) SCRIPT_MODE=1 ;;
( "--script="* ) fail -h "--script takes no arguments" ;;
( "-S"?* ) SCRIPT_MODE=1; next="-${a:2}" ;;
( "--"* ) fail -h "unknown option: $a" ;;
( "-"* ) fail -h "unknown option: ${a:0:2}" ;;
( * ) rest_args+=("$a") ;;
esac; fi ;;
esac
done
}
frag_num=0
args_br=$' \\\n '
meta_re="^#(BUILD|SQUASH|INCLUDE|META(META)?)# *(.*)\$"
dirv="DOCKERDIR" qdirv="\"\$$dirv\""
filev="DOCKERFILE" qfilev="\"\$$filev\""
xfilev="BUILDXFILE" qxfilev="\"\$$xfilev\""
xtagv="BUILDXTAG" qxtagv="\"\$$xtagv\""
preamble() { # runtime (0 => called for a METAMETA block)
local runtime=$1; shift 2
local exports="export"
exports+=" $dirv=$(qstr "$DIR")"
exports+=" $filev=$(qstr "$FILE")"
exports+=" $xfilev=$(qstr "$FILEx")"
exports+=" $xtagv=$(qstr "$TAGx")"
if ((!runtime)); then echo "$exports"; return; fi
echos "#!/usr/bin/env bash" \
"" \
"$exports" \
"DOCKERBUILD() {" \
" echo \"BUILDX---> docker build \$*\"" \
" docker build \"\$@\" || exit \$?; }" \
"" \
"if [[ ! \"\$(docker version)\" =~ .*Experimental:\\ *true.* ]]; then" \
" echo \"Warning: docker experimental mode is off\" 1>&2; fi"
if ((!FORCE_MODE)); then
echos "if [[ \"\$(docker images -q $qxtagv)\" != \"\" ]]; then" \
" echo \"Error: temp tag exists: \$$xtagv\" 1>&2; exit 1; fi"
fi
}
fragment_start() { # no arguments
echo ""
if ((frag_num)); then printf "\necho \"\"\n"; fi
echo "echo \"################ >>> building fragment #$((++frag_num))" \
"<<< ################\""
echo "echo -n > $qxfilev"
if ((frag_num > 1)); then echo "echo \"FROM \$$xtagv\" >> $qxfilev"; fi
}
fragment_end() { # posn(0/1/2) args_str
local posn="$1" args_str="$2" arg has_args=0; shift 2
printf "\nDOCKERBUILD"
for arg in "${meta_args[@]}"; do
if [[ "${arg:$posn:1}" != "1" ]]; then continue; fi
if ((!has_args)); then has_args=1; printf "$args_br"; else printf " "; fi
printf "%q" "${arg:4}"
done
if [[ "x$args_str" != "x" ]]; then printf "%s%s" "$args_br" "$args_str"; fi
printf "\nrm -f $qxfilev\n"
}
add_chunk() { # chunk type
local chunk="$1" type="$2" marker="EOF"; shift 2
if [[ "x$chunk" == "x" ]]; then return; fi
if [[ "$type" == "META" ]]; then
echo "{"
printf "%s" " $chunk"
echo "} >> $qxfilev || exit \$?"
else
while [[ "$chunk" == *"$marker"* ]]; do marker="EOF_$RANDOM"; done
echo "cat >> $qxfilev <<\"$marker\""
printf "%s" "$chunk"
echo "$marker"
fi
}
generate_script() { # no arguments, only globals
local line more ltype chunk="" ctype="-" posn=0 oIFS="$IFS"
local ocase="$(shopt -p nocasematch)"
preamble 1
IFS=""; shopt -s nocasematch
fragment_start
local fd inpstack=()
exec {fd}< "$FILE"
while ((1)); do
if read -u $fd -r line; then :
elif [[ ${#inpstack[@]} == 0 ]]; then break
else exec {fd}<&-; fd=${inpstack[-1]}; FILE="${inpstack[-2]}"
unset inpstack[-1]; unset inpstack[-1]; continue
fi
more="$line"
while is_cont "$more" && read -u $fd -r more; do line+=$'\n'"$more"; done
if [[ ! "$line" =~ $meta_re ]]; then ltype="-"; more="$line"
else ltype="${BASH_REMATCH[1]}"; more="${BASH_REMATCH[3]}"
fi
case "$ltype" in
( "BUILD" | "SQUASH" )
add_chunk "$chunk" "$ctype"; chunk=""; ctype="-"
more="$(trim -r "$more")"
if [[ "$ltype" == "SQUASH" ]]; then more="--squash${more:+ }$more"; fi
more+="${more:+$args_br}--tag $qxtagv --file $qxfilev $qdirv"
fragment_end $posn "$more"; posn=1
fragment_start
;;
( "INCLUDE" )
local files=() here="$(dirname "$FILE")" i
local oldopts="$(shopt -p nocasematch globstar extglob nullglob)"
shopt -u nocasematch; shopt -s nullglob globstar extglob
eval "files=( $more ); $oldopts"
if [[ ${#files[@]} == 0 ]]; then continue; fi
for ((i = 0; i < ${#files[@]}; i++)); do
files[i]="$(realpath -m --relative-to "$PWD" "$here/${files[i]}")";
if [[ ! -r "${files[i]}" ]]; then
echo "Warning: cannot read include file: ${files[i]}" 1>&2
unset files[i]
fi
done
if [[ ${#files[@]} == 0 ]]; then continue; fi
for ((i = ${#files[@]}-1; i >= 0; i--)); do
if [[ "x${files[i]}" == "x" ]]; then continue; fi
inpstack+=("$FILE" $fd)
FILE="${files[i]}"; exec {fd}< "${files[i]}"
done
;;
( "METAMETA" )
inpstack+=("$FILE" $fd)
exec {fd}<<<"$(eval "$(preamble 0); $more")"
;;
( "$ctype" )
if [[ "$ltype" == "META" ]]; then chunk+=" "; fi
chunk+="$more"$'\n'
;;
( * )
add_chunk "$chunk" "$ctype"; chunk="$more"$'\n'; ctype="$ltype"
esac
done
add_chunk "$chunk" "$ctype"
fragment_end 2 "--file $qxfilev $qdirv"
if ((posn > 0)); then printf "\ndocker rmi %s\n" "$qxtagv"; fi
IFS="$oIFS"; eval "$ocase"
}
main() { # path
if [[ "x$1" == "x" ]]; then fail -h "missing <path> argument"; fi
DIR="$1"; shift
if [[ "$#" -gt 0 ]]; then fail -h "too many arguments: $*"
elif [[ ! -d "$DIR" ]]; then fail "directory does not exist: $DIR"
fi
if [[ "x$FILE" == "x" ]]; then FILE="$DIR/Dockerfile"; FILE="${FILE#./}"; fi
if [[ "x$FILEx" == "x" ]]; then FILEx="${FILE}x"; fi
if [[ "x$TAGx" == "x" ]]; then TAGx="0buildx-temp"; fi
if [[ ! -f "$FILE" ]]; then fail "dockerfile not found: $FILE"; fi
if ((!FORCE_MODE)) && [[ -f "$FILEx" ]]; then
fail "generated dockerfile exists: $FILEx"
fi
if ((SCRIPT_MODE)); then generate_script; else eval "$(generate_script)"; fi
}
parse_args "$@"
main "${rest_args[@]}"

2
tests/include/t08-i1 Normal file
Просмотреть файл

@ -0,0 +1,2 @@
include1 line1
include1 line2

3
tests/include/t08-i2 Normal file
Просмотреть файл

@ -0,0 +1,3 @@
include2 line1
#INCLUDE# t08-i1
include2 line2

50
tests/run-tests Executable file
Просмотреть файл

@ -0,0 +1,50 @@
#!/usr/bin/env bash
cd "$(dirname $0)"
verbose=0 vbuf=""; if [[ "$1" == "-v" ]]; then verbose=2; shift; fi
vecho() {
if [[ "$1" == "+" ]]; then
shift; if ((!verbose)); then verbose=1; echo -n "$vbuf"; fi
fi
if ((verbose)); then echo "$@";
else local x="$(echo "$@"; printf "x")"; vbuf+="${x%x}"; fi
}
rxquote() { # str
str="$1"; shift
for ch in \\ \. \* \? \+ \^ \$ \( \) \| \[ \] \{ \}; do
str="${str//"$ch"/\\$ch}"
done
echo "${str//\\\[\\.\\.\\.\\\]/.*}"
}
show_part() { # file pfx
local file="$1" pfx="$2"; shift 2
grep "^$pfx" "$file" | sed -e "s/^$pfx//"
}
ntests=0 nfails=0
if [[ "$#" == "0" ]]; then tests=(t[0-9][0-9]); else tests=("$@"); fi
for t in "${tests[@]}"; do
vecho -n "Test $t..."
expect="$(show_part "$t" ">")"
expectrx="^$(rxquote "$expect")\$"
args="$(show_part "$t" ";")"
if [[ "$args" == *$'\n'* ]]; then
vecho + " bad test: multiple args line" 1>&2; exit 1;
fi
((ntests++))
show_part "$t" "<" > "Dockerfile"
script="$(../docker-buildx --script . $args 2>&1)"
rm -f "Dockerfile"
if [[ "$script" =~ $expectrx ]]; then vecho " pass"; continue; fi
vecho + " fail"; ((nfails++))
if ((verbose > 1)); then
vecho "Expected:"; vecho " | ${expect//$'\n'/$'\n | '}"
vecho "Actual:"; vecho " | ${script//$'\n'/$'\n | '}"
fi
done
if ((!nfails)); then echo "$ntests tests passed"
else echo "$nfails/$ntests tests failed"; exit 1; fi

53
tests/t01 Normal file
Просмотреть файл

@ -0,0 +1,53 @@
Plain use.
<FROM ubuntu
<
<ENV USER=foo HOME=/home/foo
<
<RUN useradd -c Blah -U -G root -d "$HOME" -m "$USER"
<
<WORKDIR "$HOME"
<
<RUN echo xxxxxxxxxxxxxxx > x
<RUN x() { cat x x x x x x x x x x x x x x x x > xx; mv xx x; ls -lh x; } \
< && x && x && x && x && x
<
<RUN chown $USER:$USER x
<
<USER "$USER:$USER"
>#!/usr/bin/env bash
>
>export DOCKERDIR="." DOCKERFILE="Dockerfile" BUILDXFILE="Dockerfilex" BUILDXTAG="0buildx-temp"
>DOCKERBUILD() {
> echo "BUILDX---> docker build $*"
> docker build "$@" || exit $?; }
>
>if [[ ! "$(docker version)" =~ .*Experimental:\ *true.* ]]; then
> echo "Warning: docker experimental mode is off" 1>&2; fi
>if [[ "$(docker images -q "$BUILDXTAG")" != "" ]]; then
> echo "Error: temp tag exists: $BUILDXTAG" 1>&2; exit 1; fi
>
>echo "################ >>> building fragment #1 <<< ################"
>echo -n > "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>FROM ubuntu
>
>ENV USER=foo HOME=/home/foo
>
>RUN useradd -c Blah -U -G root -d "$HOME" -m "$USER"
>
>WORKDIR "$HOME"
>
>RUN echo xxxxxxxxxxxxxxx > x
>RUN x() { cat x x x x x x x x x x x x x x x x > xx; mv xx x; ls -lh x; } \
> && x && x && x && x && x
>
>RUN chown $USER:$USER x
>
>USER "$USER:$USER"
>EOF
>
>DOCKERBUILD \
> --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"

9
tests/t02 Normal file
Просмотреть файл

@ -0,0 +1,9 @@
Help flag & text.
;-Sh
<blah
>Usage: docker-buildx [options...] <path>
>
> docker-buildx is a tool [...]

16
tests/t03 Normal file
Просмотреть файл

@ -0,0 +1,16 @@
Bad flag & short usage text.
;-Qh
<blah
>docker-buildx: unknown option: -Q
>
>Usage: docker-buildx [options...] <path>
>Docker-like Basic Options:
>[...]
>Additional Options:
>[...]
>Meta-options for (--help for more info):
>[...]
>Use --help for full details.

71
tests/t04 Normal file
Просмотреть файл

@ -0,0 +1,71 @@
Buildx flags.
;-XF SomeOtherFile -T SomeOtherTag
<FROM ubuntu
<
<ENV USER=foo HOME=/home/foo
<
<RUN useradd -c Blah -U -G root -d "$HOME" -m "$USER"
<
<WORKDIR "$HOME"
<
<RUN echo xxxxxxxxxxxxxxx > x
<RUN x() { cat x x x x x x x x x x x x x x x x > xx; mv xx x; ls -lh x; } \
< && x && x && x && x && x
<
<RUN chown $USER:$USER x
<
<#SQUASH#
<
<USER "$USER:$USER"
>#!/usr/bin/env bash
>
>export DOCKERDIR="." DOCKERFILE="Dockerfile" BUILDXFILE="SomeOtherFile" BUILDXTAG="SomeOtherTag"
>DOCKERBUILD() {
> echo "BUILDX---> docker build $*"
> docker build "$@" || exit $?; }
>
>if [[ ! "$(docker version)" =~ .*Experimental:\ *true.* ]]; then
> echo "Warning: docker experimental mode is off" 1>&2; fi
>
>echo "################ >>> building fragment #1 <<< ################"
>echo -n > "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>FROM ubuntu
>
>ENV USER=foo HOME=/home/foo
>
>RUN useradd -c Blah -U -G root -d "$HOME" -m "$USER"
>
>WORKDIR "$HOME"
>
>RUN echo xxxxxxxxxxxxxxx > x
>RUN x() { cat x x x x x x x x x x x x x x x x > xx; mv xx x; ls -lh x; } \
> && x && x && x && x && x
>
>RUN chown $USER:$USER x
>
>EOF
>
>DOCKERBUILD \
> --squash \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #2 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>
>USER "$USER:$USER"
>EOF
>
>DOCKERBUILD \
> --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>docker rmi "$BUILDXTAG"

80
tests/t05 Normal file
Просмотреть файл

@ -0,0 +1,80 @@
Meta options.
;--first: A1 A2 --last: Z1 Z2 --middle: M1 M2 --all: 1 2 :-- -t blah --first: A3
<line1
<#BUILD#
<line2
<#BUILD# more args
<line3
<#SQUASH#
<line4
>#!/usr/bin/env bash
>
>export DOCKERDIR="." DOCKERFILE="Dockerfile" BUILDXFILE="Dockerfilex" BUILDXTAG="0buildx-temp"
>DOCKERBUILD() {
> echo "BUILDX---> docker build $*"
> docker build "$@" || exit $?; }
>
>if [[ ! "$(docker version)" =~ .*Experimental:\ *true.* ]]; then
> echo "Warning: docker experimental mode is off" 1>&2; fi
>if [[ "$(docker images -q "$BUILDXTAG")" != "" ]]; then
> echo "Error: temp tag exists: $BUILDXTAG" 1>&2; exit 1; fi
>
>echo "################ >>> building fragment #1 <<< ################"
>echo -n > "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line1
>EOF
>
>DOCKERBUILD \
> A1 A2 1 2 A3 \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #2 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line2
>EOF
>
>DOCKERBUILD \
> M1 M2 1 2 \
> more args \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #3 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line3
>EOF
>
>DOCKERBUILD \
> M1 M2 1 2 \
> --squash \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #4 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line4
>EOF
>
>DOCKERBUILD \
> Z1 Z2 1 2 --tag blah \
> --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>docker rmi "$BUILDXTAG"

28
tests/t06 Normal file
Просмотреть файл

@ -0,0 +1,28 @@
A single build is considered last.
;--first: A1 A2 --last: Z1 Z2 --middle: M1 M2 --all: 1 2 :-- -t blah --first: A3
<FROM foo
>#!/usr/bin/env bash
>
>export DOCKERDIR="." DOCKERFILE="Dockerfile" BUILDXFILE="Dockerfilex" BUILDXTAG="0buildx-temp"
>DOCKERBUILD() {
> echo "BUILDX---> docker build $*"
> docker build "$@" || exit $?; }
>
>if [[ ! "$(docker version)" =~ .*Experimental:\ *true.* ]]; then
> echo "Warning: docker experimental mode is off" 1>&2; fi
>if [[ "$(docker images -q "$BUILDXTAG")" != "" ]]; then
> echo "Error: temp tag exists: $BUILDXTAG" 1>&2; exit 1; fi
>
>echo "################ >>> building fragment #1 <<< ################"
>echo -n > "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>FROM foo
>EOF
>
>DOCKERBUILD \
> Z1 Z2 1 2 --tag blah \
> --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"

85
tests/t07 Normal file
Просмотреть файл

@ -0,0 +1,85 @@
Parsing meta annotations.
<line1 \
<#BUILD#
<line2 \\\
<#BUILD#
<line3 \ \\
<#BUILD#A \
< B
<line4 EOF
<#BLAH#
<line5
<#SQUASH# C \
< D \\\
< E \\\\\
< F
<#BLAH# \
<#BUILD#
<line6
< line7
>#!/usr/bin/env bash
>
>export DOCKERDIR="." DOCKERFILE="Dockerfile" BUILDXFILE="Dockerfilex" BUILDXTAG="0buildx-temp"
>DOCKERBUILD() {
> echo "BUILDX---> docker build $*"
> docker build "$@" || exit $?; }
>
>if [[ ! "$(docker version)" =~ .*Experimental:\ *true.* ]]; then
> echo "Warning: docker experimental mode is off" 1>&2; fi
>if [[ "$(docker images -q "$BUILDXTAG")" != "" ]]; then
> echo "Error: temp tag exists: $BUILDXTAG" 1>&2; exit 1; fi
>
>echo "################ >>> building fragment #1 <<< ################"
>echo -n > "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line1 \
>#BUILD#
>line2 \\\
>#BUILD#
>line3 \ \\
>EOF
>
>DOCKERBUILD \
> A \
> B \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #2 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF_[...]"
>line4 EOF
>#BLAH#
>line5
>EOF_[...]
>
>DOCKERBUILD \
> --squash C \
> D \\\
> E \\\\\
> F \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #3 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>#BLAH# \
>#BUILD#
>line6
> line7
>EOF
>
>DOCKERBUILD \
> --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>docker rmi "$BUILDXTAG"

78
tests/t08 Normal file
Просмотреть файл

@ -0,0 +1,78 @@
Includes, with recursion, case insensitive annottaions.
<line1
<#BUILD#
<#bUiLd#
<line2
<#INCLUDE# include/t08-missing
<line3
<#INCLUDE# include/t08-i1
<line4
<#INCLUDE# include/t08-i2
<line5
<#INCLUDE# **/t08-*
<line6
>#!/usr/bin/env bash
>
>export DOCKERDIR="." DOCKERFILE="Dockerfile" BUILDXFILE="Dockerfilex" BUILDXTAG="0buildx-temp"
>DOCKERBUILD() {
> echo "BUILDX---> docker build $*"
> docker build "$@" || exit $?; }
>
>if [[ ! "$(docker version)" =~ .*Experimental:\ *true.* ]]; then
> echo "Warning: docker experimental mode is off" 1>&2; fi
>if [[ "$(docker images -q "$BUILDXTAG")" != "" ]]; then
> echo "Error: temp tag exists: $BUILDXTAG" 1>&2; exit 1; fi
>
>echo "################ >>> building fragment #1 <<< ################"
>echo -n > "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line1
>EOF
>
>DOCKERBUILD \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #2 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>
>DOCKERBUILD \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #3 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>Warning: cannot read include file: include/t08-missing
>cat >> "$BUILDXFILE" <<"EOF"
>line2
>line3
>include1 line1
>include1 line2
>line4
>include2 line1
>include1 line1
>include1 line2
>include2 line2
>line5
>include1 line1
>include1 line2
>include2 line1
>include1 line1
>include1 line2
>include2 line2
>line6
>EOF
>
>DOCKERBUILD \
> --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>docker rmi "$BUILDXTAG"

62
tests/t09 Normal file
Просмотреть файл

@ -0,0 +1,62 @@
Meta instructions, skip generation when block is empty.
<line1
<#BUILD#
<line2
<#META# foo \
< x y z
<#META# bar
<
<#META# baz
<line3
>#!/usr/bin/env bash
>
>export DOCKERDIR="." DOCKERFILE="Dockerfile" BUILDXFILE="Dockerfilex" BUILDXTAG="0buildx-temp"
>DOCKERBUILD() {
> echo "BUILDX---> docker build $*"
> docker build "$@" || exit $?; }
>
>if [[ ! "$(docker version)" =~ .*Experimental:\ *true.* ]]; then
> echo "Warning: docker experimental mode is off" 1>&2; fi
>if [[ "$(docker images -q "$BUILDXTAG")" != "" ]]; then
> echo "Error: temp tag exists: $BUILDXTAG" 1>&2; exit 1; fi
>
>echo "################ >>> building fragment #1 <<< ################"
>echo -n > "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line1
>EOF
>
>DOCKERBUILD \
> --tag "$BUILDXTAG" --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>
>echo ""
>echo "################ >>> building fragment #2 <<< ################"
>echo -n > "$BUILDXFILE"
>echo "FROM $BUILDXTAG" >> "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line2
>EOF
>{
> foo \
> x y z
> bar
>} >> "$BUILDXFILE" || exit $?
>cat >> "$BUILDXFILE" <<"EOF"
>
>EOF
>{
> baz
>} >> "$BUILDXFILE" || exit $?
>cat >> "$BUILDXFILE" <<"EOF"
>line3
>EOF
>
>DOCKERBUILD \
> --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"
>
>docker rmi "$BUILDXTAG"

33
tests/t10 Normal file
Просмотреть файл

@ -0,0 +1,33 @@
Metameta, including nesting three levels.
<line1
<#METAMETA# echo 1 $DOCKERDIR $DOCKERFILE
<#METAMETA# echo "#METAMETA# echo \"2 \$DOCKERDIR \$DOCKERFILE\""
<#METAMETA# echo "#METAMETA# echo \"#METAMETA# echo \\\"3 \\\$DOCKERDIR \\\$DOCKERFILE\\\"\""
<line2
>#!/usr/bin/env bash
>
>export DOCKERDIR="." DOCKERFILE="Dockerfile" BUILDXFILE="Dockerfilex" BUILDXTAG="0buildx-temp"
>DOCKERBUILD() {
> echo "BUILDX---> docker build $*"
> docker build "$@" || exit $?; }
>
>if [[ ! "$(docker version)" =~ .*Experimental:\ *true.* ]]; then
> echo "Warning: docker experimental mode is off" 1>&2; fi
>if [[ "$(docker images -q "$BUILDXTAG")" != "" ]]; then
> echo "Error: temp tag exists: $BUILDXTAG" 1>&2; exit 1; fi
>
>echo "################ >>> building fragment #1 <<< ################"
>echo -n > "$BUILDXFILE"
>cat >> "$BUILDXFILE" <<"EOF"
>line1
>1 . Dockerfile
>2 . Dockerfile
>3 . Dockerfile
>line2
>EOF
>
>DOCKERBUILD \
> --file "$BUILDXFILE" "$DOCKERDIR"
>rm -f "$BUILDXFILE"