The recent release of MySQL 8.0.19 included some behavior changes which
affected Skeema:
* The sort order of foreign keys in SHOW CREATE TABLE has changed. This caused
some tables with multiple FKs to be unsupported for diff in Skeema.
* Integer display widths are suppressed in SHOW CREATE TABLE, as well as
information_schema, in most cases. This change did not affect Skeema itself,
but it broke a huge number of integration tests, which make assumptions
about the canonical format of DDL.
These problems are fixed in this commit. Documentation has also been updated
to reflect that the lint-display-width option is irrelevant as of
MySQL 8.0.19.
Finally, the flavor option now permits inclusion of an optional patch version,
e.g. flavor=mysql:8.0.19. Skeema commands will never auto-populate the patch
version, but users may manually use one if desired, for example when a
specific patch is needed with workspace=docker.
This commit makes numerous improvements to Skeema's handling of situations
where it is unable to parse the database server's vendor and/or version:
* If the database vendor cannot be parsed from @@global.version_comment, try
searching @@global.version too. This fixes#68, support for MariaDB 10.3
in RDS.
* If the vendor still cannot be parsed, treat it like upstream MySQL (e.g.
if the version_comment and version both do not contain the string
"mariadb") for most conditional behaviors.
* diff/push: use configured flavor in .skeema (for all behaviors) if the
server's actual flavor cannot be determined properly.
* diff/push: warn if the server's actual flavor is valid and differs from
what is in .skeema
* init, add-environment, diff/push: warn if the server's actual flavor
cannot be parsed, and no flavor is specified in .skeema
Both `skeema init` and `skeema add-environment` now persist --connect-options
to the host-level .skeema file, if it was specified on the CLI.
Additionally, `skeema add-environment` now persists --ignore-schema and
--ignore-table to the host-level .skeema file, matching the existing behavior
of `skeema init` for these options.
This PR moves much of Skeema's logic out of the main package and into several
new sub-packages, which can be imported by other applications if desired.
Functionality is largely unchanged, and no new features have been added. But a
few foundational benefits of this work include:
* The codebase no longer assumes a 1:1 mapping between *.sql files and tables.
This will eventually permit non-table object types (views, procs, grants, etc)
to be stored in the same repo as schemas, if desired. See #41 for background.
* In the upcoming Skeema 1.1.x series, it will be possible to use a local Docker
instance for temp schema operations. This performs better in high-latency,
high-table-count scenarios; see #25 for background.
* The limit on max *.sql file size has been removed. Closes#34.
* `skeema pull` now performs much better than before, as long as --normalize is
enabled (which it is by default).
* The code supporting `skeema push --concurrent-instances` is now much cleaner
and more idiomatic.
* Test coverage has been improved.
This commit adds a new "flavor" option, which in Skeema v1.0.x is just
informational (no functional impact). `skeema init`, `skeema pull`, and
`skeema add-environment` now automatically populate this option in host-level
.skeema files, to persist the database's vendor (mysql/percona/mariadb) and
major.minor versions. This information may be used in new features beginning in
Skeema v1.1.
`skeema add-environment` no longer does a connectivity test of the supplied
host; it just manipulates the .skeema file now.
`skeema lint` had a bug with how it handled --ignore-schema, now fixed.
Some exit codes have been changed from 2 (generic fatal error) to 78 (bad
config) in cases where a configuration problem is more clearly to blame.
`go test` now ignores global option files (/etc/skeema and
/usr/local/etc/skeema) and user-specific option files ~/.my.cnf and ~/.skeema,
to avoid these files affecting the result of tests.
Hidden directories are now consistently ignored everywhere, in terms of option
file parsing. This is to avoid issues with SCM metadata directories.
`skeema push` now handles "schema=*" more consistently: the alphabetically
first schema will always be used as the template.
Minor internal refactors to a few methods to make them more testable.
Split exec.go into 2 files: ddlstatement.go (code related specifically to DDL
execution) and shellout.go (code related specifically to shelling out to
external processes). Added new ShellOut struct accordingly.
Extract connect-options parse logic into its own helper function, which is also
now called from RealConnectOptions to fix a prior bug. connect-options no longer
allows bareword booleans, since drivers don't support this anyway.
Added several new unit tests relating to external command execution, as well as
connect-options parsing.
Move several option-related helper methods from various files to new file
config.go.
Now that the number of source files is growing, renamed all subcommand files to
have "cmd_" prefix. At some future point they may be moved to a separate
package, but for now this provides easier visual identification of which files
define subcommands.
Evan Elias