And moving endtoend tests from go/mysql to go/mysql/endtoend.
This breaks the go/vt/vttest circular dependency, and makes the
tests that need a MySQL instance all in the same place, cleaner.
Please refer to #2694 and #2670 for motivation and reasoning for
this change.
I've tried to follow best practice in inserting the copyright
headers. In other open source projects, not all files carry
the notice. For example documentation doesn't. I've followed
similar ground rules.
I did not change the php because there is a separate LICENSE
file there by Pixel Federation. We'll first need to notify
them our intent before changing anything there.
As for the presubmit check, it's going to be non-trivial
because of the number of exceptions, like file types,
directories and generated code. So, it will have to be
a separate project.
In the process:
- Fixing bug in SendCommand.
Need to flush the command, doh!
- Move ComBinlogDump to go/mysqlconn.
- Move ComBinlogDumpGTID to go/mysqlconn.
- Consolidating mysqlconn.{r,R}eadPacket.
- Removing replication methods from sqldb.
Removing SendCommand, ReadPacket, and Shutdown.
They are now on mysqlconn.Conn only.
- Simplifying BinlogEvent interface.
HasGTID was always equal to IsGTID (was not the case with Google MySQL
5.1, but that's not supported any more).
IsBeginGTID can then be returned by GTID().
It implements both client and server side of the MySQL binary protocol.
A few notes:
- SSL is not supported yet. It wouldn't be too hard to add, but our
current client library code doesn't support it either.
- only the usual mysql native password authentication is supported.
Again, would be easy to support more.
- vtgate now has the option to listen using the MySQL protocol.
The local examples are configured to do so.
- The API is not great, but is close to sqldb.Conn. Will be refactored
once go/mysql is retired.
That's what the actual protocol size for the field is. Also documenting
the other fields, as they are uint16 or uint8 in the protocol.
And rebuilding all proto docs too.
Removes exclude_field_name from ExecuteOptions, replacing with an
IncludedFields enum. Enum values allow specifying whether to return
all fields, just type and name (default), or just type (perf optimization)
Adds and updates tests
In the process:
- fixing binlog replay starting from a timestamp: we just cannot skip
the events at a lower level, as the binlogs may contain format event,
binlog GTID set events, and so on. We need to parse them all.
- adding support for MySQL 5.6 PREVIOUS_GTIDS_EVENT event. It contains a
SIDBlock, adding parsing for that (we already had code to generate it,
as it's used to start replication from a GTIDSet).
- Some code in vtmysql_internals.h was incorrectly compiling for MySQL
5.6 and higher, when it should have been 5.5.
The mysql API bug seems to trigger for very large data sizes,
likely greater than 1<<20. This is because we make go think
that those slices can't be bigger than that value.
The other fix is to prevent a divide by 0 error.
The new Value implementation is now based on the vitess types.
* The inner interface has now been replaced by typ and val.
* All Values are expected to be consistent with their types.
For example, an Int64 type must contain a number.
* The functions that build values generally ensure consistency.
* There is a set of 'Trusted' functions that can bypass this
consistency check. They should be used with care.
* The proto3 conversion functions build the correct Value types
based on the field types.
* The bson conversion function provides a Repair function that
allows you to fix up the types after the fact. This should be
deleted after bson is deprecated.
* The building of Values from a QueryResult is non-trivial because
the field info is not part of the QueryResult for streaming
queries. So, the API requires fields to be explicitly passed in.
* Fuctions that encode or convert to native types expect Value
to be consistent. If not, they panic.
* proto3.QueryResult is considered to be trusted. If it contains
inconsistent data, it will cause panics.
* The EventStreamer has been fixed to ensure that the fields and
rows it publishes are trustable: They can used as parameters
to the Trusted API.
* The Raw() function usage has been minimized. We should see if
it can be deprecated. This way, we can make Result truly read-only.
There are a few more tweaks that need to be done:
* The Proto3ToResult call plumbing was hacked in to make everything
work. That part needs cleaning.
* The bind vars don't need to be converted to their native types
any more.
This part 1 of the QueryResult revamp. With this change,
vitess does not see any MySQL types. The BSON layer still
sends mysql types for backward compatibility, which will
be deleted once deprecated.
This change also introduces constants in mysql package for flags
and types. Additionally, I mask out flag bits that are not relevant.
This will help us be deterministic about what we do or don't support.
I found a few problems and shortcomings in the mocks:
1. A lock was being held while connecting to mysql, which should
be avoided.
3. Tests depended on a global variable (DefaultDB) being initialized
to different values at the beginning of each test. This would prevent
parallelization.
2. There was no way to use a real mysql connection in tests once
a fakesqldb was setup, which is needed for the queryservice tests
to be ported to go.
In the new scheme, I've added an Engine field to ConnParams. If it's
empty, then the default (MySQL) connection is used without locks.
If there is a name, then we use the map where other test engines
are registered.
The next change will convert the memcache faking to a similar scheme.
Alain Jobart
Sugu Sougoumarane
Bryan Beaudreault
Anthony Yeh
Sugu Sougoumarane
Ammar Aijazi