* Support REPLACE INTO for unsharded tables
* merge fixes
* Make REPLACE INTO use PASS_DML
* Fix tests
* update tests
* add replace support to preview
* Remove unneeded tests now that PASS_DML. Add test of PASS_DML/RBR for Replace
UNION mergeability was previously using the same logic as
subquery mergeability. It turns out that it doesn't always
work because subqueries can have outer query dependencies
whereas unions don't have such relationship. So, the logic
has now been separated.
initial WIP union commit
WIP: non-compiling, known not done; before branch for planbuilder.route.Select->SelectStatement type change
WIP checkpoint
WIP checkpoint...compiles
WIP on merge primitive
trivial test fix
support UNION in subqueries
preliminary tests
tests, responses to PR comments
copy queryConstruct to avoid data race
added tests for unsharded union
union distinct in unsharded cases; default in switch in from.go
This change allows two extensions to the SQL syntax:
1. UNION statements can be followed by ORDER BY, LIMIT and LOCK clauses.
2. SELECT statements that are part of a UNION can be parenthesized.
The main purpose of allowing parenthesis for selects was to
support the following two constructs: (SELECT...) UNION (SELECT...)
and INSERT INTO a (SELECT...).
The initial approach was to allow parenthesizing of all SELECT
statements. However, just adding that as an additional rule
created conflicts. In particular, `((SELECT * FROM t))` could
be seen as a subquery with a parenthesized select, or a row
tuple with a subquery. We've avoided this conflict by allowing
parenthesis only for selects that are part of a union.
This change allows you to do the UNION construct. Additionally,
we go one step further and allow ORDER BY and LIMIT clauses
for unions which we didn't allow before. The tabletserver
planbuilder has also been extended accordingly. VTGate
support of unions will also need to be extended.
The construct for inserts will be addresed in the next PR.
If necessary, we could allow the top level statement to be
parenthesized, but that has no practical value. So, we can
skip that for now.
For now, the current grammar has 1 shift/reduce conflict, which will
be resolved once the support for the INSERT...(SELECT...) is added.
vtgate now gets the groups and sends them in the immediate caller id.
we store the time of the ldap query and refresh (in another goroutine) if
older than configurable threshold. this happens upon ComQuery. the simpleacl
implementation is now group aware, and the ldapacl plugin has been removed.
During BeginExecute() do not start a second transaction if another BeginExecute() for the same row is already in progress. This avoids that updates targeted for a single row can consume all transaction pool slots.
Note that the serialization is bound to the lifetime of the BeginExecute() call and not to the lifetime of the transaction. This made the implementation simpler since no additional plumbing was required. Due to the additional MySQL locking, this should result into two transaction pool slots per row at most. (One pending on COMMIT, one waiting for MySQL in BEGIN+EXECUTE.) If this is not effective enough, we will have to revise this decision.
By default it is disabled.
It can be fully enabled (-enable_hot_row_protection) or enabled in dry-run mode (-enable_hot_row_protection_dry_run). If one of the two flags is specified, most of the code will be exercised. If none of the two flags is set, the code will be completely skipped.
The list of the last 1000 queries which triggered the hot row protection are available at /hotrows on the vttablet HTTP port.
Other change:
- Extended fakesqldb to run a callback before it responds to MySQL. This is required to properly synchronize the concurrent transactions. Without, they may not race and not get serialized.
* Reload schemas on all DDLs. Support table qualifiers in DDL queries.
* For now only support table_name in non-create because the lookahead conflicts with force_eof
* create tables cant have a row count, so no point checking. This avoids an NPE on Table.Name, since creates are the only one to only have a NewName
* Make table_name work for CREATE and VIEWs. Fix alter view test, the syntax in the test failed to parse and was incorrect according to docs
* This test does not apply since the removal of TableWasDropped
* If we're in a transaction, don't start a new one
* Reload schema on commit, if necessary
* recycle connection when using transaction
* utilize existing autocommit function
* simplify
* unneeded function
* If an transaction is open, we should commit it before running the DDL -- DDL's have an implicit commit.
* implement and use BeginAgain when we implicit commit a running transaction due to DDL
* Move BeginAgain to TxConnection
* dont lowercase qualifier
* a few parser additions:
- DATE Function
- LIKE pattern ESCAPE char syntax
- && as AND
- Charset Introducers on integral values
- pound-sign comments
- INSERT with no values
- support all keywords as identifiers where possible
* simplify
* Add to WalkSubtree
* Sample parser change: take 2
* more work to integrate booleans and values:
- unify all the BoolExpr and ValExpr as just Expr
- unify the tuples to avoid grammar conflicts
* Adding collate operator support
* Adding collate operator support
* Changing to ComparisonExpr
* Changing to ComparisonExpr
* Revert to BinaryExpr
* moving to collate value_expression
* reverting to value
* changing to ```value_expression COLLATE value_expression```
* remove PK - 4 from main_test
* removed z from lowezr
* fixing reviews by sugu
* fixing reviews by sugu
* adding tests per http://dev.mysql.com/doc/refman/5.7/en/charset-collate.html
* adding tests per http://dev.mysql.com/doc/refman/5.7/en/charset-collate.html
* support for OR operator without it being used for concatenation
* remove OR_PIPE and reuse OR
* order by complex expression
* revert order by complex expression
* revert order by complex expression
* collation support
* collation support
* collation support
* collation support
* simplify yacc for collation
* adding tests for order by
* handling one more test case of order by column number with collate
* fixing recommendations from sugu
* fixing recommendations from sugu
Tying most of the tabletserver pieces together:
* Changed insert plan. It's easier to read the row from the
db than to try to build it in memory. The read should be
efficient because we're just pulling it from the buffer cache.
* DML plan. DMLs on the message table should invalidate the cache.
* Make sure id is part of the pk. We need it for the DMLs.
* Update the cache on commit.
* tabletserver endtoend test.
* Include on clauses in impossible queries, to avoid only_full_group_by in aggregations with joins
* remove now-unnecessary handling of left/right join. fix endtoend tests
When a group by is available, ensure it's part of field query to avoid issues with aggregations
Refactor to use a unified code path between vtgate and vttablet
Addresses: #1861.
The new resolution algorithm better mimics MySQL behavior: GROUP BY,
HAVING and ORDER BY search column aliases as well as table names.
It's not possible to exactly mimic MySQL's behavior without knowledge
of the full schema. However, such cases happen only if the SQL attempts
to hide table column names with column aliases, which is uncommon
and not recommended.
The most common application constructs where the post-process constucts
reference table columns should now work.
Addresses #2373.
',' operators will now be allowed for joins. They will be treated
as JOIN operators without an ON clause. This will work for unsharded
joins as well as cross-shard joins. For now, we will not look for
join conditions in the WHERE clause to identify single-shard queries.
Printf formatting doesn't work for names that themselves
contain backticks. Since we now have a function that esacapes
those correctly, I've changed all places to use the new function.
Tokenizer has been changed to allow.
AST has been changed to escape correctly.
All bind vars use CompliantName function.
Made col & table ref matching mimic MySQL's behavior: When
qualified names are used, keyspace & table names have to be
cases sensitive, and col names have to be insensitive.
Table names are mostly stored as TableIdent. This allows for
better tracking of where it's used, and making sure that
we escape it correctly, and more easily add special handling
in the future as necessary.
* MultiRow Insert with different KeyspaceIds
* Fixing the Router Tests
* Rewrite Queries using the Rows Object
* Making Separate Changes for InsertSharded and Fixing Tests
* Fixing Tests in router_dml_test.go
* Fixing PlanBuilder TestCases
* Refactoring Code for MultiRowInsert
* Incorporating Changes as discussed today
* Incorporating Review Comments as discussed
* Incorporating Review Comments as discussed
* Incorporating Review Comments as discussed
* Resolving Merge Conflicts
* Resolving Merge Conflicts
* fixing the Values Format
* Incorporating Review Comments and Removing ExecuteMulti
* Refactoring getInsertShardedRoute() and Adding tests in vtgatev3_test.py
* Making routing a map of map[string][]string
ORDER BY NULL is a special-case construct that serves as hint
to the optimizer that the client is not expecting the result
in any particular order. So, it's safe to push it down to
all routes without regard to relational rules.
Additional changes:
An unsharded insert could have a select statement. If so, that
statement has to be validated to make sure it targets the same
keyspace.
Values supplied inside the row values can themselves be subqueries.
If so, the code detects this and returns an error.
Sharded and unsharded insert plans now have separate code
paths because they have many differences.
Intermixing of row values with queries is not allowed. I've
changed the parser to not allow it any more. This is has led
to some code simplification.
On the execution side:
handleGenerate has been upgraded to fetch all values in one call.
It also handles the case where some values are supplied and some not.
It works the same way for both sharded and unsharded tables.
This change addresses #2254. The ORDER BY clause is allowed
to reference columns in the FROM clause that are not necessarily
present in the SELECT expressions.
The new construct allows you to use the `SELECT NEXT N VALUES
FROM t`. Consequently, the increment value in the sequence
table is deprecated (and ignored). We still support the previous
`SELECT NEXT VALUE...` construct, which is now essentially
`SELECT NEXT 1 VALUES...`. Once the new construct is adopted,
we'll deprecate the old one.
The returned value is the first of the N values.
As mentioned in the issue, I still allow 0x literals,
but will make future changes to prevent Vitess from
mis-interpreting those values.
I've added some missing tests in the process.
We found a situation where subqueries that used FQ column names
would cause V3 code to be confused about their vindexes. For
example, a query like:
select ... from (select t.id from t) as t1...
would end up creating a vindex named "t.id". But a reference
to t1.id would fail to match against "t.id" because the string
comparison would fail.
This PR fixes the issue by creating aliases named "id" along with
fully qualified names as "t.id", which would allow any kind of
reference to correctly find the columns referenced.
This behavior still doesn't fully match MySQL's behavior. However,
it comes closer, while still preserving SQL name resolution rules.
The particular use case where we deviate is as follows:
select id as foo from t having id = 1
MySQL allows the above construct. However, we'll fail because you
can reference that expression either as 'foo' or 't.id', but not
as 'id'.
Implementation notes:
The colsym already had two values to match against. I have now changed
the meaning of those:
1. The Alias is the alias of the column. If one is not provided,
then we assign the base name of the column. Previously we used to
assign the FQ name.
2. The ExprName (now renamed to QualifiedName) always contains
the FQ name. This used to have a value only if there was already
an alias provided.
The matching logic has been changed accordingly. Some new errors
have been added to handle possible ambiquities that can now happen
because we match using base names. Otherwise, most things should
work like before.
Bryan Beaudreault
Sugu Sougoumarane
Alex Charis
Alain Jobart
Sugu Sougoumarane
Michael Demmer
Michael Berlin
Amit Khare
Ashu
Chris Gillis