Using regexp syntax.
Also supporting them in binlog player, and in blacklisted tables.
Meaning we can use wildcards in vertical splits.
Also adding -table_names_only to 'vtctl GetSchema', so it's easy
to test a regexp.
They allow the status page of vttablet to be better.
Also testing this code in vtctld_test, so we make sure the
templates work. And manual inspection works very well too.
When re-building a Keyspace that is ServedFrom another
keyspace, we need to save the SrvKeyspace object in all the
cells from the ServedFrom keyspace as well.
Used to get the list of cells by looking at the cells that
contain a serving tablet in shard 0. Now use the intersection
of the union of all shard.Cells and the passed-in cell list.
After a failed action, we were recording the failure, but not
cleaning up the original action. And a test. Also some cleanup:
- added a 'vtctl ReadTabletAction command
- using it in tests to not wait random amounts of time
(the added test time is less than the time gained!)
- also fixing parameter passing in tabletmanager.py test.
The rowcache invalidator was launched sepearately from QueryEngine
by SqlQuery. This had some potential race conditions that were
hard to resolve with this code structure.
Now, the invalidator is part of QueryEngine. This allows us to
close the invalidator as part of QueryEngine's close. We are
guaranteed that no requests will be allowed when Close gets called.
The previous scheme had some subtle race conditions that could lead to unexpected behavior.
This new scheme addresses all of them. There are still some race conditions to be resolved
with the rowcache invalidator, which will be addressed in the next CL.
- All state transitions require a write lock.
- Any section of code that requires a state not to change during an action will require a read lock.
- No long-running locks.
- Managing NOT_SERVING -> SERVING is straightforward. Since nothing else is allowed to happen during this time, we can make an exception and hold the lock for the entire operation.
- Once we're SERVING, there are two phases that we have to manage when disallowQueries gets called:
1. Wait for existing transactions to complete.
2. Wait for existing queries to complete.
Waiting for transactions:
Before we start waiting, we have to ensure that no new transactions can get created. Otherwise, we may introduce a race condition that ends up allowing the creation of a new transaction after we conclude that all existing transactions are completed.
How do we prevent this:
disallowQueries:
- obtain write lock
- transition SERVING to SHUTTING_TX
- release write lock
- wait for existing transactions to complete
Begin:
- obtain read lock
- verify SERVING state
- create a transaction
- release read lock
This guarantees that new calls to Begin will not go through when we start waiting for transactions to complete.
Waiting for queries:
disallowQueries:
- obtain write lock
- transition SHUTDOWN_TX to SHUTTING_DOWN
- release write lock
- wg.Wait()
Execute:
- obtain read lock
- verify SERVING or SHUTDOWN_TX
- wg.Add(1)
- release read lock
- execute query
- wg.Done()
This guarantees that no new queries will be allowed once disallowQueries starts wg.Wait(). When the function returns, we know that all existing queries have completed, we're free to transition state to NOT_SERVING.
Alain Jobart
Peter Lobsinger
Alain Jobart
Sugu Sougoumarane
Liang Guo