Граф коммитов

5711 Коммитов

Автор SHA1 Сообщение Дата
Alain Jobart 0979197a8c Removing *Mysqld from Agent now. 2015-05-22 11:12:33 -07:00
Alain Jobart ce8aed335e tabletserver now uses MysqlDaemon too. 2015-05-22 11:09:36 -07:00
Ammar Aijazi 85b960e0fc Merge pull request #712 from youtube/aaijazi_worker_test_check_extra_data
Add checks in worker test to make sure that there's no extra data on destination shards
2015-05-22 10:11:49 -07:00
Ammar Aijazi 7307085cd9 Add checks in worker test to make sure that there's no extra data on destination shards 2015-05-22 10:10:03 -07:00
Alain Jobart bf3777c6b0 Now converting binlog player to MysqlDaemon. 2015-05-22 09:52:41 -07:00
Alain Jobart 2fbb54133d Converting a couple more methods to MysqlDaemon. 2015-05-22 09:33:23 -07:00
Alain Jobart cc148e81d9 Converting FindSlaves to MysqlDaemon. 2015-05-22 09:20:50 -07:00
Ammar Aijazi ba9f5ab20b Merge pull request #710 from youtube/aaijazi_batch_insert_in_worker_test
Make test/worker.py do batch insert at setup, so that worker copy and…
2015-05-22 08:09:16 -07:00
Alain Jobart b0c2337210 Converting one more call to MysqlDaemon. 2015-05-21 19:52:43 -07:00
Ammar Aijazi b575f2f89f Make test/worker.py do batch insert at setup, so that worker copy and initial insert take similiar amounts of time 2015-05-21 19:19:59 -07:00
Alain Jobart a5b3f2aa53 Removing unused mysqld parameter for method. 2015-05-21 19:12:46 -07:00
Alain Jobart 80678fa46b Removing now empty {Enable,Disable}BinlogServer methods.
(they were useful before we switched to using a MySQL connection).
2015-05-21 19:01:53 -07:00
Alain Jobart 08e0d27dc7 Merge pull request #709 from youtube/resharding
Resharding
2015-05-21 17:53:41 -07:00
Alain Jobart 7ac9766e46 Fixing TabletType:
- removing SNAPSHOT_SOURCE
- renaming CHECKER to WORKER
2015-05-21 16:36:16 -07:00
Alain Jobart f76208ec52 Submitting another change from Ric. Will need fixing. 2015-05-21 15:42:24 -07:00
Alain Jobart 48e84a97c6 Merge pull request #708 from youtube/log
Log
2015-05-21 15:18:26 -07:00
Alain Jobart 987ec108c6 Merge pull request #705 from youtube/resharding
Resharding
2015-05-21 15:18:12 -07:00
Alain Jobart fb10d05bfd Exporting a couple methods for other modules to use. 2015-05-21 14:46:15 -07:00
Alain Jobart 8f2a4c71aa Removing extra log line. 2015-05-21 14:41:48 -07:00
Alain Jobart 794e034a08 Moving filebackupstorage into its own package. 2015-05-21 11:15:29 -07:00
Shengzhe 3df76885f9 Merge pull request #706 from yaoshengzhe/fix_schema_rework
schema rework
2015-05-21 11:13:27 -07:00
Alain Jobart d89f042c5a Fixing backup timestamp format to be UTC, human readable, and correctly comparable.
Example backup name: test_nj-0000062345.2015-05-21.163734
2015-05-21 09:39:28 -07:00
Alain Jobart a5fd0206c4 Making GetBackupStorage return en error, instead of Fatalf-ing out. 2015-05-21 09:28:45 -07:00
Alain Jobart 1d1d71354a Merge branch 'master' into resharding 2015-05-21 09:22:52 -07:00
Alain Jobart 65aa8a6022 Merge pull request #707 from youtube/log
Fixing contexts in these file. Short context is now
2015-05-21 09:14:29 -07:00
Alain Jobart 3fd4524b3a Fixing contexts in these file. Short context is now
always named shortCtx, and never overrides the ctx variable.
Fixes vertical_split.py.
2015-05-21 09:03:15 -07:00
Alain Jobart d0cc07ae69 Simplifying file backup storage (implementation and plugins). 2015-05-21 08:54:52 -07:00
Shengzhe Yao f777f49938 monitor schema change dir in vtctld
1. Add two flags in vtctld: schemaChangeDir and schemaChangeController.
   schemaChangeDir specifies a parent directory that contains schema
   changes for all keyspaces.
   schemaChangeController controls how to get schema change sqls from
   schemaChangeDir and handle relevant schema change events.
2. Add RegisterControllerFactory in schemamanager that allows developers
   to register different Controller implementations.
3. Add test case to test when schema change failed on some shards.
2015-05-20 23:35:39 -07:00
Shengzhe Yao 318ebfc310 add GetKeyspace in the schemamanager.Controller interface
The idea is that sometimes a Controller implementation will take care all
keyspaces. In such case, schemamanager.Executor is not able to know keyspace
ahead.
2015-05-20 23:07:47 -07:00
Shengzhe Yao c1c3383834 remove uihandler and create UIController 2015-05-20 21:37:43 -07:00
Shengzhe Yao 1172c89697 merge DataSourcer and EventHandler interface
DataSourcer and EventHandler interfaces are good but often a concrete
EventHandler implementation needs some information only exists in DataSourcer.
e.g. A DataSourcer reads schema changes from a file system and an EventHandler
wants to move this file around to response different schema change events.
The information exchange would be hard to do with two separate interfaces, as
this level of abstraction introduces more boilerplate code.

This change combines these two interface into a single one: schemamanager.Controller,
which reduces one level of abstraction and the code looks more cleaner.
2015-05-20 21:07:18 -07:00
Alain Jobart 4c743918c0 Merge branch 'master' into resharding 2015-05-20 16:40:33 -07:00
Alain Jobart 90ff0d8535 Merge pull request #701 from youtube/log
Log
2015-05-20 16:39:45 -07:00
Alain Jobart cf568fdc28 Adding unit test for findFiles for backup.
Making other unit test simpler. Removing special case
for symlinks, unused.
2015-05-20 16:18:28 -07:00
Alain Jobart 85c2d155d6 Merge branch 'master' into resharding 2015-05-20 13:56:35 -07:00
Alain Jobart 9776e112ce Removing couple unused methods. 2015-05-20 13:54:55 -07:00
Alain Jobart a4bb4d36ad Removing now unused mysqld.GetMasterAddr and related uses. 2015-05-20 13:43:47 -07:00
Shengzhe 4dfb7e992f Merge pull request #702 from yaoshengzhe/fix_add_pk_column_in_split_query
add SplitColumn field to SplitQuery
2015-05-20 13:25:39 -07:00
Alain Jobart 979f0fdc4e Adding a unit test for backup/restore. Had to export a couple
flags so it's unit-testable.
2015-05-20 13:13:57 -07:00
Alain Jobart 9b9f96abd8 Deleting old Snapshot/Restore/Clone code. 2015-05-20 09:26:52 -07:00
Alain Jobart e15f38cd39 Removing clone.py & primecache.py, adding backup.py. 2015-05-20 09:00:23 -07:00
Alain Jobart dea26442d9 Merge branch 'master' into resharding 2015-05-20 08:57:06 -07:00
Alain Jobart bbf0916778 Few cosmetic changes, and addressing cancellation better. 2015-05-20 08:03:01 -07:00
Michael Berlin 35ed97de1a Merge pull request #669 from youtube/automation_first_minimal_version
Initial version of automation framework.
2015-05-19 18:23:47 -07:00
Michael Berlin 452265dadd Merge pull request #703 from youtube/fix_docker_run_permissions_root
docker/test/run.sh: Make repo world readable for access from docker.
2015-05-19 18:19:47 -07:00
Michael Berlin 1b8cbb90f1 Initial version of automation framework.
The automation framework allows to automate cluster operations which
require a series of manual steps e.g. resharding.

A Cluster Operation has a list of task containers which are processed
sequentially. Each task container can contain one or more tasks which
will be executed in parallel.

Here's an example of a cluster operation with two task containers. The
second task container has two tasks:
- step 1
- step 2a | step 2b

If the task container contains one task, the task can emit new task
containers which will be inserted after the current task container. This
mechanism is used to fully expand Cluster Operations by special tasks
which emit new task containers e.g. "ReshardingTask".

This patchset implements the minimal steps to automate "resharding"
whereas task implementations for "vtctl" and "vtworker" are missing.
These will be added in later, separate commits.
2015-05-19 18:03:59 -07:00
Michael Berlin 201532b076 Merge pull request #624 from youtube/mysql_flags_java_client
java/vtgate-client: Process new "Flags" MySQL field in result set.
2015-05-19 17:56:08 -07:00
Michael Berlin a8785556be docker/test/run.sh: Make repo world readable for access from docker.
Previously, the permissions of all items *within* the repository were already updated, but not the repository itself.

This resulted into the error 'cp: cannot stat `/tmp/src/*': Permission denied' when running "docker/test/run.sh mariadb".
2015-05-19 17:43:18 -07:00
Shengzhe Yao 199aa60881 add SplitColumn field to SplitQuery
1. Add SplitColumn field to SplitQuery so that caller could hint SplitQuery endpoint
   which column is best for splitting query.
2. The split column must be indexed and this will be verified on the server side.
3. coding style fixes suggested by golint.
2015-05-19 17:13:43 -07:00
Alain Jobart b342856305 Moving a few things to MysqlDaemon. 2015-05-19 15:04:04 -07:00