Граф коммитов

297 Коммитов

Автор SHA1 Сообщение Дата
Scott Lanning 6de6108dae minor fix for deprecation warning in `kubectl port-forward`
Just happened to notice a deprecation notice when trying to debug something:

  -p POD is DEPRECATED and will be removed in a future version. Use port-forward POD instead.

The output is going to a tmpfile, so people wouldn't normally see it.
2017-05-11 19:30:17 +02:00
Sugu Sougoumarane 7af23e1dde license: BSD->Apache v2.0
Please refer to #2694 and #2670 for motivation and reasoning for
this change.

I've tried to follow best practice in inserting the copyright
headers. In other open source projects, not all files carry
the notice. For example documentation doesn't. I've followed
similar ground rules.

I did not change the php because there is a separate LICENSE
file there by Pixel Federation. We'll first need to notify
them our intent before changing anything there.

As for the presubmit check, it's going to be non-trivial
because of the number of exceptions, like file types,
directories and generated code. So, it will have to be
a separate project.
2017-05-06 00:38:56 -07:00
Sugu Sougoumarane ea12c8f0f6 V3: new session-based protocol
This addresses #2723.

This change is essentially a new protocol for V3. Although
backward compatible, it changes the connection model.
Basically, the newer V3 features will work only if you used
the new protocol.

The new model deprecates keyspace_shard, tablet_type and options
from ExecuteRequest and moves them into the Session. This means
that the Session is generally not empty, and may be updated
by any call to Execute or ExecuteBatch, even if the statements are
not transactional. Consequently, transactional methods like Begin,
etc. are deprecated in favor of Execute("begin").

Transaction modes will now be supported by new `SET` syntax, which
will correspondingly update the Session variable.

This also makes a connection that contains a session non-multiplexable.
We'll need to resolve whether it's still worth exposing this flexibility
to the clients, and if so, how.

For now, I've updated the Go driver to use the "modern" protocol.
However, the low level rpc (Impl) continues to support the older
functions like Begin, etc. This allows us to test the legacy
functionality.

All other clients: Python, PHP and Java are currently unchanged.

Given that this is a major protocol change, it hints at a 3.0,
but the changes are 2.1 compatible with the following exceptions:

* Go driver uses the new protocol.
* vtclient binary requires `-target` instead of `-tablet-type`, etc.

All tests are passing including PHP & Java clients. In terms
of upgrade:

* PHP and Java can be upgraded by just updating the code.
* Python will probably require a brand new library. The existing
  vtdb contains way too much baggage, and it may not be worth
  retrofitting this new incompatible protocol onto what's
  currently there. I'm looking for a name. `vtdb2`, `vitessdb`?
2017-04-28 13:08:44 -07:00
wangyipei01 523ff22322 workflow: Add tutorial for resharding workflow in Kubernetes environment. (#2722)
* workflow: Add tutorial for resharding workflow in kubernentes
environment.

* worklow: Add resharding workflow tutorial in kubernetes environment.

* publish site Fri Mar 31 14:51:01 PDT 2017

* workflow: resolve comments.

* publish site Mon Apr  3 11:41:35 PDT 2017
2017-04-03 18:19:01 -07:00
Michael Berlin 612e6f82f2 Merge pull request #2726 from michael-berlin/java_rename
Move all Java code under io.vitess.*.
2017-03-31 16:46:47 -07:00
Michael Berlin a1268cce13 java: Move "example" under io.vitess.* package hierarchy. 2017-03-30 19:51:03 -07:00
thompsonja d2a4135fcb Merge pull request #2727 from thompsonja/k8s
Only add --server to kubectl commands when KUBERNETES_API_SERVER is s…
2017-03-30 19:46:09 -07:00
Joshua Thompson b19c681afd Address comments. 2017-03-30 19:43:18 -07:00
Joshua Thompson cb1e1e63e1 Only add --server to kubectl commands when KUBERNETES_API_SERVER is specified. 2017-03-30 19:18:03 -07:00
Alain Jobart 54d19dd243 Merge pull request #2639 from HubSpot/auth-server-ldap
WIP: AuthServerLdap
2017-03-29 07:56:01 -07:00
Yipei Wang 01aaa2e074 workflow: Add tutorial for resharding workflow for local environment. 2017-03-27 14:17:00 -07:00
Alex Charis 509cae4b8e fix local_example test 2017-03-24 13:11:00 -04:00
Sugu Sougoumarane 5aa4c04f43 mysqlconn: add authServerNone for easier testing
The authServerNone config lets you connect using no username & password
which is convenient for testing and demos. The example demo has been
changed to use this feature. This will allow people to just connect
to vtcombo using mysql and run ad-hoc queries.
2017-03-23 11:32:42 -07:00
davygeek 104bee0199 Add kubernetes api server config (#2603) 2017-03-02 17:34:56 -08:00
thompsonja c97d97e654 Add keyspace parameter to guestbook app. (#2612) 2017-03-01 14:13:14 -08:00
thompsonja 678ea9f449 Add flags to kubernetes guestbook app. (#2605)
Add flags to kubernetes guestbook app.
2017-02-28 14:05:38 -08:00
Alain Jobart 1ff6d011c6 Adding interface for custom auth plugin.
And a config-file based implementation.
Using it in all tests.
2017-02-24 13:13:13 -08:00
Michael Demmer 70913e9c03 Kubernetes examples: Enable MySQL protocol support in vtgate. (#2547)
By default, vtgate listens on the standard MySQL port 3306 for MySQL protocol requests.

This allows to point any MySQL driver to vtgate directly.

Note that this feature is currently in beta and not fully supported and documented yet.
2017-02-10 15:26:53 -08:00
Steve Perkins 22d5d380ca Adds support to VitessJDBCUrl for parsing optional parameter to enable SSL (#2349)
Adds remaining SSL-related optional parameters to JDBC URL strings

Moves JDBC driver layer closer to passing TLS credentials to the gRPC client layer

Hands off TLS from the Java/JDBC layer to the gRPC client layer

Code review updates

First cut at unit testing of SSL connection from gRPC layer to VTGate

Not setting keyManager on gRPC client when client authentication is unused

Adds TLS to client side of Grpc unit test

Adds netty-tcnative to gRPC client classpath, and couple of bugfixes

Fixes bugs... first successful test of SSL connection from client to vtgateclienttest mock.

Adds unit test coverage for SSL connection with client authentication enabled.

Updates 'grpc-all' and 'netty-tcnative' dependencies to latest versions

Improves null-check and handling

Adds optional argument to 'examples/local' scripts, to allow end-to-end testing of TLS connection to VTGate

Builds optional 'fat JAR' of JDBC driver, for use in standalone situation such as DBeaver, SQuirreL, etc.

Adds JavaDocs, and misc cleanup

Temporarily ignores TLS unit tests, to diagnose issue with Travis CI

Redirects IO from external OpenSSL processes to Travis CI console log

Improves logging

Temporarily disables SSL in TLS unit test

Temporarily disables SSL in TLS unit test

Restores disabled unit tests

Testing Travis CI issues

Travis testing

Adds explicit delays to test race condition theory

Travis testing

Testing Travis

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Restores state prior to Travis CI testing

Adds support to VitessJDBCUrl for parsing optional parameter to enable SSL

Adds remaining SSL-related optional parameters to JDBC URL strings

Moves JDBC driver layer closer to passing TLS credentials to the gRPC client layer

Hands off TLS from the Java/JDBC layer to the gRPC client layer

Code review updates

First cut at unit testing of SSL connection from gRPC layer to VTGate

Not setting keyManager on gRPC client when client authentication is unused

Adds TLS to client side of Grpc unit test

Adds netty-tcnative to gRPC client classpath, and couple of bugfixes

Fixes bugs... first successful test of SSL connection from client to vtgateclienttest mock.

Adds unit test coverage for SSL connection with client authentication enabled.

Updates 'grpc-all' and 'netty-tcnative' dependencies to latest versions

Improves null-check and handling

Adds optional argument to 'examples/local' scripts, to allow end-to-end testing of TLS connection to VTGate

Builds optional 'fat JAR' of JDBC driver, for use in standalone situation such as DBeaver, SQuirreL, etc.

Adds JavaDocs, and misc cleanup

Temporarily ignores TLS unit tests, to diagnose issue with Travis CI

Redirects IO from external OpenSSL processes to Travis CI console log

Improves logging

Temporarily disables SSL in TLS unit test

Temporarily disables SSL in TLS unit test

Restores disabled unit tests

Testing Travis CI issues

Travis testing

Adds explicit delays to test race condition theory

Travis testing

Testing Travis

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Restores state prior to Travis CI testing

Adds support to VitessJDBCUrl for parsing optional parameter to enable SSL

Adds remaining SSL-related optional parameters to JDBC URL strings

Moves JDBC driver layer closer to passing TLS credentials to the gRPC client layer

Hands off TLS from the Java/JDBC layer to the gRPC client layer

Code review updates

First cut at unit testing of SSL connection from gRPC layer to VTGate

Not setting keyManager on gRPC client when client authentication is unused

Adds TLS to client side of Grpc unit test

Adds netty-tcnative to gRPC client classpath, and couple of bugfixes

Fixes bugs... first successful test of SSL connection from client to vtgateclienttest mock.

Adds unit test coverage for SSL connection with client authentication enabled.

Updates 'grpc-all' and 'netty-tcnative' dependencies to latest versions

Improves null-check and handling

Adds optional argument to 'examples/local' scripts, to allow end-to-end testing of TLS connection to VTGate

Builds optional 'fat JAR' of JDBC driver, for use in standalone situation such as DBeaver, SQuirreL, etc.

Adds JavaDocs, and misc cleanup

Temporarily ignores TLS unit tests, to diagnose issue with Travis CI

Redirects IO from external OpenSSL processes to Travis CI console log

Improves logging

Temporarily disables SSL in TLS unit test

Temporarily disables SSL in TLS unit test

Restores disabled unit tests

Testing Travis CI issues

Travis testing

Adds explicit delays to test race condition theory

Travis testing

Testing Travis

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Adds more logging

Restores state prior to Travis CI testing

Trying unit tests with larger SSL key size

Increased logging to identify java.security settings

Increased logging to identify java.security settings

Increased logging to identify java.security settings

Increased logging to identify java.security settings

Increase CA cert size to 2048 bits

Increase CA cert size to 2048 bits

Disables basic TLS tests

Re-enables  but disables

Somewhat decouples JDBC TLS unit tests

Ensures that vtgateclienttest mocks terminate after each test suite

Adds delay in unit tests to make sure VTGate has time to start and stop

Adds logging of vtgateclienttest output

Restores  as a subclass of

bugfix

Reverts temporary changes

Trying unit tests with larger SSL key size

Increased logging to identify java.security settings

Increased logging to identify java.security settings

Increased logging to identify java.security settings

Increased logging to identify java.security settings

Increase CA cert size to 2048 bits

Increase CA cert size to 2048 bits

Disables basic TLS tests

Re-enables  but disables

Ensures that vtgateclienttest mocks terminate after each test suite

Adds delay in unit tests to make sure VTGate has time to start and stop

Restores  as a subclass of

bugfix

Reverts temporary changes

Cleanup after git squash

Experimenting with removing sleeps from

Experimenting with removing sleeps from  and
2017-02-02 15:40:27 -08:00
thompsonja 12cf93459e Initial cluster sandbox code. (#2413)
* Initial cluster sandbox code.

BUG=33969078
2017-01-18 09:56:48 -08:00
Alain Jobart ebe7c69af6 Changing default port to not conflict. 2017-01-16 16:21:15 -08:00
Alain Jobart 094184e415 New go native MySQL protocol package.
It implements both client and server side of the MySQL binary protocol.
A few notes:
- SSL is not supported yet. It wouldn't be too hard to add, but our
  current client library code doesn't support it either.
- only the usual mysql native password authentication is supported.
  Again, would be easy to support more.
- vtgate now has the option to listen using the MySQL protocol.
  The local examples are configured to do so.
- The API is not great, but is close to sqldb.Conn. Will be refactored
  once go/mysql is retired.
2017-01-16 10:29:09 -08:00
Alain Jobart 98e7aafeb5 Now the local examples use 'zk2' topology.
Instead of the soon-to-be-deprecated 'zookeeper' one.
2017-01-05 10:51:56 -08:00
Alain Jobart 7af425b096 Various DBConfig fixes.
1. CreadentialsServer was not always used. In particular, schema changes
and backups were not using it. Re-organized the code a bit so it's
easier to see where they're used, and fixed it.

2. mysqlctl and mysqlctld only basically start and stop mysqld. So they
only need the dba user credentials. Fixed the code everywhere for that
to work.

3. the mysql password was passed as a command line parameter to mysql /
mysqladmin / mysql_upgrade, which is not secure (visible via PS, and in
the logs). Now we only pass it when necessary, and we put it in a
temporary config file.
2016-12-06 14:51:36 -08:00
Dalton 310cb4a303 create orchestrator configmap (#2341)
* create orchestrator configmap

Added a Kubernetes configmap to the orchestrator wrapper scripts. Makes it easier to change the default settings while using the default image.

* hotfixes

* Orchestrator up includes the ${script_dir}
* Orchestrator down deletes configmap
* Removed items map
2016-12-06 11:48:57 -08:00
Stefan a762477463 examples/kubernetes: Clean up unnecessary variables in scripts (#2285)
* fixes typo

* Fix: source from env.sh before local param subst

* Removes VITESS_NAME from scripts that source env.sh

env.sh defines VITESS_NAME with a default so for any script
which sources from env.sh, these statements are unreachable.

* Factors CELL var out to env.sh

* Dehardcodes vtctld local cell
2016-11-21 18:12:34 -08:00
sougou cf0c3484c4 parser: support for new NEXT VALUES (#2242)
The new construct allows you to use the `SELECT NEXT N VALUES
FROM t`. Consequently, the increment value in the sequence
table is deprecated (and ignored). We still support the previous
`SELECT NEXT VALUE...` construct, which is now essentially
`SELECT NEXT 1 VALUES...`. Once the new construct is adopted,
we'll deprecate the old one.
The returned value is the first of the N values.
2016-11-14 11:10:51 -08:00
Sugu Sougoumarane bef31d6405 go1.8 compliant functions in vitessdriver
I've left out the row support functions because most of them are
not supported by the protocol. The most important changes
are supported: contexts and named arguments.

* Deleted the deprecated shard specific connection support.
* Built go1.8, fixed compilation errors and added tests.
* Fixed documentation that was still stating support for v1 API.
* Fixed custom_sharding test that was relying on v1 API.
* Doc comments updated to explain isolation levels and named args.
* Code review comments addressed.
2016-11-04 13:28:56 -07:00
Michael Berlin fde2644dc0 examples: Fix building the guestbook Docker image after switching to gRPC 1.0.0. 2016-11-02 15:42:44 -07:00
Anthony Yeh 2523fa230c k8s: Make it easier to use a custom Docker image. (#2206) 2016-11-01 17:10:49 -07:00
Pavel Ivanov ccd8972ec4 Add allprivs user to vttablet flags in k8s and local examples. (#2197) 2016-10-30 18:07:08 -07:00
Pavel Ivanov 60882ece1f Allow to use app2 and workflows in the k8s and local examples. (#2194)
- Add vtctld2 app files into vitess/lite docker image used in the k8s example.
- Add flags to vtctld to use those files to render app2 and allow to use workflows.
- Add the same flags in the local example.
2016-10-30 14:02:21 -07:00
Anthony Yeh ee5ea9e3d8 gcsbackupstorage: Deprecate unused project name flag. (#2137)
The GCS API no longer requires us to specify the project name for any of
the calls we use.
2016-10-11 14:29:23 -07:00
harshit-gangal ea09eb79e8 Ceph changes (#2023)
* Ceph changes

* doc changes
2016-09-07 14:47:34 -07:00
Anthony Yeh 531a030b92 Merge pull request #1831 from youtube/cephImprove
bucket name based on keyspace and shard, removing bucket info from static config file
2016-07-28 19:03:20 -07:00
Alain Jobart a130b4b537 Remove all C deps on zk lib. 2016-07-14 13:03:24 -07:00
Pavel Ivanov 03e370bda0 Fix bug in passing conditional test_flags in vtctld k8s template.
When test_flags were empty that resulted in an empty line in the template.
Kubernetes seem to have special treatment for that and it resulted in a new line
after the -etcd_global_addrs flag before the backup flags. And that resulted in
vtctld not seeing the backup flags at all (new line == next command).
For some reason putting test_flags and backup_flags on the same line in that
order resulted in the same behavior (when test_flags is empty the line has extra
space in the beginning of the line and Kubernetes inserts new line before it),
so I'm reversing their order.

The bug was introduced by https://github.com/youtube/vitess/pull/1809, in the
commit https://github.com/youtube/vitess/commit/3d82402706.
2016-06-29 23:48:05 -07:00
raunak-agarwal-flipkart ab2dd0a828 changed documentation around ceph, removed ceph related scripts and created function for correcting bucket name 2016-06-29 18:35:34 +05:30
raunak-agarwal-flipkart a36c57d138 bucket name based on keyspace and shard, bucket from static config file 2016-06-29 18:01:37 +05:30
Anthony Yeh 53de75e7b1 Merge pull request #1823 from enisoc/vtctld2
vtctld: Add skeleton for Angular 2 UI.
2016-06-27 14:59:50 -07:00
Anthony Yeh 402a3b1c96 vtctld: Add skeleton for Angular 2 UI. 2016-06-27 13:56:30 -07:00
David Slater b0384b9fb1 Merge branch 'master' into shard-organizer 2016-06-24 16:10:20 -07:00
David Slater c894e27f5d Changed the concurrency pattern and refactored the srv_keyspace handler. 2016-06-24 14:49:50 -07:00
thompsonja fa1c762560 Merge pull request #1809 from thompsonja/reparent_test_e2e
Update end2end Reparent test (kubernetes)
2016-06-24 14:40:27 -07:00
Joshua Thompson 3d82402706 Conditionally add -enable_queries to k8s vtctld template if in test mode. Updated comments. 2016-06-24 13:05:31 -07:00
Alain Jobart 6391c07c95 Changing vttest to load vschema from schema_dir.
Each keyspace can have it own vschema file then. This is more consistent
with 'vtctl ApplyVSchema' that takes the same input. Less confusing.
2016-06-14 09:27:08 -07:00
Michael Berlin dc9f880548 examples: Remove *_protocol flags from the examples.
The code defaults to "grpc" for all flags now. Therefore, there's no
point in setting it explicitly to "grpc" anymore.

Only test code is still setting the flag. Verified with:

$ ack-grep -- "-[-a-zA-Z0-9_]+_protocol"
2016-06-13 12:03:05 -07:00
Joshua Thompson 37f39117ca Update e2e reparent test 2016-06-09 11:46:04 -07:00
Alain Jobart 4e490415f3 Deprecating --topology in run_local_database.py. 2016-06-07 08:30:38 -07:00
Anthony Yeh 1129d69282 Merge pull request #1626 from enisoc/orchestrator
Orchestrator Integration
2016-06-06 14:30:49 -07:00