publish site Wed Dec 14 09:07:01 PST 2016

This commit is contained in:
Sugu Sougoumarane 2016-12-14 09:07:01 -08:00 коммит произвёл sougou
Родитель ba6f368f46
Коммит 5abd5140cf
2 изменённых файлов: 415 добавлений и 7 удалений

Просмотреть файл

@ -22,23 +22,23 @@
<loc>http://vitess.io/user-guide/horizontal-sharding.html</loc>
</url>
<url>
<loc>http://vitess.io/search/</loc>
<loc>http://vitess.io/getting-started/</loc>
</url>
<url>
<loc>http://vitess.io/terms/</loc>
</url>
<url>
<loc>http://vitess.io/about/</loc>
</url>
<url>
<loc>http://vitess.io/search/</loc>
</url>
<url>
<loc>http://vitess.io/overview/</loc>
</url>
<url>
<loc>http://vitess.io/contributing/</loc>
</url>
<url>
<loc>http://vitess.io/getting-started/</loc>
</url>
<url>
<loc>http://vitess.io/terms/</loc>
</url>
<url>
<loc>http://vitess.io/</loc>
</url>
@ -93,6 +93,9 @@
<url>
<loc>http://vitess.io/user-guide/troubleshooting.html</loc>
</url>
<url>
<loc>http://vitess.io/user-guide/twopc.html</loc>
</url>
<url>
<loc>http://vitess.io/user-guide/upgrading.html</loc>
</url>

405
docs/user-guide/twopc.html Normal file
Просмотреть файл

@ -0,0 +1,405 @@
<!DOCTYPE html>
<html lang="en" itemscope itemtype="https://schema.org/WebPage">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Vitess / 2PC Guide</title>
<!-- Webfont -->
<link href='http://fonts.googleapis.com/css?family=Roboto:400,100,100italic,300,300italic,400italic,500,500italic,700,700italic,900,900italic' rel='stylesheet' type='text/css'>
<!--<link rel="shortcut icon" type="image/png" href="/favicon.png">-->
<!-- Bootstrap -->
<link href="/libs/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
<!-- Styles -->
<link rel="stylesheet" type="text/css" href="/css/site.css" />
<!-- Font Awesome icons -->
<link href="/libs/font-awesome-4.4.0/css/font-awesome.min.css"
rel="stylesheet"
type="text/css">
<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script>
<!-- Include all compiled plugins (below), or include individual files as needed -->
<script src="/libs/bootstrap/js/bootstrap.min.js"></script>
<!-- metadata -->
<meta name="og:title" content="Vitess / 2PC Guide"/>
<meta name="og:image" content="/images/vitess_logo_with_border.svg"/>
<meta name="og:description" content="Vitess is a database clustering system for horizontal scaling of MySQL."/>
<link rel="icon" href="/images/vitess_logo_icon_size.png" type="image/png">
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-60219601-1', 'auto');
ga('send', 'pageview');
</script>
</head>
<body class="docs" id="top_of_page">
<span id="toc-depth" data-toc-depth="h2,h3"></span>
<nav id="common-nav" class="navbar navbar-fixed-top inverse">
<div class="container">
<!-- Brand and toggle get grouped for better mobile display -->
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar-collapse-1">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="/">
<img class="logo" src="/images/vitess_logo_with_border.svg" alt="Vitess">
</a>
</div>
<!-- Collect the nav links, forms, and other content for toggling -->
<div class="collapse navbar-collapse" id="navbar-collapse-1">
<ul class="nav navbar-nav navbar-right" id="standard-menu-links">
<li><a href="/overview/">Overview</a></li>
<li><a href="/user-guide/introduction.html">Guides</a></li>
<li><a href="/reference/vitess-api.html">Reference</a></li>
<li><a href="http://blog.vitess.io">Blog</a></li>
<li><a href="https://github.com/youtube/vitess"><i class="fa fa-github"></i> GitHub</a></li>
<!-- Hide link to blog unless we have actual posts -->
<!-- <li><a href="/blog/" title="">Blog</a></li> -->
</ul>
<ul class="nav nav-stacked mobile-left-nav-menu" id="collapsed-left-menu">
<li class="submenu">
<h4 class="arrow-r no-top-margin">Overview</h4>
<ul style="display: none">
<li><a href="/overview/">What is Vitess</a></li>
<li><a href="/overview/scaling-mysql.html">Scaling MySQL with Vitess</a></li>
<li><a href="/overview/concepts.html">Key Concepts</a></li>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">Getting Started</h4>
<ul style="display: none">
<li style="padding-bottom: 0"><a href="/getting-started/">Run Vitess on Kubernetes</a>
<ul style="display: block">
<li style="padding-bottom: 0"><a href="/getting-started/docker-build.html">Custom Docker Build</a></li>
</ul>
</li>
<li><a href="/getting-started/local-instance.html">Run Vitess Locally</a></li>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">User Guide</h4>
<ul style="display: none">
<li><a href="/user-guide/introduction.html">Introduction</a>
<li><a href="/user-guide/client-libraries.html">Client Libraries</a>
<li><a href="/user-guide/backup-and-restore.html">Backing Up Data</a>
<li><a href="/user-guide/reparenting.html">Reparenting</a></li>
<li style="padding-bottom: 0"><a href="/user-guide/schema-management.html">Schema Management</a></li>
<ul style="display: block">
<li style="padding-bottom: 0"><a href="/user-guide/schema-swap.html">Schema Swap (Tutorial)</a></li>
</ul>
<li style="padding-bottom: 0"><a href="/user-guide/sharding.html">Sharding</a>
<ul style="display: block">
<li><a href="/user-guide/horizontal-sharding.html">Horizontal Sharding (Codelab)</a></li>
<li style="padding-bottom: 0"><a href="/user-guide/sharding-kubernetes.html">Sharding in Kubernetes (Codelab)</a></li>
</ul>
</li>
<li><a href="/user-guide/vitess-replication.html">Vitess and Replication</a></li>
<li><a href="/user-guide/topology-service.html">Topology Service</a></li>
<li><a href="/user-guide/transport-security-model.html">Transport Security Model</a></li>
<li style="padding-bottom: 0"><a href="/user-guide/launching.html">Launching</a>
<ul style="display: block">
<li><a href="/user-guide/scalability-philosophy.html">Scalability Philosophy</a></li>
<li><a href="/user-guide/production-planning.html">Production Planning</a></li>
<li><a href="/user-guide/server-configuration.html">Server Configuration</a></li>
<li style="padding-bottom: 0"><a href="/user-guide/troubleshooting.html">Troubleshooting</a></li>
</ul>
</li>
<li><a href="/user-guide/upgrading.html">Upgrading</a></li>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">Reference Guides</h4>
<ul style="display: none">
<li><a href="/reference/vitess-api.html">Vitess API</a>
<li><a href="/reference/vtctl.html">vtctl Commands</a>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">Other Resources</h4>
<ul style="display: none">
<li><a href="/resources/presentations.html">Presentations</a>
<li><a href="http://blog.vitess.io/">Blog</a>
<li><a href="/resources/roadmap.html">Roadmap</a>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">Contributing</h4>
<ul style="display: none">
<li><a href="/contributing/">Contributing to Vitess</a>
<li><a href="/contributing/github-workflow.html">GitHub Workflow</a>
<li><a href="/contributing/code-reviews.html">Code Reviews</a>
</ul>
</li>
<div>
<form method="get" action="/search/">
<input id="search-form" name="q" type="text" placeholder="Search">
</form>
</div>
<li><a href="https://github.com/youtube/vitess" id="collapsed-left-menu-repo-link"><i class="fa fa-github"></i> GitHub</a></li>
</ul>
</div><!-- /.navbar-collapse -->
</div><!-- /.container-fluid -->
</nav>
<div id="masthead">
<div class="container">
<div class="row">
<div class="col-md-9">
<h1>2PC Guide</h1>
</div>
</div>
</div>
</div>
<div class="container">
<div class="row scroll-margin" id="toc-content-row">
<div class="col-md-2" id="leftCol">
<ul class="nav nav-stacked mobile-left-nav-menu" id="sidebar">
<li class="submenu">
<h4 class="arrow-r no-top-margin">Overview</h4>
<ul style="display: none">
<li><a href="/overview/">What is Vitess</a></li>
<li><a href="/overview/scaling-mysql.html">Scaling MySQL with Vitess</a></li>
<li><a href="/overview/concepts.html">Key Concepts</a></li>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">Getting Started</h4>
<ul style="display: none">
<li style="padding-bottom: 0"><a href="/getting-started/">Run Vitess on Kubernetes</a>
<ul style="display: block">
<li style="padding-bottom: 0"><a href="/getting-started/docker-build.html">Custom Docker Build</a></li>
</ul>
</li>
<li><a href="/getting-started/local-instance.html">Run Vitess Locally</a></li>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">User Guide</h4>
<ul style="display: none">
<li><a href="/user-guide/introduction.html">Introduction</a>
<li><a href="/user-guide/client-libraries.html">Client Libraries</a>
<li><a href="/user-guide/backup-and-restore.html">Backing Up Data</a>
<li><a href="/user-guide/reparenting.html">Reparenting</a></li>
<li style="padding-bottom: 0"><a href="/user-guide/schema-management.html">Schema Management</a></li>
<ul style="display: block">
<li style="padding-bottom: 0"><a href="/user-guide/schema-swap.html">Schema Swap (Tutorial)</a></li>
</ul>
<li style="padding-bottom: 0"><a href="/user-guide/sharding.html">Sharding</a>
<ul style="display: block">
<li><a href="/user-guide/horizontal-sharding.html">Horizontal Sharding (Codelab)</a></li>
<li style="padding-bottom: 0"><a href="/user-guide/sharding-kubernetes.html">Sharding in Kubernetes (Codelab)</a></li>
</ul>
</li>
<li><a href="/user-guide/vitess-replication.html">Vitess and Replication</a></li>
<li><a href="/user-guide/topology-service.html">Topology Service</a></li>
<li><a href="/user-guide/transport-security-model.html">Transport Security Model</a></li>
<li style="padding-bottom: 0"><a href="/user-guide/launching.html">Launching</a>
<ul style="display: block">
<li><a href="/user-guide/scalability-philosophy.html">Scalability Philosophy</a></li>
<li><a href="/user-guide/production-planning.html">Production Planning</a></li>
<li><a href="/user-guide/server-configuration.html">Server Configuration</a></li>
<li style="padding-bottom: 0"><a href="/user-guide/troubleshooting.html">Troubleshooting</a></li>
</ul>
</li>
<li><a href="/user-guide/upgrading.html">Upgrading</a></li>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">Reference Guides</h4>
<ul style="display: none">
<li><a href="/reference/vitess-api.html">Vitess API</a>
<li><a href="/reference/vtctl.html">vtctl Commands</a>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">Other Resources</h4>
<ul style="display: none">
<li><a href="/resources/presentations.html">Presentations</a>
<li><a href="http://blog.vitess.io/">Blog</a>
<li><a href="/resources/roadmap.html">Roadmap</a>
</ul>
</li>
<li class="submenu">
<h4 class="arrow-r">Contributing</h4>
<ul style="display: none">
<li><a href="/contributing/">Contributing to Vitess</a>
<li><a href="/contributing/github-workflow.html">GitHub Workflow</a>
<li><a href="/contributing/code-reviews.html">Code Reviews</a>
</ul>
</li>
<div>
<form method="get" action="/search/">
<input id="search-form" name="q" type="text" placeholder="Search">
</form>
</div>
</ul>
</div>
<div class="col-md-3" id="rightCol">
<div class="nav nav-stacked" id="tocSidebar">
<div id="toc"></div>
</div>
</div>
<div class="col-md-7" id="centerCol">
<div id="centerTextCol">
<h1 id="2pc-user-guide">2PC User guide</h1>
<h1 id="overview">Overview</h1>
<p>Vitess 2PC allows you to perform atomic distributed commits. The feature is implemented using traditional MySQL transactions, and hence inherits the same guarantees. With this addition, Vitess can be configured to support the following three levels of atomicty:</p>
<ol>
<li><strong>Single database</strong>: At this level, only single database transactions are allowed. Any transaction that tries to go beyond a single database will be failed.</li>
<li><strong>Multi database</strong>: A transaction can span multiple databases, but the commit will be best effort. Partial commits are possible.</li>
<li><strong>2PC</strong>: This is the same as Multi-database, but the commit will be atomic.</li>
</ol>
<p>2PC commits are more expensive than multi-database because the system has to save away the statements before starting the commit process, and also clean them up after a successful commit. This is the reason why it&#39;s a separate option instead of being always on.</p>
<h2 id="isolation">Isolation</h2>
<p>2PC transactions only guarantee atomicity: Either the whole transaction commits, or it&#39;s rolled back entirely. It does not guarantee ACID Isolation. This means that a third party that performs cross-database reads can observe partial commits while a 2PC transaction is in progress.</p>
<p>Guaranteeing ACID isolation is very contentious and has high costs. Providing it by default would have made vitess impractical for the most common use cases.</p>
<p>However, it is possible for the application to judiciously request ACID isolation where critical: If <code class="prettyprint">SELECT</code>s are performed with <code class="prettyprint">LOCK IN SHARE MODE</code>, then you&#39;re guaranteed that the data will not be modified by anyone else until the transaction is complete.</p>
<h1 id="configuring-vtgate">Configuring VTGate</h1>
<p>The atomicity policy is controlled by the <code class="prettyprint">transaction_mode</code> flag. The default value is <code class="prettyprint">multi</code>, and will set it in multi-database mode. This is the same as the previous legacy behavior.</p>
<p>To enforce single-database transactions, the VTGates can be started by specifying <code class="prettyprint">transaction_mode=single</code>.</p>
<p>To enable 2PC, the VTGates need to be started with <code class="prettyprint">transaction_mode=twopc</code>. The VTTablets will require a few more flags, which will be explained below.</p>
<p>The VTGate <code class="prettyprint">transaction_mode</code> flag decides what to allow. The application can independently request a specific atomicity for each transaction. The request will be honored by VTGate only if it does not exceed what is allowed by the <code class="prettyprint">transaction_mode</code>. For example, <code class="prettyprint">transacion_mode=single</code> will only allow single-db transactions. On the other hand, <code class="prettyprint">transaction_mode=twopc</code> will allow all three levels of atomicity.</p>
<h1 id="driver-apis">Driver APIs</h1>
<p>The way to request atomicity from the application is driver-specific.</p>
<h2 id="go-driver">Go driver</h2>
<p>For the Go driver, you request the atomicity by adding it to the context using the <code class="prettyprint">WithAtomicity</code> function. For more details, please refer to the respective GoDocs.</p>
<h2 id="python-driver">Python driver</h2>
<p>For Python, the <code class="prettyprint">begin</code> function of the cursor has an optional <code class="prettyprint">single_db</code> flag. If the flag is <code class="prettyprint">True</code>, then the request is for a single-db transaction. If <code class="prettyprint">False</code> (or unspecified), then the following <code class="prettyprint">commit</code> call&#39;s <code class="prettyprint">twopc</code> flag decides if the commit is 2PC or Best Effort (<code class="prettyprint">multi</code>).</p>
<h2 id="java-amp-php-todo">Java &amp; PHP (TODO)</h2>
<h2 id="adding-support-in-a-new-driver">Adding support in a new driver</h2>
<p>The VTGate RPC API extends the <code class="prettyprint">Begin</code> and <code class="prettyprint">Commit</code> functions to specify atomicity. The API mimics the Python driver: The <code class="prettyprint">BeginRequest</code> message provides a <code class="prettyprint">single_db</code> flag and the <code class="prettyprint">CommitRequest</code> message provides an <code class="prettyprint">atomic</code> flag which is synonymous to <code class="prettyprint">twopc</code>.</p>
<h1 id="configuring-vttablet">Configuring VTTablet</h1>
<p>The following flags need to be set to enable 2PC support in VTTablet:</p>
<ul>
<li><strong>twopc_enable</strong>: This flag needs to be turned on.</li>
<li><strong>twopc_coordinator_address</strong>: This should specify the address (or VIP) of the VTGate that VTTablet will use to resolve abandoned transactions.</li>
<li><strong>twopc_abandon_age</strong>: This is the time in seconds that specifies how long to wait before asking a VTGate to resolve an abandoned transaction.</li>
</ul>
<p>With the above flags specified, every master VTTablet also turns into a watchdog. If any 2PC transaction is left lingering for longer than <code class="prettyprint">twopc_abandon_age</code> seconds, then VTTablet invokes VTGate and requests it to resolve it. Typically, the <code class="prettyprint">abandon_age</code> needs to be substantially longer than the time it takes for a typical 2PC commit to complete (10s of seconds).</p>
<h1 id="configuring-mysql">Configuring MySQL</h1>
<p>The usual default values of MySQL are sufficient. However, it&#39;s important to verify that <code class="prettyprint">wait_timeout</code> (28800) has not been changed. If this value was changed to be too short, then MySQL could prematurely kill a prepared transaction causing data loss.</p>
<h1 id="monitoring">Monitoring</h1>
<p>A few additional variables have been added to <code class="prettyprint">/debug/vars</code>. Failures described below should be rare. But these variables are present so you can build an alert mechanism if anything were to go wrong.</p>
<h2 id="critical-failures">Critical failures</h2>
<p>The following errors are not expected to happen. If they do, it means that 2PC transactions have failed to commit atomically:</p>
<ul>
<li><strong>InternalErrors.TwopcCommit</strong>: This is a counter that shows the number of times a prepared transaction failed to fulfil a commit request.</li>
<li><strong>InternalErrors.TwopcResurrection</strong>: This counter is incremented if a new master failed to resurrect a previously prepared (and unresolved) transaction.</li>
</ul>
<h2 id="alertable-failures">Alertable failures</h2>
<p>The following failures are not urgent, but require someone to investigate:</p>
<ul>
<li><strong>InternalErrors.WatchdogFail</strong>: This counter is incremented if there are failures in the watchdog thread of VTTablet. This means that the watch dog is not able to alert VTGate of abandoned transactions.</li>
<li><strong>Unresolved.Prepares</strong>: This is a gauge that is set based on the number of lingering Prepared transactions that have been alive for longer than 5x the abandon age. This usually means that a distributed transaction has repeatedly failed to resolve. A more serious condition is when the metadata for a distributed transaction has been lost and this Prepare is now permanently orphaned.</li>
</ul>
<h1 id="repairs">Repairs</h1>
<p>If any of the alerts fire, it&#39;s time to investigate. Once you identify the <code class="prettyprint">dtid</code> or the VTTablet that originated the alert, you can navigate to the <code class="prettyprint">/twopcz</code> URL. This will display three lists:</p>
<ol>
<li><strong>Failed Transactions</strong>: A transaction reaches this state if it failed to commit. The only action allowed for such transactions is that you can discard it. However, you can record the DMLs that were involved and have someone come up with a plan to repair the partial commit.</li>
<li><strong>Prepared Transactions</strong>: Prepared transactions can be rolled back or committed. Prepared transactions must be remedied only if their root Distributed Transaction has been lost or resolved.</li>
<li><strong>Distributed Transactions</strong>: Distributed transactions can only be Concluded (marked as resolved).</li>
</ol>
</div>
</div>
</div>
</div>
<div class="page-spacer"></div>
<footer role="contentinfo" id="site-footer">
<nav role="navigation" class="menu bottom-menu">
<a href="https://groups.google.com/forum/#!forum/vitess" target="_blank">Contact: vitess@googlegroups.com</a>&nbsp;&nbsp;<b>·</b>&nbsp;&nbsp;
<a href="https://groups.google.com/forum/#!forum/vitess-announce" target="_blank">Announcements</a>&nbsp;&nbsp;<b>·</b>&nbsp;&nbsp;
&#169; 2016 <a href="/">Vitess</a> powered by <a href="https://developers.google.com/open-source/">Google Inc</a>
</nav>
</footer>
<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
<!-- <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script> -->
<!-- Include all compiled plugins (below), or include individual files as needed -->
<!--
<script src="/libs/bootstrap/js/bootstrap.min.js"></script>
-->
<!-- Include the common site javascript -->
<script src="/js/common.js" type="text/javascript" charset="utf-8"></script>
</body>
</html>