gecko-dev/taskcluster/docs/caches.rst

44 строки
1.4 KiB
ReStructuredText
Исходник Обычный вид История

.. taskcluster_caches:
=============
Common Caches
=============
There are various caches used by the in-tree tasks. This page attempts to
document them and their appropriate use.
Version Control Caches
======================
Bug 1312475 - Add a version parameter to checkouts cache; r=dustin ff5a4bab0813 (bug 1311791) and 332a08725ed0 (bug 1292071) changed behavior of the VCS caches. First, the store cache / path was merged into the checkouts cache. Then the path of the Mercurial shared store was moved within the cache to always be rooted at the cache root. Caches are shared across tasks. Tasks can execute on any revision configured to use a cache. So, when interacting with caches, it is important to consider how every revision configured to use that cache will interact with it. Take this scenario for example. A worker executes a task where the hg shared store is rooted at /home/worker/checkouts/src/hg-shared. Then the worker executes a task where the hg shared store is rooted at /home/worker/checkouts/hg-store. `hg robustcheckout` will see the checkout from the first task. But then it sees that the store it is pointing to is at an unexpected location (checkouts/src/hg-shared instead of checkouts/hg-store). `hg robustcheckout` aggressively normalizes state to ensure consistency. So when it sees this mismatch, it blows away the checkout and creates one from checkouts/hg-store to replace it. That's a lot of overhead. And this cycle can repeat itself if the right combination of revisions run on the worker! A solution to this problem is to create a clean break from caches when cache semantics change. In TaskCluster, that means using a different cache. This commit introduces a "version" component to the checkouts cache name. By doing so, we create a clean break from all previous caches, ensuring all revisions this point forward won't encounter an hg shared store at an unexpected location. This also paves the road for easily making additional clean breaks in the future. MozReview-Commit-ID: JT8yuULKpch --HG-- extra : rebase_source : 2e5d321e4314ff7543423d3c7837b770b7c8a3a3
2016-10-24 19:58:01 +03:00
``level-{{level}}-checkouts-{{version}}``
This cache holds version control checkouts, each in a subdirectory named
after the repo (e.g., ``gecko``).
Checkouts should be read-only. If a task needs to create new files from
content of a checkout, this content should be written in a separate
directory/cache (like a workspace).
Bug 1312475 - Add a version parameter to checkouts cache; r=dustin ff5a4bab0813 (bug 1311791) and 332a08725ed0 (bug 1292071) changed behavior of the VCS caches. First, the store cache / path was merged into the checkouts cache. Then the path of the Mercurial shared store was moved within the cache to always be rooted at the cache root. Caches are shared across tasks. Tasks can execute on any revision configured to use a cache. So, when interacting with caches, it is important to consider how every revision configured to use that cache will interact with it. Take this scenario for example. A worker executes a task where the hg shared store is rooted at /home/worker/checkouts/src/hg-shared. Then the worker executes a task where the hg shared store is rooted at /home/worker/checkouts/hg-store. `hg robustcheckout` will see the checkout from the first task. But then it sees that the store it is pointing to is at an unexpected location (checkouts/src/hg-shared instead of checkouts/hg-store). `hg robustcheckout` aggressively normalizes state to ensure consistency. So when it sees this mismatch, it blows away the checkout and creates one from checkouts/hg-store to replace it. That's a lot of overhead. And this cycle can repeat itself if the right combination of revisions run on the worker! A solution to this problem is to create a clean break from caches when cache semantics change. In TaskCluster, that means using a different cache. This commit introduces a "version" component to the checkouts cache name. By doing so, we create a clean break from all previous caches, ensuring all revisions this point forward won't encounter an hg shared store at an unexpected location. This also paves the road for easily making additional clean breaks in the future. MozReview-Commit-ID: JT8yuULKpch --HG-- extra : rebase_source : 2e5d321e4314ff7543423d3c7837b770b7c8a3a3
2016-10-24 19:58:01 +03:00
A ``version`` parameter appears in the cache name to allow
backwards-incompatible changes to the cache's behavior.
``level-{{level}}-{{project}}-tc-vcs`` (deprecated)
This cache is used internally by ``tc-vcs``. This tool is deprecated and
should be replaced with ``hg robustcheckout``.
Workspace Caches
================
``level-{{level}}-*-workspace``
These caches (of various names typically ending with ``workspace``)
contain state to be shared between task invocations. Use cases are
dependent on the task.
Other
=====
``tooltool-cache``
Tooltool invocations should use this cache. Tooltool will store files here
indexed by their hash, and will verify hashes before copying files from
this directory, so there is no concern with sharing the cache between jobs
of different levels.