Improve notebook validation (#159)
* Add the ability to test individual notebooks (even if they are blacklisted) to `validate-notebooks` script. * Document `validate-notebooks` script (including blacklisted katas). * Add a section on testing the katas to CONTRIBUTING.md. * Edit testing-oriented tasks of DeutschJozsaAlgorithm kata to pass validation unmodified. * Modify testing harness of GHZGame kata to pass validation.
This commit is contained in:
Mariia Mykhailova
GitHub
Родитель
debff12623
Коммит
5f4dea443a
|
|
@ -8,11 +8,11 @@ We're so glad you asked!
|
|||
|
||||
* [Improving Documentation](#improving-documentation)
|
||||
|
||||
* [Contributing New Tasks](#contributing-new-tasks)
|
||||
* [Contributing New Tasks to the Existing Katas](#contributing-new-tasks-to-the-existing-katas)
|
||||
* [Contributing Code](#contributing-code)
|
||||
* [Improving Existing Katas](#improving-existing-katas)
|
||||
* [Contributing New Katas](#contributing-new-katas)
|
||||
|
||||
* [Style Guide](#style-guide)
|
||||
* [Testing the Katas](#testing-the-katas)
|
||||
* [Style Guide](#style-guide)
|
||||
|
||||
* [Contributor License Agreement](#contributor-license-agreement)
|
||||
|
||||
|
|
@ -43,10 +43,12 @@ This will also ensure that you're not working on the same thing as somebody else
|
|||
|
||||
We're always happy to discuss new ideas and to offer advice, be it on the test harness implementation or on the best breakdown of a topic into tasks.
|
||||
|
||||
#### Contributing New Tasks to the Existing Katas
|
||||
#### Improving Existing Katas
|
||||
|
||||
Each kata is a sequence of tasks on the topic progressing from very simple to quite challenging. If you have an idea for a task which fits nicely in the sequence, filling a gap between other tasks or expanding the sequence with harder tasks, bring it forward!
|
||||
|
||||
Note that most of the katas have a Jupyter Notebook front-end, so if you are modifying a task or adding a new one in the Q# project, you have to update the Jupyter Notebook for this kata as well.
|
||||
|
||||
You are also welcome to browse through the list of issues labeled as ["help wanted"](https://github.com/Microsoft/QuantumKatas/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) and pick up any of them to work on it.
|
||||
|
||||
#### Contributing New Katas
|
||||
|
|
@ -62,6 +64,20 @@ If you want to create a kata for some topic, start by checking the roadmap to se
|
|||
If somebody is already working on this topic, you can try to find them (using the repository issues) and coordinate with them.
|
||||
If the topic you want is not claimed, or is not on the list, go ahead and let us know you'll be working on it by creating an issue.
|
||||
|
||||
### Testing the Katas
|
||||
|
||||
You can use Jupyter Notebook front-end of the kata you're working on to validate the kata (i.e., to check that all tasks have correct reference solutions for them, and that all tests used in the notebook actually exist in the project.
|
||||
|
||||
To validate the kata, use the [`scripts/validate-notebooks.ps1`](../scripts/validate-notebooks.ps1) script.
|
||||
For example, to validate BasicGates kata run the following command from the PowerShell prompt from the root directory of the QuantumKatas project:
|
||||
|
||||
```powershell
|
||||
PS> ./scripts/validate-notebooks.ps1 ./BasicGates/BasicGates.ipynb
|
||||
```
|
||||
|
||||
To use this script, you need to be able to [run Q# Jupyter notebooks locally](https://docs.microsoft.com/quantum/install-guide/jupyter)
|
||||
and to [have PowerShell installed](https://github.com/PowerShell/PowerShell#get-powershell).
|
||||
|
||||
### Style Guide
|
||||
|
||||
* We try to adhere to [the general Q# Style Guide](https://docs.microsoft.com/quantum/contributing/style-guide) in our Q# code.
|
||||
|
|
|
|||
|
|
@ -460,8 +460,9 @@
|
|||
"open Microsoft.Quantum.Diagnostics;\n",
|
||||
"\n",
|
||||
"operation Run_DeutschJozsa_Algorithm () : String {\n",
|
||||
" // You can use EqualityFactB function to check that the return value of DJ_Algorithm operation matches the expected value\n",
|
||||
" EqualityFactB(DJ_Algorithm(4, Oracle_Zero), true, \"f(x) = 0 not identified as constant\");\n",
|
||||
" // You can use the Fact function to check that the return value of DJ_Algorithm operation matches the expected value.\n",
|
||||
" // Uncomment the next line to test the algorithm on the oracle for f(x) = 0.\n",
|
||||
" // Fact(DJ_Algorithm(4, Oracle_Zero), \"f(x) = 0 not identified as constant\");\n",
|
||||
" \n",
|
||||
" // Run the algorithm for the rest of the oracles\n",
|
||||
" // ...\n",
|
||||
|
|
@ -557,10 +558,11 @@
|
|||
"}\n",
|
||||
"\n",
|
||||
"operation Run_BernsteinVazirani_Algorithm () : String {\n",
|
||||
" // Now use AllEqualityFactI to verify the results of the algorithm\n",
|
||||
" if (not AllEqualityFactI(BV_Algorithm(3, Oracle_Zero), [0, 0, 0])) {\n",
|
||||
" return \"Incorrect result for f(x) = 0\";\n",
|
||||
" }\n",
|
||||
" // Now use AllEqualityFactI to verify the results of the algorithm.\n",
|
||||
" // Uncomment the following lines to test the algorithm on the oracle for f(x) = 0.\n",
|
||||
" // if (not AllEqualityFactI(BV_Algorithm(3, Oracle_Zero), [0, 0, 0])) {\n",
|
||||
" // return \"Incorrect result for f(x) = 0\";\n",
|
||||
" // }\n",
|
||||
" \n",
|
||||
" // Run the algorithm on the rest of the oracles\n",
|
||||
" // ...\n",
|
||||
|
|
|
|||
|
|
@ -272,10 +272,10 @@ namespace Quantum.Kata.DeutschJozsaAlgorithm {
|
|||
// Goal: use your implementation of Deutsch-Jozsa algorithm from task 3.1 to test
|
||||
// each of the oracles you've implemented in part I for being constant or balanced.
|
||||
operation DJ_Test () : Unit {
|
||||
// Hint: you will need to use partial application to test ones such as Oracle_Kth_Qubit and Oracle_ProductFunction;
|
||||
// Hint: you will need to use partial application to test oracles such as Oracle_Kth_Qubit and Oracle_ProductFunction;
|
||||
// see task 2.3 for a description of how to do that.
|
||||
|
||||
// Hint: use EqualityFactB function to assert that the return value of DJ_Algorithm operation matches the expected value
|
||||
// Hint: use the Fact function to assert that the return value of DJ_Algorithm operation matches the expected value
|
||||
|
||||
// DJ_Test appears in the list of unit tests for the solution; run it to verify your code.
|
||||
|
||||
|
|
|
|||
|
|
@ -261,9 +261,7 @@
|
|||
"their quantum strategies and given their respective qubits from the entangled triple.\n",
|
||||
"The players have already been told what their starting bits (r, s and t) are.\n",
|
||||
"\n",
|
||||
"**Goal:** Return an array of players' output bits (a, b and c).\n",
|
||||
"\n",
|
||||
"Note that this task uses QuantumStrategy which you've implemented in task 2.2."
|
||||
"**Goal:** Return an array of players' output bits (a, b and c)."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -51,8 +51,7 @@ namespace Quantum.Kata.GHZGame {
|
|||
//////////////////////////////////////////////////////////////////
|
||||
|
||||
// Task 2.1. Entangled triple
|
||||
operation CreateEntangledTriple_Reference (qs : Qubit[]) : Unit
|
||||
is Adj {
|
||||
operation CreateEntangledTriple_Reference (qs : Qubit[]) : Unit is Adj {
|
||||
X(qs[0]);
|
||||
X(qs[1]);
|
||||
|
||||
|
|
|
|||
|
|
@ -118,8 +118,6 @@ namespace Quantum.Kata.GHZGame {
|
|||
// their quantum strategies and given their respective qubits from the entangled triple.
|
||||
// The players have already been told what their starting bits (r, s and t) are.
|
||||
// Goal: Return an array of players' output bits (a, b and c).
|
||||
//
|
||||
// Note that this task uses QuantumStrategy which you've implemented in task 2.2.
|
||||
operation PlayQuantumGHZ (strategies : (Qubit => Bool)[]) : Bool[] {
|
||||
// ...
|
||||
fail "Task 2.3 not implemented yet";
|
||||
|
|
|
|||
|
|
@ -127,7 +127,9 @@ namespace Quantum.Kata.GHZGame {
|
|||
operation T23_PlayQuantumGHZ_Test () : Unit {
|
||||
for (i in 0..1000) {
|
||||
let rst = (RefereeBits())[RandomInt(Length(RefereeBits()))];
|
||||
let strategies = [QuantumStrategy(rst[0], _), QuantumStrategy(rst[1], _), QuantumStrategy(rst[2], _)];
|
||||
let strategies = [QuantumStrategy_Reference(rst[0], _),
|
||||
QuantumStrategy_Reference(rst[1], _),
|
||||
QuantumStrategy_Reference(rst[2], _)];
|
||||
let abc = PlayQuantumGHZ(strategies);
|
||||
EqualityFactB(WinCondition_Reference(rst, abc), true,
|
||||
$"Quantum strategy lost: for rst={rst} the players returned abc={abc}");
|
||||
|
|
|
|||
|
|
@ -108,7 +108,8 @@
|
|||
|
||||
// Hide Jupyter Notebooks artifacts.
|
||||
"**/*.ipynb": true,
|
||||
"**/.ipynb_checkpoints": true
|
||||
"**/.ipynb_checkpoints": true,
|
||||
"**/Check.html": true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ namespace Quantum.Kata.GroversAlgorithm {
|
|||
open Microsoft.Quantum.Arrays;
|
||||
open Microsoft.Quantum.Intrinsic;
|
||||
open Microsoft.Quantum.Canon;
|
||||
open Microsoft.Quantum.Convert;
|
||||
|
||||
|
||||
//////////////////////////////////////////////////////////////////
|
||||
|
|
@ -255,7 +256,7 @@ namespace Quantum.Kata.GroversAlgorithm {
|
|||
oracle(register, output);
|
||||
if (MResetZ(output) == One) {
|
||||
set correct = true;
|
||||
set answer = BoolArrFromResultArr(res);
|
||||
set answer = ResultArrayAsBoolArray(res);
|
||||
}
|
||||
ResetAll(register);
|
||||
} until (correct or iter > 100) // the fail-safe to avoid going into an infinite loop
|
||||
|
|
|
|||
|
|
@ -1,17 +1,36 @@
|
|||
# Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
# Licensed under the MIT License.
|
||||
|
||||
# Script to validate Jupyter notebooks.
|
||||
<#
|
||||
|
||||
.SYNOPSIS
|
||||
This script validates katas represented as Jupyter notebooks.
|
||||
|
||||
.NOTES
|
||||
A kata is validated as follows:
|
||||
* corresponding Jupyter notebook is copied into a "Check.ipynb",
|
||||
* %kata magic is replaced with %check_kata magic that runs
|
||||
the tasks with their reference implementations instead of the stubs.
|
||||
* if the execution of the notebook with %check_kata magic fails,
|
||||
the validation of the kata fails.
|
||||
|
||||
.PARAMETER Notebook
|
||||
Path to the notebook to be validated.
|
||||
If not set, defaults to validating all notebooks in the current directory.
|
||||
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
Param(
|
||||
[Parameter(Position=1)]
|
||||
$Notebook = ""
|
||||
)
|
||||
|
||||
|
||||
& "$PSScriptRoot/install-iqsharp.ps1"
|
||||
$all_ok = $True
|
||||
|
||||
# This function takes a folder with Katas. Copies the corresponding
|
||||
# jupyter notebook into a "Check.ipynb" that replaces the %kata magic
|
||||
# with a %check_kata magic that runs the kata not with the user-supplied code
|
||||
# but with the Reference implementation.
|
||||
# If the execution fails or has warnings, the execution of the notebook
|
||||
# will fail.
|
||||
function validate {
|
||||
function Validate {
|
||||
Param($Notebook)
|
||||
|
||||
# Name of the notebook that will be used for checking katas.
|
||||
|
|
@ -42,26 +61,36 @@ function validate {
|
|||
popd
|
||||
}
|
||||
|
||||
# List of Notebooks that can't be validated as they required user-input
|
||||
# other than the kata answer:
|
||||
|
||||
# List of Notebooks that can't be validated for various reasons:
|
||||
# * Check.ipynb is a validation artifact and not an actual kata notebook.
|
||||
# * CHSH and MagicSquare games require implementing two code cells at once before running the test,
|
||||
# so the first of the cells implemented is guaranteed to fail.
|
||||
# * GraphColoring and SolveSATWithGrover have tasks for which the correct solution fails or times out with relatively high probability.
|
||||
#
|
||||
$not_ready =
|
||||
@(
|
||||
'Check.ipynb',
|
||||
'CHSHGame.ipynb',
|
||||
'DeutschJozsaAlgorithm.ipynb',
|
||||
'GHZGame.ipynb',
|
||||
'GraphColoring.ipynb',
|
||||
'MagicSquareGame.ipynb',
|
||||
'SolveSATWithGrover.ipynb'
|
||||
)
|
||||
|
||||
$errors = $false
|
||||
Get-ChildItem (Join-Path $PSScriptRoot '..') `
|
||||
-Recurse `
|
||||
-Include '*.ipynb' `
|
||||
-Exclude $not_ready `
|
||||
| ForEach-Object { $errors = (validate $_) -or $errors }
|
||||
|
||||
if ($Notebook -ne "") {
|
||||
# Validate only the notebook provided as the parameter (do not exclude blacklisted notebooks)
|
||||
Get-ChildItem $Notebook `
|
||||
| ForEach-Object { $errors = (Validate $_) -or $errors }
|
||||
} else {
|
||||
# Validate all notebooks in the folder from which the script was executed
|
||||
Get-ChildItem (Join-Path $PSScriptRoot '..') `
|
||||
-Recurse `
|
||||
-Include '*.ipynb' `
|
||||
-Exclude $not_ready `
|
||||
| ForEach-Object { $errors = (Validate $_) -or $errors }
|
||||
}
|
||||
|
||||
if (-not $all_ok) {
|
||||
Write-Host "##vso[task.logissue type=error;]Validation errors for Jupyter notebooks."
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче