зеркало из https://github.com/microsoft/pybryt.git
added docs
This commit is contained in:
Chris Pyles
Родитель
3136bb0bc0
Коммит
b7cf4ef815
|
|
@ -12,6 +12,7 @@ PyBryt Documentation
|
|||
|
||||
annotations/index
|
||||
reference_implementations
|
||||
student_implementations
|
||||
api_reference
|
||||
|
||||
PyBryt is an open source Python auto-assessment library for teaching and learning. Our goal is to empower students and educators to learn about technology through fun, guided, hands-on content aimed at specific learning goals. PyBryt is designed to work with existing autograding solutions and workflows such as `Otter Grader`_, `OkPy`_, and `Autolab`_.
|
||||
|
|
|
|||
|
|
@ -144,3 +144,28 @@ sequence generator:
|
|||
|
||||
Interacting with Reference Implementations
|
||||
------------------------------------------
|
||||
|
||||
The :py:class:`ReferenceImplementation<pybryt.ReferenceImplementation>` class defines an API for
|
||||
working with reference implementations. The core method for reconciling a student implementation,
|
||||
encoded as a list of 2-tuples, is
|
||||
:py:meth:`ReferenceImplementation.run<pybryt.ReferenceImplementation.run>`. This method is
|
||||
abstracted away by the :py:meth:`StudentImplementation.check<pybryt.StudentImplementation.check>`
|
||||
method, which calls it for that student implementation.
|
||||
|
||||
|
||||
.. _storing_refs:
|
||||
|
||||
Storing Reference Implementations
|
||||
---------------------------------
|
||||
|
||||
Reference implementation objects can be saved to a file by calling
|
||||
:py:meth:`ReferenceImplementation.dump<pybryt.ReferenceImplementation.dump>`, which takes in the
|
||||
path to the file and uses the ``dill`` library to serialize the object. To load a reference
|
||||
implementation, or a list of reference implementations, from a file, use the static method
|
||||
:py:meth:`ReferenceImplementation.load<pybryt.ReferenceImplementation.load>`.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ref = pybryt.ReferenceImplementation([...])
|
||||
ref.dump() # defaults to filename 'reference.pkl'
|
||||
ref = pybryt.ReferenceImplementation.load('reference.pkl')
|
||||
|
|
|
|||
|
|
@ -0,0 +1,87 @@
|
|||
Student Implementations
|
||||
=======================
|
||||
|
||||
For tracking and managing student implementations, stored in PyBryt as a list of 2-tuples of the
|
||||
objects observed while tracing students' code and the timestamps of those objects, PyBryt provides
|
||||
the :py:class:`StudentImplementation<pybryt.StudentImplementation>` class. The constructor for this
|
||||
class takes in either a path to a Jupyter Notebook file or a notebook object read in with
|
||||
``nbformat``.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
stu = pybryt.StudentImplementation("subm.ipynb")
|
||||
|
||||
# or...
|
||||
nb = nbformat.read("subm.ipynb")
|
||||
stu = pybryt.StudentImplementation(nb)
|
||||
|
||||
|
||||
Notebook Execution
|
||||
------------------
|
||||
|
||||
The constructor reads the notebook file and stores the student's code. It then proceeds to execute
|
||||
the student's notebook using ``nbformat``'s ``ExecutePreprocessor``. The memory footprint of the
|
||||
student's code is constructed by executing the notebook with a trace function that tracks every
|
||||
value created and accessed **by the student's code** and the timestamps at which those values were
|
||||
observed.
|
||||
|
||||
To trace into code written in specific files, use the ``addl_filenames`` argument of the constructor
|
||||
to pass a list of absolute paths of files to trace inside of. This can be useful for cases in which
|
||||
a testing harness is being used to test student's code and the student's *actual* submission is
|
||||
written in a Python script (which PyBryt would by default not trace).
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
stu = pybryt.StudentImplementation("harness.ipynb", addl_filenames=["subm.py"])
|
||||
|
||||
PyBryt also employs various custom notebook preprocessors for handling special cases that occur in
|
||||
the code to allow different types of values to be checked. To see the exact version of the code that
|
||||
PyBryt executes, set ``output`` to a path to a notebook that PyBryt will write with the executed
|
||||
notebook. This can be useful e.g. for debugging reference implementations by inserting ``print``
|
||||
statements that show the values at various stages of execution.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
stu = pybryt.StudentImplementation("subm.ipynb", output="executed-subm.ipynb")
|
||||
|
||||
|
||||
Checking Implementations
|
||||
------------------------
|
||||
|
||||
To reconcile a student implementation with a set of reference implementations, use the
|
||||
:py:meth:`StudentImplementation.check<pybryt.StudentImplementation.check>` method, which takes in
|
||||
a single :py:class:`ReferenceImplementation<pybryt.ReferenceImplementation>` object, or a list of
|
||||
them, and returns a :py:class:`ReferenceResult<pybryt.ReferenceResult>` object (or a list of them).
|
||||
This method simply abstracts away managing the memory footprint tracked by the
|
||||
:py:class:`StudentImplementation<pybryt.StudentImplementation>` object and calls the
|
||||
:py:meth:`ReferenceImplementation.run<pybryt.ReferenceImplementation.run>` method for each provided
|
||||
reference implementation.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ref = pybryt.ReferenceImplementation.load("reference.ipynb")
|
||||
stu = pybryt.StudentImplementation("subm.ipynb")
|
||||
stu.check(ref)
|
||||
|
||||
To run the references for a single group of annotations, pass the ``group`` argument, which should
|
||||
be a string that corresponds to the name of a group of annotations. For example, to run the checks
|
||||
for a single question in a reference that contains multiple questions, the pattern might be
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
stu.check(ref, group="q1")
|
||||
|
||||
|
||||
Storing Implementations
|
||||
-----------------------
|
||||
|
||||
Because generating the memory footprints of students' code can be time consuming and computationally
|
||||
expensive, :py:class:`StudentImplementation<pybryt.StudentImplementation>` objects can also be
|
||||
serialized to make multiple runs across sessions easier. The
|
||||
:py:class:`StudentImplementation<pybryt.StudentImplementation>` class provides the
|
||||
:py:class:`dump<pybryt.StudentImplementation.dump>` and
|
||||
:py:class:`load<pybryt.StudentImplementation.load>` methods, which function the same as with
|
||||
:ref:`reference implementations<storing_refs>`.
|
||||
:py:class:`StudentImplementation<pybryt.StudentImplementation>` objects can also be serialized to
|
||||
base-64-encoded strings using the :py:class:`dumps<pybryt.StudentImplementation.dumps>` and
|
||||
:py:class:`loads<pybryt.StudentImplementation.loads>` methods.
|
||||
|
|
@ -113,6 +113,7 @@
|
|||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../reference_implementations.html">Reference Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../student_implementations.html">Student Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../api_reference.html">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
|
|
|
|||
|
|
@ -113,6 +113,7 @@
|
|||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../reference_implementations.html">Reference Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../student_implementations.html">Student Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../api_reference.html">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
|
|
|
|||
|
|
@ -113,6 +113,7 @@
|
|||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../reference_implementations.html">Reference Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../student_implementations.html">Student Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="../api_reference.html">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
<!doctype html>
|
||||
<html class="no-js">
|
||||
<head><meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1"/><link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="prev" title="Reference Implementations" href="reference_implementations.html" />
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1"/><link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="prev" title="Student Implementations" href="student_implementations.html" />
|
||||
|
||||
<meta name="generator" content="sphinx-3.1.2, furo 2021.02.21.beta25"/>
|
||||
<title>API Reference - PyBryt documentation</title>
|
||||
|
|
@ -113,6 +113,7 @@
|
|||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="reference_implementations.html">Reference Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="student_implementations.html">Student Implementations</a></li>
|
||||
<li class="toctree-l1 current current-page"><a class="current reference internal" href="#">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
|
|
@ -812,6 +813,10 @@ present, this is the maximum satisfying timestamp of all child results</p>
|
|||
<code class="sig-name descname">end</code><em class="property">: float</em><a class="headerlink" href="#pybryt.StudentImplementation.end" title="Permalink to this definition">¶</a></dt>
|
||||
<dd></dd></dl>
|
||||
<dl class="py method">
|
||||
<dt id="pybryt.StudentImplementation.load">
|
||||
<em class="property">static </em><code class="sig-name descname">load</code><span class="sig-paren">(</span><em class="sig-param"><span class="n">file</span><span class="p">:</span> <span class="n">str</span></em><span class="sig-paren">)</span> → pybryt.student.StudentImplementation<a class="headerlink" href="#pybryt.StudentImplementation.load" title="Permalink to this definition">¶</a></dt>
|
||||
<dd></dd></dl>
|
||||
<dl class="py method">
|
||||
<dt id="pybryt.StudentImplementation.loads">
|
||||
<em class="property">classmethod </em><code class="sig-name descname">loads</code><span class="sig-paren">(</span><em class="sig-param"><span class="n">data</span><span class="p">:</span> <span class="n">str</span></em><span class="sig-paren">)</span> → pybryt.student.StudentImplementation<a class="headerlink" href="#pybryt.StudentImplementation.loads" title="Permalink to this definition">¶</a></dt>
|
||||
<dd></dd></dl>
|
||||
|
|
@ -836,14 +841,14 @@ present, this is the maximum satisfying timestamp of all child results</p>
|
|||
|
||||
<div class="related-pages">
|
||||
|
||||
<a class="prev-page" href="reference_implementations.html">
|
||||
<a class="prev-page" href="student_implementations.html">
|
||||
<svg><use href="#svg-arrow-right"></use></svg>
|
||||
<div class="page-info">
|
||||
<div class="context">
|
||||
<span>Previous</span>
|
||||
</div>
|
||||
|
||||
<div class="title">Reference Implementations</div>
|
||||
<div class="title">Student Implementations</div>
|
||||
|
||||
</div>
|
||||
</a>
|
||||
|
|
|
|||
|
|
@ -112,6 +112,7 @@
|
|||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="reference_implementations.html">Reference Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="student_implementations.html">Student Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="api_reference.html">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
|
|
@ -302,10 +303,14 @@
|
|||
<td style="width: 33%; vertical-align: top;"><ul>
|
||||
<li><a href="api_reference.html#pybryt.Annotation.limit">limit (Annotation attribute)</a>
|
||||
</li>
|
||||
<li><a href="api_reference.html#pybryt.ReferenceImplementation.load">load() (ReferenceImplementation static method)</a>
|
||||
|
||||
<ul>
|
||||
<li><a href="api_reference.html#pybryt.StudentImplementation.load">(StudentImplementation static method)</a>
|
||||
</li>
|
||||
</ul></li>
|
||||
</ul></td>
|
||||
<td style="width: 33%; vertical-align: top;"><ul>
|
||||
<li><a href="api_reference.html#pybryt.ReferenceImplementation.load">load() (ReferenceImplementation static method)</a>
|
||||
</li>
|
||||
<li><a href="api_reference.html#pybryt.StudentImplementation.loads">loads() (StudentImplementation class method)</a>
|
||||
</li>
|
||||
</ul></td>
|
||||
|
|
|
|||
|
|
@ -113,6 +113,7 @@
|
|||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="reference_implementations.html">Reference Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="student_implementations.html">Student Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="api_reference.html">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
|
|
|
|||
Двоичные данные
docs/html/objects.inv
Двоичные данные
docs/html/objects.inv
Двоичный файл не отображается.
|
|
@ -1,7 +1,7 @@
|
|||
<!doctype html>
|
||||
<html class="no-js">
|
||||
<head><meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1"/><link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="API Reference" href="api_reference.html" /><link rel="prev" title="Invariants" href="annotations/invariants.html" />
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1"/><link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Student Implementations" href="student_implementations.html" /><link rel="prev" title="Invariants" href="annotations/invariants.html" />
|
||||
|
||||
<meta name="generator" content="sphinx-3.1.2, furo 2021.02.21.beta25"/>
|
||||
<title>Reference Implementations - PyBryt documentation</title>
|
||||
|
|
@ -113,6 +113,7 @@
|
|||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Reference Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="student_implementations.html">Student Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="api_reference.html">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
|
|
@ -256,6 +257,25 @@ sequence generator:</p>
|
|||
</div>
|
||||
<div class="section" id="interacting-with-reference-implementations">
|
||||
<h2>Interacting with Reference Implementations<a class="headerlink" href="#interacting-with-reference-implementations" title="Permalink to this headline">¶</a></h2>
|
||||
<p>The <a class="reference internal" href="api_reference.html#pybryt.ReferenceImplementation" title="pybryt.ReferenceImplementation"><code class="xref py py-class docutils literal notranslate"><span class="pre">ReferenceImplementation</span></code></a> class defines an API for
|
||||
working with reference implementations. The core method for reconciling a student implementation,
|
||||
encoded as a list of 2-tuples, is
|
||||
<a class="reference internal" href="api_reference.html#pybryt.ReferenceImplementation.run" title="pybryt.ReferenceImplementation.run"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ReferenceImplementation.run</span></code></a>. This method is
|
||||
abstracted away by the <a class="reference internal" href="api_reference.html#pybryt.StudentImplementation.check" title="pybryt.StudentImplementation.check"><code class="xref py py-meth docutils literal notranslate"><span class="pre">StudentImplementation.check</span></code></a>
|
||||
method, which calls it for that student implementation.</p>
|
||||
</div>
|
||||
<div class="section" id="storing-reference-implementations">
|
||||
<span id="storing-refs"></span><h2>Storing Reference Implementations<a class="headerlink" href="#storing-reference-implementations" title="Permalink to this headline">¶</a></h2>
|
||||
<p>Reference implementation objects can be saved to a file by calling
|
||||
<a class="reference internal" href="api_reference.html#pybryt.ReferenceImplementation.dump" title="pybryt.ReferenceImplementation.dump"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ReferenceImplementation.dump</span></code></a>, which takes in the
|
||||
path to the file and uses the <code class="docutils literal notranslate"><span class="pre">dill</span></code> library to serialize the object. To load a reference
|
||||
implementation, or a list of reference implementations, from a file, use the static method
|
||||
<a class="reference internal" href="api_reference.html#pybryt.ReferenceImplementation.load" title="pybryt.ReferenceImplementation.load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ReferenceImplementation.load</span></code></a>.</p>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">ref</span> <span class="o">=</span> <span class="n">pybryt</span><span class="o">.</span><span class="n">ReferenceImplementation</span><span class="p">([</span><span class="o">...</span><span class="p">])</span>
|
||||
<span class="n">ref</span><span class="o">.</span><span class="n">dump</span><span class="p">()</span> <span class="c1"># defaults to filename 'reference.pkl'</span>
|
||||
<span class="n">ref</span> <span class="o">=</span> <span class="n">pybryt</span><span class="o">.</span><span class="n">ReferenceImplementation</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">'reference.pkl'</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
|
@ -263,12 +283,12 @@ sequence generator:</p>
|
|||
<footer>
|
||||
|
||||
<div class="related-pages">
|
||||
<a class="next-page" href="api_reference.html">
|
||||
<a class="next-page" href="student_implementations.html">
|
||||
<div class="page-info">
|
||||
<div class="context">
|
||||
<span>Next</span>
|
||||
</div>
|
||||
<div class="title">API Reference</div>
|
||||
<div class="title">Student Implementations</div>
|
||||
</div>
|
||||
<svg><use href="#svg-arrow-right"></use></svg>
|
||||
</a>
|
||||
|
|
@ -320,6 +340,7 @@ sequence generator:</p>
|
|||
</ul>
|
||||
</li>
|
||||
<li><a class="reference internal" href="#interacting-with-reference-implementations">Interacting with Reference Implementations</a></li>
|
||||
<li><a class="reference internal" href="#storing-reference-implementations">Storing Reference Implementations</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
|
|
|||
|
|
@ -111,6 +111,7 @@
|
|||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="reference_implementations.html">Reference Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="student_implementations.html">Student Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="api_reference.html">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
|
|
|
|||
Различия файлов скрыты, потому что одна или несколько строк слишком длинны
|
|
@ -0,0 +1,283 @@
|
|||
<!doctype html>
|
||||
<html class="no-js">
|
||||
<head><meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1"/><link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="API Reference" href="api_reference.html" /><link rel="prev" title="Reference Implementations" href="reference_implementations.html" />
|
||||
|
||||
<meta name="generator" content="sphinx-3.1.2, furo 2021.02.21.beta25"/>
|
||||
<title>Student Implementations - PyBryt documentation</title>
|
||||
<link rel="stylesheet" href="_static/styles/furo.css?digest=33d2fc4f3f180ec1ffc6524e273e21d7d58cbe49">
|
||||
<link rel="stylesheet" href="_static/pygments.css">
|
||||
<link media="(prefers-color-scheme: dark)" rel="stylesheet" href="_static/pygments_dark.css">
|
||||
|
||||
|
||||
|
||||
<style>
|
||||
:root {
|
||||
--color-code-background: #eeffcc;
|
||||
--color-code-foreground: black;
|
||||
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--color-code-background: #202020;
|
||||
--color-code-foreground: #d0d0d0;
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
/* For allowing end-user-specific overrides */
|
||||
.override-light {
|
||||
--color-code-background: #eeffcc;
|
||||
--color-code-foreground: black;
|
||||
|
||||
}
|
||||
.override-dark {
|
||||
--color-code-background: #202020;
|
||||
--color-code-foreground: #d0d0d0;
|
||||
|
||||
}
|
||||
</style><link rel="stylesheet" href="_static/styles/furo-extensions.css?digest=d391b54134226e4196576da3bdb6dddb7e05ba2b"></head>
|
||||
<body dir="">
|
||||
|
||||
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
|
||||
<symbol id="svg-toc" viewBox="0 0 24 24">
|
||||
<title>Contents</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
|
||||
stroke-width="1.5" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path stroke="none" d="M0 0h24v24H0z"/>
|
||||
<line x1="4" y1="6" x2="20" y2="6" />
|
||||
<line x1="10" y1="12" x2="20" y2="12" />
|
||||
<line x1="6" y1="18" x2="20" y2="18" />
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-menu" viewBox="0 0 24 24">
|
||||
<title>Menu</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
|
||||
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"
|
||||
class="feather feather-menu">
|
||||
<line x1="3" y1="12" x2="21" y2="12"></line>
|
||||
<line x1="3" y1="6" x2="21" y2="6"></line>
|
||||
<line x1="3" y1="18" x2="21" y2="18"></line>
|
||||
</svg>
|
||||
</symbol>
|
||||
<symbol id="svg-arrow-right" viewBox="0 0 24 24">
|
||||
<title>Expand</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
|
||||
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"
|
||||
class="feather feather-chevron-right">
|
||||
<polyline points="9 18 15 12 9 6"></polyline>
|
||||
</svg>
|
||||
</symbol>
|
||||
</svg>
|
||||
|
||||
<input type="checkbox" class="sidebar-toggle" name="__navigation" id="__navigation">
|
||||
<input type="checkbox" class="sidebar-toggle" name="__toc" id="__toc">
|
||||
<label class="overlay sidebar-overlay" for="__navigation"></label>
|
||||
<label class="overlay toc-overlay" for="__toc"></label>
|
||||
|
||||
|
||||
|
||||
<div class="page">
|
||||
<header class="mobile-header">
|
||||
<div class="header-left">
|
||||
<label class="nav-overlay-icon" for="__navigation">
|
||||
<i class="icon"><svg><use href="#svg-menu"></use></svg></i>
|
||||
</label>
|
||||
</div>
|
||||
<div class="header-center">
|
||||
<a href="index.html"><div class="brand">PyBryt documentation</div></a>
|
||||
</div>
|
||||
<div class="header-right">
|
||||
<label class="toc-overlay-icon toc-header-icon" for="__toc">
|
||||
<i class="icon"><svg><use href="#svg-toc"></use></svg></i>
|
||||
</label>
|
||||
</div>
|
||||
</header>
|
||||
<aside class="sidebar-drawer">
|
||||
<div class="sidebar-container">
|
||||
|
||||
<div class="sidebar-sticky"><a class="sidebar-brand" href="index.html">
|
||||
|
||||
|
||||
<span class="sidebar-brand-text">PyBryt documentation</span>
|
||||
|
||||
</a><form class="sidebar-search-container" method="get" action="search.html">
|
||||
<input class="sidebar-search" placeholder=Search name="q">
|
||||
<input type="hidden" name="check_keywords" value="yes">
|
||||
<input type="hidden" name="area" value="default">
|
||||
</form><div class="sidebar-scroll"><div class="sidebar-tree">
|
||||
<ul class="current">
|
||||
<li class="toctree-l1 has-children"><a class="reference internal" href="annotations/index.html">Annotations</a><input class="toctree-checkbox" id="toctree-checkbox-1" name="toctree-checkbox-1" type="checkbox"/><label for="toctree-checkbox-1"><i class="icon"><svg><use href="#svg-arrow-right"></use></svg></i></label><ul>
|
||||
<li class="toctree-l2"><a class="reference internal" href="annotations/relational_annotations.html">Relational Annotations</a></li>
|
||||
<li class="toctree-l2"><a class="reference internal" href="annotations/invariants.html">Invariants</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="reference_implementations.html">Reference Implementations</a></li>
|
||||
<li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Student Implementations</a></li>
|
||||
<li class="toctree-l1"><a class="reference internal" href="api_reference.html">API Reference</a></li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</aside>
|
||||
<main class="main">
|
||||
<div class="content">
|
||||
<article role="main">
|
||||
<label class="toc-overlay-icon toc-content-icon" for="__toc">
|
||||
<i class="icon"><svg><use href="#svg-toc"></use></svg></i>
|
||||
</label>
|
||||
<div class="section" id="student-implementations">
|
||||
<h1>Student Implementations<a class="headerlink" href="#student-implementations" title="Permalink to this headline">¶</a></h1>
|
||||
<p>For tracking and managing student implementations, stored in PyBryt as a list of 2-tuples of the
|
||||
objects observed while tracing students’ code and the timestamps of those objects, PyBryt provides
|
||||
the <a class="reference internal" href="api_reference.html#pybryt.StudentImplementation" title="pybryt.StudentImplementation"><code class="xref py py-class docutils literal notranslate"><span class="pre">StudentImplementation</span></code></a> class. The constructor for this
|
||||
class takes in either a path to a Jupyter Notebook file or a notebook object read in with
|
||||
<code class="docutils literal notranslate"><span class="pre">nbformat</span></code>.</p>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">stu</span> <span class="o">=</span> <span class="n">pybryt</span><span class="o">.</span><span class="n">StudentImplementation</span><span class="p">(</span><span class="s2">"subm.ipynb"</span><span class="p">)</span>
|
||||
|
||||
<span class="c1"># or...</span>
|
||||
<span class="n">nb</span> <span class="o">=</span> <span class="n">nbformat</span><span class="o">.</span><span class="n">read</span><span class="p">(</span><span class="s2">"subm.ipynb"</span><span class="p">)</span>
|
||||
<span class="n">stu</span> <span class="o">=</span> <span class="n">pybryt</span><span class="o">.</span><span class="n">StudentImplementation</span><span class="p">(</span><span class="n">nb</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="section" id="notebook-execution">
|
||||
<h2>Notebook Execution<a class="headerlink" href="#notebook-execution" title="Permalink to this headline">¶</a></h2>
|
||||
<p>The constructor reads the notebook file and stores the student’s code. It then proceeds to execute
|
||||
the student’s notebook using <code class="docutils literal notranslate"><span class="pre">nbformat</span></code>’s <code class="docutils literal notranslate"><span class="pre">ExecutePreprocessor</span></code>. The memory footprint of the
|
||||
student’s code is constructed by executing the notebook with a trace function that tracks every
|
||||
value created and accessed <strong>by the student’s code</strong> and the timestamps at which those values were
|
||||
observed.</p>
|
||||
<p>To trace into code written in specific files, use the <code class="docutils literal notranslate"><span class="pre">addl_filenames</span></code> argument of the constructor
|
||||
to pass a list of absolute paths of files to trace inside of. This can be useful for cases in which
|
||||
a testing harness is being used to test student’s code and the student’s <em>actual</em> submission is
|
||||
written in a Python script (which PyBryt would by default not trace).</p>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">stu</span> <span class="o">=</span> <span class="n">pybryt</span><span class="o">.</span><span class="n">StudentImplementation</span><span class="p">(</span><span class="s2">"harness.ipynb"</span><span class="p">,</span> <span class="n">addl_filenames</span><span class="o">=</span><span class="p">[</span><span class="s2">"subm.py"</span><span class="p">])</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
<p>PyBryt also employs various custom notebook preprocessors for handling special cases that occur in
|
||||
the code to allow different types of values to be checked. To see the exact version of the code that
|
||||
PyBryt executes, set <code class="docutils literal notranslate"><span class="pre">output</span></code> to a path to a notebook that PyBryt will write with the executed
|
||||
notebook. This can be useful e.g. for debugging reference implementations by inserting <code class="docutils literal notranslate"><span class="pre">print</span></code>
|
||||
statements that show the values at various stages of execution.</p>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">stu</span> <span class="o">=</span> <span class="n">pybryt</span><span class="o">.</span><span class="n">StudentImplementation</span><span class="p">(</span><span class="s2">"subm.ipynb"</span><span class="p">,</span> <span class="n">output</span><span class="o">=</span><span class="s2">"executed-subm.ipynb"</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="checking-implementations">
|
||||
<h2>Checking Implementations<a class="headerlink" href="#checking-implementations" title="Permalink to this headline">¶</a></h2>
|
||||
<p>To reconcile a student implementation with a set of reference implementations, use the
|
||||
<a class="reference internal" href="api_reference.html#pybryt.StudentImplementation.check" title="pybryt.StudentImplementation.check"><code class="xref py py-meth docutils literal notranslate"><span class="pre">StudentImplementation.check</span></code></a> method, which takes in
|
||||
a single <a class="reference internal" href="api_reference.html#pybryt.ReferenceImplementation" title="pybryt.ReferenceImplementation"><code class="xref py py-class docutils literal notranslate"><span class="pre">ReferenceImplementation</span></code></a> object, or a list of
|
||||
them, and returns a <a class="reference internal" href="api_reference.html#pybryt.ReferenceResult" title="pybryt.ReferenceResult"><code class="xref py py-class docutils literal notranslate"><span class="pre">ReferenceResult</span></code></a> object (or a list of them).
|
||||
This method simply abstracts away managing the memory footprint tracked by the
|
||||
<a class="reference internal" href="api_reference.html#pybryt.StudentImplementation" title="pybryt.StudentImplementation"><code class="xref py py-class docutils literal notranslate"><span class="pre">StudentImplementation</span></code></a> object and calls the
|
||||
<a class="reference internal" href="api_reference.html#pybryt.ReferenceImplementation.run" title="pybryt.ReferenceImplementation.run"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ReferenceImplementation.run</span></code></a> method for each provided
|
||||
reference implementation.</p>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">ref</span> <span class="o">=</span> <span class="n">pybryt</span><span class="o">.</span><span class="n">ReferenceImplementation</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s2">"reference.ipynb"</span><span class="p">)</span>
|
||||
<span class="n">stu</span> <span class="o">=</span> <span class="n">pybryt</span><span class="o">.</span><span class="n">StudentImplementation</span><span class="p">(</span><span class="s2">"subm.ipynb"</span><span class="p">)</span>
|
||||
<span class="n">stu</span><span class="o">.</span><span class="n">check</span><span class="p">(</span><span class="n">ref</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
<p>To run the references for a single group of annotations, pass the <code class="docutils literal notranslate"><span class="pre">group</span></code> argument, which should
|
||||
be a string that corresponds to the name of a group of annotations. For example, to run the checks
|
||||
for a single question in a reference that contains multiple questions, the pattern might be</p>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">stu</span><span class="o">.</span><span class="n">check</span><span class="p">(</span><span class="n">ref</span><span class="p">,</span> <span class="n">group</span><span class="o">=</span><span class="s2">"q1"</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="storing-implementations">
|
||||
<h2>Storing Implementations<a class="headerlink" href="#storing-implementations" title="Permalink to this headline">¶</a></h2>
|
||||
<p>Because generating the memory footprints of students’ code can be time consuming and computationally
|
||||
expensive, <a class="reference internal" href="api_reference.html#pybryt.StudentImplementation" title="pybryt.StudentImplementation"><code class="xref py py-class docutils literal notranslate"><span class="pre">StudentImplementation</span></code></a> objects can also be
|
||||
serialized to make multiple runs across sessions easier. The
|
||||
<a class="reference internal" href="api_reference.html#pybryt.StudentImplementation" title="pybryt.StudentImplementation"><code class="xref py py-class docutils literal notranslate"><span class="pre">StudentImplementation</span></code></a> class provides the
|
||||
<a class="reference internal" href="api_reference.html#pybryt.StudentImplementation.dump" title="pybryt.StudentImplementation.dump"><code class="xref py py-class docutils literal notranslate"><span class="pre">dump</span></code></a> and
|
||||
<a class="reference internal" href="api_reference.html#pybryt.StudentImplementation.load" title="pybryt.StudentImplementation.load"><code class="xref py py-class docutils literal notranslate"><span class="pre">load</span></code></a> methods, which function the same as with
|
||||
<a class="reference internal" href="reference_implementations.html#storing-refs"><span class="std std-ref">reference implementations</span></a>.
|
||||
<a class="reference internal" href="api_reference.html#pybryt.StudentImplementation" title="pybryt.StudentImplementation"><code class="xref py py-class docutils literal notranslate"><span class="pre">StudentImplementation</span></code></a> objects can also be serialized to
|
||||
base-64-encoded strings using the <a class="reference internal" href="api_reference.html#pybryt.StudentImplementation.dumps" title="pybryt.StudentImplementation.dumps"><code class="xref py py-class docutils literal notranslate"><span class="pre">dumps</span></code></a> and
|
||||
<a class="reference internal" href="api_reference.html#pybryt.StudentImplementation.loads" title="pybryt.StudentImplementation.loads"><code class="xref py py-class docutils literal notranslate"><span class="pre">loads</span></code></a> methods.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</article>
|
||||
<footer>
|
||||
|
||||
<div class="related-pages">
|
||||
<a class="next-page" href="api_reference.html">
|
||||
<div class="page-info">
|
||||
<div class="context">
|
||||
<span>Next</span>
|
||||
</div>
|
||||
<div class="title">API Reference</div>
|
||||
</div>
|
||||
<svg><use href="#svg-arrow-right"></use></svg>
|
||||
</a>
|
||||
<a class="prev-page" href="reference_implementations.html">
|
||||
<svg><use href="#svg-arrow-right"></use></svg>
|
||||
<div class="page-info">
|
||||
<div class="context">
|
||||
<span>Previous</span>
|
||||
</div>
|
||||
|
||||
<div class="title">Reference Implementations</div>
|
||||
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<div class="related-information">
|
||||
Copyright © 2021, Microsoft
|
||||
|
|
||||
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a>
|
||||
and
|
||||
<a class="muted-link" href="https://pradyunsg.me">@pradyunsg</a>'s
|
||||
<a href="https://github.com/pradyunsg/furo">Furo theme</a>.
|
||||
|
|
||||
<a class="muted-link" href="_sources/student_implementations.rst.txt"
|
||||
rel="nofollow">
|
||||
Show Source
|
||||
</a>
|
||||
</div>
|
||||
|
||||
</footer>
|
||||
</div>
|
||||
<aside class="toc-drawer">
|
||||
|
||||
|
||||
<div class="toc-sticky toc-scroll">
|
||||
<div class="toc-title-container">
|
||||
<span class="toc-title">
|
||||
Contents
|
||||
</span>
|
||||
</div>
|
||||
<div class="toc-tree-container">
|
||||
<div class="toc-tree">
|
||||
<ul>
|
||||
<li><a class="reference internal" href="#">Student Implementations</a><ul>
|
||||
<li><a class="reference internal" href="#notebook-execution">Notebook Execution</a></li>
|
||||
<li><a class="reference internal" href="#checking-implementations">Checking Implementations</a></li>
|
||||
<li><a class="reference internal" href="#storing-implementations">Storing Implementations</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
</aside>
|
||||
</main>
|
||||
</div>
|
||||
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
|
||||
<script src="_static/jquery.js"></script>
|
||||
<script src="_static/underscore.js"></script>
|
||||
<script src="_static/doctools.js"></script>
|
||||
<script src="_static/language_data.js"></script>
|
||||
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
|
||||
<script src="_static/scripts/main.js?digest=e931d09b2a40c1bb82b542effe772014573baf67"></script></body>
|
||||
</html>
|
||||
|
|
@ -12,6 +12,7 @@ PyBryt Documentation
|
|||
|
||||
annotations/index
|
||||
reference_implementations
|
||||
student_implementations
|
||||
api_reference
|
||||
|
||||
PyBryt is an open source Python auto-assessment library for teaching and learning. Our goal is to empower students and educators to learn about technology through fun, guided, hands-on content aimed at specific learning goals. PyBryt is designed to work with existing autograding solutions and workflows such as `Otter Grader`_, `OkPy`_, and `Autolab`_.
|
||||
|
|
|
|||
|
|
@ -144,3 +144,28 @@ sequence generator:
|
|||
|
||||
Interacting with Reference Implementations
|
||||
------------------------------------------
|
||||
|
||||
The :py:class:`ReferenceImplementation<pybryt.ReferenceImplementation>` class defines an API for
|
||||
working with reference implementations. The core method for reconciling a student implementation,
|
||||
encoded as a list of 2-tuples, is
|
||||
:py:meth:`ReferenceImplementation.run<pybryt.ReferenceImplementation.run>`. This method is
|
||||
abstracted away by the :py:meth:`StudentImplementation.check<pybryt.StudentImplementation.check>`
|
||||
method, which calls it for that student implementation.
|
||||
|
||||
|
||||
.. _storing_refs:
|
||||
|
||||
Storing Reference Implementations
|
||||
---------------------------------
|
||||
|
||||
Reference implementation objects can be saved to a file by calling
|
||||
:py:meth:`ReferenceImplementation.dump<pybryt.ReferenceImplementation.dump>`, which takes in the
|
||||
path to the file and uses the ``dill`` library to serialize the object. To load a reference
|
||||
implementation, or a list of reference implementations, from a file, use the static method
|
||||
:py:meth:`ReferenceImplementation.load<pybryt.ReferenceImplementation.load>`.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ref = pybryt.ReferenceImplementation([...])
|
||||
ref.dump() # defaults to filename 'reference.pkl'
|
||||
ref = pybryt.ReferenceImplementation.load('reference.pkl')
|
||||
|
|
|
|||
|
|
@ -0,0 +1,87 @@
|
|||
Student Implementations
|
||||
=======================
|
||||
|
||||
For tracking and managing student implementations, stored in PyBryt as a list of 2-tuples of the
|
||||
objects observed while tracing students' code and the timestamps of those objects, PyBryt provides
|
||||
the :py:class:`StudentImplementation<pybryt.StudentImplementation>` class. The constructor for this
|
||||
class takes in either a path to a Jupyter Notebook file or a notebook object read in with
|
||||
``nbformat``.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
stu = pybryt.StudentImplementation("subm.ipynb")
|
||||
|
||||
# or...
|
||||
nb = nbformat.read("subm.ipynb")
|
||||
stu = pybryt.StudentImplementation(nb)
|
||||
|
||||
|
||||
Notebook Execution
|
||||
------------------
|
||||
|
||||
The constructor reads the notebook file and stores the student's code. It then proceeds to execute
|
||||
the student's notebook using ``nbformat``'s ``ExecutePreprocessor``. The memory footprint of the
|
||||
student's code is constructed by executing the notebook with a trace function that tracks every
|
||||
value created and accessed **by the student's code** and the timestamps at which those values were
|
||||
observed.
|
||||
|
||||
To trace into code written in specific files, use the ``addl_filenames`` argument of the constructor
|
||||
to pass a list of absolute paths of files to trace inside of. This can be useful for cases in which
|
||||
a testing harness is being used to test student's code and the student's *actual* submission is
|
||||
written in a Python script (which PyBryt would by default not trace).
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
stu = pybryt.StudentImplementation("harness.ipynb", addl_filenames=["subm.py"])
|
||||
|
||||
PyBryt also employs various custom notebook preprocessors for handling special cases that occur in
|
||||
the code to allow different types of values to be checked. To see the exact version of the code that
|
||||
PyBryt executes, set ``output`` to a path to a notebook that PyBryt will write with the executed
|
||||
notebook. This can be useful e.g. for debugging reference implementations by inserting ``print``
|
||||
statements that show the values at various stages of execution.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
stu = pybryt.StudentImplementation("subm.ipynb", output="executed-subm.ipynb")
|
||||
|
||||
|
||||
Checking Implementations
|
||||
------------------------
|
||||
|
||||
To reconcile a student implementation with a set of reference implementations, use the
|
||||
:py:meth:`StudentImplementation.check<pybryt.StudentImplementation.check>` method, which takes in
|
||||
a single :py:class:`ReferenceImplementation<pybryt.ReferenceImplementation>` object, or a list of
|
||||
them, and returns a :py:class:`ReferenceResult<pybryt.ReferenceResult>` object (or a list of them).
|
||||
This method simply abstracts away managing the memory footprint tracked by the
|
||||
:py:class:`StudentImplementation<pybryt.StudentImplementation>` object and calls the
|
||||
:py:meth:`ReferenceImplementation.run<pybryt.ReferenceImplementation.run>` method for each provided
|
||||
reference implementation.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ref = pybryt.ReferenceImplementation.load("reference.ipynb")
|
||||
stu = pybryt.StudentImplementation("subm.ipynb")
|
||||
stu.check(ref)
|
||||
|
||||
To run the references for a single group of annotations, pass the ``group`` argument, which should
|
||||
be a string that corresponds to the name of a group of annotations. For example, to run the checks
|
||||
for a single question in a reference that contains multiple questions, the pattern might be
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
stu.check(ref, group="q1")
|
||||
|
||||
|
||||
Storing Implementations
|
||||
-----------------------
|
||||
|
||||
Because generating the memory footprints of students' code can be time consuming and computationally
|
||||
expensive, :py:class:`StudentImplementation<pybryt.StudentImplementation>` objects can also be
|
||||
serialized to make multiple runs across sessions easier. The
|
||||
:py:class:`StudentImplementation<pybryt.StudentImplementation>` class provides the
|
||||
:py:class:`dump<pybryt.StudentImplementation.dump>` and
|
||||
:py:class:`load<pybryt.StudentImplementation.load>` methods, which function the same as with
|
||||
:ref:`reference implementations<storing_refs>`.
|
||||
:py:class:`StudentImplementation<pybryt.StudentImplementation>` objects can also be serialized to
|
||||
base-64-encoded strings using the :py:class:`dumps<pybryt.StudentImplementation.dumps>` and
|
||||
:py:class:`loads<pybryt.StudentImplementation.loads>` methods.
|
||||
Загрузка…
Ссылка в новой задаче