Add templates for Learning Lab courses

.github/workflows/ci.yml

@@ -40,7 +40,7 @@ jobs:
       run: docker login docker.pkg.github.com -u github-actions -p ${{ secrets.GITHUB_TOKEN }}
 
     - name: Build Images & Run Queries
-      run: cd courses/template && ../../scripts/test-course-actual.sh
+      run: cd templates/action && ../../scripts/test-course-actual.sh
   test-courses-template-latest:
     runs-on: ubuntu-latest
     steps:
@@ -48,4 +48,4 @@ jobs:
       uses: actions/checkout@v1
 
     - name: Build Images & Run Queries
-      run: cd courses/template && ../../scripts/test-course-latest.sh
+      run: cd ctemplates/action && ../../scripts/test-course-latest.sh

README.md

@@ -46,7 +46,7 @@ including links to the lines of source code on GitHub when possible:
 There are two main components to any Learning Lab course for CodeQL that uses
 the components in this repository:
 
-* **Query Checking Action:**
+* [**Query Checking Action:**](#creating-the-query-checking-action)
 
   Each course has its own GitHub Action that is designed to be used in workflows
   that run when a course participant pushes new commits to their repo.
@@ -66,7 +66,7 @@ the components in this repository:
   and made available using
   [GitHub Packages](https://github.com/features/packages).
 
-* **Learning Lab Course:**
+* [**Learning Lab Course:**](#creating-the-learning-lab-course)
 
   This is the course itself.
   It creates the initial repository the participant will use for their course,
@@ -150,7 +150,7 @@ we recommend structuring your course folder like so:
 
 *(For your convinience,
 we've created a template course that uses this file-structure
-in the folder [`courses/template`](courses/template).
+in the folder [`templates/action`](templates/action).
 You can simply copy the folder,
 and follow the instructions in the template README for what things to replace).*
 
@@ -264,6 +264,35 @@ ensure that:
   [`.github/workflows/publish.yml`](.github/workflows/publish.yml) to include
   testing and image publishing for your course.
 
+### Creating the Learning Lab Course
+
+If you have not created a Learning Lab course before,
+it is recommended to take the
+[course on creating a course](https://lab.github.com/githubtraining/write-a-learning-lab-course)!
+
+There are core repositories that need to be created as part of any learning-lab
+course:
+
+* **The course repository:**
+  All the course configuration, instructions etc...
+* **The template repository:**
+  The initial contents that populate the repository
+  created on behalf of the course participant.
+  (All courses are taken with respect to it's own repository)
+
+We've created two template directories
+that you can use as a starting point for your own CodeQL Learning Lab Course:
+
+* [`templates/learninglab/course`](templates/learninglab/course)
+* [`templates/learninglab/course-template`](templates/learninglab/course-template)
+
+Simply copy the contents of these templates into their own repositories,
+and follow the [template instructions](templates/learninglab) to get started.
+
+*(Remember that you need to create 2 separate repositories
+for your Learning Lab course,
+they can't be directories in an existing repo).*
+
 ## Example Courses
 
 * [GitHub Security Lab CTF 1: SEGV hunt](courses/cpp/ctf-segv)

templates/learninglab/README.md

@@ -0,0 +1,54 @@
+# CodeQL Learning Lab Course Templates
+
+If you have not created a Learning Lab course before,
+it is recommended to take the
+[course on creating a course](https://lab.github.com/githubtraining/write-a-learning-lab-course)!
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## Usage instructions:
+
+1. Create a repo called `<MY-COURSE-REPO>`,
+   and add the contents of the directory [`course`](course)
+   as the initial contents for that repo.
+1. Create a repo called `<MY-COURSE-REPO>-template`,
+   under the same owner as `<MY-COURSE-REPO>`,
+   and add the contents of the directory [`course-template`](course-template)
+   as the initial contents for that repo.
+1. In `<MY-COURSE-REPO>`:
+   1. Update the value of `META` in `generate-config.js`,
+      and regenerate the `config.yml` by running:
+      ```
+      node generate-config.js > config.yml
+      ```
+   1. Flesh out the details of the course in `course-details.md`
+   1. Add instructions for each of the steps of the course as individual
+      markdown files in `responses/`
+   1. Update the value of `STEPS` in `generate-config.js`
+      to add details of each of the steps that course participants need to do,
+      and regenerate the `config.yml` by running:
+      ```
+      node generate-config.js  > config.yml
+      ```
+    1. Commit your changes and push to the repository
+1. In `<MY-COURSE-REPO>-template`:
+    1. Update `.qlpack` with an appropriate pack name,
+       and the language of the database that queries will be run against.
+    1. Create a `README.md` with e.g. some initial instructions for the user to
+       go to their Issues tab to get more instructions.
+    1. Update `.github.to.move/workflows/action/Dockerfile` to reference the
+       tag of the dockerfile from your [Query Checking Action:](../../README.md#creating-the-query-checking-action)
+1. Add your course to https://lab.github.com
+
+## Current limitations & workarounds
+
+* GitHub Packages can't directly be used by an actions.yml file,
+  see: https://github.com/github/codeql-learninglab-actions/issues/14
+
+* Learning Lab can't currently create courses with Actions workflows,
+  see: https://github.com/github/codeql-learninglab-actions/issues/15
+
+  In the meantime,
+  make sure your course includes instructions on how users should rename
+  `.github.to.move` to `.github` before writing queries.

templates/learninglab/course-template/.github.to.move/workflows/action/Dockerfile

@@ -0,0 +1 @@
+FROM docker.pkg.github.com/github/codeql-learninglab-actions/<course-package>

templates/learninglab/course-template/.github.to.move/workflows/action/action.yml

@@ -0,0 +1,9 @@
+name: 'Check queries'
+description: 'Check that the queries that have been pushed'
+author: 'GitHub <opensource+codeql-learninglab-actions@github.com>'
+runs:
+  using: 'docker'
+  image: 'Dockerfile'
+branding:
+  icon: 'check-circle'
+  color: 'purple'

templates/learninglab/course-template/.github.to.move/workflows/check-queries.yml

@@ -0,0 +1,21 @@
+name: Check Queries
+
+on: [push]
+
+jobs:
+  check-answers:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v1
+
+      # TODO: delete once published action used below
+    - name: Login to docker
+      run: docker login docker.pkg.github.com -u github-actions -p ${{ secrets.GITHUB_TOKEN }}
+
+      # TODO: use published action in github/codeql-learninglab-actions/courses/...
+    - name: Check answers
+      uses: ./.github/workflows/action
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        RUN_ALL: true

templates/learninglab/course-template/.qlpack

@@ -0,0 +1,3 @@
+name: <MY-COURSE-REPO>
+version: 0.0.0
+libraryPathDependencies: codeql-<LANG>

templates/learninglab/course/config.yml

@@ -0,0 +1,95 @@
+# Generated by generate-config.js
+# DO NOT EDIT DIRECTLY
+# Instead, edit generate-config.js and re-run script
+
+title: <MY-COURSE-TITLE>
+tagline: Learn CodeQL in this course
+description: >-
+  Learn CodeQL in this course
+template:
+  repo: <MY-COURSE-REPO>-template
+  name: <MY-COURSE-REPO>
+before:
+  - type: createIssue
+    title: 'Step 1 - Your first query'
+    body: 01_function_definitions.md
+steps:
+
+  - title: "Your first query"
+    description: "Get set up using CodeQL, and run your first query"
+    event: commit_comment.created
+    link: '{{ repoUrl }}/issues/1'
+    actions:
+      # Ensure comment is posted by github-actions
+      - type: gate
+        left: '%payload.sender.login%'
+        operator: ===
+        right: github-actions[bot]
+      # Ensure comment has expected completed string
+      - type: gate
+        left: '%payload.comment.body%'
+        operator: search
+        # regex-escape then yaml-escape the expected markdown string
+        right: "/Results for `step-1\\.ql`\\: \\*\\*correct\\*\\* \\(3 results\\)/"
+
+      # Answer is correct!!
+
+      # Create Issue for next task
+      - type: createIssue
+        title: "Step 2 - alloca definition"
+        body: 02_alloca_definition.md
+        action_id: next_issue
+
+      # Make comment on current issue with link to commit that introduces correct query
+      - type: respond
+        issue: "Step 1 - Your first query"
+        with: next.md
+        data:
+          next_issue: '%actions.next_issue.data.html_url%'
+          commit: '%payload.comment.commit_id%'
+
+      # Make comment on commit with link to next issue
+      - type: octokit
+        method: repos.createCommitComment
+        owner: '%payload.repository.owner.login%'
+        repo: '%payload.repository.name%'
+        sha: '%payload.comment.commit_id%'
+        body: |
+          Congratulations, looks like the query you introduced for step 1 finds the correct results!
+
+          Take a look at the [instructions for the next step](%actions.next_issue.data.html_url%) to continue.
+
+      # Close current issue
+      - type: closeIssue
+        issue: "Step 1 - Your first query"
+
+  - title: "alloca definition"
+    description: "write a query that looks for definitions of a macro"
+    event: commit_comment.created
+    link: '{{ repoUrl }}/issues'
+    actions:
+      # Ensure comment is posted by github-actions
+      - type: gate
+        left: '%payload.sender.login%'
+        operator: ===
+        right: github-actions[bot]
+      # Ensure comment has expected completed string
+      - type: gate
+        left: '%payload.comment.body%'
+        operator: search
+        # regex-escape then yaml-escape the expected markdown string
+        right: "/Results for `step-2\\.ql`\\: \\*\\*correct\\*\\* \\(5 results\\)/"
+
+      # Answer is correct!!
+
+
+      # Make comment on current issue with final message
+      - type: respond
+        issue: "Step 2 - alloca definition"
+        with: end.md
+        data:
+          commit: '%payload.comment.commit_id%'
+
+      # Close current issue
+      - type: closeIssue
+        issue: "Step 2 - alloca definition"

templates/learninglab/course/course-details.md

@@ -0,0 +1,3 @@
+This is the markdown file presented on the course page on lab.github.com
+
+The first paragraph uses a larger font, so make it eye-catching!

templates/learninglab/course/generate-config.js

@@ -0,0 +1,225 @@
+/*
+ * Helper script to generate config.yml for CodeQL Courses
+ *
+ * To regenerate config.yml, run this in a shell:
+ * node generate-config.js  > config.yml
+ */
+
+// @ts-check
+
+/**
+ * Core metadata that is output at the start of config.yml
+ */
+const META = `
+title: <MY-COURSE-TITLE>
+tagline: Learn CodeQL in this course
+description: >-
+  Learn CodeQL in this course
+template:
+  repo: <MY-COURSE-REPO>-template
+  name: <MY-COURSE-REPO>
+`;
+
+/**
+ * File in `responses/` that is used as the content for comments that are posted
+ * when a user finishes a step.
+ *
+ * Makes use of the placeholders:
+ * * `{{next_issue}}` - URL to the issue that details the next set of instructions
+ * * `{{commit}}` - Sha that was the head of the most recent PUSH
+ */
+const NEXT_MESSAGE = 'next.md';
+/**
+ * File in `responses/` that is used as the content for the comment that is
+ * posted when a user finishes the last step.
+ *
+ * Makes use of the placeholders:
+ * * `{{commit}}` - Sha that was the head of the most recent PUSH
+ */
+const END_MESSAGE = 'end.md';
+
+/**
+ * Each of the steps of this course, each step must have:
+ *
+ * * `queryFile` - The file path of the query that the user needs to write for
+ *   this step.
+ * * `expectedResults` - The number of results this query should produce
+ * * `instructionsFile` - The file in the `responses/` directory that detail the
+ *   instructions that must be followed for this user to successfully write the
+ *   query.
+ * * `title` - A title for this step, used in the issue title and
+ *   Learning Lab UI
+ * * `description` - A description of this step to display in the
+ *   Learning Lab UI
+ *
+ * @type Array<{
+ *   queryFile: string;
+ *   expectedResults: number;
+ *   instructionsFile: string;
+ *   title: string;
+ *   description: string;
+ * }>
+ */
+const STEPS = [
+    {
+    queryFile: 'step-1.ql',
+    expectedResults: 3,
+    instructionsFile: 'step-1.md',
+    title: 'Your first query',
+    description: 'Write your first query'
+  },
+  {
+    queryFile: 'step-2.ql',
+    expectedResults: 5,
+    instructionsFile: 'step-2.md',
+    title: 'Your second query',
+    description: 'Write your second query'
+  }
+];
+
+
+console.log(`
+# Generated by generate-config.js
+# DO NOT EDIT DIRECTLY
+# Instead, edit generate-config.js and re-run script
+`.trim() + '\n');
+console.log(META.trim());
+
+/**
+ * @param step {{title: string;}}
+ * @param i {number}
+ */
+const issueTitle = (step, i) => `Step ${i + 1} - ${step.title}`;
+
+/**
+ * @param str {string}
+ */
+const escapeRegExp = (str) =>
+  str.replace(/[.*+?^${}()|[\]\:\\]/g, '\\$&');
+
+// For some reason, escaping `'` in double-quoted yaml strings
+// doesn't work on learning lab, as it throws the exception
+// "unknown escape sequence". So we should explicitly only escape double quotes
+// and backslashes
+const escapeDoubleQuoteYamlString = (str) =>
+  str.replace(/[\"\\]/g, '\\$&');
+
+console.log(`
+before:
+  - type: createIssue
+    title: '${issueTitle(STEPS[0], 0)}'
+    body: ${STEPS[0].instructionsFile}
+steps:
+`.trim());
+
+STEPS.map((step, i) => {
+  // The markdown string to look for in the comment from github-actions[bot]
+  const expectedString = `Results for \`${step.queryFile}\`: **correct** (${step.expectedResults} result${step.expectedResults === 1 ? '' : 's'})`;
+  console.log(`
+  - title: "${escapeDoubleQuoteYamlString(step.title)}"
+    description: "${escapeDoubleQuoteYamlString(step.description)}"
+    event: commit_comment.created
+    link: '{{ repoUrl }}/issues${i === 0 ? '/1' : ''}'
+    actions:
+      # Ensure comment is posted by github-actions
+      - type: gate
+        left: '%payload.sender.login%'
+        operator: ===
+        right: github-actions[bot]
+      # Ensure comment has expected completed string
+      - type: gate
+        left: '%payload.comment.body%'
+        operator: search
+        # regex-escape then yaml-escape the expected markdown string
+        right: "/${escapeDoubleQuoteYamlString(escapeRegExp(expectedString))}/"
+
+      # Answer is correct!!`);
+
+  /* The following is disabled for now as Learning Lab is using ^15.18.3 of
+   * octokit/rest.js, and listBranchesForHeadCommit was released in version
+   * v16.24.1
+   */
+  // console.log(`
+  //     # If there is a PR, merge it!
+  //     - type: octokit
+  //       method: repos.listBranchesForHeadCommit
+  //       owner: '%payload.repository.owner.login%'
+  //       repo: '%payload.repository.name%'
+  //       commit_sha: '%payload.comment.commit_id%'
+  //       action_id: get_branches
+  //     - type: gate
+  //       left: '%actions.get_branches.length%'
+  //       operator: '!=='
+  //       right: 1
+  //       required: false
+  //       else:
+  //         # Executes when there is 1 matching branch
+  //         - type: octokit
+  //           method: api.pulls.list
+  //           owner: '%payload.repository.owner.login%'
+  //           repo: '%payload.repository.name%'
+  //           head: '%payload.repository.owner.login%:%actions.get_branches.1.name%'
+  //           action_id: get_prs
+  //         - type: gate
+  //           left: '%actions.get_prs.length%'
+  //           operator: '!=='
+  //           right: 1
+  //           required: false
+  //           else:
+  //             # Executes when there is 1 matching pr
+  //             - type: octokit
+  //               method: api.pulls.merge
+  //               owner: '%payload.repository.owner.login%'
+  //               repo: '%payload.repository.name%'
+  //               pull_number: '%actions.get_prs.1.number%'
+  //               required: false
+  // `);
+
+  if (i < STEPS.length - 1) {
+    const next = STEPS[i + 1];
+    // Next Step
+    console.log(`
+      # Create Issue for next task
+      - type: createIssue
+        title: "${escapeDoubleQuoteYamlString(issueTitle(next, i + 1))}"
+        body: ${next.instructionsFile}
+        action_id: next_issue
+
+      # Make comment on current issue with link to commit that introduces correct query
+      - type: respond
+        issue: "${escapeDoubleQuoteYamlString(issueTitle(step, i))}"
+        with: ${NEXT_MESSAGE}
+        data:
+          next_issue: '%actions.next_issue.data.html_url%'
+          commit: '%payload.comment.commit_id%'
+
+      # Make comment on commit with link to next issue
+      - type: octokit
+        method: repos.createCommitComment
+        owner: '%payload.repository.owner.login%'
+        repo: '%payload.repository.name%'
+        sha: '%payload.comment.commit_id%'
+        body: |
+          Congratulations, looks like the query you introduced for step ${i + 1} finds the correct results!
+
+          Take a look at the [instructions for the next step](%actions.next_issue.data.html_url%) to continue.
+
+      # Close current issue
+      - type: closeIssue
+        issue: "${escapeDoubleQuoteYamlString(issueTitle(step, i))}"`);
+  } else {
+    // End of course
+    console.log(`
+
+      # Make comment on current issue with final message
+      - type: respond
+        issue: "${escapeDoubleQuoteYamlString(issueTitle(step, i))}"
+        with: ${END_MESSAGE}
+        data:
+          commit: '%payload.comment.commit_id%'
+
+      # Close current issue
+      - type: closeIssue
+        issue: "${escapeDoubleQuoteYamlString(issueTitle(step, i))}"`);
+  }
+})

templates/learninglab/course/responses/end.md

@@ -0,0 +1 @@
+Congratulations, you have finished the course!

templates/learninglab/course/responses/next.md

@@ -0,0 +1,3 @@
+Congratulations, looks like the query you introduced in {{commit}} finds the correct results!
+
+Take a look at the [instructions for the next step]({{next_issue}}) to continue.

templates/learninglab/course/responses/step-1.md

@@ -0,0 +1 @@
+These are instructions for step 1

templates/learninglab/course/responses/step-2.md

@@ -0,0 +1 @@
+These are instructions for step 1