Merge pull request #32 from microsoft/update-doc

Updated documentation guides

docs/CONFIGURATION_GUIDE.md

@@ -1,19 +1,32 @@
-## Configuration Guide
+# Configure the tool
 
-The following guide shows the steps to configure Moodle LMS to work with the Assessment App. Please, check if you obtained the following parameters from the Platform Registration Tool:
+The following guide shows the steps to configure several popular LMS to work with the Assessment Application. If your LMS is not listed here, consult your LMS vendor on how to configure LTI application. Regardless of the LMS, the typical workflow should remain the same:
 
-* Login URL
-* Launch URL
-* Domain URL
-* Public Key
-* Public JWK
-* Public JWK Set URL
+1. Obtain parameters from the deployed  LTI Assessment Application’s registration page
+2. Configure an LTI tool on the LMS using the parameters from step 1.
+3. Obtain parameters from the configured LTI tool.
+4. Configure the LTI Assessment Application using the parameters from step 3.
+
+By now, you should've obtained the following parameters from the LTI Assessment Application’s registration page. If not, follow the [deployment guide](./DEPLOYMENT_GUIDE.md) to deploy Assessment Application LTI application and obtain the following parameters from the registration page.
+
+- Login URL
+- Launch URL
+- Domain URL
+- Public Key
+- Public JWK
+- Public JWK Set URL
 
 If you are not the one who deployed the application, you need to obtain the parameters from that person.
 
-Complete the following steps to configure the Assessment App in Moodle:
+The configuration steps slightly differ depending on the LMS you are using. In general, they will involve registering the Assessment Application as an external tool in the LMS and registering the parameters of external tool back in the Assessment Application's registration page. The following examples show how to configure Assessment Application with three of the popular LMS.
 
-### Moodle LTI 1.3
+- [Moodle](#Moodle-LMS)
+- [Canvas](#Canvas-LMS)
+
+
+## Moodle LMS
+
+LTI 1.3
 
 1. Open your LMS and sign in with the admin account.
 2. Click **Site administration** from the left navigation pane.
@@ -86,3 +99,89 @@ The following steps show how to register the parameters back in the Platform Reg
 4. Click **Save Registration**.
 
 The Assessment App is now configured on Moodle LMS and educators will be able to use it.
+
+
+## Canvas LMS
+
+The following steps show how to configure an LTI tool on a Canvas LMS.
+
+### LTI 1.1
+
+At this time, we do not support LTI 1.1 with Canvas LMS.
+ 
+### LTI 1.3
+
+The LTI 1.3 and LTI Advantage platform requires a tool to be initially configured in the Developer Keys page, followed by being added to an account or course. First, configure the tool in the Developer Keys page.
+
+1. Open your LMS and sign in with the admin account (Users who want to manage Developer Keys must have the **Developer Keys - manage** permission).
+2. Click **Admin** from the left navigation pane, then click the name of the account.
+3. Click **Developer Keys**.
+4. Click **+Developer Key** and click **+LTI Key**.
+5. Enter the following information:
+ * **Key Name**: give the tool a name of your choice. For example: "Assessment Application".
+ * **Redirection URIs**: enter the "Launch URL" from Assessment application’s registration page. 
+ * **Method**: select **Manual Entry**
+ * **Title**: give the tool a title.
+ * **Description**: give the tool a description.
+ * **Target Link URI**: enter the "Launch URL" from Assessment application’s registration page.
+ * **OpenID Connect Initiation URI**: enter "Login URL" from the Assessment application’s registration page.
+ * **JWK Method**: select **Public JWK URL**
+ * We recommend to select the **Public JWK URL** as **JWK Method**. 
+ * If you are trying to connect to a locally hosted Assessment Application, you will need to use the **JWK Method** method. 
+ * **Public JWK URL**: enter "Public JWK Set URL" from the Assessment Application’s registration page.
+ * If you select **Public JWK**, instead of **Public JWK URL**, as **JWK Method**, you can enter the "Public JWK" from the Assessment Application's registration page instead of the "Public JWK Set URL".
+![Config.Canvas.2](/images/Config.Canvas.2.png). 
+6. Under **LTI Advantage Services**, enable the following options:
+ * Can create and view assignment data in the gradebook associated with the tool.
+ * Can view assignment data in the gradebook associated with the tool.
+ * Can view submission data for assignments associated with the tool.
+ * Can create and update submission results for assignments associated with the tool.
+ * Can retrieve user data associated with the context the tool is installed in.
+ * Can lookup Account information
+ * Can list categorized event types.
+![Config.Canvas.3](/images/Config.Canvas.3.png) 
+7. Under **Additional Settings**, select the **Privacy Level** as **PUBLIC**.
+![Config.Canvas.4](/images/Config.Canvas.4.png) 
+8. Under **Placements**, make sure **Link Selection** and **Assignment Selection** are selected.
+![Config.Canvas.5](/images/Config.Canvas.5.png) 
+9. Click **Save**. The key should now appear and listed with the name you provided. 
+10. Ensure that the newly added key is set to **Enabled**.
+11. Ensure to uncheck the "eye" for the key under the "Actions", as it allows the developer key to be used. 
+11. Take note of the following parameters:
+ * **Client ID**: the number in the **Details** column, above the **Show Key** button
+![Config.Canvas.6](/images/Config.Canvas.6.png) 
+
+At the account level, external tools must be installed in the External Apps page in Account Settings. The Assessment Application can be added via the Client ID option. Only the Client ID is required to be added.
+
+1. Click **Settings** from the left navigation pane.
+2. Click **View App Configurations**.
+3. Click **+App**.
+4. Enter the following information:
+ * **Configuration Type**: select **By Client ID**
+ * **Client ID**: enter the "Client ID" from the LTI key registration.
+![Config.Canvas.7](/images/Config.Canvas.7.png) 
+5. Click **Submit**.
+6. If the Client ID is associated with an external tool, the tool name displays in the page. The page also confirms the tool should be installed.
+![Config.Canvas.8](/images/Config.Canvas.8.png) 
+7. Click **Install**.
+8. Repeat the above installation steps to install the application to every course you wish to use it in. 
+9. Continue to configure the Assessment application, by registering the parameters back in the Assessment application's registration page.
+
+The following steps show how to register the parameters back in the Assessment application's registration page. If you are not the one who deployed the application, you need to provide these parameters to that person.
+
+1. Open the tool registration page from your browser.
+2. Enter the following information:
+  * **Display name**: give the tool a name of your choice. 
+  * **Issuer**: enter **https://canvas.instructure.com** (this should always be canvas.infrastructure.com not matter your tenant url)
+  * **JWK Set URL**: enter https://[tenant-name].instructure.com/api/lti/security/jwks
+  * **Access Token URL**: enter https://[tenant-name].instructure.com/login/oauth2/token 
+  * **Authorization URL**: enter https://[tenant-name].instructure.com/api/lti/authorize_redirect 
+   NOTE: [tenant-name] is where your Canvas tenant name hosted by instructure. For example if the url of the LMS is https://canvas1.instructure.com, then the [tenant-name] is "canvas1". If you are using self-hosted Canvas, replace https://[tenant-name].instructure.com with your canvas URL.
+  * **Client ID**: enter "Client ID" from the LTI key registration.
+  * **Audience**: This can be left blank 
+3. Optionally, you can add your Institution name and logo on the registration page.
+4. Click **SAVE REGISTRATION**.
+
+You're all set. The Assessment tool is now configured on your Canvas LMS and your Educators will be able to use it within their courses. Follow the [educator guide](./USER_GUIDE.md) to create assignments that use the Assessment Application tool.
+
+

docs/EDUCATOR_GUIDE.md

@@ -1,14 +1,49 @@
 # Educator Guide
 
-This guide is designed for educators who use [Moodle LMS](https://moodle.org/).
+This guide is designed for educators who use [Moodle LMS](https://moodle.org/) and [Canvas LMS](https://learn.canvas.net/login/canvas).
 
 ## Table of content
 1. [How to create an assessment on Moodle using the Assessment App](#create-assessment)
-2. [How to set up an assessment in the Assessment App](#assessment-setup)
-3. [How to create a new question bank](#create-question-bank)
-4. [How to create a new question](#create-question)
-5. [How to import and export question banks](#import-export-question-banks)
-6. [How to browse Assessment Analytics](#assessment-analytics)
+2. [How to create an assessment on Canvas using the Assessment App](#create-assessment-canvas)
+3. [How to set up an assessment in the Assessment App](#assessment-setup)
+4. [How to create a new question bank](#create-question-bank)
+5. [How to create a new question](#create-question)
+6. [How to import and export question banks](#import-export-question-banks)
+7. [How to browse Assessment Analytics](#assessment-analytics)
+
+## How to create an assessment on Canvas using the Assessment App <a name="create-assessment-canvas"/>
+
+To begin, you need to login to Canvas LMS and follow the steps:
+
+1. Pick the course where you would like to create an assessment.
+
+![createassessment](../images/Educator.Canvas.1.PNG)
+
+2. Verify that the Assessment Application has been added as an external tool within this course by going to settings. If it hasn't, request the LMS admin to follow the [configuration guide](../docs/CONFIGURATION_GUIDE.md)
+
+![verification](../images/Educator.Canvas.7.PNG)
+
+3. Go to the Assignments section of the course and click **+ Assignment**.
+
+![assignments](../images/Educator.Canvas.2.PNG)
+
+4. Enter title to the assignment.
+
+![addtitle](../images/Educator.Canvas.3.PNG)
+
+5. Change **Points** to 100. Change **Display Grade as** to Percentage. Verify **Assignment Group** is set to Assignments
+
+![assessmenttool](../images/Educator.Canvas.4.PNG)
+
+6. Change the submission type to **External Tool**. Click on **Find**. Select the assessment application from the list of external tools. Tick the **Load This Tool In A New Tab** checkbox. Click on **Save**.
+
+![turnoffediting](../images/Educator.Canvas.5.PNG)
+
+7. Click **Save and Publish**.
+
+![createdassessment](../images/Educator.Canvas.6.PNG)
+
+The assessment should be created. You will be redirected to the assessment page, where you have to click on "Load Assessment In A New Window".  This will direct you to the Assessment App, where you can configure the created assessment: choose the deadline, assessment duration, questions and participants.
 
 ## How to create an assessment on Moodle using the Assessment App <a name="create-assessment"/>
 
@@ -118,15 +153,21 @@ Here is an example of how the downloaded question bank might look like:
     "questions":[
         {"name":"Applications of machine learning",
         "description":"Applications of machine learning are all around us",
-        "options":["True","False"],"answer":0},
+        "options":["True","False"],"answer":["0"],"textType": "text", "questionType":"MCQ"
         {"name":"Machine learning algorithms",
         "description":"Machine learning algorithms are meant to simulate",
         "options":["intelligent machines","the human brain","orangutans"],
-        "answer":1},
+        "answer":["0"], "textType": "text", "questionType":"MCQ"},
         {"name":"Example of a classical ML technique?",
         "description":"What is an example of a classical ML technique?",
         "options":["natural language processing","deep learning","neural networks"],
-        "answer":0},
+        "answer":["0"], "textType": "text", "questionType":"MCQ"},
+        {"name": "Examples of machine language techniques", 
+        "description": "Which of the following are machine language techniques", 
+        "options": ["Linear Regression", "Log Functions", "Random Forest"], 
+        "answer": ["0", "2"], 
+        "textType": "text", 
+        "questionType": "MCQ"}
     "assessmentType":"Quiz"
 }]
 ```
@@ -138,7 +179,48 @@ Click **Upload** on the top of the Home page. You will see a dialog window where
 ![selectqbtoupload](../images/selectqbtoupload.png)
 
 The question bank will appear in the list of question banks. 
-Please, note that currently, the Assessment App supports question banks only in JSON format. 
+There are currently four different question bank formats that are supported by the Assessment Application. 
+
+1. QTI format - This format is questions banks being imported from Canvas, which can be exported in QTI format. More information can be found [here](https://www.imsglobal.org/activity/qtiapip#:~:text=QTI%20is%20an%20open%20format,2.2%20and%20APIP%20version%201.1.)
+2. GIFT format - This format is for question banks being imported from Moodle, which can be exported in GIFT format. More information can be found [here](https://docs.moodle.org/311/en/GIFT_format#:~:text=GIFT%20format%20allows%20someone%20to,format%20available%20in%20Question%20bank.)
+3. Microsoft Open Source Curriculum quiz format - This format is for questions from the Microsoft Open Source Curriculum quizzes. More information can be found [here](https://docs.moodle.org/311/GIFT_format#:~:text=GIFT%20format%20allows%20someone%20to,format%20available%20in%20Question%20bank.)
+4. Assessment Application JSON format - This format is native to the assessment app and can be used if an Educator wishes to upload their questions via JSON format. The following JSON format must be followed. The following example shows one True/False question. 
+```
+[{
+    "name":"Introduction to Machine Learning",
+    "description":"Machine Learning for Beginners (Microsoft)",
+    "questions":[
+        {"name":"Applications of machine learning",
+        "description":"Applications of machine learning are all around us",
+        "options":["True","False"],"answer":["0"],"textType": "text", "questionType":"MCQ"
+    "assessmentType":"Quiz"
+}]
+```
+Please consider the following keys for the **questionType** attribute of the JSON. 
+If the question is of type True/False or Multiple Choice, questionType="MCQ". 
+If the question is of type Long answer/Short Answer, questiponType="QA"
+
+Please consider the following keys for the **textType** attribute of the JSON. 
+TextType indicates whether the question entered contains HTML embeddings or not. Some question banks exports such as those from Moodle LMS(GIFT), use HTML embeddings - and in that case the textType will be set to "html". In most cases, the textType should be set to "text". 
+
+Please consider the following information about the **Description** attribute of the JSON
+This attribute should describe the entire question as this will be shown to students before showing them the options/textbox. 
+
+Please consider the following information about the **Name** attribute of the JSON
+This attribute is for convenience purposes for the Educator, as they can get a quick summary of the question by looking at the Name, especially in the Question Bank view. 
+
+
+## What are the different question types supported <a name="question-type"/>
+1. Multiple Correct Answer - Educators can set a question description with multiple correct options for student to pick from. Student can select one or more correct options that they deem appropriate.  
+How is this marked? 
+Educators need to provide one or more correct answers to the question. Only if the student selects all provided answers do they receieve full on this question (score of 1), else 0. 
+2. True/False questions- Similar to Multiple Correct Answers, but the options are set to True and False. 
+How is this marked? 
+Educators need to provide the correc answer to this. Only if the student selects the correct answer, do they receive full on this question (score of 1), else 0. 
+3. Long/Short Answer - Educators can provide a question description, and students get to enter correct answers.
+How is this marked? 
+Educators can provide one or more correct answers to this question. If any of the correct answers match with the answer provided by the student, the student receives a full score (score of 1), else 0.  
+
 
 ## How to browse the Assessment Analytics <a name="assessment-analytics"/>
 

docs/TROUBLESHOOTING.md

@@ -0,0 +1,360 @@
+# For issues with the guides and tool
+
+For any and all issues, please raise a [GitHub issue](https://github.com/microsoft/AzureLTIAssessmentApp/issues/new/choose) so we can help you.
+
+## General Deployment Issues
+
+If your **deployment fails and the resource group has been created**; an IT admin needs to delete the resource group in their subscription and re-run the script again.
+
+If your deployment has an error look [here](https://docs.microsoft.com/azure/azure-resource-manager/templates/common-deployment-errors?WT.mc_id=learnlti-github-cxa) for common errors.
+
+## Ensure you install all the prerequisties 
+
+
+To begin, you will need to install the Prerequisites and reboot your machine
+
+To begin, you will need:
+- [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest?WT.mc_id=learnlti-github-cxa)
+- [DotNet Core SDK](https://dotnet.microsoft.com/download?WT.mc_id=learnlti-github-cxa)
+- [Node.js](https://nodejs.org/en/download/)
+- [Powershell](https://docs.microsoft.com/powershell/scripting/install/installing-powershell?view=powershell-7?WT.mc_id=learnlti-github-cxa)
+- [Git](https://git-scm.com/downloads)
+- An Azure subscription
+
+
+## Failed deployment or removing the services
+
+To remove resources simply delete the Assessment Application resource group and Assessment Application Registration in the Azure portal
+ 
+- ResourceGroupName = "AssessmentAppLTI"
+- AppName = "Assessment-Application-Lti-Tool-App"
+
+If you have changed the name of the Resource Group or AppName simply replace the actual names with the placeholder in the script.
+
+## Check the Failed Deployment status and output
+
+From the Azure Portal, simply select deployments. See the following screenshot.
+
+![FailedDeployments](../images/FailedDeployment.png)
+
+## Errors deploying install scripts
+
+Insufficient permissions if you try deploying the installation scripts from an account which is not your tenant AAD Admin or Azure Subscription Admin/Owner you will recieve an error
+
+```
+ERROR: Directory permission is needed for the current user to register the application. For how to configure, please refer 'https://docs.microsoft.com/azure/azure-resource-manager/resource-group-create-service-principal-portal'. Original error: Insufficient privileges to complete the operation.
+```
+To debug the error look at the Transcript-date-time.log file and the permissions error will be clearly shown.
+
+The personas/responsibilities for setup are:
+- Central IT (Azure tenant Owner, AAD Owner and have permissions to create service principals)
+- LMS (Learning Management System Owner who needs to configure the LTI Application)
+- Educator (who can create assessments into their course)
+
+## Azure CLI Tool not installed
+
+Exception.Message [ The term "az" is not recognized as a command applet name, function, script file, or executable program. Check the spelling of the name, or if a path exists, check that the path is correct and try again. ]
+
+To run the install you need the Azure Command Line Extension tool installed see Instructions at: https://docs.microsoft.com/cli/azure/install-azure-cli-windows?tabs=azure-cli for instructions. Please ensure you have all the prerequisites installed. 
+
+## Purging Key Vaults
+
+Your Azure Key Vault may be set to soft delete enabled. 
+
+As such if you encounter a issue in the deployment you will have to delete your resource and deploy again you may be presented with this following error.
+
+```bash
+ERROR: Deployment failed. Correlation ID: ************-****-****-**********. {
+  "error": {
+    "code": "ParentResourceNotFound",
+    "message": "Can not perform requested operation on nested resource. Parent resource 'kv-******' not found."
+```
+
+Due having soft delete enabled on the Key vault. The Key vault could not be recreated after the install initially fails.
+See more details on [Azure Key Vault recovery overview | Microsoft Docs](https://docs.microsoft.com/azure/key-vault/general/key-vault-recovery?tabs=azure-portal)
+
+Key Vault Recovery features are designed to prevent the accidental or malicious deletion of your key vault and secrets, keys, and certificate stored inside key-vault. So under soft delete the key vault is still available for 90 days after deletion therefore you need to purge the deleted Key Valut before it can be recreated.  
+
+### To purge a deleted Key vault
+
+- Go to key vaults-> manage deleted vault
+- Select the kv and select purge
+
+See the following screenshot
+
+![KeyVaultPurge](../images/Keyvaultpurge.png)
+
+## LTI Application does not load error: {Message:"Oops Something went wrong"}
+
+LMS Requirement for Secure Token SSL https
+
+If you are deploying the service in a test environment please ensure that your test LMS environment had a valid SSL certificate installed we require all traffic to utilise SSL.
+
+You will receive this error if you have are using http:// and not https:// for your LMS Environment. Oops something went wrong.
+
+![NoSSLDeployment](../images/Oopswentwrong.png)
+
+If you using a standard web server there a great tool for supporting installing SSL certs https://certbot.eff.org/lets-encrypt simply choose the web server and platform and it will give you instructions.
+
+## LTI Application Login error: You are not enrolled in this course
+
+If you receive the following error.
+
+![Learnltiadd](../images/LearnLTIAADIssue.png)
+
+User not registered on the LMS or logged into Web Browser with the incorrect account
+If using AAD 
+- Ensure if you are using an Azure AD connected account, that the user is logged into their web browser with the same account they used to sign into the LMS.
+- Please ensure that AAD sign for your LMS is enabled and you are signing into your LMS with a AAD account. 
+- The emails of the users in the LMS and AAD should match.
+- Ensure the users are active on the course.
+
+For troubleshooting see *Azure Functions Tracing* and *Failure is in Users Function App* below.
+
+## LTI Application does not load error: {"Message":"Could not validate nonce."}
+
+If you receive a browser message: {"Message":"Could not validate nonce."}
+
+We have seen this is an intermittent issue, it usually happens if the user is trying to replay an old call or if the nonce and state value don't match. If the user simply refreshes the browser the page will load.
+
+## LTI Application does not load error - 
+{"Message": “401 authentication – azure key vault secrets (unauthorized) invalid issuer” error}
+
+This error occurs when the "admin" deploying the assessment application does not have "owner" status on the subscription used to deploy the application. Using the command “az account set –subscription “subscription name”” can be used to change ownership of the subscription.
+
+
+
+
+## LTI Application does not load error: {"Message":"Could not validate request"}
+
+This issue is typically related to one of the following:
+
+- Check if you have a valid 3rd Party Signed SSL. This services requires a valid 3rd party SSL certificate, self signed SSL certificates are not valid. Please ensure your service is using https:// with a valid SSL certification.
+- Check the Launch URL, please make sure that all the fields are correctly filled while registering the tool and filling tool's platform registration page. 
+
+- Check the Azure Function/Azure Logs see https://github.com/microsoft/Learn-LTI/blob/main/docs/TROUBLESHOOTING.md#azure-functions-tracing
+
+- If the logs indicate "Issue id token not present", the issue is with the authentication workflow. Please read the following for direction. 
+
+  1. For single-sign-on to work, it is important to create an Azure Active Directory registration for the learning management tool and add users to it to use OpenID /Auth2. This ensures that a token id is generated when the user logs into the LMS itself, and so no more authentication is needed. 
+  2. It is necessary to add users into the Assessment App AAD (which is automatically created during deployment). The email IDs of these users must the same as the email addresses of users in the LMS, as that is used to match up user details on both platforms.
+
+## LTI Application does not load error: No sufficient permissions to view this page
+
+After you have successfully installed the Assessment application by running the deployment steps you try to access the created Platform Registration Page results and recieve the message 'No sufficient permissions to view this page'.
+
+Solution:
+
+- Go to the 'platforms' azure function inside of the resource group which was just created.
+- Go to 'Configuration' one the left side
+- Look into the 'AllowedUsers' environment variable.
+- Verify the email address of the user you are logged in with is included in the environment variable (!THIS IS CASE SENSITIVE!)
+- If it does not match or exist, update the environment variable and save the change.
+- Wait a minute to let change take effect and retry!
+
+We have seen cases where the email address was added there but the case is different i.e. capital first letters while the email address of the login user had lowercase)
+
+## Adding SSL Certificates to your Development Environment
+
+You can use Let’s Encrypt, if you don't have a valid SSL certificates for your development environment, Let Encrypt will allow you to setup a [Free SSL](https://letsencrypt.org/)
+
+If you using a standard web server there a great tool for supporting installing SSL certs https://certbot.eff.org/lets-encrypt simply choose the web server and platform and it will give you instructions.
+
+## Moodle: Bitnami Image and setting up SSL
+
+If your moodle deployment does not have a valid SSL you will receive the following error Oops something went wrong 
+
+![NoSSLDeployment](../images/Oopswentwrong.png)
+
+You simply need to ensure your LMS installation has a valid DNS name and SSL. Use link for the [instructions Generate and Install a Let's Encrypt SSL Certificate for a Bitnami Application](https://docs.bitnami.com/aws/how-to/generate-install-lets-encrypt-ssl)
+
+If you need help configuring your Bitnami HTTPS Configuration see the following [configure HTTPS certificates](https://docs.bitnami.com/aws/how-to/understand-bncert/)
+
+If you still receive this error after these changes check *Failures (Exceptions) in App Insights* and if you see a 401 error see Solution: *Error Unauthorized 401 when using Moodle*
+
+## Moodle: Users not active - Not Current
+
+All users need to be Active on a course if you see Not Current next to the status.
+
+![image](https://user-images.githubusercontent.com/2511341/108256622-a3406f80-7155-11eb-9ef2-b9b42faf324f.png)
+
+We have noticed that users having this issue
+
+![ErrorNoValidUsers](https://user-images.githubusercontent.com/31847926/94229941-2b726e00-ff44-11ea-8343-30ad6bab348e.png)
+
+We have seen this where students/teacher may of been manually assigned or automatically assigned as showing  "Not Current".
+
+![NotCurrent](https://user-images.githubusercontent.com/3178741/94397091-b8603600-0195-11eb-9e76-f0213684c1aa.png)
+
+The solution is simply make the students and teachers active and they can then log in. 
+To activate users: Click on the settings wheel in the top right hand corner and select "Enrollment methods" then click the crossed out eye to allow manual enrollment.
+
+When you go back to the course participants they will show as active and users will be able to load the Assessment application.
+
+## Canvas: LMS Issuer
+
+If your using on premise, hosted or cloud implementations of Canvas. Ensure you register the parameters back in the Assessment application's registration page. Please ensure you always state the  Issuer as https://canvas.instructure.com see the final steps in the Canvas instructions 
+
+## Canvas: Unable to create developer key
+Canvas requires the JWK key to be in lowercase. If using the JWK key, ensure that all the key:value pairs are in lowercase. If using the JWK URL, ensure that the URL is public facing. 
+
+## Canvas: Unable to view the assessment application as an "external tool" on the assessments page 
+Ensure that you have "installed" the assessment application within the module (indicated in the configuration guide). 
+Additionally, ensure that you have checked the "eye" icon in the developer key section configuration section (indicated in the configuration guide). 
+
+## Canvas: Error - {Message}:{"Could not validate request."}? 
+
+We have seen this issue when users are using the Canvas Bitnami image in Azure.
+
+You need to create the following config files files in canvas system:
+/opt/bitnami/apps/canvaslms/htdocs/config/dynamic_settings.yml copying the content from existing /opt/bitnami/apps/canvaslms/htdocs/config/dynamic_settings.yml.example and added following lines below (include your secrets adjusted):
+
+production:
+
+```bash
+production:
+  store:
+    canvas:
+      lti-keys:
+        jwk-past.json: "{\"p\": \"12345MySecret67890\",\"kty\": \"RSA\",\"q\": \"12345MySecret67890\",\"d\": \"12345MySecret67890\",\"e\": \"AQAB\",\"use\": \"sig\",\"kid\": \"sig-16mysig48\",\"qi\": \"12345MySecret67890\",\"dp\": \"12345MySecret67890\",\"alg\": \"RS256\",\"dq\": \"12345MySecret67890\",\"n\": \"12345MySecret67890\"}"
+        jwk-present.json: "{\"p\": \"12345MySecret67890\",\"kty\": \"RSA\",\"q\": \"12345MySecret67890\",\"d\": \"12345MySecret67890\",\"e\": \"AQAB\",\"use\": \"sig\",\"kid\": \"sig-16mysig48\",\"qi\": \"12345MySecret67890\",\"dp\": \"12345MySecret67890\",\"alg\": \"RS256\",\"dq\": \"12345MySecret67890\",\"n\": \"12345MySecret67890\"}"
+        jwk-future.json: "{\"p\": \"12345MySecret67890\",\"kty\": \"RSA\",\"q\": \"12345MySecret67890\",\"d\": \"12345MySecret67890\",\"e\": \"AQAB\",\"use\": \"sig\",\"kid\": \"sig-16mysig48\",\"qi\": \"12345MySecret67890\",\"dp\": \"12345MySecret67890\",\"alg\": \"RS256\",\"dq\": \"12345MySecret67890\",\"n\": \"12345MySecret67890\"}"
+```
+
+## Missing Name Role Provisioning Service in LTI1.1
+
+NRPS is something that has been a part of LTIAdvantage Specification this service is not available in LTI1.1
+- Moodle implementation as such we recommed you update your moodle environment.
+- OpenEdx, There is work happening in the OpenEdX community see the following request and workitems for LTIAdvantage support in OpenEdx
+- Is Anyone Working on LTI 1.3? - [Development / Collaborative Proposals](https://discuss.openedx.org/t/is-anyone-working-on-lti-1-3/798/12)
+- [TNL-7314] [BD-24] [LTI Advantage & other improvements](https://id.atlassian.com/login?continue=https%3A%2F%2Fopenedx.atlassian.net%2Flogin%3FredirectCount%3D1%26dest-url%3D%252Fbrowse%252FTNL-7314%26application%3Djira&application=jira)
+- [BD-24] [LTI v1.3 Improvements: Advantage Support - Open edX Community](https://openedx.atlassian.net/wiki/spaces/COMM/pages/1545076784/BD-24+LTI+v1.3+Improvements+Advantage+Support)
+
+## Error Unauthorized 401 when using Moodle
+
+An Unauthorized (401) Exception occurred when access the Moodle's LTI services. The 401 error is found by running *Failures (Exceptions) in App Insights*
+
+See /mod/lti/services.php/CourseSection/39/bindings/2/memberships
+
+The problem can be related to your Apache Config.
+
+Solution:
+- Edit your httpd.conf file or alternatively in the vhosts file:
+- SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
+- SetEnvIf Content-Type "(.*)" HTTP_CONTENT_TYPE=$1
+- SetEnvIf Accept "(.*)" HTTP_ACCEPT=$1
+
+Reference: https://moodle.org/mod/forum/discuss.php?d=389429
+
+## How to Edit your Httpd.conf file
+
+How To: Access and Edit Your httpd.conf
+
+For Apache VPServers running Linux. 
+
+Login into your Putty. Putty SSH can be googled. Enter your username, then password and then type
+
+```bash
+su -
+```
+Copy this code
+
+```bash
+vi /etc/httpd/conf/httpd.conf
+```
+To save a file and exit Vim:
+
+- Switch to command mode by pressing the ESC key.
+- Press : (colon) to open the prompt bar in the bottom left corner of the window.
+- Type x after the colon and hit Enter. This will save the changes and exit.
+
+To exit Vim without saving changes:
+
+- Switch to command mode by pressing the ESC key.
+- Press : (colon) to open the prompt bar in the bottom left corner of the window.
+- Type q! after the colon and hit Enter to exit without saving the changes.
+
+## Step by step guide to connect the SSO experience with AAD
+
+The details provided at [Planning Active Directory SSO Applications](https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/plan-sso-deployment)
+
+## Steps if your an Office365/Microsoft365 institution but unsure of your Active Directory Details
+
+Microsoft 365 uses Azure Active Directory (Azure AD) to manage user identities behind the scenes. Your Microsoft 365 subscription includes a free Azure AD subscription so that you can integrate your on-premises Active Directory Domain Services (AD DS) to synchronize user accounts and passwords or set up single sign-on.
+
+Azure AD also offers other functionality, like managing integrated apps, that you can use to extend and customize your Microsoft 365 subscriptions.
+
+You can use the Azure AD deployment advisors for a guided setup and configuration experience in the Microsoft 365 admin center (you must be signed in to Microsoft 365):
+
+- [Azure AD Connect advisor](https://aka.ms/aadconnectpwsync)
+- [AD FS deployment advisor](https://aka.ms/adfsguidance)
+- [Azure AD setup guide](https://aka.ms/aadpguidance)
+
+If you have a paid subscription to Microsoft 365, you also have a free Azure AD subscription. You can use Azure AD to create and manage user and group accounts. To activate this subscription, you have to complete a [one-time registration.](https://docs.microsoft.com/en-us/microsoft-365/compliance/use-your-free-azure-ad-subscription-in-office-365?view=o365-worldwide) Afterward, you can access Azure AD from your Microsoft 365 admin center.
+
+Don't go directly to azure.microsoft.com to sign up or you'll end up with a trial or paid subscription to Microsoft Azure that is separate from your free Azure AD subscription with Microsoft 365.
+
+With the free subscription you can synchronize with on-premises directories, set up single sign-on, and synchronize with many software as service applications.
+
+If you want enhanced AD DS functionality, bi-directional synchronization, and other management capabilities, you can upgrade your free subscription to a paid premium subscription. For the details, see [Azure Active Directory editions.](https://azure.microsoft.com/pricing/details/active-directory/)
+
+## Troubleshooting Azure Resource Manager Deployments
+
+In order to understand the issue in more detail, one way could be to go through the deployment details which could be accessed via Azure Portal.
+
+- Go to Azure Portal
+- Choose appropriate Subscription
+- Select Resource Groups blade from Left Rail
+- Choose appropriate Resource Group
+- Select Deployments Blade from Left Rail.
+- The log will show which service deployments failed
+
+## Azure Resources Deployments errors
+
+You could consider trying to re-deploy the same RG/Identity/AppName combination by simply re-executing run.bat or deploy a new RG inside a different region than the one tried previously. Running the same script and resource group names will also create a error if the services and resources are already present for those regions and names.
+
+## Using Hosted services on Cloudfront
+
+We have seen a issue where the "Authorization" header was not being forwarded to the backend by AWS CloudFront. See the following documentation 
+[Configure CloudFront to Forward Authorization Headers](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/add-origin-custom-headers.html#add-origin-custom-headers-forward-authorization?WT.mc_id=edna)
+
+## Debugging Errors related to http500 indicated in Browser with Oops! Something went wrong.. We suggest you go to our help page
+
+This results in you being unable to open the Assignments page in Assessment application. Users access failing with http500 which essentially has been the source of the Oops! Something went wrong error message in the browser window.
+
+Please see browser logs if indicate that the membership call being sent to the LMS (Moodle) is returning a 401 Unauthorized and the Assessment Application tool in unable to get the course members from the LMS and hence the it is giving a 500 response. 
+
+Please make sure that while configuring the tool in Moodle, Under Services, IMS LTI Names and Role Provisioning: `Use this service to retrieve members’ information as per privacy settings.` is selected as mentioned in the [Configuration Guide](CONFIGURATION_GUIDE.md)
+
+# Troubleshooting Errors 
+
+## Azure Functions Tracing
+
+This should provide details related to the execution context and state of the function execution when it has returned http 204 error.
+
+- Goto your Resource Group inside Azure and select Function App matching users-XXXXXXX.
+- Select Functions Blade in Left Hand Pane.
+- Select GetUserDetails afterwards.
+- Choose Monitor Blade in LHP and you should see the Invocation Traces.
+- Clicking on the failing trace should provide more details related to Server logs for that function invocation to help you.
+
+## Failures (Exceptions) in App Insights
+
+- Go to your Resource Group inside Azure and select Application Insights resource matching users-XXXXXXX.
+- Select Failures Blade in LHP and then choose Exceptions Tab.
+- The subsequent screen should indicate any exceptions that were thrown as a part of function execution and should provide more insights into what might've gone wrong on server when executing GetUserDetails api.
+- We'd request you to please share the above details with us in order for us to be able to help you in a better way. Since the details might contain some private information as well. 
+
+## Failure is in Users Function App
+
+Function app which is not being able to find the user (signed-in via AAD) enrolled into the current course. 
+
+Please check that the return code for the API in the Chrome DevTools network tab is http204 (i.e. No-Content).
+
+In our experience, the only case when this happens is when user signs into Assessment-Application-LTI app with an onmicrosoft.com account which does not map to the email of any of the enrolled users of the course.
+
+## Understanding what the Azuredeploy.json implements when running the run.bat/Deploy.ps1
+
+The Deployment utilizes an ARM Azure Resource Manager Template this deploys the following Azure Services into your Azure Subscription
+
+