Merge pull request #250 from donny-wong/release_2.9.0

donny-wong · web-flow · commit 116a42921afa · 2025-12-15T14:33:28.000-05:00
Release 2.9.0
diff --git a/Developer-Guide--Configure-environment-to-use-VSCode.md b/Developer-Guide--Configure-environment-to-use-VSCode.md
@@ -0,0 +1,80 @@
+# Developer Guide: Configure VSCode
+
+To configure your development environment on VSCode the following steps are required:
+
+## Setup
+
+Install the dev containers [extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) inside VSCode.
+
+### Create a `devcontainer` Configuration
+
+Devcontainer files define the vscode environment inside a docker container. It allows us to configure VSCode extensions, environment definitions, etc when running VSCode inside the docker container.
+
+Place the following file: `devcontainer.json` at the root of your project inside a `.devcontainer` folder.
+
+```json
+{
+  "name": "MarkUs Dev",
+  "dockerComposeFile": ["../compose.yaml"],
+  "service": "rails",
+  "workspaceFolder": "/app",
+  "remoteUser": "markus",
+  "overrideCommand": true,
+  "forwardPorts": [3000],
+  "mounts": ["source=${localWorkspaceFolder}/.ruby-lsp,target=/home/vscode/.ruby-lsp,type=bind"],
+  "customizations": {
+    "vscode": {"extensions": ["Shopify.ruby-lsp", "eamodio.gitlens", "koichisasada.vscode-rdbg"]},
+    "settings": {
+      "terminal.integrated.defaultProfile.linux": "bash",
+      "rubyLsp.useBundler": true,
+      "rubyLsp.rubyVersionManager": "none",
+      "rubyLsp.bundleGemfile": "/app/Gemfile",
+      "rubyLsp.exclude": [
+        "**/.git/**",
+        "node_modules/**",
+        "vendor/**",
+        "log/**",
+        "tmp/**",
+        ".bundle/**"
+      ],
+      "git.openRepositoryInParentFolders": "always",
+      "git.detectSubmodules": false,
+
+      "[ruby]": {
+        "editor.defaultFormatter": "Shopify.ruby-lsp",
+        "editor.formatOnSave": true
+      }
+    }
+  },
+  "containerEnv": {
+    "HOME": "/home/markus",
+    "LISTEN_POLLING": "1"
+  }
+}
+```
+
+Brief explanation of what is happening above:
+
+- On startup, let's open up VSCode inside the `/app` folder of our appplication by defining our workspace folder `workspaceFolder` to point to the `/app` directory of our container.
+- We are installing `ruby-lsp` and `gitlens`, both VSCode extensions inside the dev container and specifying their configuration, such as ignoring certain folders from indexing specific paths. Notice that when we open up markus outside the dev container, these extensions will be absent.
+- Force the `listen` gem to poll for changes by setting the `LISTEN_POLLING` flag. This removes flakyness in our autoreloading.
+
+### Enable the `ruby-lsp` Gem
+
+To enable modern programming features such as `go-to`, `code completion`, etc, an LSP server is required, which the default RubyMine IDE already comes preconfigured with. To work eficiently in VSCode we must enable the optionally defined `ruby-lsp` gem.
+
+To install optional gem groups, we must pass in the `BUNDLE_WITH` enviroment variable with the optional groups we wish to install.
+
+Inside the `docker-compose.override.yml`, we must define the following environment variable:
+
+```yaml
+deps-updater:
+  environment:
+    - BUNDLE_WITH=development_extra
+```
+
+When must then run: `docker compose run --rm deps-updater` to install the dependency.
+
+## Execution
+
+To run your code inside the dev container extension open up the command palette, either by pressing down `cmd + Shift + P` (on MAC OS) or by going to `view > Command Palette` and typing in `Dev Containers: Reopen in Container`
diff --git a/Developer-Guide--Set-Up-With-Docker.md b/Developer-Guide--Set-Up-With-Docker.md
@@ -102,7 +102,7 @@ Hooray! You have MarkUs up and running. Please keep reading for our recommended
 
 ## Installing and Configuring RubyMine
 
-We strongly recommend RubyMine (a JetBrains IDE) for all MarkUs development.
+We strongly recommend RubyMine (a JetBrains IDE) for all MarkUs development. If you prefer to use VSCode, please follow the guide outlined [here](https://github.com/MarkUsProject/Wiki/blob/master/Developer-Guide--Configure-Environment-to-use-VSCode.md).
 
 1. First, install RubyMine from [here](https://www.jetbrains.com/ruby/download). Note that if you are a current university student, you can obtain a [free license](https://www.jetbrains.com/student/) for all JetBrains software.
 
@@ -359,3 +359,58 @@ rm -rf ~/.docker
 ```
 
 Finally, install Docker Engine by following the instructions on [this page](https://docs.docker.com/engine/install/).
+
+### Q7
+
+When setting up the autotester, the script might often fail to find the required information to populate the schema required to run the automated tests.
+
+```plaintext
+Set up testing environment for autotest
+Creating sample autotesting assignment autotest_custom
+bin/rails aborted!
+NoMethodError: undefined method `[]' for nil (NoMethodError)
+
+      schema_data['definitions']['files_list']['enum'] = files
+                                ^^^^^^^^^^^^^^
+/app/app/helpers/automated_tests_helper.rb:34:in `fill_in_schema_data!'
+```
+
+This often happens when running the last step (#10) during the autotester [setup](https://github.com/MarkUsProject/Wiki/blob/master/Developer-Guide--Set-Up-With-Docker.md#setting-up-the-autotester) process, as a result of missing or corrupted database entries. The easiest and simplest solution is to restart the setup process with a clean docker environment.
+
+Save the following functions to your `.bashrc`, `.zshrc` or other shell configuration file and execute the `nuke_docker` command.
+
+```bash
+# Docker functions
+# *****************************************************************************
+function stop_containers() {
+  docker stop $(docker ps -a -q)
+}
+
+function remove_containers() {
+  docker rm $(docker ps -a -q)
+}
+
+function remove_volumes() {
+  docker volume rm $(docker volume ls -qf dangling=true)
+}
+
+function remove_buildx_cache() {
+  docker builder prune -af
+  docker buildx prune -af
+}
+
+function clean_containers() {
+  echo "Cleaning existing containers"
+  stop_containers
+  remove_containers
+  remove_volumes
+  remove_buildx_cache
+}
+
+function nuke_docker() {
+  clean_containers
+  docker system prune -a --volumes
+}
+```
+
+We can now restart the setup process with a clean docker environment.
diff --git a/Home.md b/Home.md
@@ -19,7 +19,7 @@
         - [Starter Files](Instructor-Guide--Assignments--Starter-Files.md)
         - [Peer Review](Instructor-Guide--Assignments--Peer-Review.md)
         - [Assigning Graders](Instructor-Guide--Assignments--Assigning-Graders.md)
-        - [Summary](Instructor-Guide--Assignments--Summary.md)
+        - [Summary](Instructor-Guide--Assignments--Grades.md)
     - [Timed Assessments](Instructor-Guide--Timed-Assessments.md)
     - [Scanned Exams](Instructor-Guide--Scanned-Exams.md)
     - Marks Spreadsheets
diff --git a/Instructor-Guide--Assignments--Grades.md b/Instructor-Guide--Assignments--Grades.md
@@ -1,14 +1,14 @@
-# Summary
+# Grades
 
 ## Table of Contents
 
-- [Summary Tab](#summary-tab)
+- [Grades Tab](#grades-tab)
     - [Summary Table](#summary-table)
     - [Summary Statistics](#summary-statistics)
 
-## Summary Tab
+## Grades Tab
 
-The summary tab can be used to view/visualize a variety of general statistics for a particular assignment. In addition, you may also view a summary of the grade breakdown for each submission. These are highlighted below.
+The grades tab can be used to view/visualize a variety of general statistics for a particular assignment. In addition, you may also view a summary of the grade breakdown for each submission. These are highlighted below.
 
 ### Summary Table
 
diff --git a/Instructor-Guide--Assignments--Late-Submission-Policies.md b/Instructor-Guide--Assignments--Late-Submission-Policies.md
@@ -29,19 +29,41 @@ For group work, *all* members of the group must have enough grace credits availa
 
 ## Use Penalty Decay Formula
 
-This option will allow you to use a built-in function that deducts X grade percentage every Y hours for a duration of Z hours. You are able to create multiple time periods in case you want to deduct 5% every hour for the first four hours and then 10% every 6 hours for 12 more hours:
+This option will allow you to use a built-in function that deducts X amount of penalty every Y hours for a duration of Z hours. You are able to create multiple time periods in case you want to deduct 5%/marks every hour for the first four hours and then 10%/marks every 6 hours for 12 more hours.
+
+### Penalty Type
+
+You must select a **Penalty Type** to determine how penalties are calculated (default is Percentage of assignment total):
+
+- **Percentage of assignment total**: Deduct a percentage of the total possible marks for the assignment (e.g., 5%)
+- **Percentage of earned mark**: Deduct a percentage of the student's earned mark (e.g., 5% of student's earned mark)
+- **Marks**: Deduct a fixed number of marks (e.g., 5 marks)
+
+The deduction unit displayed next to the input field will automatically update based on your selection (% for percentage types, marks for the marks option).
+
 ![Website Penalty Decay Formula](images/late-submission-policy-penalty-decay-formula.png)
 Additional time periods are added by clicking the "Add late period" button and removed by using the delete link. MarkUs will automatically adjust the real time periods for you when adding and deleting.
 
 When grading a late submission, the penalty will be automatically applied to the group's mark, but will not decrease the mark below 0. This penalty will appear in the "Summary" tab of the grading view.
 
-Penalties are applied as a percentage of the total mark. For example, a 10% penalty for an assignment out of 90 total marks will reduce a groups's overall score by 9 marks. If a group would have received 55/90 before the penalty is deducted, they will receive 46/90 after the penalty is deducted.
+Penalties are applied as a percentage of the total mark. For example, a 10% penalty for an assignment out of 90 total marks will reduce a group's overall score by 9 marks. If a group would have received 55/90 before the penalty is deducted, they will receive 46/90 after the penalty is deducted.
 
 ## Set Manual Penalty Periods
 
-This option allows you to set percentage penalty amounts for specified time periods.
+This option allows you to set percentage/marks penalty amounts for specified time periods.
+
+### Penalty Type
+
+You must select a **Penalty Type** to determine how penalties are calculated (default is Percentage of assignment total):
+
+- **Percentage of assignment total**: Deduct a percentage of the total possible marks for the assignment (e.g., 5%)
+- **Marks**: Deduct a fixed number of marks (e.g., 5 marks)
+- **Percentage of earned mark**: Deduct a percentage of the student's earned mark (e.g., 5% of their score)
+
+The deduction unit displayed next to the input field will automatically update based on your selection (% for percentage types, marks for the marks option).
+
 ![Website Manual Penalty Periods](images/late-submission-policy-penalty-period.png)
-Additional time periods are added by clicking the "Add late period" button and removed by using the delete link. MarkUs will automatically adjust the "From" and "To" times depending on the number of hours you specify in the "After" column.
+Additional time periods are added by clicking the "Add late period" button and removed by using the delete link. MarkUs will automatically adjust the "From" and "To" times depending on the number of hours you specify in the "Period" column.
 
 Note that for each late submission method, in order for the "From" and "To" times to be calculated, a due date must be specified in the "Properties" section. If a date is not specified then the "From" and "To" sections will be empty or read "Invalid Date"
 
diff --git a/Instructor-Guide--Assignments--Setting-Up.md b/Instructor-Guide--Assignments--Setting-Up.md
@@ -33,10 +33,17 @@ This section allows you to set the name and due date for the assignment as well
 **2. Assignment description**: This is the longer, more descriptive name of the assignment used as its full title.
 **3. Message**: This section allows you to include any additional information the students may need to know about the assignment.
 **4. Due Date**: This section lets you set the due date for the assignment. You are able to configure the time (down to the minute) at which you would like the assignment to be due. The due date will be visible to students when they view the assignment. Note that the due date may be changed later on (ex/ to accommodate class-wide extensions). Changing the due date after submissions have been collected will not affect submitted assignments.
-**5. Section Due Dates:** Checking the "Enable section due dates" box will allow you to set different due dates for different lecture sections. Note that if a due date is not specified the time in the "Due Date" section will be used by default.
+
+**5. Visibility:** Controls when students can see the assignment:
+
+- **Hidden**: Students cannot see this assignment
+- **Visible**: Students can always see this assignment
+- **Visible on/until**: Assignment is only visible between specified start and end datetimes
+
+**6. Section-Specific Settings:** Checking the "Enable section specific settings" box will allow you to set different due dates and visibility settings for different lecture sections. Each section can override the default due date and visibility settings.
 > :spiral_notepad: **NOTE:** Students not assigned to any section will only be allowed to form groups with other students not assigned to any section.
 
-**6. Check boxes:** The rest of this section includes check boxes that may be selected or deselected depending on your preferences. Note that in order for students to submit online, the "Allow students to submit through the web interface" box must be checked. If you prefer students to submit through a version control system, then uncheck the "web interface" box and select "version control system".
+**7. Check boxes:** The rest of this section includes check boxes that may be selected or deselected depending on your preferences. Note that in order for students to submit online, the "Allow students to submit through the web interface" box must be checked. If you prefer students to submit through a version control system, then uncheck the "web interface" box and select "version control system".
 
 If you wish to allow students to submit URLs check the "Allow students to submit URLs" box. This is especially useful if you plan on requiring students to submit videos or large files since preview support is available for content from YouTube and Google Drive/Docs.
 > :spiral_notepad: **NOTE:**
diff --git a/RESTful-API.md b/RESTful-API.md
@@ -1058,6 +1058,151 @@ NOTE: collect_current value meanings:
 - true: collect most recent files submitted, regardless of assignment due date or late period.
 - false: collect most recent files submitted before the due date, including any late period.
 
+### POST /api/courses/:course_id/assignments/:assignment_id/groups/:group_id/add_test_results
+
+- description: Submit automated test results for the given group for the given assignment. This endpoint is used by the autotesting service to report test execution results.
+- required parameters:
+    - test_results (object)
+        - status (string: test execution status, e.g., "pass", "fail", "finished")
+        - error (string or null: error message if test execution failed)
+        - test_groups (array of objects)
+            - time (integer or null: execution time in milliseconds)
+            - tests (array of objects)
+                - name (string: test name)
+                - status (string: one of "pass", "partial", "fail", "error", "error_all")
+                - marks_earned (integer: points earned)
+                - marks_total (integer: maximum points)
+                - output (string: test output/feedback)
+                - time (integer or null: execution time in milliseconds)
+            - extra_info (object)
+                - name (string: test group name)
+                - test_group_id (integer: ID of the test group)
+                - display_output (integer: 0=instructors, 1=instructors_and_student_tests, 2=instructors_and_students)
+                - criterion (string or null: associated criterion name)
+- optional parameters (within test_groups):
+    - timeout (integer: timeout value if test timed out)
+    - stderr (string: standard error output)
+    - malformed (string: malformed output information)
+    - annotations (array of objects: code annotations to add)
+        - content (string: annotation content)
+        - filename (string: file to annotate)
+        - type (string: "TextAnnotation", "ImageAnnotation", etc.)
+        - line_start, line_end, column_start, column_end (integers: for text annotations)
+        - x1, x2, y1, y2 (integers: for image/PDF annotations)
+    - feedback (array of objects: feedback files to attach)
+        - filename (string)
+        - mime_type (string)
+        - content (string: file content, may be base64 encoded)
+        - compression (string: "gzip" if content is compressed)
+    - tags (array of objects: tags to apply to the grouping)
+        - name (string: tag name)
+        - description (string: tag description)
+    - overall_comment (string: overall comment to add to the result)
+- example request body (json):
+
+```json
+{
+  "test_results": {
+    "status": "finished",
+    "error": null,
+    "test_groups": [
+      {
+        "time": 1250,
+        "extra_info": {
+          "name": "Correctness Tests",
+          "test_group_id": 42,
+          "display_output": 2,
+          "criterion": "Correctness"
+        },
+        "tests": [
+          {
+            "name": "test_addition",
+            "status": "pass",
+            "marks_earned": 2,
+            "marks_total": 2,
+            "time": 125,
+            "output": "All test cases passed"
+          },
+          {
+            "name": "test_subtraction",
+            "status": "fail",
+            "marks_earned": 0,
+            "marks_total": 2,
+            "time": 98,
+            "output": "Expected 5, got 3"
+          }
+        ],
+        "annotations": [
+          {
+            "content": "Consider edge cases for negative numbers",
+            "filename": "calculator.py",
+            "line_start": 10,
+            "line_end": 12,
+            "column_start": 0,
+            "column_end": 20
+          }
+        ],
+        "tags": [
+          {
+            "name": "needs_review",
+            "description": "Requires manual review"
+          }
+        ],
+        "overall_comment": "Good attempt, but edge cases need work"
+      }
+    ]
+  }
+}
+```
+
+- example success response (json):
+
+```json
+{
+  "status": "success",
+  "test_run_id": 1234
+}
+```
+
+NOTE: This endpoint creates a TestRun record with associated test results, annotations, feedback files, and tags. All operations are performed atomically within a transaction.
+
+NOTE: The request body is validated against a schema and must not exceed 10MB in size.
+
+NOTE: Authentication is required via API key. The authenticated user's role is used as the creator of the test run.
+
+### GET /api/courses/:course_id/assignments/:assignment_id/groups/:id/test_results
+
+- description: Get automated test results for the given group for the given assignment. Returns only the latest test run, grouped by test group name. Matches the UI download format.
+- supported content types: `application/json`, `application/xml`
+- example response (json):
+
+```json
+{
+  "Test Group 1": [
+    {
+      "name": "Test Group 1",
+      "test_groups_id": 7,
+      "group_name": "group1",
+      "test_result_name": "test_addition",
+      "status": "pass",
+      "marks_earned": 3.0,
+      "marks_total": 5.0,
+      "output": "All test cases passed",
+      "extra_info": null,
+      "error_type": null
+    }
+  ]
+}
+```
+
+NOTE: This endpoint returns only the most recent test run results for the group, not historical test runs.
+
+NOTE: Results are grouped by test group name (the keys in the JSON object are test group names).
+
+NOTE: Supports XML responses by setting the `Accept` header to `application/xml` or using the `.xml` extension.
+
+NOTE: Returns 404 if the group has no test results.
+
 ### POST /api/courses/:course_id/assignments/:assignment_id/groups/:group_id/extension
 
 - description: Create an extension for the given group for the given assignment
diff --git a/images/assignment-creation-properties-field.png b/images/assignment-creation-properties-field.png
diff --git a/images/late-submission-policy-penalty-decay-formula.png b/images/late-submission-policy-penalty-decay-formula.png
diff --git a/images/late-submission-policy-penalty-period.png b/images/late-submission-policy-penalty-period.png
