Installing Canvas Flexible Assessment

Developer installation guide for the Flexible Assessment application.

The application is built on the Django with a PostgreSQL backend (psycopg2) and web services provided by mod_wsgi and Apache.

Installation

Install required packages

$ sudo apt-get update
$ sudo apt-get install python3-pip python3-dev libpq-dev postgresql postgresql-contrib apache2 apache2-dev

Clone GitHub repository

$ git clone https://github.com/UBC-LFS/Canvas-Flexible-Assessment.git

Using Anaconda

This guide makes use of Anaconda for Python environments. The application can also be setup using virtualenv.

~$ cd Canvas-Flexible-Assessment
~/Canvas-Flexible-Assessment$ conda create -n django_env python=3.9
~/Canvas-Flexible-Assessment$ conda activate django_env
~/Canvas-Flexible-Assessment$ pip install -r requirements.txt

Using the Python3 virtual environment

~/$ sudo apt install python3-venv
~/$ python3 -m venv venv
~/$ source venv/bin/activate

(venv) ~/$ cd Canvas-Flexible-Assessment
(venv) ~/Canvas-Flexible-Assessment$ pip3 install -r requirements.txt

Database Setup

Setup PostgreSQL database for the Django application. This database holds tables for Users, Courses, Assessments, etc. related to the application.

$ sudo -u postgres psql

postgres=# CREATE DATABASE flexible_assessment_db;
postgres=# CREATE USER flexible_assessment_user WITH encrypted PASSWORD '<password>';
postgres=# GRANT ALL PRIVILEGES ON DATABASE flexible_assessment_db TO flexible_assessment_user;

# If you get database permission issues you can try the following. Be careful as this creates a superuser:
$ sudo -u postgres createuser -s -i -d -r -l -w super_user 
$ sudo -u postgres psql -c "ALTER ROLE super_user WITH PASSWORD '<password>';"

Create log/info.log

~/../flexible_assessment$ mkdir log
~/../flexible_assessment$ touch log/info.log

Migrate the database

~$ cd Canvas-Flexible-Assessment/flexible_assessment
~/../flexible_assessment$ python manage.py migrate

Static files

Collecting static files.

~/../flexible_assessment$ python manage.py collectstatic --noinput

Configurations

There are two parts to configuring the application (LTI and environment).

LTI Configuration

We will make a configs/ directory with the following structure.

configs
├── flexible_assessment.json
├── private.key
├── public.jwk.json
└── public.key

Within this flexible_assessment.json is used for launching the LTI app, private.key, public.key, and public.jwk.json are used by Canvas to verify the application.

Creating configs directory.

~$ cd Canvas-Flexible-Assessment/flexible_assessment
~/../flexible_assessment$ mkdir configs
~/../flexible_assessment$ nano configs/flexible_assessment.json

{
    "https://canvas.instructure.com": [{
        "default": true,
        "client_id": "<client_id>",
        "auth_login_url": "https://<canvas_domain>/api/lti/authorize_redirect",
        "auth_token_url": "https://<canvas_domain>/login/oauth2/token",
        "key_set_url": "https://<canvas_domain>/api/lti/security/jwks",
        "key_set": null,
        "private_key_file": "private.key",
        "public_key_file": "public.key",
        "deployment_ids": ["<deployment_id>"]
        }]
}

Add RSA public and private key files to configs/ and generate a JWK to add to public.jwk.json.

For values of <client_id> and <deployment_id> see Canvas Setup.

How to create public.key and private.key files as well as a public.jwk.json file.

$ pip install pycryptodome jwcrypto

import json
from Crypto.PublicKey import RSA
from jwcrypto.jwk import JWK


def get_private_key():
    key = RSA.generate(4096)
    f = open('private.key','wb')
    f.write(key.export_key())
    f.close()    

def get_public_key():
    key = RSA.generate(4096)
    f = open('public.key','wb')
    f.write(key.publickey().exportKey())
    f.close()    

def get_jwks():
    """
    # Don't add passphrase
    $ ssh-keygen -t rsa -b 4096 -m PEM -f jwtRS256.key
    
    $ openssl rsa -in jwtRS256.key -pubout -outform PEM -out jwtRS256.key.pub
    $ cat jwtRS256.key
    $ cat jwtRS256.key.pub
    """
    
    f = open("jwtRS256.key.pub", "r")
    public_key = f.read()
    f.close()
    
    jwk_obj = JWK.from_pem(public_key.encode('utf-8'))
    public_jwk = json.loads(jwk_obj.export_public())
    public_jwk['alg'] = 'RS256'
    public_jwk['use'] = 'sig'
    public_jwk_str = json.dumps(public_jwk)
    print(public_jwk_str)
    

if __name__ == '__main__':
    get_private_key()
    get_public_key()
    get_jwks()

Another method

These are used to verify the LTI application

~$ python
>>> from jwcrypto.jwk import JWK
>>> key = JWK.generate(kty='RSA', size=2048, alg='RS256', use='sig', kid=<random_string>)
>>> key.export_public()
{
    "alg": "RS256",
    "e": "AQAB",
    "kid": <random_string>,
    "kty": "RSA",
    "n": ...,
    "use": "sig"
}

Add this to public.jwk.json under configs/

Environment Configuration

Copy the environment example file as below.

~/Canvas-Flexible-Assessment$ cp .env.example .env
~/Canvas-Flexible-Assessment$ nano .env

# Database and user credentials (from Database Setup)
DB_ENGINE="django.db.backends.postgresql"
DB_NAME="flexible_assessment_db"
DB_USERNAME="flexible_assessment_user"
DB_PASSWORD=<password>

# Default settings for database
DB_HOST="localhost"
DB_PORT=""

DJANGO_SECRET_KEY=

# Canvas configurations (see Canvas Setup below)
CANVAS_OAUTH_CLIENT_ID=
CANVAS_OAUTH_CLIENT_SECRET=""
CANVAS_DOMAIN="https://canvas.example.com"

# Used to create hashes (see Generating Hash Salt and Password below)
ENCRYPT_SALT=""
ENCRYPT_PASSWORD=""

# Marks internal IP if application is running on a server, remove line if not being used
INTERNAL_IP=

Canvas Setup

We generate LTI and API keys to add the application to Canvas. The LTI key is used for the purposes of registering the app within Canvas and the API key is used for OAuth 2.0 flow providing the Instructor's token to get course data such as assignment groups, and to retrieve/submit student grades.

The keys can be generated from Admin>Developer Keys

Creating developer LTI Key

Change method to Paste JSON (Recommended since Manual Entry does not have all fields).
Set Key Name: Flexible Assessment LTI Key and Redirect URIs: https://<django_server>/launch/

Add JSON config below, change <django_server> to the domain of the app and <JWK> to the generated JWK json. The custom fields allows the app to recognise the user that has launched the app.

  {
      "title": "Flexible Assessment",
      "scopes": [
          "https://purl.imsglobal.org/spec/lti-nrps/scope/contextmembership.readonly"
      ],
      "extensions": [
          {
              "platform": "canvas.instructure.com",
              "settings": {
                  "platform": "canvas.instructure.com",
                  "placements": [
                      {
                          "placement": "course_navigation",
                          "display_type": "full_width_in_context",
                          "message_type": "LtiResourceLinkRequest",
                          "windowTarget": "_blank",
                          "default": "disabled", 
                          "target_link_uri": "https://<django_server>/launch/"
                      }
                  ]
              },
              "privacy_level": "anonymous"
          }
      ],
      "public_jwk": <JWK>,
      "description": "Flexible assessment app",
      "custom_fields": {
          "role": "$Canvas.membership.roles",
          "user_id": "$Canvas.user.id",
          "login_id": "$Canvas.user.sisSourceId",
          "course_id": "$Canvas.course.id",
          "course_name": "$Canvas.course.name",
          "user_display_name": "$Person.name.display"
      },
      "target_link_uri": "https://<django_server>/launch/",
      "oidc_initiation_url": "https://<django_server>/login/"
  }

Creating the developer API Key
1. Set Key Name: Flexible Assessment API Key, Redirect URIs and Redirect URIs (Legacy): https://<django_server>/oauth/oauth-callback/. Change <django_server> accordingly.
2. Set toggle for Enforce Scopes off. (Application uses GraphQL endpoint which is not scoped yet by Canvas)
Deploying the app
1. From Developer Keys, copy the Client ID under details for the LTI key. Add this to <client_id> in the flexible_assessment.json config file.
1. Go to Settings>Apps and add an app with Configuration Type By Client ID. Paste the LTI Client ID and Submit.
1. Get the Deployment ID from the Flexible Assessment app dropdown on the same page and add this to <deployment_id> in flexible_assessment.json.
1. From Developer Keys, copy the Client ID and Secret Key (click Show Key) under details for the API key and set CANVAS_OAUTH_CLIENT_ID and CANVAS_OAUTH_CLIENT_SECRET to these respectively in the .env config file.

Note: When testing on a local installation of Canvas LMS, <django_server> can be set to localhost:443 or 127.0.0.1:443. This app has been tested on stable/2022-06-22 branch of Canvas LMS. For instructions on how to setup Canvas locally see Installing Canvas.

Generating Hash Salt and Password

These are used to encrypt the Canvas OAuth 2.0 access tokens

~$ python
>>> from cryptography.fernet import Fernet
>>> salt = Fernet.generate_key()
>>> password = Fernet.generate_key()
>>> salt, password
Two long binary strings show up e.g. (b'23fva4-3187dagjf=sdf0-8973u2rel=', b'3291afiu=ahdw5783d231-ja9008aq=')

Add the strings within quotations to .env as ENCRYPT_SALT and ENCRYPT_PASSWORD

mod_wsgi and Apache Setup

We use mod_wsgi and Apache to deliver the application over secure. This is required as Canvas will only launch LTI apps over https.

mod_wsgi is a WSGI interface for Python web applications and Apache provides the web server for the content.

mod_wsgi

mod_wsgi needs to be configured with your specific Python version.

~$ wget https://github.com/GrahamDumpleton/mod_wsgi/archive/refs/tags/4.9.3.tar.gz
~$ tar xvfz 4.9.3.tar.gz
~$ cd mod_wsgi-4.9.3

~/mod_wsgi-4.9.3$ ./configure --with-python=<python_path>
~/mod_wsgi-4.9.3$ sudo make
~/mod_wsgi-4.9.3$ sudo make install

To get <python_path>

~$ conda activate django_env
~$ whereis python
python: <python_path>

Apache

Enable ssl module and create a Virtual Host configuration

~$ a2enmod ssl
~$ nano /etc/apache2/sites-available/flexible-assessment.conf

LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so
<VirtualHost *:443>
    ServerName flexible-assessment.com
    DocumentRoot /<path_to_project>/Canvas-Flexible-Assessment

    Alias /static /<path_to_project>/Canvas-Flexible-Assessment/flexible_assessment/static
    <Directory /<path_to_project>/Canvas-Flexible-Assessment/flexible_assessment/static>
        Options All
        AllowOverride All
        Require all granted
    </Directory>
    <Directory /<path_to_project>/Canvas-Flexible-Assessment/flexible_assessment>
        <Files wsgi.py>
            Options All
            AllowOverride All
            Require all granted
        </Files>
    </Directory>

    WSGIDaemonProcess Canvas-Flexible-Assessment python-path=/<path_to_project>/Canvas-Flexible-Assessment/flexible_assessment python-home=/<path_to_env>/anaconda3/envs/django_env/
    WSGIProcessGroup Canvas-Flexible-Assessment
    WSGIScriptAlias / /<path_to_project>/Canvas-Flexible-Assessment/flexible_assessment/flexible_assessment/wsgi.py process-group=Canvas-Flexible-Assessment

    SSLEngine on
    SSLCertificateFile <path_to_certificate>
    SSLCertificateKeyFile <path_to_key>

    ErrorLog /var/log/apache2/flexible_assessment_errors.log
    CustomLog /var/log/apache2/flexible_assessment_access.log combined
</VirtualHost>

Make sure to modify <path_to_project>, <path_to_env>, <path_to_certificate>, and <path_to_key>. Enable the site and reload Apache.

~$ a2ensite flexible-assessment
~$ systemctl reload apache2

If you are running a local installation of Canvas LMS on port 443 already, assign the Flexible Assessment app a different Virtual Host port and ensure Apache is listening on the port within ports.conf.

Note: You may have to change the group ownership of the project to www-data to avoid permission errors.

Troubleshooting

Creating a superuser

The primary id of a user is their Canvas user id so when creating a superuser we cannot use positive non-zero integers for the id. Furthermore, the Role of the superuser is set to 1 for Admin.

~/../flexible_assessment$ python manage.py createsuperuser
User id: 0
Login id: <login_id>
Display name: <display_name>
Role: 1
Password: 
Password (again):

WSGI Module not loading in Apache

You may need the appropriate header files and static library for your Python environment. You can install the Python 3.9 headers as such

~$ sudo apt-get install libpython3.9-dev

CA Bundle not in Apache environment

You may have to add the path to your CA bundle for Apache depending on your environment.

~$ sudo nano /etc/apache2/envvars

Append export REQUESTS_CA_BUNDLE=<path_to_bundle> to the file.

Environment variables not detected on local version

If the environment variables on your local machine are not detected (e.g. you receive errors that envvars such as DB_NAME are not defined), you may need to load them with the following:

First navigate to the location of your environment variables file (usually .env) the run the following:

~$ set -a
~$ source .env
~$ set +a

This should add them to the environment variables and you can check that they are active using the following command

~$ printenv

Running without Apache

To run the Django application without Apache, we can make use of runserver_plus from the django-extensions library.

Install dependencies

~$ pip install django-extensions==3.1.5 Werkzeug==2.1.2 pyOpenSSL==22.0.0

Then run the following command

~/../flexible_assessment$ python manage.py runserver_plus --key-file <path_to_key> --cert-file <path_to_certificate> --keep-meta-shutdown <django_server>

This method should not be used in production but it may be useful for simple testing.

Installing Chromium driver

Selenium might not run initially when running tests - if this happens, the problem may be that the Chromium driver is not installed on your system. To fix this, run the following command:

~$ sudo apt install chromium-chromedriver

Initially Connecting Django to the Database

If you have trouble connecting to the database with the default user postgres, instead create a new superuser named flexible_assessment_user with:

postgres=# CREATE USER flexible_assessment_user WITH SUPERUSER PASSWORD 'your_password_here';

Make sure to put the username and password into the .env file under DB_USERNAME and DB_PASSWORD.

Testing

Make sure postgresql is activated by running sudo service postgresql start. Also make sure inside settings.py you change SECURE_SSL_REDIRECT = True to SECURE_SSL_REDIRECT = False while testing.

Running tests

To run all the tests use the command python manage.py test. However, some of these tests are slow and some are used to view pages. To exclude tests tagged as 'slow', use the command python manage.py test --exclude-tag=slow. You can also use the command python manage.py test --tag=view to only run view tests. To run specific tests you can specify the directory. Such as to run the tests for instructors you can use the command python manage.py test instructor. Some tests are just to view the page, such as python manage.py test --tag=instructor_view and python manage.py test --tag=double_view (open for both instructor and student in the same course).

Writing tests

Functional tests: This uses selenium webdriver to view the pages. To get past the initial login authentication we use the code self.client.force_login(user) from django. Then we can retrieve the session id cookie session_id = self.client.session.session_key and add the cookie to the webdriver self.browser.add_cookie({'name': 'sessionid', 'value': session_id}). You can use functional tests to automate the testing of user interactions with the web page and/or use input("Enter key in terminal to continue") so you can keep the page open for you to interact with.

Unit tests: Some pages include calls to the Canvas api and therefore you must have the client keys and Canvas set up in order to test the full functionality. However, if we are just trying to test the functionality of our program around the Canvas api then we can mock the Canvas api to return dummy data. To do this you can use the custom decorator like this example:

    @tag('slow')
    @mock_classes.use_mock_canvas()
    def test_setup_course(self, mocked_flex_canvas_instance):

If you plan to add any canvas objects or their respective methods, you can override these in the mock_classes.py file. By writing methods for Mockcanvas, you can have it return user-defined objects that mimic the behaviour of Canvas object. You should refer to the Canvas API documentation to ensure that they meet the requirements for inputs and outputs so that all locally run unit tests pass and it remains functional in production as well. An example of this using Calendar Events is as follows:

class MockCalendarEvent(object):
    def __init__(self, dict):
        self.id = 12345
        self.title = dict['title']
        self.start_at = dict['start_at']
        self.end_at = dict['end_at']

    def edit(self, dict):
        return self

Here, the initialization takes a dictionary as an argument like the actual Canvas object. Additionally it outputs a MockCalendarEvent object when the edit method is called, similar to how the actual object returns a CalendarEvent object as per the documentation. (https://canvas.instructure.com/doc/api/calendar_events.html#method.calendar_events_api.update)

Notice how we added mock_flex_canvas as a parameter in the test function. How this works is we targeted FlexCanvas inside instructor.views now whenever FlexCanvas is called inside instructor.views, it gets replaced by MockFlexCanvas which we can modify to return whatever we want. View the documentation here: https://docs.python.org/3/library/unittest.mock.html

With regards to testing, certain functions of the code cannot be tested locally and must be tested on the development server, examples of these are:

Exporting grades
Setting the calendar dates
Deleting the calendar events

Provide feedback

Saved searches

Use saved searches to filter your results more quickly