Drumbeat/p2pu/Assessment and Accreditation/Assessment Schema 2

This document attempts to outline the possible relational schema for organizing assessment types, as nested subsets of properties. It is a crude form of a relational (semantic) database. Ideally, we should work through these relationships using Semantic Mediawiki or something similar.

Additional notes regarding these schemas

While there are undoubtedly many different ways to build pathways and connections among content items, assessments of that content, and visible outcomes suitable for external evaluation (and accreditation), it is simplest to think of your options as either running forwards or backwards. Going forward, you start with the content you wish to teach (or learn), determine how you will assess that learning, and then brainstorm the possible outputs. Going backwards, you start with the ultimate outcomes, determine what levels of skill and understanding are necessary to get there, and then figure out what knowledge is needed.

There are many good reasons to employ one approach or the other, or more likely, some combination of both approaches. For the sake of illustration, we have worked through simple and mostly generic examples of both methods below.

Read below for specific thoughts as they relate to course design in P2PU.

Working forward

• Assessment type (generic) - e.g., one-item queries, exams, writing samples, etc.

has properties

• Delivery (or implementation) timing - e.g., just-in-time, scheduled or partition checks, etc.
• Delivery scope - e.g., group, individual, think-pair-share, etc.
• Delivery format - e.g., stand-alone, integrated, pop-up, third-party app, etc.

has properties

• Info origin - e.g., from course organizer, from course participants, external source, etc.
• Amenable to adaptation - e.g., by organizer/designer, by participants (problem manipulation), with minor or major expertise in design and evaluation, etc.
• Formative or summative – depends on handling of the data.
  • If formative
    • Should be subject to revision or re-testing.
  • If summative
    • Should contribute meaningfully to portfolio.
• Tied to specific interventions - e.g., some items will be better suited to specific compensating resources than others.

has properties

• Required or optional for course.
• Portfolio-building or stepping stone.
  • If stepping stone:
    • Provides bridge to what?

Working backward

• Ultimate products or outcomes - e.g., a well designed website (note - using this for continued example below), a piece of working code, the completion of a well received plan of action for development, etc.

will require

• Writing of code (in X language) that accomplishes X, subject to coding standards as specified by X.
  • will require
    • Competence in coding in X language.
    • Diagrammatic evidence of understanding regarding how to accomplish X.
    • Knowledge of coding standards from X and why they are important.
• High usability value for the site.
  • will require
    • Application of usability metrics for website development, as specified by X.
      • will require
        • Understanding of usability issues and different approaches.
        • Capacity to implement metrics-gathering tools.
    • Research-based evidence of site improvements.
      • will require
        • Capacity to analyze and build on metrics.
        • Understanding of the strengths and limitations of different site metrics.
• Positive feedback from target audience.
  • will require
    • Identification of target audience(s), and establishment of working relationship with that audience at an appropriate time in project development.

will require

• Good collaboration and communication skills.
  • will require
    • Appropriate use of communicative and collaborative technologies.
      • will require
        • Identification of needs, matched to potential tools.
          • will require
            • Access to the best tools for the job.
            • Knowledge of how to use those tools most effectively, for self and others.
    • Management of collaborative process.
• Proficient use of existing technologies and processes.
  • will require
    • Sufficient knowledge of the field and evidence of reasoning behind the use or rejection of existing resources.
    • Provision of evidence regarding the success or failure of any component resource or process as an input to the ultimate goal.
• Appropriate responses to feedback and self-checks during development.
  • will require
    • Honest consideration of outside expertise.
    • Excellent meta-cognitive skills.
    • Appropriate use of evidence.

Course design for P2PU

We would never presume to dictate a single methodology of course design for P2PU, nor indeed a single pedagogy, course of study, area of interest, etc. However, the particular emphases of P2PU - peer production, distributed collaboration, individualized learning, openness - lends itself best to backwards course design.

We envision that most courses will be built around some desired outcome - a public web page, a compendium of course-related writings, and so on - which means that the course will be built with an eye to those specific goals. From there, it makes most sense to step backwards to determine how the course should be structured, including the expected schedule, the content and skills needed, and educational habits that are preferred.

Next step here is to identify existing templates for backwards course design which can serve as general guides for P2PU course construction. We'll then test out the templates using planned outcomes for Open Web Craft courses gathered from the community.