Showing posts with label assessment. Show all posts
Showing posts with label assessment. Show all posts

Thursday, July 11, 2013

Assignments, Assessment and Badges

I was asked to quickly put some video resources together to provide an overview of my work with assignments, assessment and open badges. There is a day-long conference happening at La Trobe University in Bundoora and given I enjoy putting together resources for the work I am doing toward open education I was happy to take the time.

Deep apology for the quality of these resources. The moving trucks come tomorrow and I'm down to working off my five year old netbook. All these resources are issued with a CC-BY so feel free to use them as you will; edit them, mash them up. I believe these three videos cover the why, what and how of assignments, peer assessment and open badges.

Open Badges and Peer Assessment
Quick discussion of open badges, their meta-data and how they can be utilised within courses where peer assessment is a part of overall assessment.


Flipped Assessment
Flipped assessment has people earlier in a learning journey assess the work of someone further down the learning journey. This creates an environment where people currently involved with a subject reach-out to assist others who are also involved. This encourages peer involvement and adds to assessment.


Flipped Assessment Implemented
So this is what we did to implement flipped assessment and how the assessment criteria was grounded in the rubric that the course was based upon.


Relevant References



Creative Commons Licence
This post is licensed under a Creative Commons Attribution 2.0 Canada License.

Monday, April 08, 2013

Flipped assessment implemented

The 301 - Badge System Design course being built for the P2Pu School of Badges will also include flipped assessment. The basic idea of flipped assessment is to have people early on in a shared learning journey assess those who are a few lessons ahead. The thinking behind this is the people most invested in giving and receiving collaborative assessment and peer review are those currently active in a learning journey.


Within the P2Pu Badge System Design course the flipped assessment occurs twice. Once during task 3 where early learners review a badge system design created by someone almost finished the P2Pu course. And again as a peer assessment of another learners compare and contrast task.



The collaboration is supported by both early and later learners having to reach out to each other to complete the challenge. The fun part is how the early learners have to find an open badges quick issuing site to award badges to the later learners for completing the tasks they are reviwing. and without these in-course awards the learner will not achieve overall completion of the P2Pu challenge.

Tuesday, February 05, 2013

Flipping assessment

I've been exploring the idea of flipping assessment for a while. It started with my work as a TA during my M.Ed graduate studies, also explored it during my time with WikiEducator, it was confirmed as I reflected upon my graduate studies and every community of practice since, and I designed it into a language learning portal I architected a while back. Why do we have people who have already completed the work (and possibly long since forgotten the details) assess those currently working on building their understanding. Why not have the cohort of learners most current with the subject provide the bulk of the assessment. Why not flip the assessment and have the novice learners assess those a few steps ahead on the same learning journey? I believe the cohort of novice learners (working with master learners) would be the most motivated to develop understanding and invest the most time in the assessment of another learners work. Particularly, if the assessment event is a formative event and just a part of the overall assessment.

This is what I envision for flipping assessment. This wouldn't be a complete flip, where novice learners do all of the assessment, but they would have an increased role in the assessment of the learning cohort. As I see it the elements of the diagram are as follows;
  • Working from left to right a person masters a subject.
  • A subject within a knowledge domain is exactly that, a complete subject that fits within a larger domain of knowledge. An example would be baking bread is a subject within the food preparation (or cooking, or baking) knowledge domain. And the subject of bread baking is large enough to be considered a subject within itself where someone would develop an expertise.
  • An Interested learner is a person with an interest in the subject, who never actually studies the subject, but they will read about it and not shy away from a discussion.
  • A Novice learner is a person committed to learning a subject. They actively seek out information, read and consume information about the subject. They participate in learning events (online and otherwise) to increase their understanding. They would even take a course from a traditional institution.
  • A Master learner is someone nearing completion of their studies of a subject. They could be considered to have "almost" mastered the subject.
  • Teaching assistants are people who are rewarded for providing mentorship to learners. This role is a part of traditional institutional learning, and plays a role in helping learners master a subject. Teaching assistants are often much closer to a subject than a professor as their learning was more recent and, given their role, they are closer to the learning cohort due to their frequency of personal interaction with the learners.
  • A Seasoned learner is someone who has mastered a subject, and is not actively involved in learning the subject. They do continue to read about the subject and actively seek out new and related information. They are members of communities of practice regarding a subject and occasionally engage and/or contribute to the community. They are very current with the subject and provide feedback and guidance to other community members.
  • The Professors are well, professors. They are most often people with Ph.D`s who are very active in furthering and adding to a domain of knowledge. Their focus is not always on teaching or the details of a specific subject within their chosen domain of knowledge, unless that is their area of research. I guess what I want to say is that a professor won`t be focused on the subject matter (and the learners study) of an undergraduate level course and its specific subjects. This is not to say professors can`t be an amazing resource for someone learning a new subject. A professors time should be used very wisely, but finding ways to encourage their engagement is good.
  • The Experts are people who, through a number of different methods, have become experts in a subject. They continue to be experts by ongoing reading, personal research tasks, and involvement with online communities. They sometimes lurk (even engage) in online learning communities. Getting experts to contribute to assessment would be great, and finding ways to include them is time well spent.
My plan to implement flipped assessment
I want to experiment with the idea of having novice learners assess the work of master learners. Both the novice and master learners need to be active within the same learning journey, where the master learner is only a few lessons (or tasks) ahead of the novice. The idea being the novice is beginning their study of the subject and the master is almost finished.  I want the novice learners to read and understand the work of a master learner and give it an assessment. If the novice has questions, they ask them of the master learner. The master learner also has to encourage the novice learners to assess their work. Without this assessment neither the novice or the master will progress to completion. If novice learners want to team up to assess a master learners work, they can. This flipped assessment will not be the only assessment the learners will receive as a peer based summative assessment will occur when a learner has finished their study of the subject.

I plan to implement this flipped assessment in the Badge System Design challenge I am creating in P2Pu. The design of the challenge is iterative in that we will be covering the topics more than once with increasing depth as the challenge progresses. This iterative approach allows for a task earlier in the challenge to include the assessment of another learners completed works later in the challenge. Essentially, a novice learner is assessing the work of a master learner. Flipped Assessment.

If you are interested in the previous background chat discussing this idea it can be found in Google+.

Friday, March 23, 2012

OER Assessment

I believe new approaches to open assessment need to be built to handle the growing global demand for education and learning. I am not alone in my thinking. I believe there is no shortage of great OER and a growing number of approaches toward open accreditation. These approaches to accreditation follow both traditional models and those more innovative. The point I am wanting to make is; there is no shortage of good open educational materials and there are some solid open accreditation services emerging. What I believe is missing is open assessment, for assessment closes the gap between studying educational materials and receiving accreditation for that study.

Inaugural ICT4D event participants.
This post is about open assessment and the approaches that I believe will work toward creating open assessment for the masses. I also think some background on myself and why I have developed my way of perceiving open assessment is important. I haven't taken the traditional route in obtaining an education; yet, in the end I have received a Bachelors of Technology from the Open Learning Agency of BC (OLA) and a Masters of Education from Memorial University of Newfoundland (MUN). But to think it was a straight line toward these credentials would be wrong. My OLA B.Tech degree was by cobbling together a couple of technology diplomas, with University credits from three different institutions. My MUN M.Ed IT was all online, even though I spent a year on campus, all my coursework was done online. Needless to say I have often taken alternate routes toward a traditional destination.

I know one of the biggest influences in my adult life was my participation at the inaugural ICT4D event at Royal Holloway in 2006. Ever since then I have been focusing a large part of my working and volunteer efforts in blending technology and education for the greater good. The paper I presented was about what I called a "critical technologist". It is really about the role someone could take in assisting to develop capacity with ICT in developing countries. The "critical technologist" is a play on Paulo Friere's Critical Pedagogy. I am strong believer in Paulo Friere's work. The three areas of work I have done within this capacity since 2006 have been;
  • WikiEducator (2007 - 2010) - I will always look fondly upon my work with WikiEducator. I was a large  contributor at its early stages and wrote a few key pieces that are still in use today. To see my participation, review my profile; http://wikieducator.org/User:Prawstho. I was honoured and fortunate to be nominated to WikiEducator council, and in 2010 we parted ways. On occasion I still contribute to WikiEducator, though my focus is more in Wikiversity.
  • Continuing Legal Education of BC (2008 - 2011) - I was Senior Project Manager / Lead Technical Architect for a large grant to increase access to education for the lawyers of British Columbia. We used a lot of open source software and open approaches to provide a services orientation for building learning communities; This 2011 post titled "Increasing access to education", does good job describing what we did.
  • Progressive Video Assessment - I came up with this learning systems architecture in 2008 and had the opportunity to implement it in a couple of projects; AIM Language Professional and CLE Engagements Questions.
How does my educational and work experience influence how I believe OER assessment should be built? I believe it can be distilled into eight important points;
  1. It should be peer-based
  2. It should be scalable
  3. It should be fun and enjoyable
  4. It should provide many reusable instruments
  5. It should play well with others
  6. It should be reference-able
  7. It should support open data
  8. It should feed into open credentialing
I believe open assessment should be easily integrated with other approaches of assessment and credentialing. It should assist learners who are attending traditional institutions as well as alternative approaches like OERu and P2Pu. It should also support those who want to learn independently without ever stepping foot in a traditional institution (online, virtual or otherwise) or those independent learners who dont seek credentials. I really like the parallel universe approach being offered by OERu; yet, I am also concerned about the need for fees and my internal critical pedagogue is tingling. I have extended the parallel universe approach being proposed by OERu.

OERu parallel learning universe with an additional dimension.

Approaches to Open Assessment:
  • In the immediate term we should build peer-assessment to utilize and promote the use of badges (see wikieducator: http://wikieducator.org/User:Prawstho or Mozilla badges for example). This will get us going quickly and then build upon this with greater automation and other assessment approaches.
  • In the near term we should build rubrics to allow people to perform self-assessments and then submit portfolio for a peer-review. Peer reviewers should also be given badges for executing N peer reviews. This does not require a lot of automation, can be implemented quite quickly.
  • In the close term we should build formative and summative assessment instruments that can be baked into websites, apps and tools. People should be able to edit / improve on instruments in a community kind of way.
  • Beginning immediately and through the next year we should start building apps and experimenting with mass collaboration for assessment.
  • In the medium term we should build a repository of guidelines, templates, assessment instruments and approaches.
  • In the long term we should build a fun community around assessment and broadening peoples skills and knowledge regarding assessment and accreditation.
  • As an ongoing initiative we should encourage research and centers of excellence around open assessment and its relationship with open accreditation.