Eric Shepherd’s Blog is now at http://blog.eric.info

Eric Shepherd’s blog is now at http://blog.eric.info

February 21, 2011 at 1:21 pm Leave a comment

Learning Mashups

Every day I am asked to explain mashups and their impact on the world of learning and assessments and I felt that it was about time that I shared something more widely.  In the context of learning and assessments mashups can be categorized as follows:

  1. Portlet – configured by an end user
  2. Page (Web, Wiki or Blog) – configured by a web author
  3. Data – configured by a programmer

Portlet Based Mashup

image Portlet based mashups provide different “windows” that interact with different and separate applications and users can move these portlets around their screen to meet their own personal requirements.

This is explained in the YouTube video below:

How is this applicable to learning and assessment?

As organizations standardize and align on their IT architecture there will be a framework in which to view applications in the context of someone’s job role.  The job roles for CLOs, Training Managers, Instructors, Teachers, Professors, Instructional Designers, Psychometricain, students and other stakeholders are different and have different preferences, priorities and organizational styles.  Portlet based user interfaces allows users to view their screens in the way that they find them most useful to perform their tasks.

Page Based Mashup

Page based mashups are configured by the author of the web page, wiki and or blog to determine where application data and interactions might occur.  As an author of a blog I have inserted a YouTube video above and another below that explains how to embed a video and assessment into the same wiki page:

To learn more about inserting assessments into web pages, blogs and wikis check out: http://questionmarkdemo.wikispaces.com/

Back to mashups…

This blog posting is presented as an HTML web page, with HTML text and a video feed from YouTube and a assessment feed from a Questionmark server.

image The diagram to the left shows this diagrammatically.  The main page contents are coming in from the web server, and in this case, a Questionmark assessment is coming in from another server.  The author of this web page is the person that chose how this design would be rendered.

How is this applicable to learning and assessment?

Documentation, learning content and assessments are often available throughout an organization’s infrastructure but challenging to find and they are especially difficult to find in context.  Wikis, blogs and web pages can be used to expose existing resources, wrap context, to provide a meaningful learning and/or assessment experience. Learning mashups can improve learner experiences and learning outcomes.

Data Mashups

image

A data mash up is where data is drawn from different data sources and displayed within a single page.  In this style a programmer constructs a program to retrieve the data, from different data sources, and then display it within a web page.

image

The diagram to the right illustrates the data that is mashed up to provide a presentation of the price of taxis in New York City.

This form of data mashup used to require that the application had direct access to the database.  These days, with web-services, it is possible to read and write data anywhere in the world with a single application.

How is this applicable to learning and assessment?

With more data being accessible via web-services, applications can now provide powerful reports to assist decision making to improve learning outcomes, drive qualifications when required and boost performance.

Conclusion

Standard tools and technologies supporting blogs, wikis and web pages can provide new styles of easy-to-maintain stimulating and powerful learning environments.

Did you learn something from this mashup?

September 1, 2009 at 1:40 pm 3 comments

We’ll Kick Off the Questionmark European User Conference at Manchester United’s Old Trafford!

imageSometime things get surreal!

As I kid most of my friends were soccer fans or should I say football fans! I didn’t support any particular team but I did see Chelsea, Arsenal and Crystal Palace games live and saw many more on TV.

I never quite made it to see Old Trafford – but now, thanks to Questionmark, I will!

image

The 2009 Questionmark European Users Conference is coming up next month and this year we’re holding an evening event actually in the Old Trafford and I’m excited about it! 

Whether or not you’re a Manchester United fan, the Old Trafford is one of England’s most impressive stadiums. It’s going to be fun and I’ll be a little humbled by the experience.  Most importantly I’ll be able to share the experience with customers, partners and employees which is going to make it a night to remember. 

Hopefully I’ll see you there! There is still time to register at::
    http://www.questionmark.com/uk/conference/index.aspx 

imageimageimage

August 27, 2009 at 8:17 am Leave a comment

Added Vocabulary Page

Added the first draft of a vocabulary page to the Assessment Maturity Model website at:
  http://www.assessmentmaturitymodel.org/vocabulary.html

August 23, 2009 at 4:12 pm Leave a comment

Keeping In Touch with The Assessment Maturity Model

I have been busy today finalizing links, logos and systems to help people keep in touch with what we’re doing with the Assessment Maturity Model.  Added links for email subscriptions (using Constant Contact), Twitter and an RSS feed to the blog.  Also made the blog to be a sub-domain for easier access (i.e. http://blog.assessmentmaturitymodel.org).

If you want to keep in touch click here http://www.assessmentmaturitymodel.org/keepintouch.html to discover your options.

August 23, 2009 at 12:34 am Leave a comment

Assessment Maturity Model

Over the last few years I have been working with customers and friends on an Assessment Maturity Model. It started with brainstorming, it was developed using a wiki and it has been tested by representing the idea to a number of groups around the world.  Now I feel that it is ready to take to the next step.

Over the last few weeks I have been formalizing the model to make it easy to understand and building a web site http://www.assessmentmaturitymodel.org/ 

The premise behind the Assessment Maturity Model is that if you can’t measure it you can’t manage it. But to measure it you need to know what "it" is. 

image

 

The Assessment Maturity Model proposes that there are six key performance indicators, known as Measures, within the three key Areas of Assessments, namely:

  • Assessment Development
    This area includes all aspects of authoring (creating and maintaining) the items and assessments.
  • Assessment Delivery
    This area includes all aspects of administering the assessment to candidates, respondents, participants, etc.
  • Presenting Results
    This areas deals with presenting results in a trustworthy way to the stakeholder in meaningful context.

Six key performance indicators, known as Measures, are tracked to provide an indication of maturity and these are:

  • Stakeholder Satisfaction
  • Security
  • Strategic Goals
  • Processes
  • Data Management
  • Communications with Stakeholders

image By tracking these Measures an organization can determine where they are and can plan for where they want to be. These can be tracked by a single Area, as shown in the graphic to the left, or with the three Areas combined for an overview. The graphic to the left shows how an organization can track by area based on "Quality" and "Efficiency".

If you manage any type of assessment program I’d encourage you to take time to learn more about the Areas, Measures and Phases of the Assessment Maturity Model.  or just

Please feel free to link to the Assessment Maturity Model web site at http://www.assessmentmaturitymodel.org/, cross link from other web sites and blogs, twitter about it, email me about it, comment here about it and/or even tell your friends about it! 

Watch out for more – this isn’t finished yet!

August 14, 2009 at 12:14 am Leave a comment

Types of Assessments – Learn, Qualify, Perform

image I wanted to share some work with you on how to logically organize assessments and came up with this simple model which seems to resonate with the people that I present it to.  You might also be interested in my previous article on Types Of Assessments (Formative, Diagnostic, Summative, and Surveys)..

There are assessments for learning and assessments to qualify people for certain activities or job roles; then they perform tasks. Throughout the process of learning, qualifying and performing, we accumulate evidence and feedback that help us make decisions. 

The definitions and distinction of the various assessment types that I refer to in this article are documented within the Assessment Maturity Model at http://assessmentmaturitymodel.wikispaces.com/Assessment+Solutions:

Learn

image Assessments Through The Learning Process can be distinguished and linked together by thinking of:

  • Pre-Learning Assessments
  • In-Learning Assessments
  • Of-Learning Assessments

Pre-Learning Assessments

The following assessments are used prior to learning events:

  • Job Task Analysis
  • Needs Analysis Surveys
  • Diagnostic assessments
  • Pre Tests
  • Placement tests
  • Self-diagnostic tools
  • In-Learning Assessments
  • Of-Learning Assessments

In-Learning

The following assessments are used within learning events to assist the learner learn:

  • Formative assessments
  • Quizzes during Learning
  • 360 Learner Peer Review
  • Practice Tests

Of-Learning

The following assessments are used to detect what a learner has learned:

  • Level 1 Surveys
  • Course evaluations
  • 360 (Level 3) Surveys
  • Post Course Tests
  • Graduation Exams
  • Internal exams
  • Open exams
  • Licensing exams

Qualify

image I struggled with this for a long time on whether qualify should be its own category but finally I came to the conclusion that it should because it provides sufficient distinctions on its own and could be separated from learning experiences. 

There are three basic categories of Assessments To Qualify and these have various styles of assessments associated with this:

  • Academic: Graduation Exams
  • Pre-Employment: Pre-employment Screening, Pre-employment Skills Tests, Personality assessments, Psychological assessment
  • Certification & Licensing: Graduation exams, Internal exams, Open exams, Licensing exams

Perform

imageAfter learning and qualifying we end of up performing manual and intellectual tasks. And as we do things evidence surfaces that can be referenced to help us make decisions about more learning, more qualifications and our performance.

This evidence can be used and assessed within an organizations’ Performance and Talent Management processes to help people become more successful in their work 

Performance and Talent Management

image Performance Management is a method used to influence and manage behaviors and results with the goal of bringing out the best in people.  Talent Management refers to the phases of finding, developing and retaining people to perform activities. 

Within these phases people engage in learning and qualifications and their activities tends to accumulate evidence that assists them and others assess their performance.

Talent Management assessments can be categorized with the three phases of Talent Management:

  • Talent Acquisition – finding and employing the right people
    • Job Task Analysis
    • As per Pre-Employment assessments i.e. Pre-employment Screening, Pre-employment Skills Tests, Personality assessments, Psychological assessment
  • On-Boarding – orienting new people to the workplace
    • As per Assessments Through The Learning Process
  • Performance, Talent and Team Management – to help people be successful
    • As per Assessments Through The Learning Process
    • Appraisal (360s)
    • Employee Attitudes
    • Opinion Surveys

I hope you have found this posting useful in showing how the types of assessments relate to Learning, Qualifying and Performing as well as getting things done!

August 3, 2009 at 12:09 pm Leave a comment

Older Posts


Add to Technorati Favorites

Recent Posts

Categories


Follow

Get every new post delivered to your Inbox.