Posts tagged ‘Learning’

Learning Mashups

Every day I am asked to explain mashups and their impact on the world of learning and assessments and I felt that it was about time that I shared something more widely.  In the context of learning and assessments mashups can be categorized as follows:

  1. Portlet – configured by an end user
  2. Page (Web, Wiki or Blog) – configured by a web author
  3. Data – configured by a programmer

Portlet Based Mashup

image Portlet based mashups provide different “windows” that interact with different and separate applications and users can move these portlets around their screen to meet their own personal requirements.

This is explained in the YouTube video below:

How is this applicable to learning and assessment?

As organizations standardize and align on their IT architecture there will be a framework in which to view applications in the context of someone’s job role.  The job roles for CLOs, Training Managers, Instructors, Teachers, Professors, Instructional Designers, Psychometricain, students and other stakeholders are different and have different preferences, priorities and organizational styles.  Portlet based user interfaces allows users to view their screens in the way that they find them most useful to perform their tasks.

Page Based Mashup

Page based mashups are configured by the author of the web page, wiki and or blog to determine where application data and interactions might occur.  As an author of a blog I have inserted a YouTube video above and another below that explains how to embed a video and assessment into the same wiki page:

To learn more about inserting assessments into web pages, blogs and wikis check out:

Back to mashups…

This blog posting is presented as an HTML web page, with HTML text and a video feed from YouTube and a assessment feed from a Questionmark server.

image The diagram to the left shows this diagrammatically.  The main page contents are coming in from the web server, and in this case, a Questionmark assessment is coming in from another server.  The author of this web page is the person that chose how this design would be rendered.

How is this applicable to learning and assessment?

Documentation, learning content and assessments are often available throughout an organization’s infrastructure but challenging to find and they are especially difficult to find in context.  Wikis, blogs and web pages can be used to expose existing resources, wrap context, to provide a meaningful learning and/or assessment experience. Learning mashups can improve learner experiences and learning outcomes.

Data Mashups


A data mash up is where data is drawn from different data sources and displayed within a single page.  In this style a programmer constructs a program to retrieve the data, from different data sources, and then display it within a web page.


The diagram to the right illustrates the data that is mashed up to provide a presentation of the price of taxis in New York City.

This form of data mashup used to require that the application had direct access to the database.  These days, with web-services, it is possible to read and write data anywhere in the world with a single application.

How is this applicable to learning and assessment?

With more data being accessible via web-services, applications can now provide powerful reports to assist decision making to improve learning outcomes, drive qualifications when required and boost performance.


Standard tools and technologies supporting blogs, wikis and web pages can provide new styles of easy-to-maintain stimulating and powerful learning environments.

Did you learn something from this mashup?


September 1, 2009 at 1:40 pm 3 comments

Types of Assessments – Learn, Qualify, Perform

image I wanted to share some work with you on how to logically organize assessments and came up with this simple model which seems to resonate with the people that I present it to.  You might also be interested in my previous article on Types Of Assessments (Formative, Diagnostic, Summative, and Surveys)..

There are assessments for learning and assessments to qualify people for certain activities or job roles; then they perform tasks. Throughout the process of learning, qualifying and performing, we accumulate evidence and feedback that help us make decisions. 

The definitions and distinction of the various assessment types that I refer to in this article are documented within the Assessment Maturity Model at


image Assessments Through The Learning Process can be distinguished and linked together by thinking of:

  • Pre-Learning Assessments
  • In-Learning Assessments
  • Of-Learning Assessments

Pre-Learning Assessments

The following assessments are used prior to learning events:

  • Job Task Analysis
  • Needs Analysis Surveys
  • Diagnostic assessments
  • Pre Tests
  • Placement tests
  • Self-diagnostic tools
  • In-Learning Assessments
  • Of-Learning Assessments


The following assessments are used within learning events to assist the learner learn:

  • Formative assessments
  • Quizzes during Learning
  • 360 Learner Peer Review
  • Practice Tests


The following assessments are used to detect what a learner has learned:

  • Level 1 Surveys
  • Course evaluations
  • 360 (Level 3) Surveys
  • Post Course Tests
  • Graduation Exams
  • Internal exams
  • Open exams
  • Licensing exams


image I struggled with this for a long time on whether qualify should be its own category but finally I came to the conclusion that it should because it provides sufficient distinctions on its own and could be separated from learning experiences. 

There are three basic categories of Assessments To Qualify and these have various styles of assessments associated with this:

  • Academic: Graduation Exams
  • Pre-Employment: Pre-employment Screening, Pre-employment Skills Tests, Personality assessments, Psychological assessment
  • Certification & Licensing: Graduation exams, Internal exams, Open exams, Licensing exams


imageAfter learning and qualifying we end of up performing manual and intellectual tasks. And as we do things evidence surfaces that can be referenced to help us make decisions about more learning, more qualifications and our performance.

This evidence can be used and assessed within an organizations’ Performance and Talent Management processes to help people become more successful in their work 

Performance and Talent Management

image Performance Management is a method used to influence and manage behaviors and results with the goal of bringing out the best in people.  Talent Management refers to the phases of finding, developing and retaining people to perform activities. 

Within these phases people engage in learning and qualifications and their activities tends to accumulate evidence that assists them and others assess their performance.

Talent Management assessments can be categorized with the three phases of Talent Management:

  • Talent Acquisition – finding and employing the right people
    • Job Task Analysis
    • As per Pre-Employment assessments i.e. Pre-employment Screening, Pre-employment Skills Tests, Personality assessments, Psychological assessment
  • On-Boarding – orienting new people to the workplace
    • As per Assessments Through The Learning Process
  • Performance, Talent and Team Management – to help people be successful
    • As per Assessments Through The Learning Process
    • Appraisal (360s)
    • Employee Attitudes
    • Opinion Surveys

I hope you have found this posting useful in showing how the types of assessments relate to Learning, Qualifying and Performing as well as getting things done!

August 3, 2009 at 12:09 pm Leave a comment

Wikis and Social Constructivism: “Learning is best when learners collaborate to create…”

I wanted to share an article with you, that is short, meaningful and useful, on how wikis not only provide the benefits of knowledge sharing but also assists learners learn: Practical Applications of Research: Wikis for learning – what to expect by Dr. Irene Boland

From Dr. Boland’s research the key learning benefits of wikis appear to me to be:

  • Receive immediate feedback from others
  • Curiosity about how others are responding to, or modifying, their work causes learners to return to the wiki and refresh their memory regularly.
  • Learners can chart their own course for learning based on their interests.
  • Learn by reading other’s postings even if the leaner does not post.
  • Develop their own methods of classifying content in ways that make sense to them, rather than using existing systems of classification.
  • Help their peers find relevant content by assigning tags or keywords to pages


July 6, 2009 at 9:24 am 1 comment

A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives


A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives

by Lorin W. Anderson (Author), David R. Krathwohl (Author), Peter W. Airasian (Author), Kathleen A. Cruikshank (Author), Richard E. Mayer (Author), Paul R. Pintrich (Author), James Raths (Author), Merlin C. Wittrock (Author)

Here is the link to A Taxonomy for Learning, Teaching, and Assessing. I don’t yield any money from this I just like to share the knowledge.

May 18, 2009 at 12:15 pm Leave a comment

Trends in Assessment as they relate to Learning Professionals – 1st YouTube Interview!

I wanted to share the link to the first YouTube video interview that I have been featured in which was directed and produced by Dr. Will Thalheimer of Work-Learning Research.  Will, a great guy and one of the world leading experts and meta researchers in the field of memory recall, interviewed me on Trends in Assessment as they relate to Learning Professionals.  

April 11, 2009 at 11:38 am 3 comments

Formative Assessments for Medical Students

One of my friends, traveling in Australia and attending the LearnX conference,  alerted me to a study by researchers from the Faculty of Medicine at The University of New South Wales that I thought might be useful after my post on the various Types of Assessments.

The study set out to determine whether participation and performance in integrated online formative assessments in the biomedical sciences has measurable effects on learning by junior medical students. Students enrolled in Phase 1 (Years 1 and 2) of an undergraduate Medicine program were studied in 2006 and 2007. The researchers analyzed end-of-course summative exam marks in seven consecutive courses to study the effect of participation and performance in voluntary online formative assessments. Students shared their perceptions about the formative assessments using online evaluation surveys – and rated them highly.

imageAn example of the data being presented in the report is shown to the level.  What’s more, participation in formative assessments had a statistically significant positive relationship with EOC marks in all courses. The results of the study support the contention that "well designed formative assessments can have significant positive effects on learning. There is untapped potential for use of formative assessments to assist learning by medical students and postgraduate medical trainees."

You will find an abstract of this study on BioMed Central (but beware there is at least one “medical” graphic in the actual report):

April 2, 2009 at 4:29 pm Leave a comment

As Learning and Assessment Professionals What Could We Have Done to Prevent the Financial Crisis

As I prepare my various presentations for our Questionmark Users Conferences in Memphis, Tennessee in the context of an economic downturn but a thriving assessment market I can’t help thinking about how we as learning and assessment professionals could have done more to prevent the economic crisis.

A radical thought but take this journey with me.  The current crisis stems from:

  • Unethical behaviors in the mortgage sales process
    • Where were our regulators?  Did they know what they were meant to be doing? Were they assessed?
    • Were these mortgage brokers tested on ethics?
    • Was the chain of command trained on how to spot and prevent fraud and deceptive practices?
    • Do our certification methodologies need to move into the 21st century? We have so many tools to raise people’s levels of consciousness and thereby their accountability. But are we using them as effectively as we could?
  • Lack of Consumer Education
    • Businesses are out to make money. Credit helps us borrow against our future in order to have something now but borrowing excessively causes stress while making others wealthy. How many borrowers fail to understand this and why?
    • I wish I had learned fiscal responsibility earlier on in my life. Did I miss that class at school or college or did someone forget to put that into the curriculum?
    • Should/could there be elearning courses and assessments that could be part of the lending process to ensure that consumers are competent to take on their responsibilities?
  • Lack of understanding of new (unstable) financial instruments (e.g. Credit Default Swaps)
    • It seems to me that the financial instruments, or weapons of mass financial destruction, were poorly understood.  We don’t let children play with fireworks for very good reasons!  Many good people’s financial future has been wrecked by financial expert’s incompetence.
    • Just as we don’t let our pilots fly complex machines without educating them and testing them I don’t think that we should have amateurs pretending that they understand these complex instruments.
    • Is there any process to confirm that complex financial instruments are understood by the people who construct and trade them? Are they competent?
    • As assessment professionals we know that when someone says “Trust me I know” it often means that they don’t!
    • Should there have been more effective learning around these financial instruments?
    • Should they have tested to see if they had sufficient competence to play the high-risk financial games that they were playing?
  • C level executives exposing their organizations to unacceptable levels of risk
    • Some organizations were so focused on training and certifying new employees being on-boarded to handle their growing businesses that they forgot about educating and assessing middle managers and above.
    • I’m CEO of Questionmark.  I go on training courses. I do tests. I have to take our internal tests.
    • Should we not have batteries of test instruments to confirm that our C levels have the business understanding and ethics to run the organizations (banks, etc) that we as a society depend on?

Okay, so you now get what I’m talking about and you get where I am going!

So Learning and Assessment professionals stand up and be counted!  You have knowledge, skills, abilities and attitudes that can help us learn from recent experiences and help prepare us for a more ethical and stable future.  Your organization employs you for your expertise and so I ask that you use your knowledge and skills to help and benefit us all.

Anyone that knows me gets that I am passionate about assessments and I love learning. I especially love learning from our customers.  At this year’s User Conference I’d love to learn what we could have done to prevent the unethical/uneducated behaviors and questionable practices that led us on this journey of mass financial destruction not to agonize over what has happened but to help our profession be a part of the solution.

Lets make a difference and stand up for our profession!

March 28, 2009 at 3:37 pm 6 comments

Add to Technorati Favorites

Recent Posts