Definition

A competency-based method whereby abilities are measured in most direct, real-world approach. Systematic measurement of overt demonstration of acquired skills.

Advantages

• Provide a more direct measure of what has been learned (presumably in the program).
• Go beyond paper-and-pencil tests and most other assessment methods in measuring skills.
• Preferable to most other methods in measuring the application and generalization of learning to specific settings, situations, etc.
• Particularly relevant to the goal and objectives of professional training programs and disciplines with well defined skill development.

Disadvantages

• Ratings of student performance are typically more subjective than standardized tests.
• Requires considerable time and effort (especially frontloading), thus being costly.
• Sample of behavior observed or performance appraised may not be typical, especially because of the presence of observers.

Ways to Reduce Disadvantages

• Develop specific, operational (measurable) criteria for observing and appraising performance.
• Provide training for observers/appraisers.
• Conduct pilot-testing in which rate of agreement (inter-rater reliability) between observers/appraisers is determined.
• Continue training and/or alter criteria for more specificity until acceptable consistency of measurement is obtained.
• Conduct observations/appraisals in the least intrusive manner possible (e.g., use of one-way observational mirrors, videotaping, etc.).
• Observe/appraise behavior in multiple situations and settings.
• Consider training and utilizing graduate students, upper level students, community volunteers, etc. as a means of reducing the cost and time demands on faculty.
• Cross-validate results with other measures, multiple methods should be used to validate the results of appraisals.

Bottom Line

Generally the most highly valued but costly form of student outcomes assessment. However, it is usually the most valid way to measure skill development.

Bibliographic References

Burke, Kay, ed. Authentic Assessment: A Collection. Illinois: Skylight Training and Publishing, Inc., 1992.
Hart, Diane. Authentic Assessment: A Handbook for Educators. New York: Addison-Wesley, 1994.
Ryan, Alan G. "Towards Authentic Assessment in Science via STS." Bulletin of Science, Technology & Society. 1994, v 14, n 5/6, p 290.
Wiggins, Grant. "The Case for Authentic Assessment." ERIC Digest. December 1990.