Definition:

Objective and/or subjective tests designed by faculty of the program or course sequence being evaluated.

Advantages

• Content and style can be geared to specific goals, objectives, and student characteristics of the program, curriculum, etc.,
• Specific criteria for performance can be established in relationship to curriculum.
• Process of development can lead to clarification/crystallization of what is important in the process/content of student learning.
• Local grading by faculty can provide relatively rapid feedback.
• Greater faculty/institutional control over interpretation and use of results.
• More direct implication of results for program improvements.

Disadvantages

• Require considerable leadership/coordination, especially during the various phases of development.
• Cannot be sued for benchmarking, or cross-institutional comparisons.
• Costly in terms of time and effort (more frontloaded effort for objective exams; more backloaded effort for subjective exams.
• Demands expertise in measurement to assure validity/reliability/utility.
• May not provide for externality.

Ways to Reduce Disadvantages:

• Enter into consortium with other programs, departments, or institutions with similar goals and objectives as a means of reducing costs associated with developing instruments. An element of externality is also added through this approach.
• Utilize on-campus measurement experts whenever possible for test construction and validation.
• Contract with faculty “consultants” to provide development and grading.
• Incorporate outside experts, community leaders, etc. into development and grading process.
• Embed in program requirements for maximum relevance with minimum disruption (e.g., a capstone course).
• Validate results through use of multi-method approach.

Bottom Lines:

Most useful for individual coursework or program evaluation, with careful adherence to measurement principles. Must be supplemented for external validity.

Bibliographic Reference:

Banta, T.W., “Questions Faculty Ask About Assessment,” Paper presented at the Annual Meeting of the American Association for Higher Education (Chicago, IL April 1989).
Banta, T.W. and J.A. Schneider, “Using Locally Developed Comprehensive Exams for Majors to Assess and Improve Academic Program Quality,” Paper presented at the Annual Meeting of the American Educational Research Association (70th, San Francisco, CA, April 16-20, 1986).
Burton, E. and R.L. Linn, “Report on Linking Study – Comparability Across Assessments: Lessons from the Use of Moderation Procedures in England. Project 2.4: Quantitative Models to Monitor Status and Progress of Learning and Performance”, National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA 1993.
Lopez, C.L., “Assessment of Student Learning,” Liberal Education, 84(3), Summer 1998, 36-43.
Warren, J., “Cognitive Measures in Assessment Learning,: New Directions for Institutional Research, 15(3), Fall 1988, 29-39.