Definition

Asking individuals to share their perceptions about the study target -- e.g., their own or others skills/attitudes/behavior, or program/course qualities and attributes.

• Typically yield the perspective that students, alumni, the public, etc., have of the institution that may lead to changes especially beneficial to relationships with these groups.
• Can cover a broad range of attributes within a brief period of time.
• Results tend to be more easily understood by lay persons.
• Can cover areas of development, which might be difficult or costly to assess more directly.
• Can provide accessibility to individuals who otherwise would be difficult to include in assessment efforts (e.g., alumni, parents, employers).

• Can provide unique stakeholder input, valuable in its own right (especially employers and parents). How is the course/program/college serving their purposes?
• Offer different perspectives, presumably less biased than either student or assessor.
• Enable recognition and contact with important, often under-valued constituents. Relations may improve by just asking for their input.
• Can increase both internal validity (through convergent validity triangulation with other data) and external validity.
• Convey a sense of importance regarding the opinions of stakeholder groups.

Disadvantages

• Results tend to be highly dependent on wording of items, salience of survey or questionnaire, and organization of instrument. Thus, good surveys and questionnaires are more difficult to construct than they appear.
• Frequently rely on volunteer samples, which can be biased.
• Mail surveys tend to yield low response rates.
• Require careful organization in order to facilitate data analysis via computer for large samples.
• Commercially prepared surveys tend not to be entirely relevant to an individual institution and its students.
• Forced response choices may not provide opportunities for respondents to express their true opinions.
• Results reflect perceptions which individuals are willing to report and thus tend to consist of indirect data.
• Locally developed instrument may not provide for externality of results.

• As with any indirect data, inference and reports can contain a high degree of error.
• Third-parties can be biased too, in directions more difficult to anticipate than self-reports.
• Less investment by third-parties in assessment processes often means lower response rates, even lower than student/alumni rates.
• Usually requires logistical details (e.g., identifying sample, making contact, getting useful response, etc.,), therefore more costly than it looks.
• If information about specific individuals as requested, confidentiality becomes an important and sometimes problematic issue that must be addressed carefully.

Ways to Reduce Disadvantages:

• Use only carefully constructed instruments that have been reviewed by survey experts.
• Include open-ended, respondent worded items along with forced-choice.
• If random sampling or surveying of the entire target population is not possible, obtain the maximum sample size possible and follow-up with non-respondents (preferably in person or by phone).
• If commercially prepared surveys are used, add locally developed items of relevance to the institution.
• If locally developed surveys are used, attempt to include at least some externally-referenced items (e.g., from surveys for which national data are available).
• Word reports cautiously to reflect the fact that results represent perceptions and opinions respondents are willing to share publicly.
• Use pilot or "try out" samples in local development of instruments and request formative feedback from respondents on content clarity, sensitivity, and format.
• Cross-validate results through other sources of data through triangulation.

Ways to Reduce Third Party Disadvantages

• Very careful, explicit directions for types of responses requested can reduce variability.
• Attain informed consent in cases where information about specific individuals is being requested.
• Coordinate contacts with other campus organizations contacting the same groups, to reduce and increase response rates.

A relatively inexpensive way to collect data on important evaluative topics from a large number of respondents. Must always to treated cautiously, however, since results only reflect what subjects are willing to report about their perception of their attitudes and/or behaviors.

Bibliographic References:

Sudman, Seymour & Norman Bradburn (1982). Asking Questions: A Practical Guide to Questionnaire Design. San Francisco: Jossey-Bass Publishers. Suskie, Lida (1996). Questionnaire Survey Research: What Works? Association for Institutional Research, Resources for Institutional Research, Number Six.