Self-developed measures and qualitative measurement
Although all research approaches can use self-developed instruments, descriptive research, single-subject research, and action research are the approaches with which researchers are most likely to develop their own instruments. Self-developed instruments are measures created by the researcher for a specific setting or group of participants.
Descriptive-survey researchers design and develop their own surveys to gather the perceptions of their sample participants on current educational issues. Single-subject researchers and action researchers often develop tools to record quantitative data from their observations.
Action researchers also develop other instruments or use materials developed for their practice to measure student performance in either quantitative or qualitative ways. Qualitative researchers develop their own interview and observational protocols.
Researchers who use self-developed instruments have to take time during the research process to design and develop them. The point in the research process at which a researcher decides to do this depends on the type of research that is being conducted. Survey researchers develop their instruments following a thorough review of the literature in which they identify key variables and themes and then develop sub questions and survey items.
On the other hand, qualitative researchers might develop an initial protocol after they have entered the research setting and have become more familiar with the study’s participants. In addition, qualitative researchers would likely modify or refine their protocols as they collect and analyze their data and realign the focus of their study.
Even though they develop their own instruments, survey researchers would not think of changing their instruments halfway through their study because they want participants to answer the exact same questions.
Developing Scales for Use in Surveys
Developed by Rensis Likert for his doctoral thesis, the Liker scale is the most widely used scale in survey research and certainly the one that has found its way into popular culture. Many people actually use the term “Likert scale” without knowing the scale’s complexities and uses. The classic use of the Likert scale was to pose questions or items to participants and have them respond using an agreement scale by selecting a number that best represented their response.
Presented in Exhibit 5.1 are some items taken from a survey of preserve teacher candidates. The purpose of this section of the survey was to document candidates’ expectations of their field placement experience and the role of technology
Semantic Differential Scales
Another type of rating scale is the semantic differential scale. Unlike the Liker scale, which uses sentence statements, in a semantic differential scale, participants are asked to make judgments regarding words or phrases describing persons, events, activities, or materials. The ratings are made by checking a point along a line indicating a continuum between two polar opposites.
The scale is usually set up so that positive and negative responses occur at each end of the scale with equal frequency. See Exhibit 5.2 for an example of a semantic differential scale that might be used as a quick evaluation of a workshop. One advantage to using semantic differential scales is that responses can be made quickly because little reading is required. However, the information produced is superficial.
As part of the piloting process, the researcher would analyze pilot participants’ before and after responses to determine the degree of consistency and report such findings using a correlation coefficient. From time to time, a researcher may find that participants lack consistency on a particular item or series of items.
For example, a group of participants might answer “strongly agree” on the first survey administration but answer “strongly disagree” the second time a few weeks later. In cases such as this, the researcher will want to do some further investigating before coming to the conclusion that the item is unstable.