Chapter 12

USING QUESTIONNAIRES AND SCALES

Learning Objectives:
1. To discuss the importance of using questionnaires and scales.
2. To discuss the how to design questionnaires and scales properly.
3. To explain the issues of validity and reliability in questionnaires
4. To give and explain many examples of questionnaires and scales.

Overview:
1. Introduction
2. Questionnaires and Scales
3. Issues of Validity and Reliability
4. Designing Questionnaires and Scales
5. A Pilot Study
6. Response Rate

1. INTRODUCTION:
As discussed in the previous chapters, methodological framework is one of the four
framework approach in research project that helps the researcher create a logical, consistent and
comprehensive research project. Methodological framework is usually done after the conceptual
framework or the literature review. Figure 1 shows the initial three frameworks in research.
Under the methodological framework contains the methodology of the research and the data
collections methods. The data collection method will based on the chosen methodology of the
research project. Questionnaires and scales are examples of data collection methods widely used
in research projects.
The
Conceptual
framework

The The
theoretical methodological
framework framework

Figure 1 Three initial frameworks of the four frameworks approach to research

1
 Questionnaires and scales are basically data gathering methods used in research. They are
usually done in research projects that requires a survey methodology. In qualitative research,
only open-ended questionnaires can be utilized because of its subjective nature while in
quantitative research, both questionnaire and scales can be utilized with the consideration that the
questionnaire contains objective type questions that can be converted to numbers. As discussed
in the previous chapter qualitative research falls under the constructivism and interpretivism
philosophy while quantitative falls under positivism.
Questionnaires and scales can be used anywhere, not just in conducting research. You for
instance have already answered questionnaires for your final exams in your university or filled
up scales after eating in a restaurant conducting a survey about the quality of food.
Questionnaires and scales are widely used because of its simplicity and efficiency. Essential
information and data can be obtained from one participant in usually less than an hour by the use
of scales and questionnaires.
At some point of our lives most of us were able to fill in a questionnaire for a researcher, we
participate by being one of the respondent needed for the researcher to achieve the overall
measure of the attitude and opinions of the respondents. By reading this, you’ll know why we
have to willingly participate those surveys as needed and how could you earn the willingness of
the respondents, if you are a researcher.
In the research process shown in figure, the devising of data collection method is after the
selection of methodology. In the beginning of the research project, you may have already have a
little idea on the outcome and the process of the research. For example, you want to conduct a
study about the satisfaction of the workers in a certain company, you may already have thought
about what methods you will do in order to conduct the study, but only after reviewing many
literature and selecting methodology can you truly know what data collection methods are
appropriate to be utilized in that study.

2. QUESTIONNAIRES AND SCALES

In every research project, there is a different appropriate methodology that must be selected.
If a research project has a large quantity of population, an appropriate methodology would be a
survey methodology. Under the survey methodology, questionnaires and scales are the best fit to
be used as data collection method because of its simplicity and efficiency. In a research project
with a large amount of population, you cannot thoroughly research each of them one by one
because of time concerns. As such, you can just give the respondents questionnaires or scales
which are easy to answer that can generate informative data.
Questionnaires can be utilized in both quantitative and qualitative type of research while
scales can only be utilized in quantitative type of research. Questionnaire for quantitative and
qualitative type research tend to be different. For qualitative research, questionnaires contains
open-ended questions and are subjective in nature while in quantitative type of research,
questionnaires contains much simpler and objective type of questions that can be quantified. For
a large amount of population, researchers usually avoid conducting qualitative type of
questionnaires for the reasons that data gathered from it can be hard and time-consuming to

2
analyse. As such, they utilized questionnaires that gathered more quantitative data than
qualitative data. Then they usually only select few qualitative data samples to be analysed as a
representative of the whole population
Questionnaires and scales are essential instruments that are used to gather primarily
quantitative data needed for the research. It is very significant, for the questionnaire is an
instrument in the research that consist a series or sequences of questions for the purpose of
collecting such data and informations from the respondents. Moreover, questionnaire really
affects the validity, generality and reliability of the research outcomes. By doing this approach, it
lets the researcher to easily perform and accomplish comparison, summarization and even
generalization of the data gathered. Different people take things differently. With that being said,
it only explains that the questionnaires and the responds of the people who became the
respondents might be able to elucidate the researcher about the real meaning of the answers that
the respondents provided throughout the studies. There are three kinds of questions that could be
used in a questionnaire, these are fixed alternative question, scale question and open ended
question. Where in fixed alternative question offers multiple choice answers; scale question is
designed to obtain the respondent’s opinion, feeling or sentiment; and open-ended questions are
questions that is not answerable by “yes” or “no”, it is questions that are formulated as a
statement that requires more thought response and more than a keen and a simple one word
answer which is classified as a qualitative data. On the other hand, scales are used to quantify,
categorize and classify variables. It actually describes the nature of data, information, and
answers given within the numbers or scores assigned to the questions.
. Scales can only be utilized in a quantitative type of research because the data obtained from
it are quantifiable. Scales are usually much simpler than questionnaires in nature and most scales
usually represents data as numbers. There are many different examples of scales one of which is
the Likert scale. Likert scale is coined after the person who created it: Rensis Likert. Likert scale
is a number point scale which asked the respondent how much they agree or disagree with a
certain statement. Table 1 shows different Likert Scale examples.

Table 1 Likert Scale Examples

Agreement
Strongly Agree
Agree
Undecided
Disagree
Strongly Disagree
Frequency
Very Frequently
Frequently
Occasionally
Rarely
Never

3
 This kind of scale is very useful and efficient to use because of its simplicity. Moreover,
Likert scale questions can aid in avoiding open-ended and fill in the black questions that can
generate hard to analyse results. Respondents can usually finish answering the Likert scale in
less than an hour and the data obtained from it can be easily analysed. Figures 2 and 3 shows
different example of Likert scale questions. Figure 2 is a likert scale question asking for a
feedback about the quality of a product while figure 3 is about the satisfaction of the customer.

Figure 2 Likert scale question about the quality of material

Figure 3 Likert scale question about the satisfaction of the customer

4
 in making Likert scales, you must first decide what you will measure. This will be your main
basis for all your questions and the answer scale. Next is you make the Likert scale indicator
questions. In creating likert scale questions, it is best if you ask specific questions. This will help
the respondents answer the questions quickly. It is also preferable if you use terms that you know
most of the respondents will understand in order for them not to have different interpretations
and lastly avoid long, complex questions instead, you can make the long question into two
simple questions. Lastly you can now make the Liker scale responses, in making Likert scale
response it is best if you have odd number of responses and it should not be less than 5
responses. You should also label the numbers with words because you may confuse the readers
which of the other end is the positive or negative. (Liedke 2019)
Semantic differential scale is another widely used scale. Semantic differential scale involves
multi point questions with opposite adjectives at the two ends to decide which of the adjective
best describe a situation under observation. Figure 5 shows an example of Semantic differential
scale.

Figure 5 Example of a Semantic differential scale

3. THE ISSUES OF VALIDITY AND RELIABILITY

Validity
In conducting research, validity is the measure of how true, how logical, how consistent,
how comprehensive, and how it can contribute to the body of knowledge. In data collection
method, validity issue refers to the “measurement validity”, the degree of how it was designed,
if it can obtain the data required, and what data can it obtain. Content validity, face validity,
criterion related validity and construct validity are the different ways of developing measurement
validity.
In the content validity, the researcher makes sure that the data gathering method can truly
obtain the data required for the research project. In face validity, the research ask for the opinion
of an expert on the field of study being researched if the data collection method used is valid for
the research. The data collection method is then change or revised based on the recommendation
of the expert. In criterion related validity, the researcher uses other’s data gathering method as a
measure if his/her data gathering method is valid. The data gathering method used as basis must

5
be already validated by others. In construct validity, researcher verify if all the questions are
essential for obtaining the data required for the research project. Construct validity are usually
established if there are multiple questions of issues. These questions must be in line with the
research project and related to each other.
The questionnaire and/or scale created by the designer must be essential and in line with the
research project. In order to achieve this, conducting pilot test would help in developing the
validity of the data gathering instrument.

Reliability
In conducting research, reliability means that it can be used by others as basis for their
research as well. Repeating the research whilst getting consistent results is a measure that it is a
reliable research. In data collection, it is the same, the data collection method is reliable if it can
be repeatedly utilized and still get consistent results. One specific example of this is a tape
measure, if you measure your height today you will get a result. If you measure you height again
on the other days considering your height did not change and the result is still the same, then the
tape measure is reliable instrument.
There are three various kinds of reliability that must be considered in questionnaires and
scales. The first one is the “stability reliability” or sometimes can be called as the retest
reliability, it means that when the questionnaire is utilized repeatedly over time, it can still
generate consistent results. The second one is the “representative reliability”, a questionnaire is
representative reliable if when it is utilized in different groups in the same population it can still
produce the same results. The last one is the “equivalence reliability”. Equivalence reliability
means that the items in the questionnaire are all essential to the phenomenon being research on.
There are three different methods that can test the reliability of the questionnaire and these three
methods corresponds to the three various kinds of reliability. The first method is the split-half
method. The split-half method measures the equivalence reliability of the questionnaire. In split-
half method, the questionnaire is divided into two halves then, the items in the questionnaire are
evenly distributed into the two and tests whether or not the two halves produce same
results. The second method is the retest method. The retest method is used to measure the
stability reliability of the questionnaire. In the retest method, a pilot test is conducted for a
certain group. One set of questionnaire will be given to the group for them to answer then the
result must be collected. After sometime another set of questionnaire with same content as of the
first one will be given to the same group again for them to answer. A questionnaire is considered
stability reliable if the two sets of questionnaire generated consistent results. However, there is a
problem in this method, because the group are given the same questionnaire. They can have
different answers on the second questionnaire due to some factors. One factor is that some
respondents in the group might have more mature response on the second time because they
already reflected on their answers on the first time they answer the same questionnaire. The last
method is the inter coder reliability. In the inter coder reliability, the questionnaire is studied by
more than one researcher, observers or coders. Then if their analysis agreed with each other, the
questionnaire is considered reliable.

6
 To improve reliability the researcher must meticulously design the questionnaire. The words
or terms used in the questionnaire must be easy to understand for the respondents in order for
them not to have different interpretations that can yield to different results. The same with the
issue of validity, conducting pilot test can aid in improving reliability.
4. DESIGNING QUESTIONNAIRES AND SCALES
Questionnaires and scales that is used in research must be valid and reliable and will be able
to obtain the data required for the research project. The researcher can study different samples of
questionnaires and scales in other studies and in the literature related to their research.
How you ask, is what you get. So if the researcher was about to design a questionnaire
for his/ her research, the researcher needs to do it effectively by deciding what precisely that he
or she needs to know. To effectively design a questionnaire the researcher have to make the
respondents feel special, genuinely express his appreciation for their participation and let the
respondents know that the researcher wants to know what they think, emphasize the fact that the
researcher value their point of view. In addition, save their time! The researcher needs to put
himself on the shoes of the respondents, make the questions simple, professional and keep the
questions relevant to the respondents. Another significant point is that the researcher must be
critically engage with language, words and concepts that he was about to use in the questionnaire
and scale. Prevent the following: creating an ambiguous questions (questions that could have
more than one potential meaning); constructing complicated questions; merging two questions in
one; potentially asking embarrassing questions; probing unnecessary questions; putting loaded
words (words that mainly has a strong emotional impact such as fascist, disease, mad, class, and
poor); vague or unclear concepts (such as old, fat and medical conditions like obesity); using
insulting words; humour; colloquialisms and slang; ethnocentrism (viewing the world from the
researcher’s own cultural or ethnic perspective); using racist, sexist, or ageist language; using
jargons and abbreviations; being respondent bias (social desirability, prestige bias and
acquiescent bias); talking down to respondents (patronizing the respondents or disrespecting
them); and using double negatives for example “it is not true that”. The key issues in designing a
questionnaire are the content and order of the questions, the presentation and construction of the
questions and the length of the questionnaire. There’s a lot of channel or method on collecting
informations from the respondents these are computer questionnaire, conducting questions over
the telephone, doing an in-house survey, by mailing the questionnaire, pen and paper
questionnaire, and last would be publishing an online survey or questionnaire, which is the most
efficient one. Because conducting surveys and questionnaires has a lot of advantages. It is cost
efficient, for no money will be spent on paper, postage or printing and it also lessens the
administration and set up cost. Online method saves large amount of time where in the researcher
could easily administer, create, analyse and collect information. It is very convenient for the
respondent can answer the questionnaire whenever they want, sometimes at their spare time
which will also help to increase the response rates. It is very accessible because sending
questionnaires was just one click away, it gives the respondents a variety of ways to access the
questionnaire such as tablets, desktop, mobile devices, laptops, etc. Additional advantage would
be reach and scalability, for the researcher can send the questionnaire to a thousands of

7
respondents as fast as he can send it to one and the researcher can send the questionnaires across
the world. Another advantage was anonymity, confidentiality or secrecy that let the respondents
feel more comfortable in answering the questions honestly for it is assured that their identity was
hidden. Perhaps, objectivity is also an advantage for it eliminates any influence of the researcher
to the respondents unlike if the questionnaire was conducted over the phone or using a pen and
paper method where in the interviewer could somehow affect the answers of the respondents.
Spreading the questionnaire using online method is more accurate, for the answer of the
respondents go straight to the online database rather than being manually entered by an
individual. And the major benefit was it would give the researcher quick results because as soon
as the respondents were able to answer the questionnaire, informations can rapidly be presented
and tabulated in a variety of report layouts.
Furthermore, scale is designed to obtain the respondent’s opinion, feeling or sentiment. It
is used to quantify, categorize and classify variables. It actually describes the nature of data,
information, and answers given within the numbers or scores assigned to the questions. There are
three types of scale given in the chapter, these are Likert scale, Semantic differential scale and
Bogardus social distance scale. To begin with, a measurement device that is used
psychologically to measure opinions, attitudes and values is called Likert Scale which was so-
called after Rensis Likert, the creator of the scale. It is an orderly gauge with a series or
sequences of statements which the respondents would choose the preference that best support
their point of view, the respondent just have to specify the extent or level to which they agree or
disagree. Using a general method of collecting data is the main advantage of this scale, meaning
that it’s easy to understand for once the reactions has been established it is very easy to analysed
them. Yet if there’s also a disadvantage of using this scale, because it’s possible that the
respondents could either lean towards indicating the most extreme choice or no opinion at all that
could lead the outcomes around the middle or each end of the scale, that makes it hard to
differentiate between strong and weakly held sentiments, point toward the space between each
leeway is equidistant, which is primarily not a real thing. Other scale is that used to measure
attitude towards something is called semantic differential scale which was developed by Charlesa
Osgood. It is also used by the researcher to ask the respondent using two opposite adjectives and
let the respondents select which of the two adjectives best describes their attitude. It is very
reliable for semantic differential scale comes with popularity and ease of understanding that
really helps the researcher to gather information easily. Last scale that was mentioned on the
chapter measures fluctuating degrees of closeness in people towards other member of various
social, racial or ethnic groups called Bogardus social distance scale termed after the creator of
the scale which was Emory Bogardus. Degrees of hostility, warmth, intimacy and indifference
between racial or ethnic socials are measured on this scale.
Measurement is essential in research because it aims to find out the quantity, capacity or
dimensions of the events or behaviours that the researcher desires and needs to explore. There
are two types of measurement, and these are quantitative measurement and qualitative
measurement respectively. A quantifiable report of characteristics or events which involves
numerical extent is called quantitative measurement. It is commonly used by the researcher to

8
create comparison between characteristics or events. However, qualitative measurement are ways
of attaining a deeper understanding of a topic. It is used by the researcher in order for him to find
out the meaning behind certain occurrence, it is often used in grounds that concerns human
behaviour, for example social sciences. Provided that there are two types of measurement,
quantitative measurement and qualitative measurement there will be an issue with regards to
reliability and validity. Reliability which is synonym to repeatability and consistency, defines the
point to which the test scores are free from errors of measurement. It is very important for it
enables the researchers to have some confidence and assurance that the extent they took are close
to the real measure. There following are the two major types of reliability: Reliability as
temporal stability that is related to the times the researcher collects the data; and Reliability as
Internal Consistency which includes Coefficient Alpha, Interscorer Reliability and the one that is
mentioned on the chapter, Split-half method that evaluates the internal consistency of a
assessment, such as questionnaires and psychometric tests. Split half method measures the
degree to which all parts of the assessment contributes equally to what is being restrained or
measured. Split half method is done to match the result of one half of an assessment with the
outcome from the other half. On the other hand, validity is also important for validity tells the
researcher that the measure he took is actually measures what the researcher hope it does. There
are four major types of validity, these are Criterion validity, predictive validity, construct validity
and convergent validity. Validity that measure is related to the result or to an outcome is called
concrete validity or criterion validity. Validity that is obtained in the future is predictive validity.
However, if the researcher will construct a hypothetical, intangible trait or quality in which an
individual differ is called construct validity. Last would be convergent validity, it actually
defined as an assessment highly compares with other variables which have overlap or same
constructs. So, if researchers would like to know how good the qualitative or quantitative
measurement is, the researcher should depend on the validity and the reliability of the
measurement. And take note that a good validity needs to have a good reliability proven first.
Wants to ensure that the research is a quality user research? To which it allows you to
generate meaningful and deep user insight, it doesn’t sounds that easy for sometimes the
respondents doesn’t know what they have been asked to. Fortunately, there is what we called
pilot study that would greatly help the researchers to ensure a quality user research. A pilot
testing is a rehearsal and a preparation of the research study, it will allow the researcher to test
his research approach with a small sum of respondents before the researcher conducts his main
study. It is needed to take time to test, iteratively improve and critique the research design,
before the main research would be performed. Although pilot study is an additional step, by
doing it the researcher will definitely ensure that his user research will dramatically develop and
improve the output of the study and surely that the research will run smoothly.
If the researcher was able to use and design the questionnaire and scales effectively,
provided that the qualitative measurement and quantitative measurement matters without any
issues on reliability and validity and was able to conduct a pilot study prior to him conducting his
main study it will now result to a response rate. The number of respondents who answered the
questionnaire and scales divided by the number of respondents in the sample is called response

9
rate, also known as return rate or completion rate. Usually researcher’s expressed the response
rate in percentage. It is observed as a central indicator of survey quality, for it will show how the
researcher was able to get the willingness of the respondents to give valid response to the
questionnaire and scales provided. Unfortunately speaking, the smaller amount the valid
responses are, the less confident the researcher would be. If you’re a researcher and wants to
improve the response rates, be polite and considerate. Show courtesy.
In designing questionnaires and scales, it is best if you focus on your conceptual framework.
Your conceptual framework contains the aims of your study. Your questionnaire and/or scale
must be always in line with the aim of your study. Then you can refine your questionnaire and/or
scale by conducting your literature review. You can use the questionnaire and/or scales of other
researches as basis for your own.
Essential points to be considered when making questionnaires:
1. What are the content of the questions? Are all of the questions relevant to the research project?
2. How are the questions presented?
3. Are the questions connected to each other and needs to be in a certain order?
4. How long is the questionnaire?

5. A PILOT STUDY
Prior to conducting the actual data collection method, a pilot study must first be done to
check the affectivity of the data collection method. This for the reason that the assumption of the
researcher of what will the result of the data collection method is not always correct. For
instance, the researcher cannot know if the questions in their questionnaire can be understood by
their respondents or not, the only way they can know is if they give them the questionnaire.
For a large amount population, pilot study is usually conducted on a small scale of
respondents only that can represent the whole population. This respondent can be called pilot
study respondents and they cannot be respondents again for the actual data collection in the
study. The pilot study respondents is consist of only a maximum of 15 respondents. The goal of
the researcher in the pilot study is to observe how the pilot study respondents will respond to the
questionnaire. The researcher must take note if they fully understood the questionnaire and if
they have recommendation of how to improve the questionnaire in terms of its validity.
The issues provided by the pilot study respondents must be addressed before the actual
study. Any recommendation for the validity of the questionnaire must be done.

6. RESPONSE RATES
In data collection, the response rate is basically important. The more valid response the
researcher can get the more valid is his research. If tall the member of the population responded

10
properly to the given data gathering instrument then the research will have a perfect data.
However, it is very unusual to have a 100 percent response rate this is due to many factors. One
factor is the way the researcher deal with the respondents. Maybe the respondents did not like
how the researchers handled the interview and in turn not responding to the researcher. Or maybe
the some of the respondents did not like some of the questions in the questionnaire due to ethical
issues.
There are many ways to yield a higher response rate. One way is to give incentives to the
respondents. Any incentives would do, you can give the respondents gift or foods if they finished
the questionnaire for instance. Another way is to make questionnaires simple and clear, many
respondents would not like to answers questionnaires that are very complex and long because
they would not want to waste their time. Timing is also a factor in generating a higher response
rate, it is basically not preferable to gather data during public and religious holidays.

End of Chapter Questions:

 Discuss extensively the importance of using questionnaires in gathering data for a large
population.
 Discuss the four essentials points that need to be considered in designing a questionnaire.
 What is a Likert Scale? Outline a research project that utilizes a Likert Scale.
 Discuss the term response rate
 Discuss the importance of a pilot study.
 Explain the practicalities of asking a question
 Give five examples on how you can generate a high response rate and explain.

11
References
Jackson, Sherri L., (2015). “Research Methods: A Modular Approach”. Retrieved from Shelf 72
7F Mezzanine (Circulation) Henry Sy Sr. Hall, De La Salle University

Liedke, L (2019). “Beginner’s Guide: What is a Likert Scale and How to Use it” Retrieved from
https://wpforms.com/beginners-guide-what-is-a-likert-scale-and-how-to-use-it/
https://businessjargons.com/semantic-differential-scale.html

Silva, D. (2016). “Research Methods Structuring Inquiries and Empirical Investigations” JO-ES
Publishing House

Trochim, William M. K., (2016). “Research Methods: The Essential Knowledge Base”.
Retrieved from Shelf 72 7F Mezzanine (Circulation) Henry Sy Sr. Hall, De La Salle University

White, Theresa L., (2013). “Research Methods”. Retrieved from Shelf 26, 7F Mezzanine
(Circulation), Henry Sy Sr. Hall, De La Salle University

12