1

Pilot Test

Student’s Name

Department, University

Course

Instructor’s Name

Due Date
 2

Pilot Test

Crowd management is a critical aspect of event planning and execution, particularly

in large-scale events like the Riyadh Season in Saudi Arabia. Effective crowd management

can have a significant impact on visitors' experience, satisfaction, and safety. On the other

hand, poor crowd management can lead to overcrowding, long wait times, and potential

safety hazards, which can negatively impact visitors' experience and reduce their likelihood

of returning to the event in the future. To ensure the quality of crowd management at Riyadh

Season events, it is essential to gather data and feedback from visitors about their experience.

This pilot test aims to measure the reliability of a questionnaire designed to collect such data,

to ensure its validity and effectiveness in measuring the quality of crowd management and its

impacts on visitors’ experience at Riyadh Season events. The results of this pilot test will

inform the refinement of the questionnaire, ensuring that it is a reliable tool for collecting

data on this critical aspect of event planning and execution.

A pilot test, is a preliminary study conducted to evaluate the reliability and validity of

research methods, instruments, and procedures before conducting a larger study. Conducting

a study that is adequately powered often requires large sample size, which can be costly in

terms of time and resources. There are several steps for conducting successful pilot test and

establish questionnaire reliability, which include sample size determination, selection of the

respondents, administration of the questionnaire, and data analysis to check for reliability.

The first step in pilot test procedure is the determination of the pilot test sample size.

Pilot testing a questionnaire on a smaller scale can be an effective way to identify any validity

and reliability issues. Although there is only limited guidance regarding the appropriate pilot

sample size, a general rule of thumb, a sample size of at least 30 participants is typically

recommended for pilot testing. Therefore, the study will administer the questionnaire and

obtain the data from 30 visitors at Riyadh Season events. Since the goal of pilot testing is not
 3

to obtain statistically significant results, but rather to identify potential reliability issues with

questionnaire, a sample size of 30 respondents will be sufficient for the test.

The second step is selection of the respondents. Researchers must carefully consider

the target population and select a sample that is representative of the population of interest.

For this pilot test, visitors at Riyadh Season events will be selected randomly. Random

recruitment of the respondents will ensure that the sample is representative of the target

population, which includes individuals who attend events in Riyadh Season. Prior to

recruitment, the respondents will be informed about the study and given a consent form,

which will allow them to agree to participate in filling the questionnaire. Additionally,

obtaining informed consent from the selected respondents will ensure that they understand

the purpose of the study and are willing to participate voluntarily.

The third step is administration of the questionnaires. The method chosen for

administering a questionnaire can have a significant impact on the quality of data collected,

and researchers must carefully consider factors such as the target population, research

question, and available resources to select the most appropriate approach. In this pilot test,

the questionnaires will be administered in person using face-to-face interviews. This

approach will allow the researchers to clarify any questions that the respondents may have

and to ensure that they understand the instructions and questions. It will also enable the

researcher to observe the respondents’ reactions and behaviors as they answer the questions,

which can provide valuable insights into the questionnaire’s validity and effectiveness. After

administration and collection of the questionnaires, the researcher will manually enter the

data into Microsoft Excel and later into Statistical Packages for Social Sciences (SPSS) for

reliability analysis.

The fourth step of pilot test is reliability analysis. The concept of reliability refers to

the extent to which a questionnaire produces consistent results over time (Cooper &
 4

Schindler, 2014). One important aspect of reliability is the assessment of whether a

measuring instrument consistently produces the same results across multiple uses. To achieve

this goal, the current study will utilize the internal consistency method, which is deemed

more stable and reliable than other approaches (Bryman, 2012).

Although there are various ways of establishing reliability, such as, inter-rater, test-

retest and internal consistency, this pilot test will focus on the latter method. This approach

involves using Cronbach's Alpha to assess the extent to which the questions in the

questionnaire are measuring the same construct or concept. This method is efficient, requiring

less time and resources compared to other methods, such as test-retest. It is also

comprehensive, allowing researchers to examine the relationship between multiple items on a

questionnaire and assess how well they are measuring a specific construct. Additionally, the

internal consistency method is flexible, making it applicable to various types of

questionnaires.

Cronbach’s alpha is a statistical measure that evaluates the consistency of a research

instrument by assessing the interrelatedness of items on the instrument. When running

Cronbach's Alpha using SPSS, there are several outputs that are generated. The first output is

the overall Cronbach's Alpha coefficient, which ranges from 0 to 1 and measures the internal

consistency of the items in the questionnaire. A value of 0.7 or above is generally considered

acceptable for research purposes. Table 1 further shows the Alpha values and the decision

associated with the range.

Table 3.3

Cronbach Alpha Value Range

Alpha Values Characterization

0-6 Poor

0.6-0.7 Marginally Acceptable

0.7-0.8 Acceptable and Reliable

0.8-0.9 Good and Reliable

0.9+ Very Good and Reliable

Source: Cronbach (2004).

The second output is the item-total statistics, which provides information on the

correlation between individual items and the total score of the questionnaire. The researcher

will use this output to identify items that may not be contributing to the internal consistency

of the questionnaire. The third output is the scale reliability statistics, which provides

information on how the overall reliability of the scale would change if each item were to be

deleted from the questionnaire. This information will help the researcher identify the most

important items in the questionnaire and to determine if any items need to be removed.

In addition, SPSS will also provide a histogram and a scatterplot to visualize the

distribution of responses and to identify potential outliers or errors in the data. The histogram

displays the frequency distribution of the responses, while the scatterplot shows the

relationship between two variables. The examination of these graphs will help identify any

outliers or errors in the data that may need to be addressed before conducting further

analyses.
 6

The final step in a pilot test will be revision of the questionnaire, if the Cronbach's

alpha value is low, indicating poor internal consistency, or if there are outliers or errors in the

data, the questionnaire. The researcher will examine the specific items with low reliability

and consider revising or removing them. Additionally, any outliers or errors in the data

should be addressed by investigating any potential issues with the administration or wording

of the questionnaire. Revision of the questionnaire based on the results of the pilot test, will

ensure that the final version of the questionnaire is reliable and valid for use in a larger study.
 7

References