Standard Operating Procedure (SOP)
Data Processing Team
Introduction
This SOP outlines the process to establish standardized procedures for data processing in a
market research firm. Its objective is to ensure accuracy, efficiency, and compliance with
industry standards and client requirements.
Responsibilities
1. Data Validation and Review
Accurately input survey responses into the database or data management software (SPSS).
Perform double-checks to ensure correct data entry and eliminate errors.
Validate responses against predefined logic and research objectives.
2. Data Cleaning & Formatting
Identify and remove duplicate, incomplete, or inconsistent responses.
Standardize data formats (e.g., dates, numbers, categories) to maintain uniformity.
Apply data weighting or adjustments to ensure sample representation if required.
3. Data Processing & Structuring
Transform raw survey data into structured formats for analysis (Excel, SPSS, Wincross, Q, etc.).
Apply coding and categorization to open-ended responses.
Conduct basic statistical checks to ensure data consistency.
4. Quality Assurance & Compliance
Adhere to client standards and data privacy regulations (GDPR, CCPA).
Identify anomalies and flag discrepancies for review.
Maintain accurate documentation of processes and changes made to datasets.
5. Data Security & Storage
Ensure secure handling and storage of sensitive research data.
Follow encryption and access control measures to prevent unauthorized data exposure.
Perform periodic backups as per company IT policies.
1
6. Collaboration & Reporting
Work closely with Survey Programmers, and Research Managers to ensure accurate data
processing.
Generate data summaries, tables, and initial insights for further analysis.
Provide processed datasets in client-specified formats for final reporting.
Process Flow
Project Specification: Look out for all project-related documents from the client including:
Table specification (Tab spec): Detailed description of the tables, banners required by
the client.
Questionnaire: The survey or set of questions designed for data collection.
Previous wave/tracker project: If applicable, review the previous questionnaire, table
spec and data file for comparison.
Other supporting documents: Max-diff, Conjoint and Other excel files.
Questionnaire and Table Spec Review:
Review project documents: Examine the table specifications and questionnaire, drafting
any questions or concerns.
Provide an Estimation: - After reviewing the materials, provide an estimation for the
data tables delivery.
Important questions: Take note of critical questions, especially open-ended ones.
Discussion: If necessary, arrange a call to quickly address any queries with
client/researcher.
Required Data File and Tool:
Primary research data file: Obtain this from the client, which includes:
o Data file (in SPSS format)
o Final q’re and table spec
Tools for data work: Use the following tools for handling the primary research data file:
o Excel
o SPSS
o Python
Primary Research Data Review:
Review the raw data which received from project manager.
Match the combined data with the secondary research data points and share your findings with
the project manager.
2
Data Table Preparation:
Tables creation tool: With the help of Wincross prepare the job file which contains the
structure of data tables.
Modifying the tables as per requirement: Update the tables as per the client
requirement like as mentioned below
1. Update the multi-response variables.
2. Filter the tables where the base is not “All respondents”.
3. Apply necessary statistics for rating scale questions.
4. Create the necessary “Summary tables” (Top-2 Box, Bottom-2 Box, Mean) for all
the rating scale questions or Grid questions.
5. Apply filters on rows if they are on different base (look QNR for this)
6. Check with the client if they want sorted tables, then sort the tables.
7. Create all the banners requested from the client.
8. In case of final data (close end + open end), work on the Open-end questions.
Weighting: Apply weighting adjustments if required to ensure sample representation.
Calculate the weights and apply weighting on the tables if client had requested
weighted banners.
Self-Quality Checks:
1. After creating the tables, thoroughly check the table base, counts and %’s.
2. Check the table labels such as table title, base wordings, stubs wordings.
3. Maintain documentation of errors and corrective actions for future reference.
Quality check from QA team: Send the tables to the QA team for another through
check.
Feedback from QA team: Implement the feedbacks received from the QA team.
Client delivery: After implementing the valid feedbacks points from QA team, run all the
banners and deliver it to the client.
Feedback Process:
Client feedback: Address any client feedback or queries on the data tables.
Explanation: Provide explanations for queries with supporting data views or arrange a
call if necessary.
Table updates: Update the tables as per the client feedbacks.
Final review: Share the updated data tables with the client.
