Breadcrumb navigation

How to Create the Survey Form

In this section, we describe the creation process of our Organizational Engagement Survey Form.

Expanding on a conceptual model backed by previous research

Work engagement research was initially viewed as the positive antithesis of burnout. The Job Demands-Resources Model (JD-R Model) was proposed as a mechanism to examine the relationship between burnout and work engagement and their associated factors and effects. The validity of the JD-R model has been confirmed in many countries around the world, including the Netherlands and Spain.

According to the JD-R model, (1) people with high work engagement are less likely to experience burnout (the opposite of engagement), (2) work engagement has a positive impact on work performance, (3) job resources (learning opportunities and expectations of support from superiors and colleagues) increase work engagement, and (4) job demands (workload and lack of time) are not directly related to work engagement.

Among research conducted around the world, however, only a few have studied organizational culture, employee personalities (personal attributes), and worker productivity as factors that can impact work engagement and workplace environment. In view of this fact, we constructed our own model focused on “engagement” by removing the elements of burnout and stress reaction from the JD-R model, and adding instead the elements of organizational culture, personality, organizational citizenship behavior (discretionary effort put forth to help the organization and colleagues to reach objectives), and productivity.

Expanding on a conceptual model backed by previous research

Collecting questionnaires used in previous research

When developing the initial survey, we collected questionnaires used in research based on the JD-R model and from academic papers on research conducted using questionnaires that added proprietary questions. From the 40 types of surveys we collected, we constructed a massive questionnaire comprised of 608 questions.

Types of questionnaires collected

  • Work engagement
  • Organizational culture
  • Personal engagement
  • Organizational fairness
  • Psychological engagement
  • Team-based incentives
  • Teamwork engagement
  • Supportive supervisor communication
  • Big 5
  • Reward and recognition
  • Core self-evaluation
  • Role conflict
  • Psychological capital
  • Role ambiguity
  • Presenteeism
  • Norms of collegiality
  • Organizational commitment
  • Fatigue and revitalization
  • Job satisfaction
  • Relationship conflicts
  • Retirement intention
  • Organizational constraints
  • Organizational citizenship behavior
  • Quantitative workload
  • Job characteristics
  • Factorial discretionary scale
  • Leader-Member Exchange (LMX)
  • Job demands and resources

Collecting questionnaires used in previous research

Responses obtained from 1,000 individuals

With the help of a market research company, we obtained responses to our 608-question questionnaire from 1,000 individuals. However, we found that response quality was varied due to the high volume of questions. In order to solve this problem, we set a certain standard to judge the quality, and adopted 599 respondents’ standard cleared response data.

After standardizing the response data, we conducted a partial correlation analysis to check the intercorrelations among elements. The results confirmed that engagement is an important element in the relationship between an employee and their company of employment. (These results were presented at the Japan Industrial Management Association conference in 2017.)

Responses obtained from 1,000 individuals

Extracting 10 factors from the response data

Next, by using exploratory factor analysis, we attempted to extract factors from the data received from the 599 individuals who responded to the 608 questions. Our aim was to extract items with a factor loading for validity of 0.4 or more and items with a Cronbach's alpha for reliability of 0.8 or more. Using this method, we were able to extract 17 factors including work engagement.

We then conducted a partial correlation analysis using the mean values of the extracted factors, and found that 7 factors were not significantly correlated with work engagement. Therefore, we excluded these 7 factors and defined the remaining 10 factors including work engagement as metrics to gauge engagement.

Extracting 10 factors from the response data

Reducing the questionnaire from 608 to 43 questions

A questionnaire with 608 items is not so practical. Even if we narrowed it down to items related to the 10 factors we identified, more than 200 items would be remained. Cutting down to the bare minimum number was indispensable.

To do so, we performed I-T correlation analysis (Item-Total correlation expressing correlation between the score of an individual item and the overall score) that is used in the item response theory model. Specifically, we lined up the items that compose each factor in order of increasing I-T correlation, and calculated the correlation between the sum of the scores of the first and second items and the overall score, the correlation between the sum of the scores of the first, second and third items and the overall score, and so on, until we reached the number of items with a correlation coefficient of 0.95 or higher. A coefficient of 0.95 indicates that two sets of data are almost identical. By repeating this process, we were able to reduce the number of items to 43.

After that, we re-obtained response data from 1,000 individuals with the 43-item questionnaire and confirmed that the 10 factors could be reproduced through exploratory factor analysis.

Reducing the questionnaire from 608 to 43 questions

Replacing the original questions with proprietary questions

Once obtaining the factors and questions related to engagement, we replaced original questions with proprietary questions.

Using the original questions as a basis, we tripled the number of proprietary questions with a combination of the original and proprietary questions, and obtained response data from 1,000 people. Confirmatory factor analysis was conducted on the obtained response data, and we selected the proprietary questions from among those with a correlation coefficient of 0.95 or higher between the factors derived from the original questions and those derived from the proprietary questions.

We conducted exploratory factor analysis of the response data from the original questions at the same time and confirmed that the 10 factors could be extracted. This demonstrates that the results were reproducible even after nearly a year had passed.

Replacing the original questions with proprietary questions

Completion of the "Organizational Engagement Survey Form" Based on Data Analysis

In this way, the Organizational Engagement Survey Form, encompassing work engagement and organizational climate, has been completed.

Key Features

  • Capable of measuring factors equivalent to work engagement.
  • Created not based on experience or intuition, so it is not dependent on the subjectivity of the creator.
  • Measures both work and organization engagement simultaneously.
  • Includes factors related to engagement, facilitating the consideration of solutions.
  • Intended for investigating causes and solutions in the workplace, not as a comprehensive organizational survey.
Number of questions:  43 (Response time: 5 minutes)
Response Method: 7-Point Likert Scale

Completion of the "Organizational Engagement Survey Form" Based on Data Analysis