Work-related Personality Inventory Manual

2. Development

This chapter describes the history of the WPI. The development of the questionnaire is described in terms of several developmental stages. In the first stage the purpose of the questionnaire is defined and the design of the questionnaire is described. In the second stage the first analyses and the item selection for the ProSiD-PI 35[1] take place while in the third stage the structure of the WPI is represented and how this structure came about.

2.1 First stage: history

In 1998, the need for a personality questionnaire tailored to the HRM work field arose among psychologist working in the field of personnel selection: a work-related personality questionnaire. At that time, multiple personality questionnaires were used in work-related assessments. These questionnaires did not meet the requirements of psychologist in the testing process. Furthermore, for candidates, the testing process was quite time consuming.

A research team was commissioned by Ixly to investigate and get an overview of the specific information requirements in the field. Twenty psychologists were approached, ten of which were working at Ixly while the other ten were working externally. The need turned out to be for a personality questionnaire that reports on scales that say something about the job performance or competencies of a person. If there could be one single personality questionnaire to serve this purpose – instead of multiple questionnaires – then this would save considering testing time as well. After full investigation of the specific needs, a purpose and target audience for the questionnaire was defined.

Purpose
The purpose was to construct a broad, work-related personality questionnaire that covered the informational needs by psychologists in selection, career transition and career advice as fully as possible.

Target audience
The target audience of the WPI is the Dutch work force. This resulted in the following requirements:

  • Completeness
  • Relevance: applied to working situations;
  • Widely applicable: from lower to higher educational levels and for different types of functions;
  • Narrow definitions of psychological constructs that can be used to make unambiguous statements. Furthermore, this enables automatic reporting.

The aforementioned psychologists were asked, independent from each other, to state as many terms and concepts that are work-related and/or apply in an HRM-related personality questionnaire, in their opinion. All mentioned concepts were scrutinized after which 80 concepts remained. After this, these concepts were operationalized which resulted in about 800 items. During the operationalization to the item level explanatory keywords with predictive value for the relevant concepts were sought. The goals was for a keyword in an item to be only related to one single concept. Respondents are found to respond to keywords when answering questions. This was evidenced by previous observations by the research team. For the Career Values Questionnaire, the OPF and Mobility Indicator (Orga, 2007), the research team was under the impression that people are intended to respond on the word level rather than on the sentence level. Due to possible future use of the questionnaire by others in a 360 degrees feedback version it was decided to formulate the items in the third person. Furthermore, there are indications that people tend to evaluate themselves more neutrally when the items are formulated in the third person (Hendriks, Hofstee & de Raad, 1998). All items were presented to a number of foreign persons who were proficient in the Dutch language but not native speakers. According to the results, all proverbial and typically Dutch items were adapted. The resulting 800 items were presented to a first population (N = 150) in a paper and pencil version. On the basis of analyses of the results, refinements have been made where we focused on high inter-item correlation, creating narrow constructs. Eventually, 430 items remained, corresponding to 35 constructs.

2.2. Second stage: ProSiD-PI 35

Research continued with the 35 concepts and 430 corresponding items. For clarity, from now on we will call the concepts that are made up by the items, ‘scales’. To create a structure, these scales are classified in factors: Ambition, Work Attitude, Emotional Stability, Extraversion, Altruism and Culture. The factor Work Attitude matches the factor Conscientiousness of the FFM, the factor Emotional Stability matches the factor Neuroticism, Extraversion has the same name, the factor Altruism matches the factor Agreeableness and the last FFM factor Openness to experience is divided up into Ambition and Culture. It is important to note that these factors in the model of this version of the WPI have an overarching, theoretical function. The fact that the names of the five factors of the FFM are changed is in the first place because this is a work-related questionnaire for which the new names are more applicable. Second, we have chosen to coin the names in a positive manner, resulting the factor Neuroticism to be changed to Emotional Stability. Third, the factor Openness to experience is ambiguous in the FFM. Separation of this factor into Ambition and Culture is easier to substantiate theoretically. In the classification of the factors we have taken the work-related characteristic of the personality questionnaire into account, meaning that the factors should be easily interpretable for assessment psychologist in the HRM work field.

In 2001, the first version of the WPI was created: the ProSiD-PI 35. To investigate the validity and reliability of the ProSiD-PI 35, it was administered to 350 people. A correlational study (Orga, 2002) was performed, in which four personality questionnaires were used as criteria. The four questionnaires were:

  • Nederlandse Persoonlijkheid Vragenlijst (NPV: Luteijn, Starren & van Dijk, 1985)
  • Edwards Personal Preference Schedule (EPPS Nederlandse bewerking: Tjoa, 1993)
  • Guilford LTP Temperament Survey (GLTS: Akkerman & Buijk, 1994)
  • GPP/GPI (Gordon, 1963)

Two norm groups were formed for the ProSiD-PI 35, namely Selection (persons who took the questionnaire in a personnel selection situation) and Advice (persons who took the questionnaire in a career advice situation). To assess the reliability of the ProSiD-PI 35 the internal consistency (Cronbach’s alpha) for every scale, for the two norm groups was calculated. For the Selection norm group the internal consistency (α) ranged from 0.68 to 0.91. For the Advice norm group the internal consistency ranged from 0.74 to 0.92.

2.3. Third stage: Structure of the WPI

After a couple years of gathering data with the ProSiD-PI 35 and renewed statistical analyses the final version, the WPI, was finished in 2008. For users, the WPI was called the ProSiD-PI 25 after the ProSiD-PI 35. This version was also used to gather data to analyse. The number of items of the WPI is reduced in comparison with the ProSiD-PI 35 and the ProSiD-PI 25; this was done after the development of a new underlying model that supports the classification in factors, scales and items statistically. The WPI consists of 25 scales with 276 items.

In developing a psychological test, one constructs a test from theory. Subsequently, one checks whether the test meets the predetermined requirements. Unfortunately, large numbers of data are not directly available for the analyses that are necessary in order to improve the tests. In constructing a psychological test, therefore, one often works ‘backwards’. The test can only be put together and restructured correctly after years of gathering data. Only then a solid statistical foundation is possible. This process has also taken place in the development of the WPI. There was a frequently used questionnaire, the ProSiD-PI 35, with a solid theoretical basis and from which a lot of data was collected. This questionnaire was adjusted and improved through statistical analyses, which resulted in the current version of the WPI. This process is discussed in more detail below.

Step 1: The theoretical classification of the factors, scale and items of the ProSiD-PI 35 were investigated thoroughly. The scores on the scales of a temporary version, Version X (which has never effectively been in use), of 17950 respondents were studied. Version X existed of 171 items.

Step 2: The scale scores of Version X were factor analysed, using Principal Component Analysis (PCA) with varimax rotation. This analysis indicated that a five factor structure was most suitable.

Step 3: Subsequently, for every item correlations with the factor scores resulting from step 2 were calculated. On the basis of these correlations, every item was assigned to one of the five factors.

Step 4: For each factor the assigned items were factor analysed to form scales. Again, we used PCA with varimax rotation. The number of scales within a factor was established using scree plots (graphical representations of the amount of explained variance by consecutive factors) and by the interpretability of the found solutions. Items were assigned to scales based on the component loadings as well as on theoretical considerations. The reason to divide items up into scales by factor rather than by all items at once, was to reduce the number of elements on which the PCA was performed. This reduction makes the found solution more stable (more replicable) and easier to interpret.

Step 5: After assignment of the items to scales, adjustments were made using ‘trial and error’, to optimize the scales in the following ways:

  • Not too many items in one scale
  • Reliability as high as possible
  • Each scale correlates clearly with one single factor
  • Minimize ceiling effects

This optimization procedure resulted in the deletion of items that did not have a substantial influence on the reliability of a scale score or that caused a second factor. In this process, we strived for minimization of ceiling effects by keeping the items with low mean scores and to make the number of items in a scale not too small. We did not find any floor effects, so these were not considered. In some cases, scales were combined when doing so yielded a one dimensional scale.

Step 6: After optimization of the scales, another factor analysis (PCA) was performed on the created scale scores in order to reassign the scales to factors. This was done because the optimized scales differed from the scales scores of ‘version X’ (step 1) in some aspects. Now, we found a six factor solution to be optimal.

Step 7: A last factor analysis of the factors was done using the Multiple Group Method (MGM) instead of the more common known PCA. This was done because of the fact that the MGM is more applicable for testing a specific hypothesis (Nunnally, 1978). Given that we wanted to know whether the previous assignment of scales into the six factor structure held for the current data, this method was preferred: the hypothesis being of course that the structure was the same as in the analyses on previous datasets. For more information on this procedure see Stuive, Kiers, Timmerman & ten Berge (2008). One thing we have to note is that in the MGM analysis we have performed, we used the formula of Steiger (1980) in order to determine the significance of the difference between dependable correlations.

The analysis was initially done for the Advice and Selection group together. This analysis showed that there were a few items that should have been assigned to another scale, indicated by a significantly higher correlation with one of the other scales than their assigned scale. Theoretically this was explicable, so the respective items were assigned to the other scales or deleted. This was the case for twenty items. After changes, deletion or some additions to prevent ceiling effects, the MGM analysis was run again. This analysis was performed for the Advice group and the Selection group separately. The results are presented in Appendix 1. For each of the groups, two items still appeared not to correlate significantly the highest with their ‘own’ scale. We decided to keep the items in their own scales, because the concerned scales were close to each other in terms of interpretation. An overview of the differences and overlap between the ProSiD-PI 35 and the WPV is provided in Appendix 2.

Step 8: An MGM analysis was also performed on the division of scales into factors, for the Selection group as well as for the Advice group. In this analysis, we tested whether the six factor structure was still optimal. We found earlier, using PCA and scree plots, that in this composition of scales a five factor and six factor structure would be defendable. In the MGM performed with five factors we could substantiate dividing the factor Ambition and place their scales under the factors Influence and Exuberance. An overview of the changes that have taken place in the formation of scales is provided in Appendix 3.

Due to changes in the structure and item format and following results from validation research, that will be discussed later in this manual, some changes in the names of scales and factors have occurred.

All factors and scales are clearly represented in Appendix 4, as well as discussed and defined. For each scale, an example item is included. By means of the description of the history of the questionnaire’s structure and this appendix including the sample example items, one can get an idea of the content of the items and questionnaire. On account of protection of the questionnaire not all items of the WPI will be discussed in this manual. In Table 2.1, the final factor and scale structure are shown.

Table 2.1. Factor- and scale structure of the WPI
FactorScaleNumber of items
InfluenceStatus10
Dominance12
Competition 9
Self-presentation 8
SociabilityNeed for contact10
Leisure contact12
Self-disclosure10
Trust10
Friendliness12
Attentiveness19
ExuberanceEnergy12
Personal growth10
Perseverance13
Adaptability 7
Originality10
Independence11
StructureOrderliness10
Precision10
Regularity10
Conformity12
Decisiveness11
StabilitySelf-confidence12
Optimism14
Frustration-tolerance11
Resilience11

2.4 Scoring of the WPI

Using a five point scale, candidates are asked to indicate whether he or she agrees or disagrees with the statement. For each scale, there are some items that are worded negatively. In summing the candidate’s score these items are keyed in the other direction. A scale score is an unweighed sum of the items belonging to that scale. A factor score is an unweighed sum of the standardized scale scores.

[1] ProSiD 35 stands for Professional Six Dimensional Personal Inventory with 35 scales. This is the name of the first (draft) version of the WPI.