Capability assessments and cultural diversity

Where must we be careful in using capability assessments?

In an internationalising environment, we are often asked if immigrants can be assessed with our capability test material and if our tests would be applicable abroad. I would like to comment on my experience with this subject in this blog, and make recommendations on how to be careful when making a choice between capability test materials.

Research in Kenya

At the end of 2005, in the third year of my psychology degree at the University of Groningen, I conducted research into the influence of culture on non-verbal capability assessments. This test was part of a non-verbal intelligence assessment for children. We asked 394 children in The Netherlands and 331 children in Kenya – all aged between 4 and 17 years old – to take the “Categories” test. The Kenyan group scored significantly lower on this capability assessment although – naturally enough – this doesn’t mean that the Kenyan children were less intelligent.

After they had taken the test, we asked the children which pictures they did not recognise, and discovered a number of culture-related factors. Obviously, they were not familiar with a toboggan in the transport category and they were also unfamiliar with a face flannel. The test was adapted to take account of these culture-related findings. Yet it would appear that the way in which the test was conducted and the kind of assessment material used – to which some of the Kenyan children were unaccustomed – exercised greater influence on the test outcome.

This research made me realise that the material covered in any capability assessment must be honest and sincere in the sense that everybody must be able to understand each component part. And also – primarily – that it is important to create separate norms and standards for various population groups.

The term IQ-score

For me, the issue with an IQ-score is that it has become a kind of ingrained term and people regard it as the “bar” by which intelligence can be measured. If you start to measure with a ruler in various countries, whereby in one country the centimeter on the ruler is slightly larger than on the ruler in another country, it is going to be difficult to see if the shelf you have ordered is actually going to fit in the cupboard. Since people have become accustomed – and have the tendency – to measure with uniform standards, they also want to measure intelligence with a uniform standard and treat everybody in the same way. It goes against our very nature to establish separate norms for different groups.

Wat do we truly want to measure?

Let’s consider what we actually want to learn if we ask somebody to take a capability assessment or an intelligence test. This could for example be the work and thought capacity of somebody in order to assess whether the person in question is up to the requirements of a certain job function. A relatively representative sample of our population makes up the norm group of the national intelligence test, so you can read from the results of the intelligence assessment if the candidate scores higher or lower than the average Dutch person. But does that really help us? It can, if you know what the desired IQ-score is. To establish this score, you would have to let more people take the intelligence test with which you want to assess the candidate. That will suggest the score necessary to indicate the candidate’s ability to fulfill the job function.

What I’m trying to convey is that the moment you seek a capability assessment or an intelligence test to determine the work and thought capacity of a candidate, you need to have in mind the norm group to which the candidate can be compared. If you’re looking for a candidate with an average pre-university education, select capability assessments that have a norm group consisting purely of pre-university scholars. In addition, it is important that the norm group participants complete the assessment in the same circumstances; are they going for a particular job, or is this simply to try out a particular capability assessment or intelligence test? In the first case, it is likely that someone will put more effort into the test – and score higher – and that makes the norm stricter. There are assessments that are reliable and distinguishable  for both adults and children. The trick is then to ensure that separate norm groups are created for the childrens’ group and the adult group.

Measuring other cultures with a Dutch norm group

Can a Dutch company assess people from its UK subsidiary with the same test and norm group? First, to do so accurately it is again important that the assessment material is honest and available in the appropriate language. Second, it is important to know with which objective the assessment is taking place. If the objective is to compare employees of various subsidiaries with one another, it is preferable to use the same – in this case the Dutch – norm group. This will make the differences more transparent and comprehensible. If the objective is to determine a minimum assessment standard  or educational level that job candidates need to achieve in order to be taken on, then it is obviously important that the education level of the norm group is representative of the education available in the country in question.

It strikes me as better to compare everybody on a level playing field if a foreigner or an immigrant applies for a position at a Dutch company. In the case of immigrants to The Netherlands, it is of the essence that the candidate is able to understand the assessment material. So consider the degree of difficulty in the way in which the questionnaire is presented; there are questionnaires that have been specially translated to accommodate candidates with a B1 language level.  If you were to make separate norm groups for immigrants, it could unintentionally conceal evident differences with respect to a Dutch candidate – and that is exactly what you want to identify.

So don’t underestimate the value of a good and appropriate norm group when selecting a capability assessment for a candidate!