The objective of the instrument is to provide an insight into the current state of an individual’s knowledge, insight and attitude in respect of these skills. Using a series of psychometric instruments, numerous characteristics are measured in a scientifically responsible way.
21st Century Skills can be defined as “The skills, knowledge, insight and attitude required to function in and contribute to future society”. In the 21st century, these skills vary greatly from those required in the 20th century, particularly as far as work is concerned. This is primarily due to the advent of increasingly advanced information and communication techniques. Numerous models have been developed in relation to 21st Century Skills, but many correspond broadly (Dede, C. 2010). As far as what is required for the labour market in the 21st century, this does not only concern new skills exclusively. Some, such as critical thinking and problem-solving capabilities, are only becoming more important (Rotherham, & Willingham, 2010).
The Ixly 21st Century Skills programme was developed from a work-related perspective and the questions examine a combination of work-related competencies, cognitive styles and level of digital literacy. Digital literacy refers to “The ability to use digital information and communication wisely and to evaluate the outcomes thereof critically. Someone who is digitally literate must be able to understand and use information purposefully” (KNAW, 2012), and can be sub-divided into the categories of Basic ICT Skills, Information Skill and Media Literacy. A separate questionnaire has been developed to measure the level of digital literacy for each of these three categories. Using literature research and comprehensive explanations, psychometrists posited items that measure basic ICT skills, information skills and media literacy (76, 39 and 40 items respectively). A pilot study, in which 907 people in civil service roles completed one or more questionnaire, was carried out subsequently. Based on an internal consistency analysis, the Basic ICT Skills questionnaire was shortened by removing less suitable items from the scale and thus shrunk to 51 items (-33%). A factor analysis to examine unidimensionality was first carried out for the Information Skills and Media Literacy instruments. One higher order factor below which four scales could be placed was found in both questionnaires. These scales always looked at internal consistency and again, less suitable items were eliminated. This resulted in a final list of 27 Information Skills items (-30%) and 30 Media Literacy items (-25%).
The 21st Century Skills programme consists of the following combination of questionnaires:
The Adaptive Work-related Personality Inventory (AWPI)
The AWPI is an adaptive version of the Work-related Personality Inventory (WPI), a questionnaire that measures a candidate’s work-related personality. With an adaptive questionnaire, not every candidate receives exactly the same list of questions. Only questions that provide additional information about the candidate’s personality are asked. The questions that are asked depend on the answers the candidate gave to previous questions. For more information, please refer to the AWPI fact sheet.
The Normative Career Values Inventory (CV-N)
The Career Values Inventory questionnaire provides an insight into an individual’s drivers or motivators in respect of work. The reference population is representative of the Dutch workforce. For use within the 21st Century Skills instrument, this has been shortened to 26 questions considered most relevant for this application.
Digital Literacy - Basic ICT Skills
The Basic ICT Skills questionnaire provides an insight into the extent to which an individual possesses knowledge and skills in relation to the operation of computers. It specifically concerns knowledge of the internet, computer hardware, use of email, operating systems, software security, and use of smartphones and social media. The questionnaire consists of a series of dichotomous questions that can be answered with yes/no. For this reason, the results are given as percentages.
Digital Literacy – Information Skills
Information skills relate to “the ability to identify and analyse an information need and the ability to search, select, process and use (apply) relevant information on this basis” (Brand-Gruwel, & Wopereis, 2010). Twenty seven items are used to assess the extent to which candidates are competent in the proposal and application of an information strategy and a search strategy. Their level of judgement and presentation skills are also examined.
Digital Literacy – Media Literacy
Media literacy can be defined as “the body of knowledge, skills and mindset with which citizens can consciously, critically and actively navigate a complex, changeable and fundamentally medialised world” (paraphrased from The Council for Culture, 2005 from Thijs, Fisser, & van der Hoeven, 2014). The Media Literacy module uses 30 items to ascertain the extent to which candidates are engaged with the media, have an insight into it and understand its impact, and whether they deal with this in a mature manner.
Pre-existing reference populations representative of the workforce (N = 1017 & N = 309 respectively) were used for the Normative Career Values Inventory and the Adaptive Work-related Personality Inventory. The new instruments were standardised based on the aforementioned study after weighting was applied to sex, age and educational qualifications. The Information Skills and Media Literacy instruments have a reference population of N = 907 and N = 852 in size respectively.
Based on the aforementioned analyses, we can report the following reliabilities:
The 21st Century Skills programme can be applied to all issues in an advisory context where the abovementioned skills required for the 21st century are of importance or need to be assessed. In virtually all occupations, particularly those in the technical and administrative sectors, it is important to have a mastery of the elements that are measured in the 21st Century Skills programme. For example, in the collection, processing, analysis, inventorisation, and careful handling of (large volumes of) data in digital format. The skills are also of great importance for sales and especially marketing roles.
The report associated with the 21st Century Skills programme lists the results for all the modules the candidate has completed. It starts with an explanation that clearly outlines what the 21st Century Skills comprise and which the candidates should possess. The modules and their sub-categories are also briefly explained here. The report then goes into more detail about the Digital Literacy module. The Basic ICT Skills module gives a summary of the percentages that reflect the levels at which people master the different modules. Since it concerns basic skills, candidates are expected to be good or proficient in a number of modules. Below that, tips are given to enable the candidate to further hone their basic skills. A summary of the information skills then follows, including the candidate’s results against the factor as well as the underlying scales in (STEN) scores. The scores and the candidate’s deviation are again explained in writing, with tips at the end. The same applies to the results for the Media Literacy instrument. STEN scores are given for the factor and the underlying scales, followed by a written explanation and tips. Following the Digital Literacy module, there is more detail on the competencies that correspond to the Cognitive Styles. Specifically, there is an additional explanation regarding the creative thinking, critical thinking and problem-solving scores. Tips are given as per the Digital Literacy module. The final module looks at the work-related competencies of Communication, Collaboration, Socio-cultural skills and Self-regulation. Finally, the definitions of the constructs are given, along with a short explanation of the scores presented and how these can be interpreted.