The Hidden Half: Methodology

The Hidden Half: Methodology

Change the Equation (CTEq) engaged the American Institutes for Research (AIR) to analyze raw data from the 2012 Programme for the International Assessment of Adult Competencies (PIAAC), a household study conducted by the Organization of Economic Cooperation and Development (OECD) to assess “’key information processing competencies’ that are relevant to adults in many social contexts and work situations, and necessary for fully integrating and participating in the labour market, education and training, and social and civic life..” (OECD, 2013). AIR focused its analysis on results from test takers in the United States, where the study was conducted from 2011 to 2012 with a nationally-representative sample of 5,000 adults aged 16 to 64. (U.S. Department of Education, n.d.).

To estimate the total number of U.S. citizens who use computers in complex ways on the job, CTEq and AIR examined responses to question G_Q06 in the PIAAC survey: What level of computer use is/was needed to perform your job/last job?

  1. STRAIGHTFORWARD, for example using a computer for straightforward routine tasks such as data entry or sending and receiving e-mails
  2. MODERATE, for example word-processing, spreadsheets or database management
  3. COMPLEX, for example developing software or modifying computer games, programming using languages like java, sql, php or perl, or maintaining a computer network


PIAAC identifies respondents' current or immediate past occupations with codes assigned through the International Standard Classification of Occupations (ISCO). To estimate the numbers of complex computer users in STEM and non-STEM occupations, CTEq classified six ISCO codes as STEM occupational codes and the remaining 43 as non-STEM occupational codes. (See here for a spreadsheet of STEM and non-STEM ISCO codes.) CTEq and AIR was then able to estimate the total number of respondents in each occupational category--STEM and non-STEM--who indicated complex computer use.


OECD (2013), OECD Skills Outlook 2013: First Results from the Survey of Adult Skills, OECD Publishing Retrieved May 27, 2015, from

U.S. Department of Education, National Center for Education Statistics, What is PIAAC 2012? (n.d.). Retrieved May 27, 2015, fromb

© 2022 by the Education Commission of the States (ECS). All rights reserved. ECS is the only nationwide, nonpartisan interstate compact devoted to education. 700 Broadway #810, Denver, CO 80203-3442

To request permission to excerpt part of this publication, either in print or electronically, please contact the Education Commission of the States’ Communications Department at 303.299.3636 or

Your Education Policy Team