Generative AI and Jobs: A Refined Global Index of Occupational Exposure

As the final step, we use the adjusted survey scores for 2,861 tasks to develop an AI assistant capable of predicting synthetic scores for new tasks across various classification systems. This combines the strengths of human expertise with generative AI’s ability to generate consistent, large-scale responses based on precise instructions.

Text Box 3. The conceptual structure of the LLM prompt (Python code for GPT-4o)

To develop our predictive model, we evaluated two approaches: (i) fine-tuning a large language model (LLM) with the 2,861-task dataset or (ii) designing an advanced prompting strategy (prompt engineering). Fine-tuning, while promising for learning domain-specific patterns, is resource-intensive and limited by our relatively small dataset.15 In contrast, by using a sufficiently large set of example tasks within the prompt, we can provide the model with explicit “demonstrations” of scoring of similar tasks in our main dataset. To ensure a balanced approach, we provide each prompt with a sample of tasks with the highest 50 and lowest 50 scores, alongside the scores of 30 tasks with greatest semantic similarity to the tasks that the model is requested to score. To create the semantic context, we construct a rapid-search vector database with all 2,861 adjusted scores. The database is queried at the start of each API call and extracts tasks with semantic similarity to the task that is being scored. Once the 30 most similar tasks have been retrieved, a set of complete instructions is passed to the model’s API, as per the general prompt strategy illustrated in Box 3.

We first validate the predictive model using a sample of tasks from the survey that were adjusted as per sections 5-7, but not included in the semantic context provided to the model.16 Once the prompt adjustments deliver satisfactory performance, we use the same system to predict automation scores for 3,265 tasks in ISCO-08 (as in Gmyrek et al., 2023).17 This approach bridges expert-validated human assessments with the scalability of LLM-based predictions, enabling consistent and extensible scoring for tasks across classification systems.

For a further refinement of our scoring system, we filter 364 tasks where the scoring in 2025 changed by more than 25% of the absolute value of 2023 score. We review these tasks manually, using an expert consensus-building method. First, we conduct a mini survey among co-authors, in which each respondent is asked to arbitrate between the diverging scores. For 64 ISCO-08 tasks, where we still observe a divergence of expert opinions, we conduct a detailed group discussion, in order to reach consensus and agree on a justification.

Following the final manual verification process, we establish a second knowledge repository, also stored in a vector database. The main LLM prompt is then updated to incorporate this additional reference by conducting two sequential semantic context retrievals for each prediction, further improving the model’s precision in predicting scores for new tasks. Using this approach, we generate predictions for all 29,753 tasks in the Polish classification system (Table 1).

To compare these 6-digit predictions with 4-digit ISCO-08 classifications, we calculate (i) the mean of all task scores aggregated at the 4-digit level and (ii) occupational means at the 6-digit level, subsequently deriving the 4-digit mean from these 6-digit components. In both cases, the correlation between predictions based on ISCO-08 tasks and Polish 6-digit tasks is 0.92, demonstrating exceptional stability of predictions. In other words, while one might question the exact scores assigned to tasks in our knowledge repository – a challenge not unique to our study – once the repository and prompting techniques are in place, the model consistently produces stable predictions based on reference tasks and their scores, and irrespective of the occupational classification system. This approach offers promising possibilities for replication across other occupational classifications, which we intend to explore further. For the remainder of this paper, we focus primarily on revising ISCO-08 scores, leaving a detailed analysis of Poland’s 6-digit occupations for a separate study (Gmyrek et al., forthcoming, 2025).