Evaluating the Impact of AI on the Labor Market: Current State of Affairs

The current rhetoric about AI mirrors the anxiety over earlier generations of technological progress. So far, is this time different?

First, we look at how quickly the overall occupational mix changed in the first 33 months since ChatGPT’s relative to previous periods of technological change. Figure 1 compares the occupational mix by month to the mix in a baseline period at the start of a major technological development (e.g. January 1996 is the baseline month for the growing adoption of the internet). The job mix for AI appears to be changing faster than it has in the past, although not markedly so. (See the appendix for a discussion of pre-trends.)

The occupational mix refers to the distribution of workers amongst all the jobs in the economy. In this context, a percentage point difference means that, relative to the start point, that percent of workers are in new occupations. This can occur by workers changing jobs, losing jobs, or unemployed people getting a new job. As such, this metric attempts to capture how different the sum of occupations that make up the labor force is relative to another point in time. By measuring this over the time generative AI has been publicly available, we can test the claim that AI is substantially changing the workforce by any of the methods mentioned above (pushing workers from one job to another, automating workers out of a job, or creating new jobs). Note that a change over this time period simply reflects change — it does not take a stance on the cause of that change. Note that if recent grads are not getting hired, that would show up in this measure in as much as recent grads (as discussed below) are often in different occupations than older workers.

With the development of the internet and the growing prevalence of computers, the turn of the 21st century gave rise to concerns over an imminent computerization of many jobs. Despite this automation anxiety, the occupational mix by 2002 was at most around 7 percentage points different than it was in 1996; i.e., only 7 percent of workers in 2002 would need to switch occupations to match the composition of the 1996 labor market.

Changes in the occupational mix since the advent of generative AI in 2022 seem to mirror the trends seen during the three comparison periods. The recent changes appear to be on a path only about 1 percentage point higher than it was at the turn of the 21st century with the adoption of the internet. Although recent trends seemingly outpace historical shifts in the occupational mix, the potential effects of AI on the labor market so far are not out of the ordinary. In fact, taking a closer look at recent years, the data suggests that this recent trend is not necessarily attributable to AI (Figure 2). Shifts in the occupational mix were well on their way during 2021, before the release of generative AI, and more recent changes do not seem any more pronounced, even as the use of AI continues to grow in popularity.

Repeating this analysis by industry similarly suggests a limited effect of AI. Figure 3 reports the change in the occupational mix from November 2022 within different industries. The Information, Financial Activities, and Professional and Business Services sectors have all seen larger shifts in the job mix compared to the shifts in the aggregate labor market, with the largest changes in the Information sector. (As a reminder the information sector includes things like newspapers, movies, and data processing.) These industries are among those with the highest exposure to generative AI. Although at first glance these changes may seem attributable to generative AI, the data again suggests that the trends within these industries started before the release of ChatGPT (Figures 4-6). In fact, over a broader time horizon, the large shifts in the Information Industry seem to be a feature of the industry itself rather than a consequence of any one technological development (Figure 7).

Looking over an even longer horizon, labor market volatility appears rather low. As a chart Jed Kolko shows, the change in the occupational mix seen in Figure 1 is sluggish compared to the change seen in the 40s and 50s (which reflected mass labor market changes due to world events). Kolko cautioned that “we simply don’t know for sure whether automation, algorithms, and AI will ultimately create more jobs than they destroy.” 

The dissimilarity data we have examined indicates that there is no substantial acceleration in the rate of change in the composition of the labor market since the introduction of ChatGPT. Lacking that, there is nothing meaningful we can either attribute or misattribute to AI.

Taking a closer look at the trend since January 2021, the dissimilarity between older and more recent college graduates rarely deviates outside of the 30-33% range (Figure 10). This implies that these trends of growing dissimilarity may pre-date ChatGPT’s release and may not be attributable to AI. However, there is perhaps some slight upward momentum more recently, though this is consistent with both Brynjolfsson et al.’s work and the CPS’s noisiness (and a slowing labor market hitting younger workers). Further, the same caution in interpretation given the small sample sizes holds in this figure as well.

To better understand whether AI is impacting the labor market, we would want to analyze whether the share of workers in occupations that are most impacted by AI usage is changing over time. If AI were automating jobs at scale, we would expect to see a smaller share of workers in some of the jobs that are most negatively impacted.

Unfortunately, comprehensive usage data is not publicly available. The best available data we have is from OpenAI and Anthropic, respectively, that detail the occupations that are most “exposed” to genAI tools (a theoretical, forward-looking metric across all jobs) and that have the highest actual usage of one specific AI tool, Claude (a more narrow, present-focused metric). While imperfect, these data are our best approximation of AI job “risk”. (See discussion of limitations in the next section and in the appendix.)

Importantly, OpenAI and Anthropic are measuring different things and we look at them separately.

Even when specifically examining the unemployed population, there is no clear growth in exposure to generative AI. Figure 12 depicts the average percentage of tasks exposed amongst unemployed workers by duration of unemployment. AI-driven displacement might suggest a growth in the proportion of exposed tasks amongst recently unemployed workers. Irrespective of the duration of unemployment, however, unemployed workers were in occupations where about 25 to 35 percent of tasks, on average, could be performed by generative AI. Although there is some variation between months, the data demonstrate no clear upward trend and no clear difference by the duration of unemployment. 

As previously noted, the metrics from OpenAI and Anthropic are imperfect proxies for AI risk and usage, while still being the best available.

A key limitation of OpenAI’s “exposure” data is that it is not based on actual usage, and should therefore be interpreted as a theoretical estimate of the jobs and sectors that could, in theory, be impacted. In reality, actual AI usage and workplace diffusion has varied dramatically between sectors and occupations with similar levels of “exposure.” For instance, generative AI tools were adopted extremely quickly and at mass scale among coders and software developers, who are in the top quintile of exposure. Meanwhile, adoption has lagged considerably in clerical sectors, despite a similar level of exposure. Thus analyzing occupations by exposure alone likely under-estimates potential labor market disruption, as the top quintiles of exposure will include occupations that are theoretically exposed but not actively using AI at a meaningful scale, and thus unlikely to see AI impacts.

A comparison of the OpenAI “exposure” data with the Anthropic usage data makes this limitation clear. The two measures appear to have only a limited correlation with one another (Figure 20).

Figure 20 consists of occupations that have data for both the OpenAI and Anthropic measures, which amounts to about 80% of CPS occupations in our sample. Figure 21 splits the data from figure 20 into quadrants along the axes of low-high exposure and low-high usage and then groups the data into its SOC job categories. Particularly striking is the greater range of different job categories in the quadrants with low usage (the left side of figure 20) and conversely the concentration in just a handful of categories in the high usage quadrants. Across both high and low exposure, high usage occupations are dominated by scientific and quantitative professions and general business occupations. The occupations clustered in the low usage/exposure (bottom left of figure 20) tend to be production occupations with little computerization.

Just as the OpenAI metric has limitations, so too does Anthropic’s usage data. Figure 22 shows the occupational shares of all “conversations” with Claude (the AI chatbot), and illustrates the occupation groups that are over- and under- represented in this usage, compared to their exposure ranking and employment share.

It is clear from the data that Claude’s usage is heavily dominated by one occupational group — computer and mathematical, which includes coders — and that arts and media (including writers) is also considerably overrepresented. While certainly coding is among the most prominent use cases of AI, it is likely that Claude’s userbase skews more heavily to these tasks due to Claude’s stand-out reputation among LLMs as being particularly good at writing and coding. New data recently published by OpenAI shows a broad pattern of usage among ChatGPT customers across a range of industries, including not only information services (including software development) but also professional services and even manufacturing. It is entirely possible, and even likely, that usage data from other AI models like Google’s Gemini or Microsoft’s Copilot would show different and more varied patterns of usage. Thus data from Claude usage alone is not representative of how workers across the economy are using AI chatbots and tools.

To accurately measure AI’s impact on the labor force, the most important data needed is comprehensive usage data from all the leading AI companies at the individual and enterprise level, including APIs. Anthropic has led the way in transparently sharing Claude usage data, including a new release of enterprise data. To further our understanding of AI’s impact, it is important that all leading AI labs do the same in a similarly transparent and privacy-protected way. 

While anxiety over the effects of AI on today’s labor market is widespread, our data suggests it remains largely speculative. The picture of AI’s impact on the labor market that emerges from our data is one that largely reflects stability, not major disruption at an economy-wide level. While generative AI looks likely to join the ranks of transformative, general purpose technologies, it is too soon to tell how disruptive the technology will be to jobs. The lack of widespread impacts at this early stage is not unlike the pace of change with previous periods of technological disruption. Preregistering areas where we would expect to see the impact and continuing to monitor monthly impacts will help us distinguish rumor from fact.