The Measurement Trap: Financial Metrics vs. Productivity Reality
The main challenge with the article was that it focused on financial returns, not the success of the actual technology. The article highlights the difficulty in quantifying GenAI’s “micro-productivity gains”. They cite the following from a Fortune 1000 procurement executive:
"If I buy a tool to help my team work faster, how do I quantify that impact? How do I justify it to my CEO when it won't directly move revenue or decrease measurable costs?"
For those of us who advocate for GenAI, we can empathize with the executive’s dilemma. I call this “micro-productivity gains” because, although saving minutes with GenAI is hard to quantify, these small efficiencies accumulate across the economy.
A great example is using GenAI to generate images.
Let’s say we save 5 minutes per image using GenAI instead of going on the “perfect pic for my presentation hunt”. Over a handful of images, we don’t see the gains. However, over 10 million images those time savings amount to 95 years of productivity!
AI Has Already Won—Where It Can
The article itself actually testifies to the significant success that the technology is bringing to the average knowledge worker. Remarkably, the article actually said the following:
"AI has already won the war for simple work."
The core argument of the article is that standard generative AI technology is not yet equipped to fully replace human workers. For example, only 10% of respondents would entrust multi-week client management projects to AI rather than to human colleagues.
This, however, is not surprising. Anyone with a paid subscription certainly knows that GenAI needs multiple iterations to get the desired output.
The idea that we have such high expectations of the technology – for it to replace a junior lawyer – is a function of hype, the automation bias, and science-fiction movies.
From BYOD to BYOAI? AI Governance in Crisis
Perhaps the most interesting finding is that 90% of employees use generative AI regularly, regardless of official policies. The study found that “almost every single person used an LLM in some form for their work”.
History does not repeat itself, but it certainly rhymes. This is not the first time that employees have tried to impose consumer tech on enterprise IT. With the ascent of the iPhone and Android in the early 2010s, workers demanded the IT department figure out a way to make their devices work with the corporate email server. This Bring Your Own Device (BYOD) movement ultimately displaced BlackBerry's enterprise dominance.
The advent of Shadow AI, as the report aptly termed this trend, is more problematic. Formerly, it would take someone quite technically adept to figure out how to get corporate data onto their device. With Shadow AI, it is only a matter of copy and paste. Consequently, AI adoption raises a range of considerations related to privacy/confidentiality, data leakage, and regulatory compliance that organizations must address.
The main challenge with the article was that it focused on financial returns, not the success of the actual technology. The article highlights the difficulty in quantifying GenAI’s “micro-productivity gains”. They cite the following from a Fortune 1000 procurement executive:
"If I buy a tool to help my team work faster, how do I quantify that impact? How do I justify it to my CEO when it won't directly move revenue or decrease measurable costs?"
For those of us who advocate for GenAI, we can empathize with the executive’s dilemma. I call this “micro-productivity gains” because, although saving minutes with GenAI is hard to quantify, these small efficiencies accumulate across the economy.
A great example is using GenAI to generate images.
Let’s say we save 5 minutes per image using GenAI instead of going on the “perfect pic for my presentation hunt”. Over a handful of images, we don’t see the gains. However, over 10 million images those time savings amount to 95 years of productivity!
AI Has Already Won—Where It Can
The article itself actually testifies to the significant success that the technology is bringing to the average knowledge worker. Remarkably, the article actually said the following:
"AI has already won the war for simple work."
The core argument of the article is that standard generative AI technology is not yet equipped to fully replace human workers. For example, only 10% of respondents would entrust multi-week client management projects to AI rather than to human colleagues.
This, however, is not surprising. Anyone with a paid subscription certainly knows that GenAI needs multiple iterations to get the desired output.
The idea that we have such high expectations of the technology – for it to replace a junior lawyer – is a function of hype, the automation bias, and science-fiction movies.
From BYOD to BYOAI? AI Governance in Crisis
Perhaps the most interesting finding is that 90% of employees use generative AI regularly, regardless of official policies. The study found that “almost every single person used an LLM in some form for their work”.
History does not repeat itself, but it certainly rhymes. This is not the first time that employees have tried to impose consumer tech on enterprise IT. With the ascent of the iPhone and Android in the early 2010s, workers demanded the IT department figure out a way to make their devices work with the corporate email server. This Bring Your Own Device (BYOD) movement ultimately displaced BlackBerry's enterprise dominance.
The advent of Shadow AI, as the report aptly termed this trend, is more problematic. Formerly, it would take someone quite technically adept to figure out how to get corporate data onto their device. With Shadow AI, it is only a matter of copy and paste. Consequently, AI adoption raises a range of considerations related to privacy/confidentiality, data leakage, and regulatory compliance that organizations must address.
Although Shadow AI speaks to the resounding success of the tech, it also speaks to the urgent need to get AI governance in place.
Beyond the Hype: What the Study Actually Reveals
Though the headlines were laser-focused on the lack of cash flow resulting from the money invested in AI, a more careful read of the article reveals the productivity boom resulting from the technology. It's startling to think that three years ago GenAI was non-existent to most. Today, we are disappointed with it because it can't replace a junior at a professional services firm.
That said, the article offered some valuable insights into what success with GenAI can look like—a topic I'll be unpacking in a future post.
Beyond the Hype: What the Study Actually Reveals
Though the headlines were laser-focused on the lack of cash flow resulting from the money invested in AI, a more careful read of the article reveals the productivity boom resulting from the technology. It's startling to think that three years ago GenAI was non-existent to most. Today, we are disappointed with it because it can't replace a junior at a professional services firm.
That said, the article offered some valuable insights into what success with GenAI can look like—a topic I'll be unpacking in a future post.
Author: Malik D. CPA, CA, CISA. The opinions expressed here do not necessarily represent UWCISA, UW, or anyone else. This post was written with the assistance of an AI language model.
No comments:
Post a Comment