Productivity gains and hidden pitfalls in AI-assisted knowledge work

We use AI today for small tasks in both our professional and personal lives. How much does it increase our productivity and output quality?

A study conducted last year by Harvard Business School in collaboration with the Boston Consulting Group asked 758 BCG consultants to perform several tasks that were designed to be performed without AI access, with AI access, or with AI access with prompt engineering overview (instructions and techniques to get the best results).

The researchers introduced the concept of a "jagged technological frontier" for AI capabilities, where some tasks are easily done by AI, while others aren't.

The results

For tasks within the AI frontier:

⬆Consultants using AI completed 12.2% more tasks on average

⬆ They completed tasks 25.1% faster

⬆Quality improved by more than 40% compared to the control group

Lower-skilled workers benefited more (43% improvement) compared to higher-skilled workers (17% improvement)

For tasks beyond the AI frontier:

🔻 Consultants using AI were 19% less likely to produce correct solutions.

How do we use AI to accomplish these tasks?

2 main approaches were taken:

✔ Individuals dividing and delegating activities between AI and themselves.

✔ Individuals integrating their workflow with AI and interacting with the technology on an ongoing basis.

A warning, not the best tool for brainstorming:

🔴 The study found that while AI improved individual performance, it also led to more homogenised results, potentially reducing the diversity of ideas.

In general, the group that had access to AI and guidance for prompt engineering outperformed those who used AI alone without additional guidance.

If you want to know a bit more read the article: Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality.