Human-AI synergy

Two definitions from vaccaro-2024-human-ai-meta-analysis, stated precisely because the loose claim carries them both:

  • Synergy: the human-AI combination performs better than the human alone and better than the AI alone.
  • Augmentation: the combination performs better than the human alone, with no claim about the AI alone.

The paper frames why the split matters. Most people assume the combination beats either alone, because otherwise they would just use the best of the two. That assumed bar is synergy. Most published results measure augmentation. The meta-analysis took both across 370 effect sizes from 106 experiments and found augmentation on average (g = 0.64) but no synergy on average (g = −0.23 against the best single performer).

What moves the outcome:

  • Task type. Decision tasks showed losses (g = −0.27). Creation tasks showed positive averages (g = 0.19), a significantly different result from decision work.
  • Which side is better alone. When the human beat the AI alone, combinations gained (g = 0.46). When the AI beat the human alone, combinations lost (g = −0.54).
  • Division of labor. Assigning each party the subtasks it wins was tried in only 3 experiments and remains under-studied.
  • Explanations and confidence displays did not move the numbers. The lever the field pushed hardest turned out to matter least.

The design response in this vault

AGENTS puts Ryan’s judgment upstream of the agent’s work, which is a division-of-labor design in the direction the paper calls for. The open question for the garden’s own thesis: which parts of knowledge-work gardening does the combination beat either side at, and which parts should stay on one side of the line?