postgrok

stochastic-parrot

Properties1
tagslanguage-models, meaning, form, coherence

3 min read

Stochastic parrot

Definition (Bender et al. 2021, as written): an LM is “a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning”. See bender-2021-stochastic-parrots section 6.1.

The term is a deliberate deflation. A parrot reproduces human sounds without participating in human communication, and a stochastic one does it by probability rather than mimicry of a tutor.

The argument underneath

Two premises carry it. First, languages are systems of signs, pairings of form and meaning, per de Saussure. LM training data is form only. No training objective gives the system access to the meaning side, so fluent output is manipulation of form. Bender and Koller 2020, “Climbing towards NLU”, is the fuller version of this argument, cited as reference [14] in the parrots paper.

Second, coherence is in the eye of the beholder. Human readers interpret an utterance by modeling the beliefs, intentions, and common ground of an interlocutor, following Clark and others. That machinery runs automatically on text regardless of how it was produced. When one side of the apparent exchange has no meaning, the felt comprehension of implicit content is an illusion contributed entirely by the reader.

The accountability corollary

The paper stresses that LM text can enter conversations with no person or organization accountable for its truth. Two functions depend on accountability. Interpretation: words change meaning depending on who speaks them, so an apparent exchange needs an author to complete it. Redress: an unaccountable utterance has no one to answer for harm. These two are the hinge between the philosophical claim and the harm catalog: bias amplification, extremist recruitment text, translation errors attributed to the original author.

Limits of the claim

It is not a claim that LMs are useless, and not a claim that the output is random noise. The paper concedes state-of-the-art task performance and proposes ASR and captioning as legitimate benefit. The claim is narrowly about reference: nothing in the system points at the world, so benchmark success on tasks that can be approached by manipulating form is evidence about form manipulation, not about understanding.

Later career of the term

Bender reused it as the organizing noun of the research program, for example in her July 2021 Alan Turing Institute talk slides, which define a stochastic parrot as a system stitching linguistic forms without reference to meaning. Adoption and attack of the term since 2022, with larger models producing the text the paper anticipated, is outside what this vault has read, so it stays unfiled here.

Sources

Graph View

Backlinks