"A transformer does not know what a garden-path sentence is. It knows that the token "horse" shifts the probability mass for what follows, layer after layer, until the sentence resolves — no backtracking, no double take. Humans stumble over "the horse raced past the barn fell." Attention weights simply reallocate."- Claude 2026
Natural Language Processing
Language is arguably the most distinctly human cognitive act — an infinitely creative system built from a finite vocabulary. Understanding how the brain parses, disambiguates, and produces language has shaped, and been reshaped by, the AI systems now built to do the same.
Learning objectives
By the end of this page you should be able to:
- Explain cognitive models underlying natural language processing.
- Describe AI-driven language processing techniques.
- Analyze NLP applications using cognitive computing approaches.
Cognitive Models Underlying NLP
Long before neural networks parsed a sentence, the brain was already doing it — assembling sound or script into words, words into structure, and structure into meaning, almost entirely below conscious awareness.
The levels of linguistic analysis
Psycholinguists describe comprehension as a hierarchy of levels, each built on the last — from raw sound up to intention. Language analysis at tonia.ai breaks this hierarchy into concrete NLP tasks at each level, from morphology to discourse.
Each level constrains the next: a mis-tagged part of speech can send an entire parse down the wrong path — the same failure mode exploited by garden-path sentences below.
The brain's language circuitry
Broca's area
Left frontal lobe. Drives grammatical structure and fluent articulation. Damage produces Broca's aphasia: halting, agrammatic speech with comprehension largely intact.
Wernicke's area
Left temporal lobe. Drives lexical access and comprehension. Damage produces Wernicke's aphasia: fluent but meaningless speech, paired with poor understanding.
Garden-path sentences
The classic account — the garden-path model — holds that human parsing is serial and greedy: it builds the simplest structure first and reanalyzes only when forced to. Large language models fail differently: without true backtracking, they simply reweight probabilities token by token, so many garden-path sentences that trip up humans barely register as ambiguous to them, and vice versa.
Rules or statistics?
Universal Grammar
Noam Chomsky's position: humans are born with an innate, dedicated language faculty — a shared grammatical template that every human language instantiates. Language acquisition is triggered by exposure, not learned from scratch.
Statistical / connectionist learning
Elizabeth Bates and others: language emerges from general-purpose learning mechanisms — memory, attention, pattern extraction — applied to enormous amounts of linguistic input. No dedicated grammar module required.
Modern language models are, in effect, a large-scale existence proof for the statistical side: they acquire fluent grammar purely from exposure to text, with no built-in grammatical rules — though whether that settles the debate about human language acquisition remains contested.
AI-Driven Language Processing Techniques
Computational NLP mirrors the levels-of-analysis hierarchy above, but implements each stage as a trainable, statistical component chained into a pipeline.
Tokenization
Splitting raw text into words or sub-word units the model can process — the same first step the mental lexicon performs when parsing speech into words. See Language Analysis for the full breakdown of tokenization and morphological analysis.
Part-of-speech tagging
Labeling each token as a noun, verb, adjective, and so on — the statistical analog of the syntactic parsing that trips up readers of garden-path sentences. See Part-of-Speech (POS) Tagging.
Word embeddings
Representing each word as a dense vector such that semantically similar words land close together in space — "king" near "queen," far from "bicycle." See Deep Learning in NLP for how embeddings are learned.
Word sense disambiguation
Resolving which meaning of a word applies from context — "bank" as riverbank or financial institution. A direct computational analog of Wernicke's-area lexical access. See Word Sense Disambiguation.
Attention: learning what to focus on
Early neural translation models compressed an entire input sentence into one fixed-size vector — a bottleneck that degraded badly on long sentences. The attention mechanism replaced that single vector with a weighted combination of every encoder state, letting the decoder look back at whichever input words are most relevant at each step.
Self-attention extends this idea inside a single sentence: every word computes a relevance score against every other word, so "he" can pull context from a distant "dog" regardless of how many words sit between them. Stacking many attention heads in parallel — multi-head attention — lets the transformer track several kinds of relationships (grammatical role, coreference, topic) at once.
This mechanism underlies today's large language models — BERT reads bidirectionally to build rich context-aware embeddings, while GPT-style models attend only to prior tokens to generate text one step at a time.
NLP Applications Through a Cognitive Computing Lens
Each application below automates a task the brain performs effortlessly — translation, sentiment reading, summarizing, extracting facts — but does so by pattern-matching at scale rather than by understanding in any human sense.
Machine translation
Attention-based encoder-decoder models translate between languages without hand-built grammar rules. See Neural Machine Translation.
Sentiment analysis
Classifying text as positive, negative, or neutral — the computational cousin of the right hemisphere's role in reading tone and affect. See Sentiment Analysis.
Summarization
Transformer models condense long documents into short, coherent summaries. See Text Summarization with Transformers.
Information extraction
Pulling structured facts — names, dates, relationships — out of unstructured text. See Information Extraction in NLP.
Text classification
Assigning documents to categories — spam filtering, topic labeling, intent detection. See Classification in NLP.
Conversational agents
Chatbots and assistants chain the above techniques together, tracking dialogue context turn by turn to hold a coherent conversation.
Where cognition and computation still diverge
Both the brain and language models favor the simplest, most probable structure first, and both build predictions about upcoming words before they arrive — the basis of predictive coding in neuroscience and next-token prediction in LLMs.
Humans reanalyze a garden-path sentence by consciously re-reading it, and ground word meaning in embodied sensorimotor experience. Language models have neither: no re-reading loop, and "grasp" is only ever a pattern among tokens, never a motion of the hand.
Tools & Tutorials
- Advanced NLP with spaCy — a free, interactive course with 55 in-browser exercises covering tokenization, POS tagging, parsing, and named entity recognition.
- Transformer Explainer — an interactive, in-browser visualization of a GPT-2 transformer; watch tokenization and multi-head attention operate on real text in real time.
- TensorFlow Embedding Projector — explore pretrained word2vec embeddings in interactive 3D space; search a word and see its nearest neighbors.
- TensorFlow — word2vec tutorial — a runnable tutorial that trains word embeddings from scratch and visualizes them, making the geometry of meaning concrete.
Further reading
- Frazier, L. & Rayner, K. (1982). Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences. Cognitive Psychology, 14(2), 178–210. — the foundational eye-tracking study establishing the garden-path model of human parsing.
- Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. NeurIPS 2017. — introduces the transformer architecture and self-attention that underlie nearly all modern NLP models.
- Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL 2019. — the bidirectional pretraining approach that set the template for modern language understanding models.