Machine Learning

Machine Translation

Treat translation as conditional sequence generation, then predict the next target token while inspecting alignment, beam candidates, and BLEU-style evaluation traps.

status: reviewimportance: importantdifficulty 4/5math: graduateread: 24mlive demo

Back to Machine Learning Next: Structured Decoding: Token Masks From Schema Automata

Concept Structure

Machine Translation

01Intuition

Start with the picture, metaphor, or geometric mechanism.

02Math

Make the objects explicit and connect them with notation.

03Code

Mirror the equations with runnable implementation details.

04Interactive Demo

Manipulate the mechanism and watch the idea respond.

3prerequisites

1next concepts

3related links

Learner Contract

What this page should let you do.

You are here becauseTreat translation as conditional sequence generation, then predict the next target token while inspecting alignment, beam candidates, and BLEU-style evaluation traps.

This Machine Learning concept is the current idea: keep the same invariant visible across Intuition, Math, Code, Interactive Demo.

Before thisMaximum Likelihood

3 prerequisites listed; refresh them before leaning on the math or code.

Maximum Likelihood Seq2seq and Encoder-Decoder Attention (review)Beam Search and Sequence Decoding (review)

By the end4/4 sections ready | runnable code expected | live demo

Explain the mechanism, trace the main notation, and test one prediction in the live demo.

Do this firstIntuition

Read the intuition before the notation; the math should name a mechanism you already felt.

Then go nextStructured Decoding: Token Masks From Schema Automata

Follow this edge after making one prediction here; the next page should reuse the result, not restart the route.

Test the linkManipulate one control and predict the visible change.Then continue to Structured Decoding: Token Masks From Schema Automata

Claim/source review status

Substantive review recorded

1/1 claims have bounded review metadata; still check caveats and source scope.Metadata-derived; review may be AI-assisted. Not a human certification.

Claims1/1 reviewed

Sources5 cited

Codeattached

Demolive

Reviewed2026-07-02

Updatedpage 2026-07-02

Learning item flow

4/4 sections readyAsk about this Research room

ConceptMachine TranslationMachine Learning EquationMachine Translation equation 1Exact equation CodeMachine Translation runnable code 1Exact runnable code DemoMachine Translation interactive demoInteractive demo ClaimMachine translation can be taught as conditional sequence generation:...Exact claim check SourceSpeech and Language Processing, Chapter 12: Machine TranslationExact source

ConceptMachine TranslationMachine Learning

6 sources attachedLocal snapshot ready

concept:machine-learning/machine-translation

Codenearby Predictbefore reveal Roomitem handoff

01IntuitionStart with the picture, metaphor, or geometric mechanism.02MathMake the objects explicit and connect them with notation.03CodeMirror the equations with runnable implementation details.04Interactive DemoManipulate the mechanism and watch the idea respond.

Intuition

Build the mental picture first so the rest of the page has something to attach to.

Section prompt

Machine translation is not word replacement. It is conditional sequence generation.

The input is a source sentence:

x_{1:n}=\text{``The green witch arrived''}.

The output is a target sentence:

y_{1:m}=\text{``Llego la bruja verde''}.

The toy Spanish tokens omit accent marks so the code and UI stay ASCII. The learning point is not Spanish grammar expertise; it is the modeling problem.

The source and target can have different lengths, different word order, omitted words, inserted function words, and morphology that depends on context. In the example above, the English adjective "green" appears before "witch," while the Spanish adjective "verde" appears after "bruja." A model that translates one word at a time in source order will stumble.

The modern encoder-decoder mental model is:

Encode the source tokens into states.
Decode the target tokens one step at a time.
At each target step, condition on the source and the already generated target prefix.
Search over possible target continuations.

Attention or cross-attention makes the conditioning path visible. When the decoder is about to emit "verde," it should look back toward the source idea "green," even though that source token is not in the same position as the target token.

Beam search makes the output path visible. The model may have several plausible next-token candidates. A decoder keeps a small frontier of candidate translations instead of committing to the first local choice forever.

Evaluation adds one more lesson. BLEU-style overlap can be useful for fast corpus-level comparison, but it is still a reference-overlap metric. A meaning-preserving paraphrase can receive a lower n-gram score because the words or order differ. A page that teaches MT honestly should show the triangle:

\text{alignment} \quad + \quad \text{decoding} \quad + \quad \text{evaluation caveat}.

Math

Translate the story into symbols, assumptions, and a derivation you can inspect.

Section prompt

Equation 1 $p_\theta(y\mid x).$ Equation 2 $p_\theta(y_{1:m}\mid x) =\prod_{t=1}^{m}p_\theta(y_t\mid y_{<t},x).$

Let $x=(x_1,\ldots,x_n)$ be the source-token sequence and $y=(y_1,\ldots,y_m)$ be the target-token sequence. Machine translation models the conditional distribution

p_\theta(y\mid x).

With an autoregressive decoder, the target probability factors as

p_\theta(y_{1:m}\mid x) =\prod_{t=1}^{m}p_\theta(y_t\mid y_{<t},x).

Training on a parallel corpus $\mathcal D=\{(x^{(i)},y^{(i)})\}_{i=1}^{N}$ usually maximizes conditional log likelihood:

\mathcal L(\theta) =\sum_{i=1}^{N}\sum_{t=1}^{m_i} \log p_\theta\left(y_t^{(i)}\mid y_{<t}^{(i)},x^{(i)}\right).

An encoder maps source tokens to hidden states

H=\operatorname{enc}_\theta(x)=(h_1,\ldots,h_n).

At target step $t$ , the decoder has a state $s_{t-1}$ that summarizes the generated target prefix $y_{<t}$ . Cross-attention creates a source context:

e_{t,j}=a_\theta(s_{t-1},h_j),

\alpha_{t,j}= \frac{\exp(e_{t,j})}{\sum_{k=1}^{n}\exp(e_{t,k})},

c_t=\sum_{j=1}^{n}\alpha_{t,j}h_j.

Then the next-token distribution is computed from the decoder state and source context:

p_\theta(y_t\mid y_{<t},x)= \operatorname{softmax}(W_o g_\theta(s_{t-1},y_{t-1},c_t)+b_o).

Decoding asks for a high-scoring sequence:

\hat y=\operatorname*{argmax}_{y} \log p_\theta(y\mid x).

Exact search is usually infeasible, so practical decoders use greedy search, beam search, sampling, or constrained variants depending on the task. MT has historically been a natural place for beam search because the output is source-grounded and there is usually a constrained translation objective.

BLEU-style evaluation compares a candidate translation $c$ to one or more references $r$ . Its core ingredient is clipped modified n-gram precision:

p_n = \frac{\sum_{g\in G_n(c)} \min(\operatorname{count}_c(g), \max_r \operatorname{count}_r(g))} {\sum_{g\in G_n(c)} \operatorname{count}_c(g)}.

BLEU combines several $p_n$ terms with a brevity penalty. This is useful for quick system comparison, especially at corpus level, but it is not a proof that the translation preserved meaning.

Code

Keep the implementation aligned with the notation so the algorithm is legible.

Section prompt

Runnable code 1from collections import Counter import math source = ["The", "green", "witch", "arrived"] pre...python

from collections import Counter
import math

source = ["The", "green", "witch", "arrived"]
prefix = ["Llego", "la", "bruja"]
reference = ["Llego", "la", "bruja", "verde"]

candidates = {
    "verde": {"logp": -0.89, "attn": [0.05, 0.78, 0.12, 0.05]},
    "bruja": {"logp": -1.51, "attn": [0.08, 0.10, 0.73, 0.09]},
    "green": {"logp": -1.90, "attn": [0.05, 0.81, 0.10, 0.04]},
    "Llego": {"logp": -2.20, "attn": [0.05, 0.08, 0.10, 0.77]},
}

def clipped_precision(candidate, reference, n):
    cand = [tuple(candidate[i:i+n]) for i in range(len(candidate)-n+1)]
    ref = Counter(tuple(reference[i:i+n]) for i in range(len(reference)-n+1))
    counts = Counter(cand)
    clipped = sum(min(count, ref[gram]) for gram, count in counts.items())
    return clipped / max(1, len(cand))

def bleu_proxy(candidate, reference):
    ps = [clipped_precision(candidate, reference, n) for n in [1, 2]]
    bp = 1.0 if len(candidate) >= len(reference) else math.exp(1 - len(reference) / len(candidate))
    return bp * math.exp(sum(math.log(max(p, 1e-9)) for p in ps) / len(ps))

for token, info in sorted(candidates.items(), key=lambda kv: kv[1]["logp"], reverse=True):
    out = prefix + [token]
    aligned_source = source[max(range(len(source)), key=lambda i: info["attn"][i])]
    print(token, "aligns_to=", aligned_source, "score=", round(bleu_proxy(out, reference), 3))

This witness is hand-authored, but the invariant matches the math: next-token probabilities are conditioned on the source and target prefix, attention can peak on a reordered source token, and reference-overlap scores can be computed without knowing whether the translation truly preserved meaning.

Interactive Demo

Use direct manipulation to connect the explanation to a moving system.

Section prompt

Use the Translation Path Lab as a prediction check.

Start with the source sentence The green witch arrived and the target prefix Llego la bruja. Predict the next target token before revealing the model's step.

Alignment: does the next target token look back to the same-position source word, or to a reordered source concept?
Decoding: which beam candidate does the conditional model prefer at this step?
Evaluation: which output has higher reference overlap, and which caveat should make you slow down?

Before reveal, the actual next token, attention weights, candidate scores, and evaluation bars are hidden. After reveal, inspect the source alignment, the top beam candidate, and the BLEU-style reference-overlap trap. The key sentence to leave with is: translation quality is not just a token probability, an attention heatmap, or an automatic metric; it is the agreement between source-conditioned generation and meaning-preserving output.

Live Concept Demo

Explore Machine Translation

The stage is code-native and interactive. Use it to test the explanation against the mechanism.

difficulty 4/5graduatecode-aligned

Demo Prediction Checkpoint

Manipulate one control and predict the visible change.

Commit to what Machine Translation should make visible before reading the result.

After The First Pass

Turn the concept into an inspected object.

Once the invariant is visible in the intuition, math, code, and demo, use these panels to inspect the mechanism visually, check source support, practice the idea, and attach a grounded research question.

Mechanism Storyboard

See the idea move before the page explains it

Treat translation as conditional sequence generation, then predict the next target token while inspecting alignment, beam candidates, and BLEU-style evaluation traps.

Prediction open01 / Intuition

Prediction lens

Start with the picture, metaphor, or geometric mechanism.

Commit first

Before reading further, choose the kind of change Machine Translation should make visible.

Visual Inquiry

Make the image answer a mathematical question

Treat translation as conditional sequence generation, then predict the next target token while inspecting alignment, beam candidates, and BLEU-style evaluation traps.

4/4 stages readyLive demo connected

Visual cueWhich visible object should carry the first intuition?

Inspection depth2/4

Prediction

Which visible object should carry the first intuition?

Commit first

Pick the cue that should make Machine Translation easier to reason about before the page gives the answer.

Source Grounding

Canonical references for the mechanism on this page.

book · 2026Speech and Language Processing, Chapter 12: Machine TranslationJurafsky and Martin

Frames MT as mapping source sentences to target sentences, training on parallel corpora, maximizing conditional target-token probability, encoder-decoder/cross-attention architecture, and evaluation caveats.

Open source

book · 2026Dive into Deep Learning: Sequence-to-Sequence Learning for Machine TranslationZhang, Lipton, Li, and Smola

Course/book source for encoder-decoder seq2seq training, teacher forcing, decoder recurrence, and machine-translation framing.

Open source

book · 2026Dive into Deep Learning: The Bahdanau Attention MechanismZhang, Lipton, Li, and Smola

Supports fixed-context bottleneck framing and target-step-specific attention context vectors over encoder hidden states.

Open source

course-notes · 2019CS224N: Neural Machine Translation, Seq2seq and AttentionStanford CS224N

Course source for seq2seq MT, encoder-decoder context vectors, attention equations, alignment-table intuition, and decoding as translation search.

Open source

paper · 2014Sequence to Sequence Learning with Neural NetworksSutskever, Vinyals, and Le

Primary source for sequence-to-sequence MT with LSTMs, conditional log-probability training, left-to-right beam-search decoding, EOS completion, and BLEU evaluation.

Open source

paper · 2015Neural Machine Translation by Jointly Learning to Align and TranslateBahdanau, Cho, and Bengio

Primary source for soft alignment, target-step-specific context vectors, and the fixed-length bottleneck motivation.

Open source

Claim Review

Treat translation as conditional sequence generation, then predict the next target token while inspecting alignment, beam candidates, and BLEU-style evaluation traps.

Status1 substantive review recorded

Claims without a substantive review badge still need exact source-support review.

Sources6 references

jurafsky-martin-slp3-mt, d2l-2026-seq2seq, d2l-2026-bahdanau-attention, cs224n-2019-nmt-seq2seq-attention, sutskever-2014-seq2seq, bahdanau-2015-align-translate

Local checks4 local checks

Use equations, runnable code, and demos to check whether the source support is operational.

Substantively reviewedMachine translation can be taught as conditional sequence generation: train on parallel sentence pairs to model p(y|x), decode a target sequence with encoder-decoder attention and search, and treat BLEU-style reference overlap as a diagnostic metric rather than ground-truth meaning.Claim metadata: source checked

SLP3, D2L, and CS224N support the task, data, encoder-decoder, attention/cross-attention, and conditional generation framing. Sutskever et al. support conditional log-probability training and left-to-right beam decoding. Bahdanau et al. support soft alignment and dynamic context vectors. Papineni et al. support BLEU as modified n-gram precision with corpus-level reference comparison.

Sources: Speech and Language Processing, Chapter 12: Machine Translation, CS224N: Neural Machine Translation, Seq2seq and Attention, Sequence to Sequence Learning with Neural Networks, Neural Machine Translation by Jointly Learning to Align and Translate, papineni-2002-bleuThe interactive example is a tiny hand-authored Spanish-token witness with ASCII tokens and fixed probabilities; real MT quality depends on tokenizer choices, training data, decoding settings, domain, human adequacy/fluency judgments, and evaluation protocols beyond BLEU.A bounded review summary is present; still check caveats and exact reference scope.

Checked SLP3 chapter 12, Stanford CS224N NMT notes, D2L seq2seq/Bahdanau attention, Sutskever/Vinyals/Le 2014, Bahdanau/Cho/Bengio 2015, and Papineni et al. 2002 for MT as supervised conditional sequence generation, parallel corpora, encoder-decoder/cross-attention conditioning, soft alignment, beam-search decoding, EOS-completed hypotheses, and BLEU modified n-gram/reference-overlap limits. GPT Pro/Oracle publication critique remains pending because 127.0.0.1:51672 refused connection.

Reviewer: codex-local-primary-source-audit; reviewed 2026-07-02

source-span-jurafsky-martin-slp3-mt source-span-cs224n-2019-nmt-seq2seq-attention source-span-papineni-2002-bleu Equation 1 Equation 2 Runnable code 1 Interactive demo

Source support candidates

book 2026Speech and Language Processing, Chapter 12: Machine Translation

book 2026Dive into Deep Learning: Sequence-to-Sequence Learning for Machine Translation

Course/book source for encoder-decoder seq2seq training, teacher forcing, decoder recurrence, and machine-translation framing.

book 2026Dive into Deep Learning: The Bahdanau Attention Mechanism

Supports fixed-context bottleneck framing and target-step-specific attention context vectors over encoder hidden states.

course-notes 2019CS224N: Neural Machine Translation, Seq2seq and Attention

Course source for seq2seq MT, encoder-decoder context vectors, attention equations, alignment-table intuition, and decoding as translation search.

Mechanism checks

Equation 1

p_\theta(y\mid x).

Equation 2

p_\theta(y_{1:m}\mid x) =\prod_{t=1}^{m}p_\theta(y_t\mid y_{<t},x).

Runnable code 1from collections import Counter import math source = ["The", "green", "witch", "arrived"] pre...Demo stateLive mechanism probe

Practice Loop

Try the idea before it explains itself

Treat translation as conditional sequence generation, then predict the next target token while inspecting alignment, beam candidates, and BLEU-style evaluation traps.

Readiness0/3 checks ready

Predict

Before touching the demo, predict one visible change that should happen in Machine Translation.

Hint 1

Reveal when your model needs a nudge.

Hint 2

Reveal when your model needs a nudge.

Hint 3

Reveal when your model needs a nudge.

Your answer canvas

Local checks

Claim

A concrete answer is on the canvas.

Mechanism

The answer names why the claim should hold.

Bridge

It touches the page context or a neighboring idea.

Misconception check

Grounded research drawerClose

ConceptMachine TranslationMachine Learning

Runnable code comparisonMachine Translation runnable code 1source = ["The", "green", "witch", "arrived"]Prediction before revealMachine Translation interactive demoManipulate one control and predict the visible change.

Grounded room questionWhat is the smallest example that makes Machine Translation click without losing the math?Local snapshot ready

Research Room

Attach the question to a claim, equation, code, or demo

Pick the concept, equation, source, runnable code, claim, misconception, or demo state before asking for help. The handoff keeps that page item in context.

Next local actionNo local draft saved yet

Open the draft below to save one note and next action in this browser.

conceptMachine Learning

Machine Translation

Attached question

What is the smallest example that makes Machine Translation click without losing the math?

Local action draftNo local draft saved yetExpand only when ready to capture one local next action

Local action draft

This draft stays in this browser, attached to the selected learning item.

Draft noteNext action

No local draft saved.

Evidence to inspect

References to inspect: attached references on this page.
Definition, prerequisite, and contrast concept links
The equation or runnable code that makes the concept operational
One demo state that shows the invariant instead of a slogan

What would resolve this

The learner can state the mechanism in their own words
The learner can name the prerequisite that would repair confusion
The learner can predict how the mechanism changes under one perturbation

Grounded AI handoff

I am working in Continuous Function's research reading room. Object: concept - Machine Translation Selected item key: recorded for copy. Context: Machine Learning Page anchor: recorded for copy. Open question: What is the smallest example that makes Machine Translation click without losing the math? Evidence to inspect: - References to inspect: attached references on this page. - Definition, prerequisite, and contrast concept links - The equation or runnable code that makes the concept operational - One demo state that shows the invariant instead of a slogan What would resolve this: - The learner can state the mechanism in their own words - The learner can name the prerequisite that would repair confusion - The learner can predict how the mechanism changes under one perturbation Answer as a careful research tutor: stay source-grounded, separate verified evidence from assumptions, name the relevant math objects, and end with one next action.

View it in context

concept/concept-notebook/machine-learning/machine-translation
concept:machine-learning/machine-translation

Learning Map

Before / Now / Try / Next

BeforeMaximum Likelihood

NowIntuition -> Math -> Code -> Demo

TryManipulate one control and predict the visible change.

NextStructured Decoding: Token Masks From Schema Automata

Intuitionready
Mathready
Codeready
Interactive Demoready

AI Companion

Ask beside the selected object

Treat translation as conditional sequence generation, then predict the next target token while inspecting alignment, beam candidates, and BLEU-style evaluation traps.

Your question

GoalComfortStyleStuck on

Context prompt

You are my AI learning companion for Continuous Function. Current context: Machine Learning concept. Learning surface: Machine Translation. What this page says: Treat translation as conditional sequence generation, then predict the next target token while inspecting alignment, beam candidates, and BLEU-style evaluation traps. Current section: Intuition, math, code, and interactive demo. Suggested next step: Manipulate one control and predict the visible change.. Learner goal: Understand the idea. Learner comfort level: New to this. Preferred explanation style: Visual first. Task: Explain the central idea in plain language, then restate it with the exact math objects from the page. Answer in a way that helps me learn: ask one clarifying question only if needed, use intuition before notation, and end with one thing I should try on the page.

Domain

Machine Learning

machine-learningdeep-learningsequence-modelingnlpmachine-translation

Prerequisites

Maximum Likelihood Seq2seq and Encoder-Decoder Attention (review)Beam Search and Sequence Decoding (review)

Leads To

Structured Decoding: Token Masks From Schema Automata

Decoding & Sampling: Temperature, Top-p & Inference-Time Control Tokenization & Vocabulary Design Scaled Dot-Product Attention & Transformer Layers

Within this domain