Vision-Language Models: CLIP and BLIP-Style Systems

import numpy as np

def unit(v):
    return v / (np.linalg.norm(v) + 1e-9)

images = {
    "dog": np.array([0.92, 0.12, 0.20]),
    "sailboat": np.array([0.10, 0.95, 0.18]),
    "chair": np.array([0.12, 0.25, 0.94]),
}

texts = {
    "a dog playing fetch on the beach": np.array([0.88, 0.16, 0.16]),
    "a sailboat glides across the ocean": np.array([0.18, 0.90, 0.22]),
    "a modern armchair in a minimal room": np.array([0.10, 0.24, 0.96]),
    "a cat sleeping on a blanket": np.array([0.62, 0.12, 0.50]),
}

def softmax(logits):
    logits = logits - logits.max()
    probs = np.exp(logits)
    return probs / probs.sum()

image_name = "sailboat"
scores = {
    text: float(unit(images[image_name]) @ unit(vec))
    for text, vec in texts.items()
}

ranked = sorted(scores.items(), key=lambda item: item[1], reverse=True)
probs = softmax(np.array([score for _, score in ranked]) / 0.2)

print("image:", image_name)
for (text, score), prob in zip(ranked, probs):
    print(round(score, 3), round(float(prob), 3), text)

The snippet is a tiny witness for retrieval and zero-shot prompt scoring. Real systems learn the encoders from many image-text pairs; this code only exposes the scoring contract:

• normalize image and text vectors,
• compute image-text similarities,
• rank candidates or softmax over prompt candidates.

Captioning would not stop at a rank. It would feed image-conditioned state into a decoder and generate caption tokens.