Representation Alignment from a Few Paired Data via Distilled Optimal Transport

arXiv:2610.12781v1 Announce Type: new Abstract: Aligning two independently pretrained unimodal encoders typically demands millions of paired examples. In this paper, we study the regime where only $N \sim 100$--$500$ pairs are available, while unpaired data on either side is abundant. Our method expresses every representation by its similarities to the $N$ paired anchors, which places both modalities in a common coordinate system; matches the two point clouds with a single entropic optimal transport problem solved over the entire pool, whose marginal constraints counteract the hubness that defeats nearest-neighbor matching; and distills the resulting plan into two lightweight encoders, so that the correspondence becomes inductive. We additionally put this marginal constraint in relation with the one-step hub corrections of the bilingual lexicon induction literature. Pairing is required only for the anchors: the unpaired collections may be drawn independently from the two marginals, so that no item outside the anchor set needs to have both of its modalities observed. We evaluate on image-text and molecule-text alignment, where our method outperforms existing approaches at the same anchor budget on Flickr30k, MS-COCO and ChEBI-20.

Improves alignment of AI models with limited paired data.

This is an incremental technical contribution to multimodal representation learning, reducing data requirements for encoder alignment. It has no direct stated societal implication, positive or negative, beyond general efficiency gains in ML research.

This paper presents a technical advancement in AI alignment methods, specifically reducing the data requirements for aligning unimodal encoders. It demonstrates improved performance on established benchmarks. The technical nature of the paper does not directly address societal implications, either positive or negative.

SOURCE     arXiv cs.AI (tier 1)
   ↓
DOCUMENT   b22f4d15-4e84-4e91-8925-f9702c489127
           https://arxiv.org/abs/2610.12781
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  3 runs, methodology 1.0.1
   ↓
SCORE      -60  (Adverse)
   ↓
CONFIDENCE 82%