❌ Improbability 🤖 Penalty in AI training

unlikelihood

9.1

A training rule that makes a computer model avoid unlikely or repeated words

noun
/ˌʌnˈlaɪklihʊd/

Specialized

: pérdida de improbabilidad

The researchers added an unlikelihood loss to their model to reduce repetitive text generation.

Examples

In machine learning, minimizing unlikelihood helps improve performance.
During training, we noticed the unlikelihood helped discourage improbable token sequences and improved diversity.
Their toolkit includes utilities to compute unlikelihood across batches for sequence-to-sequence models.
The unlikelihood of repetitive outputs can be addressed with better algorithms.
The researchers incorporated an unlikelihood loss to their model to discourage generating repetitive phrases.
By implementing unlikelihood as a training objective, the team was able to reduce the occurrence of unlikely token predictions.
The optimization of the model's performance relied on adjusting the unlikelihood to achieve better output diversity.

Surface Forms

unlikelihood singular

unlikelihoods plural

Morphology

unlikelihood = likelihood (semi-transparent) = un + likelihood

Formally 'un-' + 'likelihood' is transparent in form, but the ML usage is a specialized technical sense (a loss/objective) that general learners may not infer from the components alone.

Etymology

Unlikelihood comes from un- ('not') and likelihood (from likely, 'chance'), and in machine learning the name keeps that meaning: an unlikelihood loss gives a penalty when the model makes 'not likely' predictions, helping it avoid strange or repeated outputs.