close fullscreen

Publications+Unsupervised Tokenization Learning

help edit space_dashboard
Draft — This content has not been approved for publication.
ai generated
more_horiz
open_in_full page
fullscreen modal
edit edit
space_dashboard advanced
Authors: Anton Kolonin, Vignav Ramesh

← Back to Publications

Unsupervised Tokenization Learning

Authors:

Anton Kolonin
Vignav Ramesh

Year: 2022
Venue: arXiv:2205.11443
Links: arXiv abstract

Summary

Introduces a “transition freedom” metric for unsupervised tokenization that outperforms statistical metrics such as mutual information and conditional probability, reaching F-measure 0.71–1.0 across multilingual corpora, with different metric offshoots needed per language. It finds that compressing models by eliminating statistically weak evidence improves tokenization, and that larger training corpora do not necessarily help.

Relevance to Hyperon

An unsupervised-language-learning source extending the Link-Grammar / Vepstas–Goertzel interpretable grammar-induction line into tokenization — relevant to the unsupervised-learning pipeline of the Semantic Parsing Deep Dive.

Key References

  • RawData source
  • Semantic Parsing Deep Dive


Tags

help edit space_dashboard
ai generated
more_horiz
open_in_full page
fullscreen modal
edit edit
space_dashboard advanced


Discussion