Do Subword Tokenizers Reproduce Blind Morpho-Orthographic Segmentation? A Convergence Test Across Three Tokenizer Families

Abstract

Masked-priming studies show that human readers decompose written words into stem- and suffix-like units even when that decomposition has no basis in real morphology (e.g., corner primes recognition of CORN nearly as strongly as cleaner primes CLEAN), evidence for an early, meaning-blind segmentation stage in reading (Rastle, Davis, & New, 2004). Subword tokenizers also perform segmentation based substantially on orthographic form rather than explicit semantic analysis, but arrive at their segmentations through a fundamentally different mechanism: statistical co-occurrence over a training corpus, with no reading system, lexical access process, or semantic representation involved. We test whether these two form-based processes converge on the same segmentation signature, using the original Rastle, Davis & New (2004) stimuli (n=147) and three architecturally distinct tokenizers: byte-pair encoding (BPE), WordPiece, and SentencePiece. Across four converging analyses — boundary match rate (all χ²<2.0, all p>.38), continuous boundary distance (all p>.2), correlation with human priming magnitude (all |r|<.22, all p>.12), and single-token segmentation rate (all p>.07) — we find no evidence of convergence between tokenizer and human segmentation behavior. A frequency-controlled regression confirms the null while validating the analysis pipeline: word frequency significantly predicted priming magnitude in the best-powered model (β=−14.41, p=.035) even as tokenizer boundary distance did not. We argue this indicates that generic form-based statistical segmentation, on its own, is not sufficient to reproduce the human blind-segmentation signature, and that the human effect depends on mechanisms beyond what ordinary subword tokenization captures.

1. Introduction

A substantial line of psycholinguistic work has established that human word recognition includes an early, semantics-blind segmentation stage: readers decompose corner into CORN+er almost as readily as they decompose cleaner into CLEAN+er, despite CORN having no semantic relationship to corner (Rastle, Davis, & New, 2004; Crepaldi, Hemsworth, Davis, & Rastle, 2016). This effect — commonly termed morpho-orthographic segmentation — is well established in the masked-priming literature and is not what the present work tests.

Our question is different: do modern subword tokenizers, which also perform segmentation based substantially on orthographic and statistical form rather than explicit semantic analysis, independently produce this same segmentation pattern? Tokenizers and human readers share a superficial similarity — both segment words on the basis of form rather than meaning — but they arrive at their respective segmentations through entirely different mechanisms. Human morpho-orthographic segmentation is thought to arise from an affix-stripping mechanism operating during early visual word recognition (Taft & Forster, 1975; Rastle, Davis, & New, 2004). Subword tokenizers instead learn segmentations from frequency- and likelihood-driven statistics over raw text corpora, with no reading-specific machinery, lexical access process, or semantic representation in the loop.

This distinction lets us treat the comparison as a convergence test rather than a replication or a morphological-accuracy check. If tokenizer segmentation reproduces the human pattern — treating pseudo-suffixed words (e.g., corner→CORN) and truly-suffixed words (e.g., cleaner→CLEAN) similarly, both distinct from non-suffixed controls (e.g., brothel→BROTH) — this would suggest that human-like blind morpho-orthographic segmentation can emerge from generic form-based statistics alone, with no reading-specific mechanism required. If tokenizers fail to reproduce this pattern despite sharing the form-over-meaning property, this would indicate that the human effect depends on something beyond what generic subword statistics capture.

This question is distinct from prior work comparing subword tokenization to gold-standard linguistic morphology. MorphBPE (Asgari, El Kheir, & Sadraei Javaheri, 2025) and MorphTok (Brahma et al., 2025) both show that standard BPE segmentation diverges from true morphological structure, and that constraining tokenizers toward gold morphology improves downstream task performance. That is a different question from ours: those works ask whether tokenizers recover true morphology (already answered, negatively). We ask whether tokenizers recover the specific blind, semantics-independent segmentation signature documented in the psycholinguistic priming literature — a signature defined not by accuracy against true morphology, but by equivalent treatment of true and pseudo-morphological structure.

Concretely, we test four predictions that follow from convergence: that match rate should be lower for non-suffixed than suffixed words in every tokenizer (not found, all p>.38); that boundary distance should track the same pattern (not found, all p>.2); that boundary proximity should predict the size of the human priming effect item-by-item (not found, all p>.12); and that non-suffixed words should be less likely to receive any internal split at all (a marginal trend for one tokenizer only, p=.070, not significant for the other two). None of the four predictions was confirmed.

2. Related Work

Human morpho-orthographic segmentation. Masked-priming studies have repeatedly shown facilitation from pseudo-suffixed primes (corner-CORN) at magnitudes comparable to truly-suffixed primes (cleaner-CLEAN), and reliably greater than form-matched non-suffixed controls (brothel-BROTH) (Rastle, Davis, & New, 2004). This pattern has been replicated and extended, including work isolating the effect to genuinely suffix-like endings via position-specific suffix priming (Crepaldi, Hemsworth, Davis, & Rastle, 2016) and work suggesting the effect can be modulated by task and semantic context (Diependaele, Sandra, & Grainger, 2005; Amenta, Marelli, & Crepaldi, 2015). The dominant theoretical account treats this as evidence for an early, form-based affix-stripping stage that operates prior to, and independent of, semantic access (Taft & Forster, 1975).

Subword tokenization. Byte-pair encoding (Sennrich, Haddow, & Birch, 2016) iteratively merges the most frequent adjacent symbol pairs in a training corpus to build a subword vocabulary. WordPiece (used in BERT; Devlin, Chang, Lee, & Toutanova, 2019) uses a related but likelihood-based merge criterion. SentencePiece (Kudo & Richardson, 2018) implements a language-independent tokenizer, commonly paired with a unigram language model segmentation objective, that operates directly on raw text without requiring language-specific pre-tokenization. All three are trained purely on statistical regularities in text, with no access to morphological annotation or semantic representation during training.

Tokenizer alignment with true morphology. A separate line of work asks whether subword tokenizers recover linguistically correct morphological boundaries, and generally finds that they do not without explicit constraint. MorphBPE (Asgari, El Kheir, & Sadraei Javaheri, 2025) modifies the BPE merge criterion to forbid merges across gold or predicted morpheme boundaries, showing improved morphological consistency and downstream training efficiency across English, Russian, Hungarian, and Arabic. MorphTok (Brahma et al., 2025) introduces morphology-aware pre-tokenization for Hindi and Marathi, improving machine translation and language modeling performance relative to standard BPE. These works establish that standard subword tokenization diverges from true morphology and that this divergence has measurable downstream costs. The present work does not test this same question; instead, it tests whether standard, unconstrained tokenizers reproduce a specific behavioral signature — semantics-blind equivalence between true and pseudo-morphological segmentation — documented independently in human reading research.

3. Method

Stimuli. 150 word triplets from the appendix of Rastle, Davis & New (2004), spanning three conditions: form (non-suffixed controls, e.g., brothel→BROTH), opaque (pseudo-suffixed, e.g., corner→CORN), and transparent (truly suffixed, e.g., cleaner→CLEAN). Each item includes the primed reaction time and a matched unprimed control reaction time from the original study; a priming-effect magnitude was computed as control RT − primed RT (positive values indicate facilitation). Three items were excluded following the original paper's own exclusions (against, tactile, textile), leaving n=147 (form: 47, opaque: 50, transparent: 50).

Tokenizers. Three architecturally distinct tokenizer families were used off-the-shelf, with no fine-tuning or task-specific adaptation: BPE (tiktoken, cl100k_base encoding, as used in GPT-3.5/GPT-4), WordPiece (bert-base-uncased), and SentencePiece (google/flan-t5-small).

Measures. For each prime word, three measures of tokenizer-human agreement were computed relative to the character position separating the target stem from its suffix (e.g., position 4 in “corner”, separating “corn” from “er”):

1. Boundary match (binary): whether the tokenizer places an exact split at that character position.
2. Boundary distance (continuous): the minimum character distance between that position and the nearest tokenizer-internal boundary, defined only for words the tokenizer splits into more than one piece.
3. Single-token rate (binary): whether the tokenizer represents the entire prime word as a single, non-internally-split token.

Statistical analyses. Match rates were compared across conditions via chi-square tests of independence, both omnibus (three-way) and pairwise (opaque vs. transparent; form vs. opaque). Boundary distance was compared across conditions via OLS regression. The relationship between boundary distance and human priming magnitude was tested via Pearson correlation and via an OLS regression of priming magnitude on condition, boundary distance, prime length, and prime frequency (Zipf scale, via the wordfreq package), the latter to rule out frequency as a confound. Single-token rate was compared across conditions via chi-square test.

4. Results

4.1 Boundary match rate

Raw match rates followed the direction predicted by the blind-segmentation hypothesis for all three tokenizers — form lowest, opaque and transparent higher and comparable to each other:

Tokenizer form opaque transparent BPE (GPT-4) 0.298 0.420 0.360 WordPiece (BERT) 0.191 0.300 0.300 SentencePiece (T5) 0.383 0.480 0.520

None of these differences were statistically reliable. Omnibus chi-square tests on condition × match found no effect for any tokenizer (BPE: χ²=1.567, p=.457; WordPiece: χ²=1.931, p=.381; SentencePiece: χ²=1.922, p=.382). The pairwise comparison the hypothesis specifically depends on — form vs. opaque, which must differ significantly for the blind-segmentation account to hold — was non-significant in every tokenizer (BPE: p=.298; WordPiece: p=.316; SentencePiece: p=.448). The opaque-vs-transparent comparison was also non-significant in every tokenizer (p=.682, 1.000, .842), consistent with the hypothesis on its own, but only informative in conjunction with a significant form-vs-opaque difference, which was not observed.

4.2 Boundary distance

Tokenizer form (n) opaque (n) transparent (n) BPE 1.189 (37) 1.220 (41) 1.318 (44) WordPiece 0.182 (11) 0.286 (21) 0.455 (22) SentencePiece 0.440 (25) 0.226 (31) 0.529 (34)

OLS regressions of boundary distance on condition found no significant condition effect for any tokenizer (BPE: opaque coef=0.030, p=.917; transparent coef=0.129, p=.652; WordPiece: opaque coef=0.104, p=.635; transparent coef=0.273, p=.212; SentencePiece pattern consistent with the other two, all p>.2). This rules out binary thresholding as the source of the null in Section 4.1.

Notably, the binary and continuous measures did not agree in direction. Match rate (4.1) was lowest for form in all three tokenizers, consistent with the predicted direction. Mean boundary distance shows the opposite pattern for BPE and WordPiece: form had the smallest mean distance (i.e., was best-matched by the continuous measure) precisely where it was worst-matched by the binary measure. SentencePiece showed a third, non-monotonic pattern (opaque lowest, transparent highest). Two operationalizations of the same underlying construct disagreeing in direction across every tokenizer tested is itself evidence against a robust effect.

Sample sizes in this analysis reflect substantial variation across tokenizers in how often a word receives any internal split at all: 122/147 words for BPE, 90/147 for SentencePiece, and only 54/147 for WordPiece (see Section 4.4).

4.3 Boundary distance and priming magnitude

Boundary distance did not correlate with human priming magnitude for any tokenizer (BPE: r=−.081, p=.374, n=122; WordPiece: r=.212, p=.123, n=54; SentencePiece: r=.118, p=.267, n=90).

A regression of priming magnitude on condition, boundary distance, prime length, and prime frequency confirmed this null while providing a validity check on the analysis pipeline. Boundary distance remained non-significant in all three models after controlling for frequency and length (BPE: β=−2.06, p=.586; WordPiece: β=10.76, p=.340; SentencePiece: β=7.18, p=.276). Word frequency itself was a significant negative predictor of priming magnitude in the best-powered model (BPE, n=122: β=−14.41, p=.035), consistent with the established finding that morphological priming effects shrink for higher-frequency words, and indicating that the regression was capable of detecting a real, independently expected effect in this data — the null on tokenizer distance is therefore not attributable to a generally underpowered or miscalibrated model. Prime length and prime frequency were only negligibly correlated (r=.015), ruling out collinearity as an explanation for either coefficient.

4.4 Single-token segmentation rate

Beyond where a boundary falls, tokenizers differ in whether they produce an internal boundary at all:

Single-token rate did not differ significantly by condition for BPE or SentencePiece. WordPiece showed a marginal trend (p=.070): form words were tokenized as a single indivisible piece substantially more often than opaque or transparent words. This did not reach conventional significance but is the closest result to significance across all analyses reported here, and suggests WordPiece may be somewhat less likely to internally segment non-suffixed words in the first place, independent of where any resulting boundary falls. We report this as a trend warranting replication, not as a finding.

5. Discussion

Both tokenizers and human readers perform segmentation that is substantially form-based rather than semantically driven, but they arrive at their respective segmentations through fundamentally different mechanisms — statistical co-occurrence over text for tokenizers, whatever cognitive process underlies the masked-priming effect for humans. This shared surface property is precisely what makes the convergence test informative: if the two processes nonetheless produced the same segmentation signature, the shared form-based property would be doing the explanatory work, and blind morpho-orthographic segmentation could be characterized as an emergent property of generic subword statistics, requiring no reading-specific mechanism to explain.

Across four converging analyses and three architecturally distinct tokenizer families, we find no such convergence. Match rate, boundary distance, and correlation with priming magnitude all failed to distinguish non-suffixed from suffixed conditions in the pattern the hypothesis predicts, and this null survived controlling for word frequency and length. This indicates that generic form-based statistical segmentation, on its own, is not sufficient to reproduce the human blind-segmentation signature. By elimination, the human effect must depend on some mechanism beyond frequency-driven segmentation of character sequences — plausibly involving prior morphological or lexical knowledge, phonological structure, or properties of the reading system not captured by co-occurrence statistics over raw text. We do not identify which mechanism is responsible; our result constrains the space of sufficient explanations rather than specifying the correct one.

The one result that came closest to significance — WordPiece’s marginal single-token-rate trend (Section 4.4) — points in a direction consistent with the broader null rather than against it: it suggests WordPiece may be less likely to split non-suffixed words at all, rather than more likely to split them in the wrong place, which is a different kind of divergence from human segmentation than the one our primary hypothesis predicted.

6. Limitations

The sample (n=147, split three ways) is modest by psycholinguistic standards, and WordPiece in particular loses substantial power in the distance and correlation analyses to single-token words (n=54, versus 122 for BPE and 90 for SentencePiece). A larger or independently collected replication — for instance using the Crepaldi et al. (2016) masked-priming stimuli as a second dataset — would strengthen confidence in the null but is not expected to change its direction, given consistency across three unrelated tokenizer training procedures and four independent measures.

The binary and continuous boundary-agreement measures did not agree in direction across tokenizers (Section 4.2), which should temper confidence in either measure individually; a preregistered choice of a single primary measure would strengthen a follow-up study. The marginal single-token-rate trend for WordPiece (p=.070) also warrants replication before being treated as a real effect.

Only English stimuli and English-trained tokenizers were tested; the relationship between tokenizer segmentation and human morpho-orthographic processing may differ for morphologically richer languages, where subword tokenization interacts with morphology differently (cf. Brahma et al., 2025, on Hindi and Marathi).

References

Amenta, S., Marelli, M., & Crepaldi, D. (2015). Consonant-vowel status affects lexical processes and morphological effects in visual word recognition. Journal of Cognitive Psychology.

Asgari, E., El Kheir, Y., & Sadraei Javaheri, M. A. (2025). MorphBPE: A morpho-aware tokenizer bridging linguistic complexity for efficient LLM training across morphologies. arXiv:2502.00894.

Brahma, M., Karthika, N., Singh, A., Adiga, D., Bhate, S., Ramakrishnan, G., Saluja, R., & Desarkar, M. S. (2025). MorphTok: Morphologically grounded tokenization for Indian languages. arXiv:2504.10335.

Crepaldi, D., Hemsworth, L., Davis, M. H., & Rastle, K. (2016). Masked suffix priming and morpheme positional constraints. Quarterly Journal of Experimental Psychology, 69(1), 113–128.

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT 2019, 4171–4186.

Diependaele, K., Sandra, D., & Grainger, J. (2005). Masked cross-modal morphological priming: Unravelling morpho-orthographic and morpho-semantic influences in early word recognition. Language and Cognitive Processes, 20(1–2), 75–114.

Kudo, T., & Richardson, J. (2018). SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. Proceedings of EMNLP 2018: System Demonstrations, 66–71.

Rastle, K., Davis, M. H., & New, B. (2004). The broth in my brother’s brothel: Morpho-orthographic segmentation in visual word recognition. Psychonomic Bulletin & Review, 11(6), 1090–1098.

Sennrich, R., Haddow, B., & Birch, A. (2016). Neural machine translation of rare words with subword units. Proceedings of ACL 2016, 1715–1725.

Taft, M., & Forster, K. I. (1975). Lexical storage and retrieval of prefixed words. Journal of Verbal Learning and Verbal Behavior, 14(6), 638–647.