Computing Life · Share · YageSep 16, 2026, 08:00 UTC+8

Perplexity and OpenAI's PII detectors are not LLMs but bidirectional encoders with classification heads

上云前先脱敏：拆开 Perplexity 和 OpenAI 的 PII 检测器，它其实不是 LLM

Perplexity's open-source pplx-pii-masking is a 0.6B-parameter bidirectional encoder built on Qwen3 with causal masking disabled, topped with a token classification head and a document sensitivity head. It uses Viterbi decoding to output start/end offsets and confidence scores for 9 PII categories. OpenAI's Privacy Filter is a 1.5B sparse MoE model with ~50M active parameters and a nominal 128K context window, but its banded attention limits each token's effective view to 257 tokens. In tests, both models missed bare API keys and produced slice offsets; pplx silently truncates inputs beyond 4096 tokens, while OpenAI mislabeled an account number 550 tokens away from its context label as a phone number. The takeaway: on-device PII protection needs small classifiers for natural-language entities plus regex and entropy checks for fixed-format secrets.

Why it matters: The author ran hands-on tests against Perplexity's open-source detector, documenting misclassification, slice offset, and missed keys, then explained why autoregressive LLMs can't natively output per-span confidence. The second half defines requirements but doesn't unpack Open...

Read the original ↗Export Markdown

Take

This piece is worth opening because it cleanly dissects two open-source PII detectors at the architecture level. Perplexity's pplx-pii-masking is a 0.6B bidirectional encoder built on Qwen3 with causal masking disabled, topped with a token classification head and a document sensitivity head, using Viterbi decoding to output start/end offsets and confidence scores for 9 PII categories. OpenAI's Privacy Filter is a 1.5B sparse MoE with ~50M active parameters and a nominal 128K context window, but its banded attention limits each token's effective view to 257 tokens.

Real-world testing surfaced several failure modes: bare API keys get mislabeled as account_number, slice offsets often capture only the middle portion, and keys buried at the end of long documents get missed. pplx silently truncates inputs beyond 4096 tokens with no warning; OpenAI's banded attention mislabeled an account number 550 tokens away from its context label as a phone number.

The author's M3 Ultra benchmarks are useful reference points: pplx handles 2816 tokens in ~0.7 seconds with an F1 of ~0.98 across 14 test documents; OpenAI takes ~39 seconds for 21820 tokens. But don't read that F1 as a general metric—the test set is only 14 documents, and the post doesn't detail its composition.

The core takeaway I'd keep: on-device PII protection can't rely on a single component. Small bidirectional classifiers handle natural-language entities with variable context, while regex and entropy checks cover fixed-format secrets. That conclusion isn't new, but the teardown makes a clear case for why you can't just throw an LLM at this problem—autoregressive models can't produce continuous character-level confidence scores on raw input text. It's an architectural limitation.

Source: Computing Life · Share · Yage · yage.ai
Published: Sep 16, 2026, 08:00 UTC+8 · 14 days ago