Java's . is not [^\n]

Here is a rewrite that looks obviously correct:

.        →    [^\n]

Any dot, expanded to "anything except a newline." That is what . means, near enough, in most engines people use daily.

In Java it is wrong. java.util.regex excludes all line terminators from ., not just \n:

\n        line feed
\r        carriage return
\r\n      the pair
U+0085    next line
U+2028    line separator
U+2029    paragraph separator

So [^\n] matches a carriage return where Java's . does not. Feed it CRLF text — a Windows file, an HTTP header, a pasted log — and the two patterns diverge. No error, no warning, just a different answer on the input you were least likely to test with.

I know this because my own tool emitted that rewrite. It got there through a transpiler, and chasing down why is what convinced me to throw the transpiler out and ship fourteen language runtimes instead.

The setup

I build a visual regex editor. You drag nodes on a canvas; the app keeps an AST and serializes it back to a pattern. It supports 21 language flavors, which means one AST has to become 21 different regex strings.

That is a compiler problem, and there is a good tool for it: Pomsky — a regex language that compiles to PCRE, Java, .NET, Python, Ruby, Rust, JavaScript and RE2. I wired it in: AST → Pomsky source → Expr::parse_and_compile() → flavor-specific regex.

Before committing I probed it properly, compiling test expressions against every flavor and recording the output. Pomsky came out ahead of my assumptions:

• atomic('foo') emits (?>foo) for Python, Ruby, Java, .NET and PCRE
• ['a'-'z'] & !['aeiou'] gives class subtraction across five flavors
• range '0'-'255' expands to a correct numeric-range alternation everywhere
• regex '...' is a raw passthrough escape hatch, so anything Pomsky does not model can still be emitted verbatim

It also refuses things correctly. It rejects [sc:Latin] for Python, because Python's re genuinely has no Unicode script support. It rejects atomic groups for RE2, because RE2 is a Thompson NFA and cannot backtrack. Those are right answers.

I want to be clear about this, because the rest of the piece is about why I removed it: Pomsky is a well-built tool, and none of what follows is a defect report.

Three bugs, and who actually caused them

Within a week, users found three. I recorded all three at the time as Pomsky emitter bugs. Rebuilding the probe to write this article, that turns out to be wrong about two of them — and the truth is more useful.

Everything below is pomsky 0.12.0, which is both the version I was on and the current crates.io release:

Construct	What came out	Whose doing
Java .	[^\n]	Mine
Python \Z	$	Mine
Python \k<tag>	(?:\1)	Pomsky's

The dot. My serializer emitted Pomsky's ![n] — "not a newline" — for any dot without the s flag. Pomsky compiles that to [^\n] in every flavor, which is precisely, correctly what ![n] means. The defect is upstream of Pomsky: I asked for "not a newline" when what I meant was "whatever . means in the target flavor", and Pomsky has no way to express the second thing. There is no token for it, because a source language has to fix its own semantics — that is what makes it a language.

The anchor. My serializer mapped end-of-line $, end-of-string \Z and absolute-end \z all onto Pomsky's single End, which emits $ everywhere. Three distinct anchors flattened into one. That looks like carelessness until you check the alternative: Pomsky has no absolute-end anchor. EndOfString and \Z are both rejected outright. There was nowhere else for those anchors to go.

The backreference. This one is Pomsky's. Given a named group and a named backreference, it emits (?:\1) for Python, PCRE, Java, JavaScript and .NET — only Ruby keeps \k<tag>. Python supports (?P=tag), so this is less faithful than the flavor allows. In fairness it is a fidelity loss and not a correctness one: (?:\1) matches exactly what (?P=tag) matches. But in a tool whose entire job is showing you your regex, giving back a number where you wrote a name is a real defect.

All three are reproducible in about ten lines, if you want to check rather than take my word for it:

use pomsky::{Expr, options::{CompileOptions, RegexFlavor}};

fn go(src: &str, flavor: RegexFlavor) -> String {
    let opts = CompileOptions { flavor, ..Default::default() };
    Expr::parse_and_compile(src, opts).0.unwrap_or("<rejected>".into())
}

fn main() {
    println!("{}", go("![n]", RegexFlavor::Java));                       // [^\n]
    println!("{}", go("'a' End", RegexFlavor::Python));                  // a$
    println!("{}", go("'a' \\Z", RegexFlavor::Python));                  // <rejected>
    println!("{}", go(":t(['a'-'z']+) ::t", RegexFlavor::Python));       // (?P<t>[a-z]+)(?:\1)
}

So: two of the three were mine, and the third is cosmetic. That is a worse result for the "the library let me down" story and a better one for what actually matters, because it means the argument does not depend on Pomsky being wrong anywhere.

Pomsky is a language you write in. It has variables, tests, its own syntax and its own semantics, and it owns the meaning of what you author in it. I was using it as an intermediate representation for a round-trip translation of someone else's tree — my parser's output, expressed in Pomsky's terms, re-emitted in Java's terms. Two lossy hops where the job needed zero.

The friction was visible before the bugs. My commit history has a whole pass adding quoting rules for character-class ranges, emitting empty strings so alternations would not fail to parse, and mapping control characters by codepoint — all of it plumbing to make a human-authored language behave as a compile target. That is the shape of using a tool off-label.

The part that generalizes

I replaced Pomsky with direct serialization plus per-flavor rewrite adapters, and all three issues went away in my implementation. Not because Pomsky was the problem — two of the three were mine — but because the AST stopped being round-tripped through a language obliged to normalize it.

But fixing the emission only got me to a better answer to the wrong question.

A transpiler answers: what regex text should I emit for Python?

What I actually needed to answer was: what will CPython do when it runs this?

Those are different questions, and only the second one matters when the pattern ships. No amount of correctness in the first gets you the second, because the interesting divergences are not syntax at all. They are the same source text behaving differently.

Some measurements from my own test corpus — 539 small test patterns, each executed on up to 16 real language runtimes, with one JSON file of recorded output per engine. The data, the methodology and the per-engine coverage are published on GitHub, so none of what follows has to be taken on trust:

\h+ against DEAD 00FF xyz. Ruby returns ["DEAD", "00FF"]; Perl and PHP return [" ", " "]. In Onigmo \h is a hex digit. In PCRE2 it is horizontal whitespace. Identical two characters, opposite meanings, both succeed.

^.+$ with the multiline flag, on CRLF text. Eight engines keep the trailing \r in each match. JavaScript and Dart strip it. Ruby returns the entire input as a single match, because Ruby's m flag means dotall, not multiline — and Ruby's ^/$ are line anchors already. PostgreSQL rejects the pattern. Four behaviours, one pattern.

A CSV field pattern. Ten engines return six matches, including empty ones. Go and Rust return three. Perl rejects the pattern outright. The cause is not match-selection semantics — Go's default engine is leftmost-first, same as Perl — but empty-match iteration policy: Go's FindAll documents that "empty matches abutting a preceding match are ignored", and Rust's find_iter behaves the same way. Reduced to its essence, a* over bab gives Go ["", "a", ""] and Python ["", "a", "", ""].

There is nothing here for a transpiler to translate. The text is already identical in every flavor. The divergence lives in the engine.

Across the corpus, 264 of the 534 test patterns that ran on more than one engine behave differently on at least one of them. Discount the 135 that are POSIX shell tools genuinely lacking the feature, and 129 remain: 83 where some engine rejects the pattern outright, and 46 where nothing is reported at all — 21 returning different matches, 25 silently matching nothing.

The 83 are fine. Your build breaks, you fix it. The 46 are the ones that ship.

What running the real engine costs

I now bundle fourteen native engines as sidecar binaries: CPython, Onigmo (the regex engine MRI Ruby embeds — compiled from C and statically linked, without the interpreter), a GraalVM-compiled JVM, .NET NativeAOT, static-php-cli's PCRE2, microperl and the rest. They run as long-lived child processes speaking JSON over stdin/stdout, 1–3 ms per call.

The payload is 16 engine binaries — 24 MB per architecture, 47 MB as universal — plus a vendored CPython tree, because Python is the one engine that genuinely needs its interpreter present rather than a statically linked library. That tree is 9.5 MB per architecture after pruning the stdlib to what the engine imports, dropping sources in favour of bytecode, and keeping 14 of CPython's 122 codecs. Call it 66 MB in the shipping universal build. A regex tool that approximates everything in JavaScript ships none of that.

The honest costs: the download is larger than a pure-JS tool by an order of magnitude, every runtime is a signing and notarization surface, and cross-compiling a universal binary means maintaining two toolchains for some of them. Ground truth is regenerated through Docker images pinned per engine, because "the version on my Mac" is not a reproducible answer.

It is a lot of machinery to answer a question that sounds simple.

When a transpiler is still the right tool

If you are authoring patterns and want them portable, use Pomsky. A source language with variables, comments and compile-time tests is strictly better than hand-writing the same regex six times, and its flavor targeting is good.

If you need to know what an engine does — the offsets it reports, the empty matches it emits, whether \h is hex or whitespace — run the engine. Nothing is more authoritative than the engine itself: not the documentation, and not a very good compiler that targets it.

I spent a week learning that distinction. The cheaper version: translation tells you what to send. Only execution tells you what happens.

---

I maintain RegexPilot, a macOS regex editor that bundles fourteen native engines so patterns run on the real interpreter for their language. The divergence corpus described here is its test suite.