SYNX 3.8 is faster everywhere, and every parser finally agrees

SYNX 3.8.0 is out, and if all you do is write config files, you won't see anything new in the files themselves, because the grammar hasn't moved at all. What moved is everything underneath it: the Rust core, the TypeScript package, the Python and Node bindings and all seven native ports were measured against 3.7.4 on the same machine and the same set of files, and then rebuilt wherever the time was actually going.

How much faster

Here’s how long the same files take with each version:

And the whole table, with Bun next to Node and the Python binding:

The biggest jump is TypeScript on large documents with markers, and the reason is a little embarrassing: every :calc key used to rescan its expression against every other key in the document, so the work grew with the square of the file. In Rust most of the win came from memory. With the Windows system allocator a plain parse spent close to half its time asking for memory, and 3.8.0 asks about a third as often on static files and a fifth as often in active mode. The binary .synxb format got the same treatment: the strings a document uses most now get the shortest indexes, so files are 3 to 20 percent smaller in Rust and up to 40 percent smaller in TypeScript, and the same document now always compiles to the same bytes, which 3.7.4 didn't promise.

The parsers didn't actually agree

This part we didn't go looking for. To make sure the faster code gave the same results, we started feeding the same randomly generated documents to the Rust core and to the TypeScript package and comparing what came out, and 17 percent of the !active documents came out different. Some of that was TypeScript being wrong and some was Rust, and some of it was Rust being wrong differently on every run: it walked objects in hash-map order, so the tax:calc / total:calc example straight from the spec failed with CALC_ERR in about half of all runs.

And once we started looking, there was more of it. Markers inside list items were never applied in Rust at all. Synx.stringify and .synxb changed data on the way back, so "007" came back as the number 7 and |+ code blocks lost their indentation, and 1,777 of 3,000 random documents didn't survive the round trip. A few documents could even take the whole process down: :clamp:nan:5 made Rust panic, and a file that included itself overflowed the stack. All 55 of these are fixed, and on the final code the Rust core and TypeScript agree on every one of 70,000 generated documents.

Seven ports, all on 3.8.0

SYNX also has native parsers for Go, Java, .NET, Dart, C++, Swift and Godot, and they had been falling a few releases behind. All of them are on 3.8.0 now, checked the same way against the Rust core on thousands of generated documents each, with nothing left that differs. Some of them changed a lot more than the others, and the active-mode column is where you see it:

What it changed on our own platform

Pretty much everything APERTURESyndicate runs reads SYNX somewhere. Every service starts from an app.synx, the profile service keeps its store, cosmetics and quests in SYNX, Apeiron’s store and patch notes are SYNX files signed with the depot key, and Rebellion loads its whole balance and both languages from it. So the day 3.8.0 came out we moved all of it over and measured again, this time on the production server and on the files each of these places actually reads, against 3.7.2, which is what our services had been running:

The two at the top run on every request, and both now take about two thirds of what they used to, which in absolute terms is around a tenth of a millisecond per page. The bigger files live in the desktop apps and the game, so those we measured on a Windows 11 machine:

Not everything moved the right way, and it’s worth saying where. Service configs didn’t get faster at all: an app.synx with !active is small, and in a fresh process the first parse on 3.8.0 is about a millisecond slower, because there’s more engine to warm up, though that happens once per start. Rebellion takes about a fifth longer to write a save, because the .NET writer now quotes strings that look like numbers, so "1" stays a string when the file is read back, and the very first read of the locale when the game starts takes about 7 ms longer. Saves written by the old version open exactly as before.

One thing we caught before it reached production. In !active mode SYNX collects [...] constraints by field name across the whole document. That’s what the spec has always said and what Rust always did, but the TypeScript package only started doing it in 3.7.4, and our services were still on 3.7.2. Five of them declared port[type:int, min:1024] for their own HTTP port, so under 3.8.0 the SMTP email.port 587 picked up the same min:1024, came back as a CONSTRAINT_ERR string, and mail from those services would have stopped. Before deploying we parsed every service config with both versions under its real production environment, which is how it showed up, and gave the HTTP port min:1 instead.

Updating

Your .synx files don't need to change. The packages are @aperturesyndicate/synx-format on npm, synx-core on crates.io and synx-format on PyPI, and the ports come through their usual registries. If you use synx-core from Rust directly, objects are now an ordered IndexMap instead of a HashMap, so code that builds its own HashMap and wraps it needs a .collect(), while everything that reads through get and iteration compiles as before.

If you’re moving the TypeScript package up from 3.7.3 or older and use !active, look for constraints on field names that repeat in different sections of the same file, because that’s exactly what we ran into above.

There's one thing the format still can't express: a list inside a list item, followed by another item, doesn't survive writing and reading back, because SYNX keeps one open list at a time. That one is planned for 4.0, together with compact strings, which in a lab copy of the core made parsing another 1.2 to 1.5 times faster.

Every change, with the numbers, is in the changelog and in the 3.8.0 release on GitHub.