The JavaScript ecosystem is busy solving made up problems

I am tired of the JavaScript ecosystem.

I don’t care how many tools a project uses. Use one. Use twenty. Use a hundred if they make the work easier.

I care whether they work together.

Most of them don’t. Some of them barely work on their own.

Every tool owns one tiny part of the application and acts like the rest of the application does not exist. The bundler has one idea of the project. The type checker has another. The test runner has another. The linter parses the same files again. The development server invents a second runtime. The deployment tool reads half of its own config and leaves the other half for me.

Then I install plugins to make the tools aware of each other.

A plugin for a plugin so one tool can read another tool’s config is not architecture. It is an apology.

I am so fucking tired of fixing the empty space between tools.

Wrangler is a good example

I recently worked on a Cloudflare project using Wrangler.

Wrangler has a configuration file. Great. That is exactly what I want.

Cloudflare even recommends treating that file as the source of truth for the Worker configuration.¹ I agree.

So why can’t I configure the whole application with it?

My application uses multiple Queues. The Worker config contained the Queue binding. I still had to create the Queue manually with the Wrangler CLI. So Wrangler is able to create the Queue, it just isn’t in the mood of using the config to do so.

First create infrastructure through another command or the dashboard. Then copy its name or ID into the config. Then hope that the external resource and the local file continue to agree.

What is the source of truth here?

The config knows that the application needs a Queue. It knows the binding name. It knows the Queue name. That is enough information to create it. Why am I doing the last step manually?

Cloudflare has since added automatic provisioning for several resources, including Queues, as a beta feature. The official Queue guide still documents creating a Queue first and adding its binding afterwards.² This is the exact gap I am talking about. The declaration and the resource lifecycle were designed as separate problems. Integration arrived later as another feature. It shouldn’t be a fancy new feature that my Cloudflare project config can configure my Cloudflare project.

Routing was worse for my project.

The routes were already in the Wrangler config. Deployment used them. Local development did not reproduce the routing behavior my application depended on. wrangler dev started a server. It did not turn its own deployment routes into my local application routes. wrangler dev only works well when you have only one Worker. Well, my project had 5 Workers on the same domain. Wrangler can’t do that.

So I wrote my own development server for a Cloudflare project.

I had to read Wrangler’s settings myself and implement the missing routing around Wrangler using a proxy. The tool had the configuration. The tool had a development mode. The tool had a deployment mode. Those pieces still did not form one workflow.

That is what I mean when I say a tool does not work. I do not care that a command starts and exits successfully. The workflow has to work.

A configuration file should configure the application. A development mode should execute that configuration locally. A deployment mode should use the same model. Deployment should be idempotent.

This feels painfully obvious.

The JavaScript ecosystem is made of config islands

Wrangler is one example. The same pattern is everywhere.

The bundler understands an import through a plugin. The linter reports that the same import cannot be resolved.

The development server serves a module. The production build rewrites it differently.

A framework generates deployment output. The platform tool generates another config for the generated output. The source config is now pointing at a generated config which points at generated files.

Great.

Each tool has its own parser, module resolver, dependency graph, cache, configuration format, plugin API, and error messages. Each tool builds its own incomplete model of the same application.

The project has no shared model. It has a committee.

And I am the integration layer.

The early tools were fine

JavaScript did not begin with the problems it has today.

Early JavaScript was small scripts in a browser. There was no standard module system. Browsers implemented features at different speeds. Network connections were slow. Shipping fewer bytes mattered a lot.

The tools built for those problems made sense.

A minifier reduced download size. A concatenation tool combined scripts. A transpiler converted newer syntax for older browsers. A module bundler made CommonJS or other module systems usable in a browser. A linter caught mistakes in a very permissive language.

Those were good tools for real problems.

Then JavaScript applications became larger. The tools grew with them. They grew sideways.

Babel got plugins and presets. Bundlers got loaders and plugins. Test runners got transforms. Linters got custom parsers. Frameworks got adapters. Development servers got their own module graphs. Deployment tools got framework integrations.

Every tool learned enough about neighboring tools to hide the seam. The seam stayed.

The ecosystem kept making the issue less noticeable. It never fixed the fundamental issue.

Now every pair of tools needs a treaty.

A TypeScript plugin for the bundler. A TypeScript parser for the linter. A transform for the test runner. A resolver plugin for aliases. A framework adapter for the deployment platform. A platform plugin for the development server.

The application code sits underneath all of this and waits for the adults to stop arguing.

The missing integration already has a name

This is why I need to talk about compilers.

The JavaScript ecosystem keeps rediscovering compiler work in separate packages. The missing integration already has a proven shape. We have worked on these problems for at least half a century.

A compiler translates a program from one representation into another. Machine code is a common target. A compiler can also emit assembly, bytecode, WebAssembly, JavaScript, SQL, or another intermediate representation.

The short definition is translation. The useful part for this discussion is the shared program model.

A classic compiler pipeline looks roughly like this:

The scanner, also called a lexer, turns characters into tokens.

The parser turns those tokens into an abstract syntax tree, usually called an AST. The tree describes the structure of the program.

Semantic analysis resolves names and checks meaning. Does this variable exist? Which function does this call refer to? Is this operation valid for the given type?

Lowering converts complex language features into simpler representations. Compilers often use an intermediate representation, usually called an IR, so later phases can work on a stable model instead of raw source syntax.

Optimization passes remove dead code, fold constants, inline functions, propagate values, and simplify control flow.

Code generation emits the target representation.

A linker combines separately compiled units and resolves references between them.³⁴

The build system knows which inputs produce which outputs. That enables incremental builds, parallel execution, caching, and reproducible results.⁵

Debug information maps generated instructions back to the source the developer wrote.

These can be separate tools. They still agree on formats, symbols, dependency information, debug data, and ownership of each phase. One binary is optional. A shared model is essential.

The number of tools is irrelevant. Integration is the feature.

JavaScript renamed the compiler

Here is the translation guide:

• Transpiler: source-to-source compiler
• Bundler: linker plus packager
• Tree shaking: dead code elimination or live code inclusion
• Minifier: size-focused optimization and code emission
• Loader: compiler frontend or preprocessor
• Plugin: compiler pass
• Source map: debug information for generated source
• Hot module replacement: incremental compilation plus runtime code replacement
• Task runner: build system
• Polyfill: runtime support library

The web adds real constraints. Browsers load URLs. Modules can have side effects. Assets include CSS, images, fonts, workers, and WebAssembly. Output can be split into chunks and loaded later.

Those details matter. The underlying problems are still old.

Transpiling is compiling

A transpiler converts one source language into another source language.

Babel calls itself a JavaScript compiler. It parses JavaScript, transforms the program, and emits JavaScript.⁶

That is a compiler. The output being readable source code does not create a new branch of computer science.

Bundling is linking with web baggage

A linker connects references between compiled units.

A bundler follows imports, resolves modules, combines code, rewrites module formats, and emits one or more output files. It also handles CSS, images, and whatever logo.svg?component is supposed to mean today.

The web baggage is real. The core job is still linking.

Tree shaking is dead code elimination with a plant metaphor

Compilers have removed unused code for decades.

JavaScript made the analysis painful because modules and property access can have side effects. Can this import mutate a global? Can this getter run code? Can dynamic access reach an export that looks unused?

These are data-flow and side-effect analysis questions. Rollup itself describes tree shaking as a form of dead code elimination.⁷

We gave the pass a cute name and made it sound like the web invented it.

I don’t care who invented it. But renaming existing ideas makes it hard to learn from past implementations.

Source maps are debug information in JSON

Generated code rarely looks like the source a developer wrote. Native toolchains store debug information for this. JavaScript uses source maps. ECMA-426 standardizes the format used to map generated JavaScript, WebAssembly, and CSS back to original sources.⁸

Same problem. JSON this time.

Then we put five isolated transforms in a row and pray that every tool composes the mappings correctly.

Again, I don’t care who invented it. And I don’t say we should replace source maps with DWARF. All I say is that the web should stop reinventing solved problems.

Plugins are compiler passes without a shared compiler

Compiler passes work because they operate on agreed representations.

JavaScript plugins often receive a tool-specific AST, tool-specific module graph, tool-specific hooks, and tool-specific metadata. A plugin written for one tool usually means nothing to another tool.

So the same transformation gets implemented several times. Or an adapter converts one tool’s result into another tool’s input and loses information on the way.

This is compiler architecture with the architecture removed.

Renaming makes us worse at solving the problem

Names matter because names connect a problem to existing knowledge.

Search for “tree shaking” and you get bundler documentation. Search for “dead code elimination” and you get decades of compiler research about liveness, reachability, purity, side effects, and whole-program analysis.

Search for “hot module replacement” and you get framework APIs. Search for “incremental compilation” and you get work on dependency graphs, invalidation, caching, and partial recomputation.

Search for “bundling” and you get configuration examples. Search for “linking” and you get symbol resolution, separate compilation, relocation, interfaces, and link-time optimization.

A new name resets the conversation. It makes old solutions harder to find. It makes the ecosystem feel unique when it is repeating an old problem. I don’t say the existing solutions are perfect. But only when we know them, we can learn from them.

Then isolation makes the new solution worse.

A compiler can parse once and share the result. JavaScript tools parse the same file repeatedly. C and C++ had the exact same problems decades ago. They solved them.

A compiler toolchain can share one dependency graph. JavaScript tools discover their own graphs and disagree about them.

A build system can cache an action from declared inputs and outputs. JavaScript tools keep independent caches and invalidate them differently.

A toolchain can define one module resolution model. JavaScript projects configure TypeScript, the bundler, the test runner, the linter, Node.js, and the deployment platform separately.

A compiler can compose diagnostics and debug information across phases. JavaScript gives me one error from the editor, another from the test runner, and a third after deployment.

The basic fix is embarrassingly obvious.

Create one project model. Use one module graph. Share syntax trees or a stable IR where that makes sense. Define the target runtime once. Let development, testing, building, and deployment use the same settings. Give each phase a clear owner.

We already know how to do this.

The ecosystem keeps choosing another plugin.

TypeScript has a program that disappears

TypeScript makes this whole story more interesting.

It adds another tool and another compiler configuration to many projects. Its language design also gives library authors a rare way to connect programs without forcing them to carry the same data at runtime.

A TypeScript file contains two connected programs.

One program becomes JavaScript. It contains values, objects, strings, arrays, functions, classes, and side effects.

The other program exists only while TypeScript checks the source. It contains types, generic parameters, conditional types, mapped types, literal types, indexed access, keyof, and type-level typeof.

TypeScript models values and types as separate spaces. Some declarations create a value. Some create a type. Classes create both.⁹

Then the compiler deletes the type program.

Interfaces disappear. Type annotations disappear. Generic arguments disappear. Type-only imports disappear. They cannot perform runtime checks because they do not exist when the program runs.¹⁰

Static types disappearing at runtime is not unique. The interesting part is how far TypeScript takes the split while staying a JavaScript library language.

A library can use generic inference and type-level computation to model relationships between calls. The editor can execute that type program continuously and offer completion. The compiler can reject an invalid sequence. Then all of that machinery can disappear from the emitted JavaScript.

The runtime does not always need a copy of the information the compiler used. Sometimes the compiler only needs it to prove that the runtime program is valid.

That distinction matters.

Many TypeScript libraries keep a runtime schema value beside the type state. That is specifically not how tsql works.

This split has a clear downside. TypeScript’s types can lie.

const strArray = ["a", "b", "c"];

function addNumber(arr: Array<string | number>) {
	arr.push(1);
}

addNumber(strArray);
//        ^? string[]
// TypeScript says that `strArray` stays a string array but at runtime we add a number to it.
// It became an array of strings and numbers without any warning, error, or type change.

for (const str of strArray) {
	console.log(str.toUpperCase());
	// throws at runtime because a number does not have `.toUpperCase()`
}

[LOG]: "A"
[LOG]: "B"
[LOG]: "C"
[ERR]: "Executed JavaScript Failed:"
[ERR]: str.toUpperCase is not a function

The type program is not the runtime program. That can create a hole between them.

Or it can remove work from the runtime entirely.

tsql has three programs

I built tsql around the opposite of the usual TypeScript SQL model.

The application runtime does not know the database schema.

It does not load a runtime schema object. It does not carry the migration history into request handling. It does not validate a query again against runtime schema metadata.

The compiler knows the schema.

That is the whole point. Completely get rid of code generation.

Tsql has three programs with different jobs:

The tools are separate. They are not isolated.

The type-level program is the contract between the other two.

Program one: the type level

This program runs in the editor and TypeScript compiler.

It provides completion. It checks query structure, table names, column names, input values, and result types. It proves that a query is valid against the schema produced by the migrations.

The migration chain is also a type-level state machine.

Conceptually, it looks like this:

Every migration receives the schema that existed immediately before it.

Migration 003 does not see the final schema. It sees exactly what migrations 001 and 002 created.

Create a table and it exists in the next migration’s type. Rename a column and the old name disappears from the next step. Drop a table and later migrations cannot use it. Try to alter something that did not exist yet and TypeScript rejects the migration.

There is a fully typed schema for every point in the migration history.

The final schema type is then used to check application queries. The editor can complete the tables and columns that exist. The selected result type can be calculated from the query. A misspelled column or incompatible value fails before the application starts.

This is similar to typestate. The type records the current state and controls which operations are valid next.¹¹

The important part is that this schema is a type.

It is proof used by the compiler. It is not metadata needed by the application runtime.

After TypeScript finishes checking the program, the schema type disappears.

Program two: the application runtime

The runtime receives the queries that passed the type checker.

It formats them as SQL. It creates the bound parameters. It sends both to the database.

It does that blindly.

There is no runtime schema lookup. There is no query validation against migration metadata. There is no runtime database introspection. There is no second version of the type checker running in production.

The type layer already did that work. The runtime trusts it.

Of course, TypeScript cannot prove that the database server is online, that a unique constraint will not reject a value, or that nobody changed production by hand. The proof is narrower and useful: if the migration chain has been applied, the query is structurally and type-correct for the schema it produced.

That is enough to remove a lot of runtime machinery.

The application runtime only needs the query itself and the values being sent. It does not need the schema that was used to prove the query.

Migrations do not exist in this program.

Program three: the migration program

Migrations mostly run independently from the application runtime.

Usually this program runs in CI or during deployment. It takes the existing migrations and applies them to the database one step at a time.

The migrations are not being validated for the first time here. The type-level program already proved each migration against the exact schema state that existed before it.

The migration program just performs the proven steps.

Depending on the hosting platform, you can run the migrations at backend startup or in CI. Edge computing like Cloudflare Workers don’t have a backend startup, so the migration program runs in CI. The application runtime does not need to own migration execution but it can.

This separation is deliberate:

Three programs. Three responsibilities. One contract.

Tsql doesn’t even care what database it is talking to. It just wants something it can send SQL strings to and get back results. Existing adapters make SQLite, MySQL, and PostgreSQL work with first party support. Other SQL databases can be supported by writing an adapter, which should not take more than a few minutes.

This is what integration should look like.

Integration does not mean putting everything in one process. It does not mean making every phase load one giant configuration object. It means that the phases agree about what they exchange and that each phase owns its part.

The type layer connects migrations and queries without coupling their runtime programs. The migration program changes the database. The application runtime uses the database. Neither has to reimplement the compiler’s proof.

That is very different from the JavaScript ecosystem’s usual pile of tools. There, every tool rebuilds an incomplete project model and tries to validate the neighboring tool after the fact.

In tsql, separation removes duplication. Isolation would add it back.

How to take advantage of TypeScript’s split

The useful lesson from tsql is that uniqueness (like TypeScript’s split program model) can always be used as an advantage. I mostly hear developers complain about how weird TypeScript is and that it is not type safe etc. True. But have you considered the upside?

We are stuck with TypeScript for now. There is no point in debating that. Focus on what benefits it brings.

TypeScript is different. This means it is a good idea to build unique tools that would be literally impossible in other languages like C#, Java, or Go.

I tried to port tsql to C# but it was impossible.

I want toolchains again

I want one module resolution model. One dependency graph. One target runtime definition. One place that owns each diagnostic.

The tools can stay separate. They need shared contracts and shared state.

We already have the terms. We already have the research. We already have proven architectures.

Stop renaming old compiler work. Stop rebuilding every phase in isolation. Stop shipping another adapter and calling the seam an ecosystem.

Keep the tools. Make them work.

I want to write software to solve real problems, not made up problems.