Skip to content

mll/clojure-rt

Repository files navigation

Clojure Real Time C++ CI Linux

Clojure Real Time (clojure-rt) is a high-performance Clojure compiler designed for deterministic execution, enabling Clojure to expand into domains beyond its traditional reach. By leveraging LLVM for aggressive optimization and platform independence, clojure-rt targets performance benchmarks significantly higher than existing implementations.


Key Technical Features

  • C++ Interoperability: Aims for seamless C++ interop, with an exception model following the C++ ABI.
  • Reference Counting: Unlike the standard JVM implementation, clojure-rt utilizes a reference-counting memory model based on Reinking et al. (MSR-TR-2020-42). This eliminates garbage collection pauses, providing predictable memory management and execution time.
  • Two-Tiered JIT: A custom, two-tiered JIT compiler optimizes code at runtime. It uses static type analysis to discover types during execution, rendering traditional Clojure type hints unnecessary while achieving near-native speeds.
  • C-Based Runtime: To maximize optimization, all core immutable data structures and the Clojure-to-Java interop layers have been reimplemented in C.

Current Architecture & Bootstrap Status

The compiler is currently in a bootstrap phase, consisting of two distinct components:

1. Frontend (Clojure)

Built on org.clojure/tools.reader and org.clojure/tools.analyzer, the frontend runs via Leiningen. It remains lightweight, primarily adding passes to compute memory management annotations.

2. Backend (C++/C)

The backend comprises the LLVM code generator, JIT infrastructure, and the core Clojure runtime. Implementing data structures in C allows for the highest possible level of optimization.

Note: Currently, these components communicate via Protocol Buffers (using .cljb files). Development is progressing toward a self-hosted compiler, which will be established once the bootstrap process can fully compile Clojure itself.

Compilation

The compilation process is tested on OS X only, but due to cmake it should work everywhere if you manage to install dependencies from begin_development using your system's package manager.

  1. ./begin_development.sh
  2. cd backend-v2
  3. cmake . -DCMAKE_PREFIX_PATH=/opt/homebrew -DCMAKE_BUILD_TYPE=Debug ;; or Release
  4. make -j 8

This should build the c/c++ compiler backend.

To debug leaks on OS X:

leaks --atExit -- ./name_of_the_test

To show llvm IR for runtime:

llvm-dis runtime_uber.bc -o runtime_uber.ll

Running

./compile.sh fib.clj

Please note that currently the backend prints out a lot of debug information. The primary info it prints is the LLVM code generated for given clojure statements.

Generating protobuf models

The models do not need to be regenerated before compilation. However, there might be a time in development when such generation can become necessary. The process is described below.

The original source of protobuf models come from org.clojure/tools.analyzer.jvm documentation of its AST representation.

We use two files to describe the AST as a data structure:

frontend/resources/ast-types.edn frontend/resources/ast-ref.edn

These ast-ref.edn can be readily obtained from the analyzer repository, they can also be autogenerated using a script found in this repository. The types file is manually created to assign protobuf types to elements of AST tree produced by the analyser (needed here as clojure is dynamically typed).

The frontend application is not only capable of compiling .clj into .cljb (protobuf representation of AST) but also can be used to generate the protobuf definition (model/bytecode.proto) from the above files.

Then, protoc can be used to generate clojure and c++ specific code for decoding / encoding .cljb

TL;DR: The whole process is automated and can be run this way:

cd model ./generate.sh

Self-hosted compiler roadmap

The self-hosted compiler will consist of:

  1. Compiler backend, exactly as (2) above.
  2. Compiled protobuf file of clojure itself. It will be generated from the main clojure repository using the bootstrap compiler (1)
  3. Compiled protobuf files of (1) with all its dependencies.
  4. Bootstrapper as main function of the C++ application:

The bootstrapper would launch the backend with pre-build protobuf files as initial input. Then, it will compile the frontend (later we could use the LLVM representation of the complete compilation result so that backend would start more rapidly).

Finally, any clojure files to be compiled and run (including possible REPL commands) would be fed to the running system, first passed through the frontend and finally to the backend for execution. The protobuf files will probably be transferred between frontend and backend as in-memory data structures at this stage, as both parts of the compiler would reside inside the same process.

License

clojure-rt is being distributed on GPLv2 license. See LICENSE.md for details.

Copyright © 2022-2026 Marek Lipert, Aleksander Wiacek

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors