Building from source

This page shows how to build Marginalia, run it on your machine while you work on it, and build the plugins and this manual. Packaging for release (the Docker image, the desktop app, CI) is covered in Packaging & releases.

Requirements

The first build downloads a lot (Maven dependencies, Node.js, npm packages) and takes several minutes. Later builds are faster.

Building the application

The repository root holds the parent POM (pom.xml): it carries the version, which the application and all plugins inherit, and lists them as modules, so mvn package there builds the application and every plugin in one go. The application is the Maven project in marginalia/:

cd marginalia
mvn package                  # builds and tests, WAR in target/marginalia-1.6.0.war
mvn package -DskipTests      # the same without tests
mvn install -DskipTests      # also installs the WAR and the classes JAR into ~/.m2 (needed by plugins)
mvn -N install                                # (repository root, once) installs the parent POM, the installed POMs refer to it
mvn install -DskipTests -Dvaadin.skip=true   # plugins only need the classes JAR: no frontend build, not a deployable WAR

What a build does, in order:

1. git-commit-id-plugin writes git.properties (commit id, time, closest tag) into the classes. utils.BuildInfo reads it together with META-INF/build-info/build-info.properties (version and build time, filtered only when the WAR is packaged, so a run from target/classes has the placeholder and reports the version dev) for the footer of the page and the login screen. The resources step also copies the Markdown pages of the user manual (../manual/user/**/*.md, unfiltered) to classpath:/manual/ for the help assistant; the folder is outside marginalia/, see Packaging.

2. vaadin-maven-plugin prepares the frontend (prepare-frontend): it generates package.json, vite config and the files in src/main/frontend/generated/ (ignored by Git) and downloads Node.js if needed.

3. maven-compiler-plugin compiles the sources, then aspectj-maven-plugin compiles them again with the AspectJ compiler and weaves the Spring aspects in (see AspectJ weaving below). The woven classes in target/classes are the ones that get packaged.

4. Tests run (maven-surefire-plugin), see Running the tests.

5. vaadin-maven-plugin builds the frontend bundle (build-frontend).

6. maven-war-plugin packages target/marginalia-1.6.0.war (about 185 MB) and, because of attachClasses, also target/marginalia-1.6.0-classes.jar - the application classes that plugins compile against.

Outputs in marginalia/target/:

Development and production mode

Vaadin runs either in development mode (frontend served from a development bundle that Vaadin builds into src/main/bundles/, ignored by Git; Vaadin's dev tools available, more checks) or in production mode (optimized frontend bundle, no dev tools). The mode is decided by META-INF/VAADIN/config/flow-build-info.json, the build info that vaadin-maven-plugin writes:

• prepare-frontend (every build, also mvn compile and mvn jetty:run) writes a development build info with the absolute paths of your checkout into target/classes;

• build-frontend (in prepare-package, so only mvn package / install) builds the optimized bundle, writes a production build info into the WAR and removes the development one from target/classes.

By default Vaadin scans the bytecode of the app and puts only the components it finds referenced into the production bundle. Extensions are loaded at runtime and may use any Vaadin component, so the plugin is configured with <optimizeBundle>false</optimizeBundle> (-Dvaadin.devmode.optimizeBundle=false on the command line): the bundle contains every component on the classpath, at the price of a bigger bundle and a slower frontend build.

So mvn package always produces a production WAR - the one the Dockerfile, the desktop profile and CI use, and the one to deploy. Nothing has to be copied around. A build info in src/main/resources is never packaged into the WAR (maven-war-plugin excludes META-INF/VAADIN/config/flow-build-info.json*).

Development mode is for running from the IDE (see IntelliJ IDEA) or mvn jetty:run. Without a build info Vaadin also starts in development mode and finds the project by the vaadin.project.basedir system property, the classpath or the working directory.

To check a WAR:

unzip -p target/marginalia-1.6.0.war WEB-INF/classes/META-INF/VAADIN/config/flow-build-info.json   # "productionMode" : true

AspectJ weaving

Marginalia relies on compile-time AspectJ weaving with the spring-aspects library: Vaadin components are created with new, not by Spring, so classes annotated @Configurable (all routes, the workspace, dialogs, parts...) get their @Autowired fields injected by the woven AnnotationBeanConfigurerAspect.

Transactions are not woven: @CommonTx, @CommonTxReadOnly and @NoTx are applied by Spring's class-based proxies around the service beans (see Services).

If the classes are compiled by plain javac, nothing fails at build time, but at runtime every @Autowired field of a UI class is null and the first screen fails with a NullPointerException. Maven always uses ajc; in your IDE, make sure the AspectJ compiler is used too (see IntelliJ IDEA).

The compiler also keeps parameter names (-parameters) and all local variables (preserveAllLocals): extensions read method arguments and local variables of @Extendable methods by name, so don't remove these options.

The IDE (with the ajc compiler) compiles into the same target/ as Maven, and a class compiled by the other one is overwritten (or left unwoven, with null @Autowired fields). To keep a Maven run apart from an open IDE, move the output with -Dbuild.dir=/tmp/marginalia-build (the property build.dir of the POM, default ${project.basedir}/target): the classes, the woven classes, the test home (<build.dir>/test-home) and the WAR go there. Use the same folder for the whole sequence (mvn install of the app and the plugins read from the local repository, not from it), delete it for a clean build, and never run two builds on the same folder.

Running locally

Marginalia is a plain WAR without an embedded server, so you run it in a servlet container: from the IDE, with mvn jetty:run, in a local Jetty, or in Docker.

Keep your development data separate

Marginalia keeps everything in ~/.marginalia (System.getProperty("user.home") + .marginalia), the same folder the desktop app uses. To keep test data away from your real books, start the container with another home folder:

-Duser.home=/path/to/dev-home

The folder /path/to/dev-home/.marginalia is created on the first start. Delete it to start over with an empty installation (you'll be asked to create the administrator again).

JVM options

The extension mechanism installs a ByteBuddy agent into the running JVM when the application starts. On a JDK this works without options, but recent JDKs print a warning about dynamically loaded agents. To silence it and allow the agent to attach to its own process, add:

-XX:+EnableDynamicAgentLoading -Djdk.attach.allowAttachSelf=true

These are the options the desktop app and the tests use.

IntelliJ IDEA

1. Open the repository root; IntelliJ imports marginalia/pom.xml (and the plugin POMs, if you add them as Maven projects).

2. Settings → Build, Execution, Deployment → Compiler → Java Compiler: set Use compiler to Ajc, with the path to aspectjtools-1.9.25.1.jar from your Maven repository, and Delegate to Javac checked (javac is then used only for modules without aspects). spring-aspects must be on the aspect path of the marginalia module (its AspectJ facet). Add -parameters to the additional command line parameters of the module.

3. Add a Tomcat Server → Local run configuration (Tomcat 11), deploy the artifact marginalia:war exploded, and add the JVM options and -Duser.home=... to VM options.

4. Run it and open the URL shown in the run configuration, e.g. http://localhost:8080/marginalia/ when the application context is /marginalia.

IntelliJ builds the exploded WAR itself, without the Vaadin plugin, so the application runs in development mode. It needs to know where the project is - either:

• add -Dvaadin.project.basedir=/path/to/Marginalia/marginalia to VM options (simplest, survives Rebuild Project), or
• run mvn vaadin:prepare-frontend in marginalia/ and copy the generated target/classes/META-INF/VAADIN/config/flow-build-info.json to the same path under src/main/resources/ (ignored by Git and never packaged into the WAR, but IntelliJ copies it into the exploded WAR on every build).

Don't deploy target/marginalia-1.6.0/ (the exploded WAR from mvn package) from the IDE - it is a production build.

Java changes need a redeploy (Update application → Restart server).

Local Jetty

To run the built WAR the way the Docker image does, download Jetty 12 (jetty-home), create a Jetty base with the same modules and drop the WAR in:

export JETTY_HOME=/path/to/jetty-home-12.1.10
mkdir -p ~/jetty-marginalia && cd ~/jetty-marginalia
java -jar $JETTY_HOME/start.jar --create-startd \
     --add-modules=server,http,ee11-deploy,ee11-websocket-jakarta,ee11-webapp,ee11-jsp
cp /path/to/Marginalia/marginalia/target/marginalia-1.6.0.war webapps/ROOT.war
java -XX:+EnableDynamicAgentLoading -Djdk.attach.allowAttachSelf=true -Duser.home=/path/to/dev-home \
     -jar $JETTY_HOME/start.jar jetty.http.port=8080

Marginalia is then at http://localhost:8080/. ee11-websocket-jakarta is required: the UI is updated through a push WebSocket.

mvn jetty:run

The quickest way to run from the sources, without an IDE or a server installation. The Jetty 12 Maven plugin (jetty-ee11-maven-plugin) runs the application from target/classes and src/main/webapp inside the Maven JVM, so the JVM options go into MAVEN_OPTS:

cd marginalia
export MAVEN_OPTS="-XX:+EnableDynamicAgentLoading -Djdk.attach.allowAttachSelf=true"
mvn -DskipTests jetty:run                       # http://localhost:8080/
mvn -DskipTests -Djetty.http.port=9090 jetty:run   # another port

The application runs in development mode with the development logging (see Logs). Stop it with Ctrl+C. Java changes need a restart.

Docker

From the repository root:

docker compose up --build

The image is built from the sources in marginalia/ (tests skipped) and the data is kept in ./data in the repository. See Server (Docker) for the configuration and Packaging & releases for how the image is built.

Logs

Marginalia logs to standard output through SLF4J with logback; the log4j-api that Apache POI brings is routed into SLF4J by log4j-to-slf4j, so the output of the libraries that use it is not lost. There are two configurations:

• src/main/resources/logback.xml - development, used from target/classes by IDE runs and mvn jetty:run. The com.github.enerccio loggers are at DEBUG, so a generation logs each pipeline step and the activated lore.

• src/main/resources-release/logback.xml - release, everything at INFO. mvn package puts it into the WAR in place of the development one, so the Docker image, the desktop app and any other deployed WAR log no story content (see Packaging & releases).

In the desktop app the output goes to ~/.marginalia/desktop/logs/.

Running the tests

mvn test                                     # all tests
mvn test -Dtest=LorebookActivationTest       # one class
mvn test -Dtest='Backup*Test'                # a pattern
mvn test -Dtest='MacroRenderingTest#name'   # one method
mvn test -Dbuild.dir=/tmp/marginalia-build  # build and run in another folder than target/

The tests start the real Spring XML configuration (without OSGi and the instrumentation agent) against a SQLite database in target/test-home/ctx-*, and talk to a fake OpenAI-compatible server, so no network, model or API key is needed. RuntimeInstrumentationTest installs the ByteBuddy agent into the test JVM; Surefire passes the needed JVM options. Testing describes the test base and how to write tests.

Building the plugins

The nineteen bundled extensions in marginalia/plugins/ are separate Maven projects. They compile against the classes JAR of the application, so install the parent POM and the application first:

mvn -N install                    # in the repository root
cd marginalia
mvn install -DskipTests

cd plugins/chaptermarker          # or anthropic, arc-outline, authorsnote, beta-readers, character-tracker, gemma4toolsfix, google-genai, image-generator, instructionexporter, lore-generator, lorebookvcs, manga-generator, name-generator, name-provider, place-provider, reviewer, sidequery, sillytavern-converter, chineselocalization
mvn package                       # target/chaptermarker-1.6.0.jar

Load the JAR in Admin → Extensions of your running instance, or copy it into .marginalia/extensions of your development home before starting. Rebuild the plugins whenever you change classes they use - they are built for one exact version of the application. See Plugin development.

In Docker

marginalia/plugins/Dockerfile and docker-compose.yml build everything in a container, so no local JDK or Maven is needed (this is how the Docker server is updated, see Packaging). The build context is the repository root; the image copies pom.xml, marginalia/pom.xml, marginalia/src, manual/user (packed into the application's classes, so a plugin build against it sees the same manual), .git and marginalia/plugins, runs mvn install -DskipTests -Dvaadin.skip=true for the application, mvn install -N for the parent POM and mvn package in every folder of plugins/ that has a pom.xml. The JARs are copied out when the container runs, because they go into a bind mount:

docker compose -f marginalia/plugins/docker-compose.yml build
docker compose -f marginalia/plugins/docker-compose.yml run --rm plugins     # JARs in marginalia/plugins/build

A new plugin folder with a pom.xml is picked up without changes to the Dockerfile.

-Dvaadin.skip=true turns off every goal of vaadin-maven-plugin, so the frontend (Node.js download, npm, Vite bundle) is not built; the plugins need only the classes JAR. The WAR of such a build has no frontend bundle and must not be deployed. Use the flag the same way for any build that only installs the classes JAR for plugins. Maven cannot tell that the goal was just install (it runs the whole lifecycle up to package first), so the flag has to be given.

Database schema changes

The schema is created and upgraded by Flyway from src/main/resources/migration/V<n>__<name>.sql; Hibernate only validates it (hibernate.hbm2ddl.auto=validate), so an entity change without a migration makes the application fail on start.

com.github.enerccio.tools.GenerateFlywayDiff (in src/main/java/com/github/enerccio/tools/) helps write the migration: it compares the entities listed in persistence.xml with an existing database and prints the SQL Hibernate would run to update it. Run it from the repository root (it reads marginalia/src/main/webapp/config/persistence.xml) with the same -Duser.home as your development instance - it uses the database in <user.home>/.marginalia. Review the output before using it: Hibernate's SQLite updates are a starting point, not a finished migration. Database & migrations explains the rules for migrations.

Building the manual

The manual is a Retype site. Sources are in manual/, the configuration is retype.yml in the repository root, and the generated site is written to docs/ and served by GitHub Pages from the master branch.

npm install --global retypeapp     # or: dotnet tool install retypeapp --global
retype start                       # live preview at http://localhost:5000
tools/retype_build.sh              # writes docs/; plain `retype build` works too, without the version in the site

Commit docs/ together with the changed sources. Images go into manual/images/; most are taken by the scripts in screenshots/ (see Contributing), the few that cannot be are listed in TODO.IMAGES.md.

Troubleshooting