---
name: regression-test
description: Run the DIYLC end-to-end regression suite, which renders just under 100 sample .diy projects and compares PNG exports and netlists against reference outputs. Use after any change to rendering, component drawing, serialization, netlist or analyzer logic, or when asked to check for regressions or regenerate reference files.
---

# Running the DIYLC regression suite

`RegressionTestRunner` (in `diylc-swing/src/test/java/org/diylc/`) loads every sample project under
`diylc-regression-data/input`, exports a PNG and extracts netlists, and compares both against
reference files produced by a known-good build. It is the only automated coverage of component
appearance and of the file format end to end — unit tests do not touch either.

**Always ask before running the full suite**, and never run `PREPARE` without explicit instruction.

## Modes

- **`TEST`** — compares against the reference files and writes a CSV report plus visual diff images.
  This is what you almost always want.
- **`PREPARE`** — *regenerates* the reference PNGs and netlists from the current build. This
  overwrites the golden copy and is correct only once a human has inspected the differences and
  decided they are intended. Never run it on your own initiative.

## Easiest path

If the maintainer is at their IDE, the `.run/RegressionTestRunner.run.xml` IntelliJ configuration
already has the arguments and JVM flags. Offer that first.

## From the terminal

The runner needs the `--add-opens` / `--add-exports` flags and a classpath spanning all three
modules; `mvn exec:java` will not work because it runs in Maven's own classloader and fails to find
Reflections. Build a classpath and fork a JVM.

The classpath has to be built in the same reactor run as the compile. Running
`dependency:build-classpath` on a single module fails with a `DependencyResolutionException`,
because `diylc-processor` is not installed in the local repository and the plugin can only resolve
it from the reactor. The relative `outputFile` gives each module its own `target/cp.txt`:

```bash
cd diylc

mvn -q -pl diylc-swing -am test-compile dependency:build-classpath \
  -Dmdep.outputFile=target/cp.txt -Dmdep.includeScope=test

java \
  --add-exports java.desktop/com.apple.eawt.event=ALL-UNNAMED \
  --add-exports java.desktop/com.apple.eio=ALL-UNNAMED \
  --add-opens java.base/java.util=ALL-UNNAMED \
  --add-opens java.base/java.lang=ALL-UNNAMED \
  --add-opens java.base/java.text=ALL-UNNAMED \
  --add-opens java.desktop/java.awt=ALL-UNNAMED \
  --add-opens java.desktop/java.awt.font=ALL-UNNAMED \
  --add-opens java.desktop/java.awt.geom=ALL-UNNAMED \
  -cp "diylc-swing/target/test-classes:diylc-swing/target/classes:diylc-core/target/classes:diylc-library/target/classes:$(cat diylc-swing/target/cp.txt):$(cat diylc-core/target/cp.txt):$(cat diylc-library/target/cp.txt)" \
  org.diylc.RegressionTestRunner \
  "$(cd .. && pwd)/diylc-regression-data" TEST [filter]
```

Sanity-check the log line `Presenter - Current DIYLC version: ...`. It should report the newest
version in `diylc-core/src/main/resources/update.xml`; if it reports `5.0.0`, the `--add-opens`
flags are missing and the run is invalid.

### Arguments

1. Path to `diylc-regression-data`
2. `TEST` or `PREPARE`
3. Optional filter — a substring matched against file paths. The full suite is just under 100
   projects; while narrowing down an issue, run one file or one directory at a time
   (e.g. `splitter2`, or `guitar`).

## Reading the results

- A CSV report is written to `diylc-regression-data/reports/`, named with the timestamp and version,
  listing each file as OK or failed with the reason (`Images do not match`, `Netlists do not match
  with switches`, `Netlists do not match without switches`).
- Rendered output and **visual diff images**, with red rectangles around every differing region, go
  to `diylc-regression-data/output/` (not under version control).
- Reports are kept as historical evidence. If a run was exploratory rather than a real verification,
  delete the generated CSV afterwards so the reports directory stays meaningful — and check
  `git status` before finishing, since the runner writes into the repo.

## Interpreting a failure

A difference is not automatically a bug, but it is always a change the maintainer must see. Report
which files differ and what kind of difference it is; where an image diff exists, point at the file
so it can be looked at. Do not conclude "no regressions" from a filtered run — say exactly which
subset was run.