diy-layout-creator/diylc/.claude/skills/regression-test/SKILL.md

name, description

name	description
regression-test	Run the DIYLC end-to-end regression suite, which renders just under 100 sample .diy projects and compares PNG exports and netlists against reference outputs. Use after any change to rendering, component drawing, serialization, netlist or analyzer logic, or when asked to check for regressions or regenerate reference files.

Running the DIYLC regression suite

RegressionTestRunner (in diylc-swing/src/test/java/org/diylc/) loads every sample project under diylc-regression-data/input, exports a PNG and extracts netlists, and compares both against reference files produced by a known-good build. It is the only automated coverage of component appearance and of the file format end to end — unit tests do not touch either.

Always ask before running the full suite, and never run PREPARE without explicit instruction.

Modes

• TEST — compares against the reference files and writes a CSV report plus visual diff images. This is what you almost always want.
• PREPARE — regenerates the reference PNGs and netlists from the current build. This overwrites the golden copy and is correct only once a human has inspected the differences and decided they are intended. Never run it on your own initiative.

Easiest path

If the maintainer is at their IDE, the .run/RegressionTestRunner.run.xml IntelliJ configuration already has the arguments and JVM flags. Offer that first.

From the terminal

The runner needs the --add-opens / --add-exports flags and a classpath spanning all three modules; mvn exec:java will not work because it runs in Maven's own classloader and fails to find Reflections. Build a classpath and fork a JVM.

The classpath has to be built in the same reactor run as the compile. Running dependency:build-classpath on a single module fails with a DependencyResolutionException, because diylc-processor is not installed in the local repository and the plugin can only resolve it from the reactor. The relative outputFile gives each module its own target/cp.txt:

cd diylc

mvn -q -pl diylc-swing -am test-compile dependency:build-classpath \
  -Dmdep.outputFile=target/cp.txt -Dmdep.includeScope=test

java \
  --add-exports java.desktop/com.apple.eawt.event=ALL-UNNAMED \
  --add-exports java.desktop/com.apple.eio=ALL-UNNAMED \
  --add-opens java.base/java.util=ALL-UNNAMED \
  --add-opens java.base/java.lang=ALL-UNNAMED \
  --add-opens java.base/java.text=ALL-UNNAMED \
  --add-opens java.desktop/java.awt=ALL-UNNAMED \
  --add-opens java.desktop/java.awt.font=ALL-UNNAMED \
  --add-opens java.desktop/java.awt.geom=ALL-UNNAMED \
  -cp "diylc-swing/target/test-classes:diylc-swing/target/classes:diylc-core/target/classes:diylc-library/target/classes:$(cat diylc-swing/target/cp.txt):$(cat diylc-core/target/cp.txt):$(cat diylc-library/target/cp.txt)" \
  org.diylc.RegressionTestRunner \
  "$(cd .. && pwd)/diylc-regression-data" TEST [filter]

Sanity-check the log line Presenter - Current DIYLC version: .... It should report the newest version in diylc-core/src/main/resources/update.xml; if it reports 5.0.0, the --add-opens flags are missing and the run is invalid.

Arguments

1. Path to diylc-regression-data
2. TEST or PREPARE
3. Optional filter — a substring matched against file paths. The full suite is just under 100 projects; while narrowing down an issue, run one file or one directory at a time (e.g. splitter2, or guitar).

Reading the results

• A CSV report is written to diylc-regression-data/reports/, named with the timestamp and version, listing each file as OK or failed with the reason (Images do not match, Netlists do not match with switches, Netlists do not match without switches).
• Rendered output and visual diff images, with red rectangles around every differing region, go to diylc-regression-data/output/ (not under version control).
• Reports are kept as historical evidence. If a run was exploratory rather than a real verification, delete the generated CSV afterwards so the reports directory stays meaningful — and check git status before finishing, since the runner writes into the repo.

Interpreting a failure

A difference is not automatically a bug, but it is always a change the maintainer must see. Report which files differ and what kind of difference it is; where an image diff exists, point at the file so it can be looked at. Do not conclude "no regressions" from a filtered run — say exactly which subset was run.