lime/docs/development.md

Live development

lime/dev is the safe route for hacking on a live Lime session. It keeps the X11 event loop on one thread and starts a Slynk server bound only to 127.0.0.1. A connected REPL must send backend work through the developer mailbox, so a source reload or command cannot race CLX event handling.

It is intended for a nested Xephyr server or a separate X login/session. Do not start Lime against the same X display already owned by dwm (or another window manager): X11 permits exactly one root-window substructure redirect owner.

For the first non-nested test, keep your normal dwm login intact and use a spare TTY/X server. A minimal ~/.config/lime/xinit-dev can be:

#!/bin/sh
exec sbcl --non-interactive --eval '(require :asdf)' \
     --eval '(asdf:load-asd "/home/adan/code/common-lisp/lime/lime.asd")' \
     --eval '(asdf:load-system "lime/dev")' \
     --eval '(lime.dev:run-development-session
               :display-name (uiop:getenv "DISPLAY")
               :config-file "/home/adan/.config/lime/config.lisp")'

Make that file executable, then start it from another TTY with startx ~/.config/lime/xinit-dev -- :1 vt8 (adjust the display/VT if either is already in use). Connect SLY from your ordinary session, and keep that ordinary session or a second TTY as your escape hatch. Launch test clients on the new display with DISPLAY=:1 your-client, or put deliberate launch forms in your configuration's existing :startup hook.

Start a session

The configured :default-layout starts fresh Head-local drafts in master/stack unless overridden. Existing drafts and saved Views keep their layouts on configuration reload. The isolated regression test for startup, multi-head first visits, adoption, and reload is:

sbcl --non-interactive --load test/x11-default-layout-smoke.lisp

It requires Xvfb and creates its own test display.

Floating drag regression (also on its own isolated Xvfb):

sbcl --non-interactive --load test/x11-floating-smoke.lisp

This checks real Super-drag input, burst coalescing and release ordering, checkpoint isolation, size hints/borders, and invalidated gestures. It also compares full-desktop versus geometry-only update timings. Timing is diagnostic; the regression asserts bounded rendering work rather than a hardware-dependent frame rate. Restart once after the drag-state change; ordinary config reload alone does not install the new code/structure.

From the Lime checkout, start SBCL without --non-interactive so its process remains available:

sbcl --eval '(require :asdf)' \
     --eval '(asdf:load-asd "/home/adan/code/common-lisp/lime/lime.asd")' \
     --eval '(asdf:load-system "lime/dev")' \
     --eval '(lime.dev:start-development-session
               :display-name ":1"
               :config-file "/home/adan/.config/lime/config.lisp")'

The default Slynk endpoint is 127.0.0.1:4005; connect Emacs/SLY to it. For a nested test display, replace :1 with your Xephyr display. The normal Lime configuration is loaded before the event thread starts.

Every live backend operation from that REPL should use the serialized boundary:

(lime.dev:with-development-backend (backend)
  (lime.x11:backend-status backend))

(lime.dev:reload-development-configuration)

The second form preserves Activities, Views, Presentations, leases, windows, and the current physical-Head mapping. It only reloads user policy such as bindings, commands, rules, and bar providers.

You can expose the same actions in your development configuration, which makes the palette/key layer useful even before connecting a REPL:

(:command reload-lime-source
 :label "Reload Lime source"
 :description "Compile and install changed function-level Lime code"
 :category "Development"
 :action (lambda (backend)
           (declare (ignore backend))
           (lime.dev:reload-development-code)))
(:command reload-lime-config
 :label "Reload Lime config"
 :description "Reload the active configuration without replacing the desk"
 :category "Development"
 :action (lambda (backend)
           (declare (ignore backend))
           (lime.dev:reload-development-configuration))

Reload code without resetting your desk

After a function-level source edit, evaluate:

(lime.dev:reload-development-code)

This compiles and loads changed files in lime/x11, reconciles the current render plan, and returns a development-reload-report. A compile failure is reported in that object and the existing event loop/world stay alive.

Safe reload territory includes ordinary function changes: commands, layout math, renderer code, palette behavior, bar drawing, and configuration helpers. It deliberately does not promise structural live migration. Restart after changing defstruct slots, package or ASDF topology, backend-thread ownership, or lime/dev itself:

(lime.dev:restart-development-session)

A backend restart now saves/restores the desk through the display-specific session file, enabled by default for lime/dev. Surviving windows reconnect with verified identities, including hidden ones. With :session-file nil, it still starts a fresh world. See session persistence for the logout boundary, custom paths, safe writes, and error recovery.

The supervisor and existing Slynk listener stay alive through a slow restart. A failed pre-restart save retains the old X connection; repair and retry, or resume the retained backend. If replacement backend startup fails after X has closed, the control session stays paused: repair and call restart again, not resume. Restart from an event-loop error uses the last good disk snapshot; resume instead validates the retained in-memory world.

Inspect the live runtime with:

(lime.dev:development-status)

When you are finished, use (lime.dev:stop-development-session) from the control REPL. It stops the event thread before closing CLX, remaps managed clients, and closes the Slynk listener.

Recovering from an event-loop error

Use the supervised lime/dev launcher above for this behavior. The bare lime.x11:run-event-loop API does not supply a recovery thread or REPL.

Expected X11 client races (a window vanishing during adoption/property reads, or becoming unviewable just before focus) are handled at the adapter boundary. They increment :expected-x-errors in backend diagnostics; they do not pause the desktop. Ownership conflicts and unrelated protocol/programming errors are not silently ignored.

An unexpected event-loop error pauses event dispatch, releases active keyboard and pointer grabs, closes the command overlay, and preserves the X connection and in-memory world. The original event is consumed, not automatically retried; the remaining queue is retained. Slynk and the serialized mailbox stay alive:

(lime.dev:development-status)          ; :paused-p, :last-error-report, etc.

;; Repair the problem; function-level changes can still be loaded while paused.
(lime.dev:reload-development-code)

(lime.dev:resume-development-session) ; explicit validation + reconciliation

Resume validates the core and adapter registrations, retires disappeared clients, completes an interrupted MapRequest adoption if needed, and renders the retained world. It does not replace Activities, saved Views, or drafts. An invalid model, broken renderer, or closed connection leaves the session paused and returns an error to the REPL. This is recovery, not transaction rollback: inspect state after a partially completed command before continuing.

While paused, WM key bindings, menus, and new window management do not run; use SLY for repair/resume or stop-development-session for an intentional exit. Existing application windows/processes are not deliberately destroyed. No automatic resume loop is used, because replaying arbitrary partial mutations can make a bug worse.

Diagnostics retain the newest 32 reports in memory and overwrite one private (mode 0600) last-fault file under $XDG_STATE_HOME/lime/, normally ~/.local/state/lime/last-error-<display>.log. Display punctuation becomes -; for example :1 becomes last-error--1.log. development-status returns the exact path and any log-write error. Reports include time, event/window id, condition, and an SBCL backtrace captured before unwinding. They omit backtrace arguments, but error messages can still contain private paths/data; review a report before sharing it. Pass :diagnostic-file #p"/chosen/path.log" to the development launcher to override it, or :diagnostic-file nil for memory only. A logging failure does not end the control session.

This protects ordinary signaled event-loop errors, not native Lisp crashes, process kills, deadlocks, or X-server failure. Disk session persistence is a separate layer, not a guarantee that application processes survive. Initial startup failure before the developer session exists still reports a normal launch error. Keep a TTY escape hatch during daily-driver testing.

The resilience change adds backend/runtime slots: install it by stopping and relaunching Lime in a fresh Lisp process once, not by reloading configuration.

Isolated fault-injection regression (requires Xvfb and the lime/dev dependencies):

sbcl --non-interactive --load test/x11-resilience-smoke.lisp

It creates its own display and private Slynk port, tests stale X requests, mid-adoption death/failure, grab release, invalid-state rejection, diagnostic write failure, slow failed restart/retry, and cleanup failure. It never connects to the user's display or REPL. Test diagnostics remain in the printed temporary directory for inspection.