loop-task

Architecture

How loop-task works internally - the daemon, state machine, persistence, and scheduling.

Overview

loop-task follows a client-daemon architecture. One long-lived daemon process manages all loops, while the CLI and TUI are thin clients that communicate with the daemon over IPC. There is no external database - all state is persisted as JSON files on disk in ~/.loop-cli/.

This design was chosen for three reasons:

  • Low overhead - no database server to install, manage, or debug. A JSON file is always readable.
  • Always-on - the daemon keeps running after the CLI exits, so loops continue to execute on their schedule even when no terminal is attached.
  • Multiple clients - the CLI, the TUI board, and the HTTP API all talk to the same daemon. Any client can inspect or control any loop.

Client-daemon model

CLI / TUI Board          HTTP Client
     │                       │
     ▼                       ▼
  IPC (socket)          HTTP (REST + SSE)
     │                       │
     ▼                       ▼
    ┌──────────────────────────────┐
    │           Daemon              │
    │  ┌──────────┐ ┌────────────┐  │
    │  │IpcServer  │ │HttpApiServer│  │
    │  │           │ │SseClientSet│  │
    │  └─────┬─────┘ └────────────┘  │
    │        │                       │
    │  ┌─────▼──────────────────┐    │
    │  │   LoopManager          │    │
    │  │   TaskManager          │    │
    │  │   ProjectManager       │    │
    │  └─────┬──────────────────┘    │
    │        │                       │
    │  ┌─────▼──────┐               │
    │  │ FileWatcher │               │
    │  └────────────┘               │
    └──────────┬───────────────────┘


     ~/.loop-cli/ (JSON files)

The flow is straightforward:

  1. The CLI or TUI serialises a request as JSON and writes it to the IPC socket.
  2. The daemon's IpcServer receives the request, parses it, and dispatches it to the appropriate manager (LoopManager, TaskManager, ProjectManager).
  3. The manager updates in-memory state and persists changes to disk.
  4. When a loop's interval elapses, the daemon spawns a child process, runs the configured command, captures stdout/stderr, and saves the result.
  5. The FileWatcher monitors the JSON files on disk so that external edits (by a text editor, a script, or a second daemon instance) are detected and reconciled without restarting.

IPC transport

The CLI and daemon communicate over a Unix domain socket on macOS and Linux, or a named pipe on Windows. The transport is managed by Node.js's net module, which abstracts the platform difference.

Protocol details:

  • JSON-lines - each request and response is a single line of JSON terminated by a newline. The daemon reads from the socket, splits on \n, and parses each complete line independently.
  • Client timeout - 10 seconds (IPC_TIMEOUT_MS). If the daemon does not respond within the window, the connection is destroyed and the CLI reports a timeout error.
  • Request-response - each client sends a single JSON object and receives a single JSON response. Streaming responses (for log tailing) use a sequence of data events followed by an end event.
  • Event subscription - clients can send a subscribe request to receive real-time push events (loop status changes, run completions) without polling.

The socket path includes a SHA-1 hash of the data directory to prevent collisions when multiple users run loop-task on the same machine:

Unix:     ~/.loop-cli/daemon-<hash>.sock
Windows:  \\.\pipe\loop-cli-<username>-<hash>

Loop controller state machine

Every loop is managed by a LoopController instance that implements a 5-state state machine. The states reflect what the loop is doing at any given moment:

StateMeaning
runningThe loop's command is currently executing in a child process.
waitingThe command completed; the daemon is sleeping until the next scheduled run.
pausedThe user manually paused the loop - no scheduling occurs until it is resumed.
idleThe loop has an interval of 0 (manual mode) and is waiting for a manual trigger.
stoppedThe loop has been permanently stopped, or its maxRuns limit has been reached.

Transitions:

                  ┌──────────────────────────────────────────────┐
                  │                                              │
                  ▼                                              │
  start ──► waiting ──► running ──► waiting (on success)        │
                │          │                                     │
                │          ├──► stopped (max-runs reached)       │
                │          │                                     │
                │          ▼                                     │
                │       (error)                                  │
                │          │                                     │
                │          ▼                                     │
                │       waiting (on failure) ────────────────────┘

                │  pause

             paused ──► resume ──► waiting

                │  stop

             idle ──► trigger ──► running

                │  playLoop

             waiting

Key details:

  • idle ↔ waiting - a loop with interval = 0 starts in idle and will not run until triggerNow() is called. Calling playLoop() with a non-zero interval moves it to waiting and begins the normal schedule.
  • paused - the delay timer is paused mid-countdown. On resume, it continues from the saved remainingDelayMs rather than restarting the full interval.
  • stopped - this is a terminal state. The loop will not run again until clearMaxRunsReached() is called (to reset the run counter) or the loop is deleted and recreated.
  • running → stopped - happens automatically when runCount >= maxRuns. The _maxRunsReached flag persists across daemon restarts.

Filesystem persistence

All state lives under ~/.loop-cli/ (or the directory pointed to by LOOP_CLI_HOME). There is no database - just plain JSON files that are always safe to read or edit with any text editor.

~/.loop-cli/
├── loops.json          # Array of all LoopMeta - every loop's state
├── tasks.json          # Array of all TaskDefinition
├── projects.json       # Array of all Project
├── logs/
│   ├── <id>.log        # Per-loop log output
│   ├── <id>.log.1      # Rotated logs (up to 3 generations)
│   ├── <id>.log.2
│   └── <id>.log.3
├── daemon.pid          # Process ID of the running daemon
└── daemon.sig          # SHA-1 hash of the compiled dist/ for liveness verification

Atomic writes

File corruption during a crash is prevented by atomic writes. The daemon writes to a temporary file (<file>.<pid>.tmp) and then renames it over the target:

loops.json.12345.tmp ──rename──► loops.json

On Windows, file locks from fs.watch can cause transient EPERM/EBUSY errors during rename. The system retries up to 5 times with backoff, then falls back to a direct synchronous write. If the fallback write produces a partial file, it is safely ignored on the next read because the JSON parser will fail (and return an empty array) rather than loading corrupted state.

Log rotation

Per-loop logs are rotated at 1 MB (MAX_LOG_BYTES) with 3 generations (MAX_LOG_GENERATIONS). When a log exceeds the limit:

<id>.log.2  →  <id>.log.3  (oldest deleted)
<id>.log.1  →  <id>.log.2
<id>.log    →  <id>.log.1  (new empty <id>.log created)

Migration (legacy format)

Versions of loop-task prior to the JSON-array format stored each loop and task as individual <id>.json files in ~/.loop-cli/loops/ and ~/.loop-cli/tasks/. On startup, the daemon checks whether loops.json (or tasks.json) exists. If it does not, but the old per-file directory does, it reads all files and merges them into a single JSON array - a one-time migration that preserves backwards compatibility.

Hot-reload

The FileWatcher monitors loops.json, tasks.json, and projects.json for changes made outside the daemon - for example, by a text editor edit, a sed command, or a sync tool. This allows users to bulk-edit loop definitions without going through the CLI or TUI.

Mechanism:

  • Uses fs.watch on the parent directory to detect file-change events.
  • On Windows, where fs.watch is unreliable, an mtime polling fallback checks each file every 2 seconds (MTIME_POLL_MS).
  • Incoming events are debounced at 300 ms (DEBOUNCE_MS) to batch rapid writes into a single reconciliation pass.
  • Before processing, the file content is SHA-1 hashed and compared to the previous hash. If the hash matches, the change is ignored - this prevents the daemon's own writes from triggering self-reconciliation.
  • Self-writes are filtered through registerSelfWrite(filePath, content), which stores the expected hash before the write completes.
  • When a legitimate change is detected, LoopManager.reconcile() compares the new file contents to its in-memory state and starts, stops, or updates loops as needed.

Spread scheduling

When multiple loops share the same interval, they would all fire at the same time if they started simultaneously - a problem known as thundering herd. To prevent this, each loop's initial delay is spread across its interval using a deterministic hash of the loop ID:

phase = hash(loopId) % interval
delay = (phase - elapsed + interval) % interval

The computePhase function uses a simple string hash (Bernstein's djb2 variant) to produce a deterministic offset from the loop's ID. This means:

  • Two loops with the same interval will almost never start at the same time.
  • Each loop's phase is stable across daemon restarts (the same loop ID always produces the same offset).
  • Users can override the automatic phase with --offset <ms> for explicit control, such as aligning a run to the top of the hour.

Daemon lifecycle

Spawn (ensureDaemon)

When a CLI command runs, ensureDaemon() is called before any IPC request:

  1. Read daemon.pid - if a PID file exists and the process is alive (process.kill(pid, 0)), a daemon is already running.
  2. Verify the code signature - read daemon.sig and compare it to a SHA-1 hash of the current dist/ directory (based on combined mtime + file count + total size). If the signature differs, the old daemon is stopped because the compiled code has changed.
  3. If no healthy daemon exists, spawn a detached child process (spawn(execPath, args, { detached: true, stdio: 'ignore' })) and unref() it so it outlives the parent.
  4. Poll for up to 5 seconds, waiting for the new daemon to write its PID and signature files.

Startup

The daemon's main() runs in sequence:

  1. Create a TaskManager and run any legacy migrations (migrateLoopsToJson, migrateTasksToJson).
  2. Initialise the LoopManager (which loads persisted loops, creates LoopController instances, and starts any loop that was not in stopped state).
  3. Bind the IPC socket - if the socket is already held, exit immediately.
  4. Start the HTTP API server on 127.0.0.1:8845 (or the port in LOOP_CLI_HTTP_PORT).
  5. Write daemon.pid and daemon.sig.
  6. Start the FileWatcher for hot-reload.
  7. Register signal handlers for graceful shutdown.

Shutdown

On SIGINT, SIGTERM, or uncaughtException:

  1. Stop the FileWatcher.
  2. Remove daemon.pid and daemon.sig.
  3. Call LoopManager.shutdown() - gracefully stops all active loops.
  4. Close the IPC server and remove the socket file.
  5. Close the HTTP API server and disconnect all SSE clients.
  6. Exit with code 0.

Task chaining and context

Tasks can be linked together to form chains - when a task finishes, the daemon automatically triggers its successor based on the exit code.

  • onSuccessTaskId - the next task to run when the current task exits with code 0.
  • onFailureTaskId - the next task to run when the current task exits with a non-zero code.

Chaining is handled by the executeChain function in the daemon. After a loop's main command finishes, it checks whether the associated task has a chain target and, if so, resolves the task definition via the TaskResolver, runs the next command, and repeats until the chain ends or a task has no successors.

Context sharing - the stdout of each command in a chain is parsed for KEY=VALUE pairs and merged into a chainContext object. Subsequent commands in the chain can reference these values using {{key}} template syntax, which is interpolated before execution.

For full details, see Task Chaining.