Skip to content

Integrated Variant

The optional webatm_integrated package used by the integrated build. It is wired into the core app only when WEBATM_INTEGRATED=1.

webatm_integrated

webatm_integrated

Provide the WebATM integrated extensions.

Shipped ONLY in the webatm-integrated build variant. Adds BlueSky server lifecycle control (start/stop/restart/kill), live, in-order log streaming of the bluesky --headless process tree to the web UI, and -- because BlueSky runs in this same container -- pre-wires WebATM's file management directly to BlueSky's own scenario / plugins / output directories (no manual base-path step).

The core webatm package never imports this package. It is wired in through a single env-guarded hook in WebATM.app.create_app that calls register only when WEBATM_INTEGRATED=1 and this package is installed.

LogStreamer

LogStreamer(
    socketio,
    max_history: int = 2000,
    batch_ms: int = 100,
    batch_max: int = 200,
)

Buffer process output and broadcast it as ordered, batched events.

Initialize the streamer.

Parameters:

Name Type Description Default
socketio SocketIO

Instance used to emit batches.

required
max_history int

Maximum lines retained for history replay.

2000
batch_ms int

Delay in milliseconds used to coalesce a batch.

100
batch_max int

Maximum lines per emitted batch chunk.

200
Source code in WebATM-integrated/webatm_integrated/log_streamer.py
def __init__(
    self,
    socketio,
    max_history: int = 2000,
    batch_ms: int = 100,
    batch_max: int = 200,
):
    """Initialize the streamer.

    Args:
        socketio (flask_socketio.SocketIO): Instance used to emit batches.
        max_history (int): Maximum lines retained for history replay.
        batch_ms (int): Delay in milliseconds used to coalesce a batch.
        batch_max (int): Maximum lines per emitted batch chunk.
    """
    self._sio = socketio
    # Identifies this streamer (and thus this backend boot) in every
    # payload, so clients can tell a fresh stream from a replay.
    self.stream_id = uuid.uuid4().hex
    self._lock = threading.Lock()
    self._history: collections.deque[dict] = collections.deque(maxlen=max_history)
    self._pending: list[dict] = []
    self._seq = 0
    self._flush_scheduled = False
    self._batch_ms = batch_ms
    self._batch_max = batch_max

feed_line

feed_line(line: str) -> None

Ingest one output line, assign its order, and schedule a flush.

Parameters:

Name Type Description Default
line str

The process output line to broadcast.

required
Source code in WebATM-integrated/webatm_integrated/log_streamer.py
def feed_line(self, line: str) -> None:
    """Ingest one output line, assign its order, and schedule a flush.

    Args:
        line (str): The process output line to broadcast.
    """
    with self._lock:
        self._seq += 1
        item = {"seq": self._seq, "t": time.time(), "line": line}
        self._history.append(item)
        self._pending.append(item)
        if not self._flush_scheduled:
            self._flush_scheduled = True
            try:
                self._sio.start_background_task(self._flush_after_delay)
            except Exception:
                # Un-wedge the scheduler: the line stays pending and the
                # next feed_line retries; a stuck True flag would silence
                # the stream forever.
                self._flush_scheduled = False
                raise

history

history() -> list[dict]

Return a snapshot of buffered lines for late-joining clients.

Returns:

Type Description
list[dict]

Buffered items with seq, t and line keys.

Source code in WebATM-integrated/webatm_integrated/log_streamer.py
def history(self) -> list[dict]:
    """Return a snapshot of buffered lines for late-joining clients.

    Returns:
        list[dict]: Buffered items with ``seq``, ``t`` and ``line`` keys.
    """
    with self._lock:
        return list(self._history)

on_process_exit

on_process_exit(return_code: int) -> None

Emit an end-of-stream marker when the server process exits.

Parameters:

Name Type Description Default
return_code int

Exit code of the BlueSky server process.

required
Source code in WebATM-integrated/webatm_integrated/log_streamer.py
def on_process_exit(self, return_code: int) -> None:
    """Emit an end-of-stream marker when the server process exits.

    Args:
        return_code (int): Exit code of the BlueSky server process.
    """
    self.feed_line(f"--- bluesky server exited (return code {return_code}) ---")

BlueSkyProcessManager

BlueSkyProcessManager(
    on_line: Callable[[str], None] | None = None,
    on_exit: Callable[[int], None] | None = None,
    spawn: Callable | None = None,
    cmd: list[str] | None = None,
)

Thread-safe lifecycle manager for the bluesky --headless process tree.

Tracks only the parent process; signals (stop/kill) address the whole process group so node children are reaped together with the server.

Initialize the process manager.

Parameters:

Name Type Description Default
on_line Callable[[str], None] | None

Callback invoked with each output line of the process tree (newline stripped).

None
on_exit Callable[[int], None] | None

Callback invoked with the return code when the server process exits.

None
spawn Callable | None

Spawn primitive for the reader task (e.g. socketio.start_background_task); defaults to a plain daemon thread.

None
cmd list[str] | None

Command to launch; defaults to ["bluesky", "--headless"].

None
Source code in WebATM-integrated/webatm_integrated/process_manager.py
def __init__(
    self,
    on_line: Callable[[str], None] | None = None,
    on_exit: Callable[[int], None] | None = None,
    spawn: Callable | None = None,
    cmd: list[str] | None = None,
):
    """Initialize the process manager.

    Args:
        on_line: Callback invoked with each output line of the process
            tree (newline stripped).
        on_exit: Callback invoked with the return code when the server
            process exits.
        spawn: Spawn primitive for the reader task (e.g.
            ``socketio.start_background_task``); defaults to a plain
            daemon thread.
        cmd (list[str] | None): Command to launch; defaults to
            ``["bluesky", "--headless"]``.
    """
    self._lock = threading.RLock()
    self._proc: subprocess.Popen | None = None
    self._on_line = on_line
    self._on_exit = on_exit
    self._spawn = spawn or _default_spawn
    self._cmd = cmd or ["bluesky", "--headless"]
    self._state = "stopped"  # stopped | running | stopping

start

start() -> dict

Spawn the headless server (in its own process group) and a reader.

The process is started in a new session with merged, line-buffered stdout/stderr so the reader receives one ordered stream for the server and all node children. A no-op if the server is already running. If a concurrent :meth:stop is mid-shutdown, waits for it to finish and then starts a fresh server instead of reporting the doomed process as "already running".

Returns:

Type Description
dict

Result with success, status, pid and message (plus error on failure).

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def start(self) -> dict:
    """Spawn the headless server (in its own process group) and a reader.

    The process is started in a new session with merged, line-buffered
    stdout/stderr so the reader receives one ordered stream for the
    server and all node children. A no-op if the server is already
    running. If a concurrent :meth:`stop` is mid-shutdown, waits for it
    to finish and then starts a fresh server instead of reporting the
    doomed process as "already running".

    Returns:
        dict: Result with ``success``, ``status``, ``pid`` and
            ``message`` (plus ``error`` on failure).
    """
    with self._lock:
        stopping_proc = self._proc if self._state == "stopping" else None
    if stopping_proc is not None:
        # A concurrent stop() owns this process's shutdown; wait it out
        # (worst case escalate_after + the 5s SIGKILL wait) rather than
        # report a pid that is about to die as "already running".
        try:
            stopping_proc.wait(timeout=15)
        except subprocess.TimeoutExpired:
            return {
                "success": False,
                "status": "error",
                "message": "BlueSky server is still stopping; retry shortly",
            }

    with self._lock:
        if self._proc is not None and self._proc.poll() is None:
            return {
                "success": True,
                "status": "running",
                "pid": self._proc.pid,
                "message": "BlueSky server already running",
            }
        # PYTHONUNBUFFERED keeps the server's (and inherited children's)
        # stdout line-buffered so log lines arrive promptly and in order.
        env = dict(os.environ, PYTHONUNBUFFERED="1")
        try:
            self._proc = subprocess.Popen(
                self._cmd,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,  # merge -> single ordered stream
                stdin=subprocess.DEVNULL,
                bufsize=1,
                text=True,
                env=env,
                start_new_session=True,  # own process group / session
                close_fds=True,
            )
        except Exception as e:
            self._state = "stopped"
            return {
                "success": False,
                "status": "error",
                "message": f"Failed to start BlueSky: {e}",
                "error": str(e),
            }
        self._state = "running"
        proc = self._proc

    # Launch the reader outside the lock.
    self._spawn(self._read_loop, proc)
    return {
        "success": True,
        "status": "running",
        "pid": proc.pid,
        "message": "BlueSky server started",
    }

stop

stop(
    sig: int = signal.SIGTERM, escalate_after: float = 5.0
) -> dict

Signal the whole process group, escalating to SIGKILL if needed.

Parameters:

Name Type Description Default
sig int

Signal sent to the process group first.

SIGTERM
escalate_after float

Seconds to wait for exit before force-killing the group with SIGKILL.

5.0

Returns:

Type Description
dict

Result with success, status and message. success is False if the process survives even SIGKILL.

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def stop(self, sig: int = signal.SIGTERM, escalate_after: float = 5.0) -> dict:
    """Signal the whole process group, escalating to SIGKILL if needed.

    Args:
        sig (int): Signal sent to the process group first.
        escalate_after (float): Seconds to wait for exit before
            force-killing the group with SIGKILL.

    Returns:
        dict: Result with ``success``, ``status`` and ``message``.
            ``success`` is False if the process survives even SIGKILL.
    """
    with self._lock:
        proc = self._proc
        if proc is None or proc.poll() is not None:
            self._state = "stopped"
            return {
                "success": True,
                "status": "stopped",
                "message": "BlueSky server is not running",
            }
        self._state = "stopping"

    try:
        return self._terminate(proc, sig, escalate_after)
    except BaseException:
        # Never leave the state stranded at "stopping": start() would wait
        # 15s and then refuse, with no control surface able to clear it.
        self._settle(proc)
        raise

kill

kill() -> dict

Force-kill the whole process group immediately (no graceful wait).

Returns:

Type Description
dict

Result with success, status and message.

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def kill(self) -> dict:
    """Force-kill the whole process group immediately (no graceful wait).

    Returns:
        dict: Result with ``success``, ``status`` and ``message``.
    """
    with self._lock:
        proc = self._proc
        was_running = proc is not None and proc.poll() is None
    result = self.stop(sig=signal.SIGKILL, escalate_after=2.0)
    # Only claim a kill if there was actually a live process to kill --
    # otherwise keep stop()'s "is not running", which the UI shows verbatim.
    if was_running and result.get("success") and result.get("status") == "stopped":
        result["message"] = "BlueSky server killed"
    return result

restart

restart() -> dict

Stop the current tree (if any) and start a fresh one.

Returns:

Type Description
dict

The start() result, with the message adjusted to "restarted" on success. If the old tree could not be stopped, its failure result is returned instead of starting a new one (start() would just report the surviving process as "already running").

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def restart(self) -> dict:
    """Stop the current tree (if any) and start a fresh one.

    Returns:
        dict: The ``start()`` result, with the message adjusted to
            "restarted" on success. If the old tree could not be stopped,
            its failure result is returned instead of starting a new one
            (``start()`` would just report the surviving process as
            "already running").
    """
    stop_result = self.stop()
    if not stop_result.get("success"):
        return stop_result
    result = self.start()
    if result.get("success"):
        result["message"] = "BlueSky server restarted"
    return result

status

status() -> dict

Report whether the server is running, with its pid and state.

Returns:

Type Description
dict

Result with success, running, status and pid (None when stopped).

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def status(self) -> dict:
    """Report whether the server is running, with its pid and state.

    Returns:
        dict: Result with ``success``, ``running``, ``status`` and
            ``pid`` (None when stopped).
    """
    with self._lock:
        proc = self._proc
        if proc is None or proc.poll() is not None:
            return {
                "success": True,
                "running": False,
                "status": "stopped",
                "pid": None,
            }
        return {
            "success": True,
            "running": True,
            "status": self._state,
            "pid": proc.pid,
        }

register

register(
    app,
    socketio,
    *,
    session_manager=None,
    bluesky_proxy=None,
)

Wire the integrated features into an existing WebATM app.

Called by WebATM.app.create_app when WEBATM_INTEGRATED=1. Points file management at BlueSky's fixed working directory, creates the process manager and log streamer (stashed on app), registers the integrated REST routes and Socket.IO handlers, reaps the BlueSky process group on worker exit, and — on the first boot only (claim_first_boot()) — schedules the background auto-start (disable with WEBATM_AUTO_START=0).

Parameters:

Name Type Description Default
app Flask

Flask application instance.

required
socketio SocketIO

Flask-SocketIO instance (async_mode="threading").

required
session_manager SessionManager

Core session manager (accepted for forward-compat; currently unused).

None
bluesky_proxy BlueSkyProxy

Core proxy, used for the first-boot auto-connect.

None

Returns:

Type Description
dict

The created manager (BlueSkyProcessManager) and streamer (LogStreamer), handy for tests.

Source code in WebATM-integrated/webatm_integrated/__init__.py
def register(app, socketio, *, session_manager=None, bluesky_proxy=None):
    """Wire the integrated features into an existing WebATM app.

    Called by ``WebATM.app.create_app`` when ``WEBATM_INTEGRATED=1``. Points
    file management at BlueSky's fixed working directory, creates the process
    manager and log streamer (stashed on ``app``), registers the integrated
    REST routes and Socket.IO handlers, reaps the BlueSky process group on
    worker exit, and — on the first boot only (``claim_first_boot()``) —
    schedules the background auto-start (disable with ``WEBATM_AUTO_START=0``).

    Args:
        app (flask.Flask): Flask application instance.
        socketio (flask_socketio.SocketIO): Flask-SocketIO instance
            (``async_mode="threading"``).
        session_manager (SessionManager): Core session manager (accepted for
            forward-compat; currently unused).
        bluesky_proxy (BlueSkyProxy): Core proxy, used for the first-boot
            auto-connect.

    Returns:
        dict: The created ``manager`` (BlueSkyProcessManager) and ``streamer``
            (LogStreamer), handy for tests.
    """
    # Imported lazily so the Flask-importing modules stay out of flask-free
    # unit-test imports of process_manager / log_streamer.
    from .auto_start import auto_start_enabled, claim_first_boot, schedule_auto_start
    from .bluesky_paths import configure_file_management
    from .routes import register_integrated_routes
    from .socket_handlers import register_integrated_socket_handlers

    configure_file_management(app)

    streamer = LogStreamer(socketio)
    manager = BlueSkyProcessManager(
        on_line=streamer.feed_line,
        on_exit=streamer.on_process_exit,
        spawn=socketio.start_background_task,
    )

    # Stash on the app so blueprint-free route/handler closures can reach them
    # via flask.current_app if needed.
    app.bluesky_process_manager = manager
    app.bluesky_log_streamer = streamer

    register_integrated_routes(app, manager)
    register_integrated_socket_handlers(socketio, streamer)

    # Reap the whole bluesky process group if the worker process exits.
    atexit.register(manager.kill)

    # First boot only (see the docstring above): auto-start BlueSky and connect
    # the proxy in the background so app creation returns promptly.
    if auto_start_enabled() and claim_first_boot():
        schedule_auto_start(socketio, manager, bluesky_proxy)

    return {"manager": manager, "streamer": streamer}

webatm_integrated.auto_start

webatm_integrated.auto_start

Auto-start the bundled BlueSky server and connect the proxy on first boot.

Integrated build only: BlueSky runs in the same container, so on start-up we spawn the bluesky --headless process tree and, once its ports accept connections, run the same connect sequence as the manual /api/server/config route. The user lands on a live, already-connected map. If a manual connect wins the race, auto-start stands down (see connect_proxy_when_ready).

Opt out with WEBATM_AUTO_START=0. The core webatm package never imports this module; it is reached only via webatm_integrated.register (env-guarded on WEBATM_INTEGRATED=1).

auto_start_enabled

auto_start_enabled() -> bool

Report whether to auto-start BlueSky and auto-connect on boot.

On by default; set WEBATM_AUTO_START=0 to disable.

Returns:

Type Description
bool

True unless the WEBATM_AUTO_START environment variable is set to "0".

Source code in WebATM-integrated/webatm_integrated/auto_start.py
def auto_start_enabled() -> bool:
    """Report whether to auto-start BlueSky and auto-connect on boot.

    On by default; set ``WEBATM_AUTO_START=0`` to disable.

    Returns:
        bool: True unless the ``WEBATM_AUTO_START`` environment variable is
            set to ``"0"``.
    """
    return os.environ.get("WEBATM_AUTO_START", "1") != "0"

claim_first_boot

claim_first_boot(marker_path: str | None = None) -> bool

Atomically claim the one-shot auto-start for this boot.

Creates the marker file with O_CREAT | O_EXCL so only the first caller per boot wins — a replaced gunicorn worker re-running register() stands down instead of resurrecting a manually-stopped server. The default marker lives on tmpfs so it clears on a fresh container start.

Parameters:

Name Type Description Default
marker_path str | None

Marker file location. Defaults to WEBATM_AUTOSTART_MARKER, then /dev/shm/webatm_autostart.done.

None

Returns:

Type Description
bool

True for the first caller to create the marker file, False thereafter. If the marker cannot be created at all (e.g. no /dev/shm), degrades to True and skips the once-per-boot guard.

Source code in WebATM-integrated/webatm_integrated/auto_start.py
def claim_first_boot(marker_path: str | None = None) -> bool:
    """Atomically claim the one-shot auto-start for this boot.

    Creates the marker file with ``O_CREAT | O_EXCL`` so only the first caller
    per boot wins — a replaced gunicorn worker re-running ``register()`` stands
    down instead of resurrecting a manually-stopped server. The default marker
    lives on tmpfs so it clears on a fresh container start.

    Args:
        marker_path (str | None): Marker file location. Defaults to
            ``WEBATM_AUTOSTART_MARKER``, then ``/dev/shm/webatm_autostart.done``.

    Returns:
        bool: True for the first caller to create the marker file, False
            thereafter. If the marker cannot be created at all (e.g. no
            ``/dev/shm``), degrades to True and skips the once-per-boot guard.
    """
    path = marker_path or os.environ.get("WEBATM_AUTOSTART_MARKER", _DEFAULT_MARKER)
    try:
        fd = os.open(path, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644)
    except FileExistsError:
        logger.info("Auto-start: already performed this boot; skipping")
        return False
    except OSError as e:
        logger.warning(
            f"Auto-start: could not create boot marker '{path}' ({e}); "
            "proceeding without the once-per-boot guard"
        )
        return True
    try:
        os.write(fd, f"pid={os.getpid()}\n".encode())
    finally:
        os.close(fd)
    return True

schedule_auto_start

schedule_auto_start(
    socketio, manager, bluesky_proxy
) -> None

Run the auto-start sequence on a background task.

Backgrounded so register() (which runs during app creation) returns immediately: waiting for BlueSky's ports can take several seconds on a cold start, and we must not block the worker from beginning to serve requests.

Parameters:

Name Type Description Default
socketio SocketIO

Socket.IO instance whose start_background_task runs the sequence.

required
manager BlueSkyProcessManager

Process manager used to start the BlueSky server.

required
bluesky_proxy BlueSkyProxy | None

Core proxy to connect once BlueSky is ready, or None to skip the auto-connect.

required
Source code in WebATM-integrated/webatm_integrated/auto_start.py
def schedule_auto_start(socketio, manager, bluesky_proxy) -> None:
    """Run the auto-start sequence on a background task.

    Backgrounded so ``register()`` (which runs during app creation) returns
    immediately: waiting for BlueSky's ports can take several seconds on a cold
    start, and we must not block the worker from beginning to serve requests.

    Args:
        socketio (flask_socketio.SocketIO): Socket.IO instance whose
            ``start_background_task`` runs the sequence.
        manager (BlueSkyProcessManager): Process manager used to start the
            BlueSky server.
        bluesky_proxy (BlueSkyProxy | None): Core proxy to connect once
            BlueSky is ready, or None to skip the auto-connect.
    """
    socketio.start_background_task(_run_auto_start, manager, bluesky_proxy)

connect_proxy_when_ready

connect_proxy_when_ready(
    bluesky_proxy,
    *,
    host: str | None = None,
    ready_timeout: float = 60.0,
    poll_interval: float = 0.5,
    is_port_listening: Callable[..., bool] | None = None,
    register_subscribers: Callable[..., None] | None = None,
    get_proxy: Callable[[], object] | None = None,
    lock: AbstractContextManager | None = None,
    sleep: Callable[[float], None] | None = None,
) -> bool

Wait for BlueSky to accept connections, then connect the WebATM proxy.

Polls BlueSky's command/data ports until one is listening (or the timeout elapses), then performs the same start_client → register_subscribers sequence as the manual /api/server/config route. The connect step runs under the shared WebATM.proxy.connect_lock and stands down if a manual connect replaced the global proxy during the port wait — connecting the stale boot-time proxy would leave a second, subscriber-less ZMQ client alive to broadcast a bogus disconnect later.

The collaborators are injectable so this can be unit-tested without a real BlueSky server or wall-clock delays.

Parameters:

Name Type Description Default
bluesky_proxy BlueSkyProxy

Core proxy to connect.

required
host str | None

BlueSky server host. Defaults to the proxy's server_ip, then BLUESKY_SERVER_HOST, then "localhost".

None
ready_timeout float

Maximum seconds to wait for a port to listen.

60.0
poll_interval float

Seconds to sleep between port probes.

0.5
is_port_listening Callable | None

Port probe (port, timeout, host); defaults to the core implementation.

None
register_subscribers Callable | None

Subscriber-registration hook; defaults to WebATM.proxy.register_subscribers.

None
get_proxy Callable | None

Returns the current global proxy; defaults to WebATM.proxy.get_bluesky_proxy.

None
lock AbstractContextManager | None

Lock held around the connect step; defaults to WebATM.proxy.connect_lock.

None
sleep Callable | None

Sleep function; defaults to time.sleep.

None

Returns:

Type Description
bool

True if the proxy connect succeeded, False if BlueSky never came up in time, the connect raised, or a manual connect won the race.

Source code in WebATM-integrated/webatm_integrated/auto_start.py
def connect_proxy_when_ready(
    bluesky_proxy,
    *,
    host: str | None = None,
    ready_timeout: float = 60.0,
    poll_interval: float = 0.5,
    is_port_listening: Callable[..., bool] | None = None,
    register_subscribers: Callable[..., None] | None = None,
    get_proxy: Callable[[], object] | None = None,
    lock: AbstractContextManager | None = None,
    sleep: Callable[[float], None] | None = None,
) -> bool:
    """Wait for BlueSky to accept connections, then connect the WebATM proxy.

    Polls BlueSky's command/data ports until one is listening (or the timeout
    elapses), then performs the same ``start_client`` → ``register_subscribers``
    sequence as the manual ``/api/server/config`` route. The connect step runs
    under the shared ``WebATM.proxy.connect_lock`` and stands down if a manual
    connect replaced the global proxy during the port wait — connecting the
    stale boot-time proxy would leave a second, subscriber-less ZMQ client
    alive to broadcast a bogus disconnect later.

    The collaborators are injectable so this can be unit-tested without a real
    BlueSky server or wall-clock delays.

    Args:
        bluesky_proxy (BlueSkyProxy): Core proxy to connect.
        host (str | None): BlueSky server host. Defaults to the proxy's
            ``server_ip``, then ``BLUESKY_SERVER_HOST``, then ``"localhost"``.
        ready_timeout (float): Maximum seconds to wait for a port to listen.
        poll_interval (float): Seconds to sleep between port probes.
        is_port_listening (Callable | None): Port probe ``(port, timeout,
            host)``; defaults to the core implementation.
        register_subscribers (Callable | None): Subscriber-registration hook;
            defaults to ``WebATM.proxy.register_subscribers``.
        get_proxy (Callable | None): Returns the current global proxy;
            defaults to ``WebATM.proxy.get_bluesky_proxy``.
        lock (AbstractContextManager | None): Lock held around the connect
            step; defaults to ``WebATM.proxy.connect_lock``.
        sleep (Callable | None): Sleep function; defaults to ``time.sleep``.

    Returns:
        bool: True if the proxy connect succeeded, False if BlueSky never came
            up in time, the connect raised, or a manual connect won the race.
    """
    # Deferred imports: keep this module light for unit tests and avoid pulling
    # the Flask/ZMQ-laden core packages unless we actually connect.
    if is_port_listening is None:
        from WebATM.server.bluesky_server_status import is_port_listening
    if register_subscribers is None:
        from WebATM.proxy import register_subscribers
    if get_proxy is None:
        from WebATM.proxy import get_bluesky_proxy as get_proxy
    if lock is None:
        from WebATM.proxy import connect_lock as lock
    if sleep is None:
        sleep = time.sleep

    host = (
        host
        or getattr(bluesky_proxy, "server_ip", None)
        or os.environ.get("BLUESKY_SERVER_HOST", "localhost")
    )

    if not _wait_for_ports(
        host, ready_timeout, poll_interval, is_port_listening, sleep
    ):
        logger.error(
            f"Auto-start: BlueSky ports {_BLUESKY_PORTS} not listening on '{host}' "
            f"after {ready_timeout:.0f}s; proxy not connected"
        )
        return False

    with lock:
        if get_proxy() is not bluesky_proxy:
            logger.info(
                "Auto-start: a manual connect replaced the proxy while waiting "
                "for BlueSky; standing down"
            )
            return False
        try:
            bluesky_proxy.server_ip = host
            bluesky_proxy.start_client(hostname=host)
            # Subscribers can only be registered once the client exists, which
            # start_client creates -- this is the same ordering the manual
            # /api/server/config route relies on. They attach to bluesky_proxy
            # explicitly, so they always land on the client just started.
            register_subscribers(bluesky_proxy)
            logger.info(f"Auto-start: WebATM proxy connected to BlueSky at '{host}'")
            return True
        except Exception as e:
            logger.error(f"Auto-start: failed to connect proxy to BlueSky: {e}")
            return False

webatm_integrated.process_manager

webatm_integrated.process_manager

BlueSky headless server process manager.

Owns the lifecycle of the bluesky --headless process tree: the headless server plus every node child process it spawns. On POSIX, BlueSky spawns node children as ordinary subprocesses that inherit the parent's stdout/stderr and live in the parent's process group, so:

  • a single merged pipe on the parent captures the server and all node-child output, already interleaved in order; and
  • launching the parent in its own session (start_new_session=True) lets us reap the entire tree with one os.killpg.

BlueSkyProcessManager

BlueSkyProcessManager(
    on_line: Callable[[str], None] | None = None,
    on_exit: Callable[[int], None] | None = None,
    spawn: Callable | None = None,
    cmd: list[str] | None = None,
)

Thread-safe lifecycle manager for the bluesky --headless process tree.

Tracks only the parent process; signals (stop/kill) address the whole process group so node children are reaped together with the server.

Initialize the process manager.

Parameters:

Name Type Description Default
on_line Callable[[str], None] | None

Callback invoked with each output line of the process tree (newline stripped).

None
on_exit Callable[[int], None] | None

Callback invoked with the return code when the server process exits.

None
spawn Callable | None

Spawn primitive for the reader task (e.g. socketio.start_background_task); defaults to a plain daemon thread.

None
cmd list[str] | None

Command to launch; defaults to ["bluesky", "--headless"].

None
Source code in WebATM-integrated/webatm_integrated/process_manager.py
def __init__(
    self,
    on_line: Callable[[str], None] | None = None,
    on_exit: Callable[[int], None] | None = None,
    spawn: Callable | None = None,
    cmd: list[str] | None = None,
):
    """Initialize the process manager.

    Args:
        on_line: Callback invoked with each output line of the process
            tree (newline stripped).
        on_exit: Callback invoked with the return code when the server
            process exits.
        spawn: Spawn primitive for the reader task (e.g.
            ``socketio.start_background_task``); defaults to a plain
            daemon thread.
        cmd (list[str] | None): Command to launch; defaults to
            ``["bluesky", "--headless"]``.
    """
    self._lock = threading.RLock()
    self._proc: subprocess.Popen | None = None
    self._on_line = on_line
    self._on_exit = on_exit
    self._spawn = spawn or _default_spawn
    self._cmd = cmd or ["bluesky", "--headless"]
    self._state = "stopped"  # stopped | running | stopping

start

start() -> dict

Spawn the headless server (in its own process group) and a reader.

The process is started in a new session with merged, line-buffered stdout/stderr so the reader receives one ordered stream for the server and all node children. A no-op if the server is already running. If a concurrent :meth:stop is mid-shutdown, waits for it to finish and then starts a fresh server instead of reporting the doomed process as "already running".

Returns:

Type Description
dict

Result with success, status, pid and message (plus error on failure).

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def start(self) -> dict:
    """Spawn the headless server (in its own process group) and a reader.

    The process is started in a new session with merged, line-buffered
    stdout/stderr so the reader receives one ordered stream for the
    server and all node children. A no-op if the server is already
    running. If a concurrent :meth:`stop` is mid-shutdown, waits for it
    to finish and then starts a fresh server instead of reporting the
    doomed process as "already running".

    Returns:
        dict: Result with ``success``, ``status``, ``pid`` and
            ``message`` (plus ``error`` on failure).
    """
    with self._lock:
        stopping_proc = self._proc if self._state == "stopping" else None
    if stopping_proc is not None:
        # A concurrent stop() owns this process's shutdown; wait it out
        # (worst case escalate_after + the 5s SIGKILL wait) rather than
        # report a pid that is about to die as "already running".
        try:
            stopping_proc.wait(timeout=15)
        except subprocess.TimeoutExpired:
            return {
                "success": False,
                "status": "error",
                "message": "BlueSky server is still stopping; retry shortly",
            }

    with self._lock:
        if self._proc is not None and self._proc.poll() is None:
            return {
                "success": True,
                "status": "running",
                "pid": self._proc.pid,
                "message": "BlueSky server already running",
            }
        # PYTHONUNBUFFERED keeps the server's (and inherited children's)
        # stdout line-buffered so log lines arrive promptly and in order.
        env = dict(os.environ, PYTHONUNBUFFERED="1")
        try:
            self._proc = subprocess.Popen(
                self._cmd,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,  # merge -> single ordered stream
                stdin=subprocess.DEVNULL,
                bufsize=1,
                text=True,
                env=env,
                start_new_session=True,  # own process group / session
                close_fds=True,
            )
        except Exception as e:
            self._state = "stopped"
            return {
                "success": False,
                "status": "error",
                "message": f"Failed to start BlueSky: {e}",
                "error": str(e),
            }
        self._state = "running"
        proc = self._proc

    # Launch the reader outside the lock.
    self._spawn(self._read_loop, proc)
    return {
        "success": True,
        "status": "running",
        "pid": proc.pid,
        "message": "BlueSky server started",
    }

stop

stop(
    sig: int = signal.SIGTERM, escalate_after: float = 5.0
) -> dict

Signal the whole process group, escalating to SIGKILL if needed.

Parameters:

Name Type Description Default
sig int

Signal sent to the process group first.

SIGTERM
escalate_after float

Seconds to wait for exit before force-killing the group with SIGKILL.

5.0

Returns:

Type Description
dict

Result with success, status and message. success is False if the process survives even SIGKILL.

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def stop(self, sig: int = signal.SIGTERM, escalate_after: float = 5.0) -> dict:
    """Signal the whole process group, escalating to SIGKILL if needed.

    Args:
        sig (int): Signal sent to the process group first.
        escalate_after (float): Seconds to wait for exit before
            force-killing the group with SIGKILL.

    Returns:
        dict: Result with ``success``, ``status`` and ``message``.
            ``success`` is False if the process survives even SIGKILL.
    """
    with self._lock:
        proc = self._proc
        if proc is None or proc.poll() is not None:
            self._state = "stopped"
            return {
                "success": True,
                "status": "stopped",
                "message": "BlueSky server is not running",
            }
        self._state = "stopping"

    try:
        return self._terminate(proc, sig, escalate_after)
    except BaseException:
        # Never leave the state stranded at "stopping": start() would wait
        # 15s and then refuse, with no control surface able to clear it.
        self._settle(proc)
        raise

kill

kill() -> dict

Force-kill the whole process group immediately (no graceful wait).

Returns:

Type Description
dict

Result with success, status and message.

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def kill(self) -> dict:
    """Force-kill the whole process group immediately (no graceful wait).

    Returns:
        dict: Result with ``success``, ``status`` and ``message``.
    """
    with self._lock:
        proc = self._proc
        was_running = proc is not None and proc.poll() is None
    result = self.stop(sig=signal.SIGKILL, escalate_after=2.0)
    # Only claim a kill if there was actually a live process to kill --
    # otherwise keep stop()'s "is not running", which the UI shows verbatim.
    if was_running and result.get("success") and result.get("status") == "stopped":
        result["message"] = "BlueSky server killed"
    return result

restart

restart() -> dict

Stop the current tree (if any) and start a fresh one.

Returns:

Type Description
dict

The start() result, with the message adjusted to "restarted" on success. If the old tree could not be stopped, its failure result is returned instead of starting a new one (start() would just report the surviving process as "already running").

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def restart(self) -> dict:
    """Stop the current tree (if any) and start a fresh one.

    Returns:
        dict: The ``start()`` result, with the message adjusted to
            "restarted" on success. If the old tree could not be stopped,
            its failure result is returned instead of starting a new one
            (``start()`` would just report the surviving process as
            "already running").
    """
    stop_result = self.stop()
    if not stop_result.get("success"):
        return stop_result
    result = self.start()
    if result.get("success"):
        result["message"] = "BlueSky server restarted"
    return result

status

status() -> dict

Report whether the server is running, with its pid and state.

Returns:

Type Description
dict

Result with success, running, status and pid (None when stopped).

Source code in WebATM-integrated/webatm_integrated/process_manager.py
def status(self) -> dict:
    """Report whether the server is running, with its pid and state.

    Returns:
        dict: Result with ``success``, ``running``, ``status`` and
            ``pid`` (None when stopped).
    """
    with self._lock:
        proc = self._proc
        if proc is None or proc.poll() is not None:
            return {
                "success": True,
                "running": False,
                "status": "stopped",
                "pid": None,
            }
        return {
            "success": True,
            "running": True,
            "status": self._state,
            "pid": proc.pid,
        }

webatm_integrated.log_streamer

webatm_integrated.log_streamer

Live, in-order log streaming of the BlueSky process tree to web clients.

A single server-wide stream (one subprocess) is broadcast to all connected browsers over the server_log Socket.IO event. Ordering is guaranteed by a monotonic sequence number assigned under a lock at ingest, before any async hop. Bursts (for example, creating many nodes at once) are coalesced into batches so a flood of lines cannot overwhelm Socket.IO.

Every payload also carries a per-streamer stream id. Sequence numbers restart at 1 whenever the backend (and with it this streamer) is replaced -- a container restart, a recycled gunicorn worker -- while an open browser tab keeps the old session's seqs and de-duplicates on them. Without the stream id the new boot's lines would collide and be silently dropped client-side.

LogStreamer

LogStreamer(
    socketio,
    max_history: int = 2000,
    batch_ms: int = 100,
    batch_max: int = 200,
)

Buffer process output and broadcast it as ordered, batched events.

Initialize the streamer.

Parameters:

Name Type Description Default
socketio SocketIO

Instance used to emit batches.

required
max_history int

Maximum lines retained for history replay.

2000
batch_ms int

Delay in milliseconds used to coalesce a batch.

100
batch_max int

Maximum lines per emitted batch chunk.

200
Source code in WebATM-integrated/webatm_integrated/log_streamer.py
def __init__(
    self,
    socketio,
    max_history: int = 2000,
    batch_ms: int = 100,
    batch_max: int = 200,
):
    """Initialize the streamer.

    Args:
        socketio (flask_socketio.SocketIO): Instance used to emit batches.
        max_history (int): Maximum lines retained for history replay.
        batch_ms (int): Delay in milliseconds used to coalesce a batch.
        batch_max (int): Maximum lines per emitted batch chunk.
    """
    self._sio = socketio
    # Identifies this streamer (and thus this backend boot) in every
    # payload, so clients can tell a fresh stream from a replay.
    self.stream_id = uuid.uuid4().hex
    self._lock = threading.Lock()
    self._history: collections.deque[dict] = collections.deque(maxlen=max_history)
    self._pending: list[dict] = []
    self._seq = 0
    self._flush_scheduled = False
    self._batch_ms = batch_ms
    self._batch_max = batch_max

feed_line

feed_line(line: str) -> None

Ingest one output line, assign its order, and schedule a flush.

Parameters:

Name Type Description Default
line str

The process output line to broadcast.

required
Source code in WebATM-integrated/webatm_integrated/log_streamer.py
def feed_line(self, line: str) -> None:
    """Ingest one output line, assign its order, and schedule a flush.

    Args:
        line (str): The process output line to broadcast.
    """
    with self._lock:
        self._seq += 1
        item = {"seq": self._seq, "t": time.time(), "line": line}
        self._history.append(item)
        self._pending.append(item)
        if not self._flush_scheduled:
            self._flush_scheduled = True
            try:
                self._sio.start_background_task(self._flush_after_delay)
            except Exception:
                # Un-wedge the scheduler: the line stays pending and the
                # next feed_line retries; a stuck True flag would silence
                # the stream forever.
                self._flush_scheduled = False
                raise

history

history() -> list[dict]

Return a snapshot of buffered lines for late-joining clients.

Returns:

Type Description
list[dict]

Buffered items with seq, t and line keys.

Source code in WebATM-integrated/webatm_integrated/log_streamer.py
def history(self) -> list[dict]:
    """Return a snapshot of buffered lines for late-joining clients.

    Returns:
        list[dict]: Buffered items with ``seq``, ``t`` and ``line`` keys.
    """
    with self._lock:
        return list(self._history)

on_process_exit

on_process_exit(return_code: int) -> None

Emit an end-of-stream marker when the server process exits.

Parameters:

Name Type Description Default
return_code int

Exit code of the BlueSky server process.

required
Source code in WebATM-integrated/webatm_integrated/log_streamer.py
def on_process_exit(self, return_code: int) -> None:
    """Emit an end-of-stream marker when the server process exits.

    Args:
        return_code (int): Exit code of the BlueSky server process.
    """
    self.feed_line(f"--- bluesky server exited (return code {return_code}) ---")

webatm_integrated.bluesky_paths

webatm_integrated.bluesky_paths

BlueSky file-management path wiring for the integrated build.

In the integrated variant BlueSky runs inside the same container, and a pip-installed BlueSky keeps its working directory at ~/bluesky (see BlueSky's pathfinder). This module points WebATM's existing file-management routes (/api/bluesky/...) at that directory and locks them there, replacing the standalone build's manual "configure base path" step, so uploads and browsing always land exactly where the running server reads them. Only reached via webatm_integrated.register (env-guarded on WEBATM_INTEGRATED=1); the core webatm package never imports it.

resolve_bluesky_workdir

resolve_bluesky_workdir() -> Path

Return BlueSky's working directory (where scenario/plugins/output live).

Deliberately mirrors the pip-package rule (~/bluesky) instead of exposing a setting, so WebATM and the BlueSky server can never point at different directories.

Returns:

Type Description
Path

BlueSky's working directory (~/bluesky).

Source code in WebATM-integrated/webatm_integrated/bluesky_paths.py
def resolve_bluesky_workdir() -> Path:
    """Return BlueSky's working directory (where scenario/plugins/output live).

    Deliberately mirrors the pip-package rule (``~/bluesky``) instead of
    exposing a setting, so WebATM and the BlueSky server can never point at
    different directories.

    Returns:
        Path: BlueSky's working directory (``~/bluesky``).
    """
    return Path.home() / "bluesky"

configure_file_management

configure_file_management(app) -> str

Pre-configure WebATM's file-management routes for the integrated build.

Sets app.bluesky_base_path -- the same attribute the standalone /api/bluesky/configure-base-path route sets, so every existing file route works unchanged -- and locks it (bluesky_base_path_locked) so that route refuses to repoint it. The managed subdirectories are best-effort created so the UI can browse and upload even before the BlueSky server's first start.

Parameters:

Name Type Description Default
app Flask

Flask application instance.

required

Returns:

Type Description
str

The configured base path (BlueSky's working directory).

Source code in WebATM-integrated/webatm_integrated/bluesky_paths.py
def configure_file_management(app) -> str:
    """Pre-configure WebATM's file-management routes for the integrated build.

    Sets ``app.bluesky_base_path`` -- the same attribute the standalone
    ``/api/bluesky/configure-base-path`` route sets, so every existing file
    route works unchanged -- and locks it (``bluesky_base_path_locked``) so
    that route refuses to repoint it. The managed subdirectories are
    best-effort created so the UI can browse and upload even before the
    BlueSky server's first start.

    Args:
        app (flask.Flask): Flask application instance.

    Returns:
        str: The configured base path (BlueSky's working directory).
    """
    workdir = resolve_bluesky_workdir()
    base_path = str(workdir)
    app.bluesky_base_path = base_path
    app.bluesky_base_path_locked = True

    try:
        workdir.mkdir(parents=True, exist_ok=True)
        for subdir in MANAGED_SUBDIRS:
            (workdir / subdir).mkdir(exist_ok=True)
    except OSError as e:
        # Non-fatal: BlueSky (re)creates these on its first start, and the
        # file routes degrade gracefully when a directory is missing.
        logger.warning(
            f"Could not pre-create BlueSky file directories under {workdir}: {e}"
        )

    logger.info(f"Integrated: BlueSky file management configured at {base_path}")
    return base_path

webatm_integrated.routes

webatm_integrated.routes

REST control routes for the integrated BlueSky server.

Namespaced under /api/integrated/ so they cannot collide with core routes and are simply absent from the default build.

register_integrated_routes

register_integrated_routes(app, manager)

Register server lifecycle-control routes on the Flask app.

Parameters:

Name Type Description Default
app Flask

Flask application instance.

required
manager BlueSkyProcessManager

Controls the bundled server.

required
Source code in WebATM-integrated/webatm_integrated/routes.py
def register_integrated_routes(app, manager):
    """Register server lifecycle-control routes on the Flask app.

    Args:
        app (flask.Flask): Flask application instance.
        manager (BlueSkyProcessManager): Controls the bundled server.
    """

    @app.route("/api/integrated/server/start", methods=["POST"])
    def integrated_server_start():
        """Start the bundled BlueSky server (POST /api/integrated/server/start)."""
        logger.info("Integrated: start BlueSky server requested")
        return jsonify(manager.start())

    @app.route("/api/integrated/server/stop", methods=["POST"])
    def integrated_server_stop():
        """Stop the bundled BlueSky server (POST /api/integrated/server/stop)."""
        logger.info("Integrated: stop BlueSky server requested")
        return jsonify(manager.stop())

    @app.route("/api/integrated/server/restart", methods=["POST"])
    def integrated_server_restart():
        """Restart the bundled BlueSky server (POST /api/integrated/server/restart)."""
        logger.info("Integrated: restart BlueSky server requested")
        return jsonify(manager.restart())

    @app.route("/api/integrated/server/kill", methods=["POST"])
    def integrated_server_kill():
        """Kill the whole BlueSky process group (POST /api/integrated/server/kill)."""
        logger.info("Integrated: kill BlueSky server requested")
        return jsonify(manager.kill())

    @app.route("/api/integrated/server/status", methods=["GET"])
    def integrated_server_status():
        """Report the server process state (GET /api/integrated/server/status)."""
        return jsonify(manager.status())

webatm_integrated.socket_handlers

webatm_integrated.socket_handlers

Socket.IO handlers for the integrated build.

Provides live-log history replay for late-joining clients. The lifecycle actions themselves are REST-only (see :mod:.routes).

register_integrated_socket_handlers

register_integrated_socket_handlers(socketio, streamer)

Register the integrated build's Socket.IO event handlers.

Parameters:

Name Type Description Default
socketio SocketIO

The Flask-SocketIO instance.

required
streamer LogStreamer

Holds the server-log history.

required
Source code in WebATM-integrated/webatm_integrated/socket_handlers.py
def register_integrated_socket_handlers(socketio, streamer):
    """Register the integrated build's Socket.IO event handlers.

    Args:
        socketio (flask_socketio.SocketIO): The Flask-SocketIO instance.
        streamer (LogStreamer): Holds the server-log history.
    """

    @socketio.on("request_log_history")
    def on_request_log_history():
        """Replay buffered server-log lines (``request_log_history`` event).

        Emits the log history to the requesting client only (flask_socketio's
        ``emit`` defaults to the sender's room), marked with ``replay: True``;
        clients de-duplicate by ``seq`` within the payload's ``stream``.
        """
        emit(
            EVENT,
            {
                "stream": streamer.stream_id,
                "lines": streamer.history(),
                "replay": True,
            },
        )