Modal Assistant

Bots that ship, debug and tune your Modal apps.

Ask in plain words. A team of bots runs the modal commands, reads your logs, metrics and bill, and comes back with the answer and the fix. Nothing in your workspace changes until you approve it.

One request, recorded live: a new bot writes arcade-fifty, you approve the deploy, and 50 Sandboxes each play a different game. Sped up only while bots wait.

The work it takes off your plate.

Six real requests on a demo workspace, each with what the bot showed back. Under each one are the Modal commands it wrote and ran.

  1. Ship a new app from one request

    “Make a new bot called Game Shipper that builds and ships Modal apps. Its first job: an app called arcade-fifty where each of 50 Sandboxes plays a different game…”

    It wrote and tested the app, deployed it once you approved, and started all 50 games.

    arcade-fifty in Game Shipper’s browser: 50 Sandboxes, each playing a different Gymnasium or MiniGrid game, all live
    Ran 1 Modal command
    1. modal deploy bots/3570/arcade/app.py -e can-devafter you approved

    Looked up modal.Sandbox.create and modal.Dict.put in the Modal docs before writing the app.

  2. Know which Apps need you

    “Chart end-to-end p99 latency for each demo-* app over the last 24 hours, in one chart, and tell me in one line which one needs attention.”

    demo-gpu-burn comes first: some of its calls wait minutes for a GPU.

    Moody’s chart of end-to-end p99 latency for three demo apps over 24 hours, with demo-gpu-burn spiking above 200 seconds
    Ran 3 Modal commands
    1. modal function metrics demo-llm-api/generate --group calls --since 24h --bucket-size 1h --json -e can-dev
    2. modal function metrics demo-gpu-burn/burn --group calls --since 24h --bucket-size 1h --json -e can-dev
    3. modal function metrics demo-batch-embed/embed_batch --group calls --since 24h --bucket-size 1h --json -e can-dev

    Read the CLI reference for modal function metrics first.

  3. Get the root cause, not just the error

    “Find the app here with the most recent errors, read its logs, and tell me the root cause and the fix.”

    First tokens slow to 1.7 s every 20 minutes. The cause: no warm containers. The fix: one line.

    Incident Responder’s chart of demo-llm-api’s mean time to first token over 12 hours, spiking every 20 minutesThe log viewer on demo-llm-api’s worst burst: first tokens of 1,100 to 2,400 ms, then back under 350 ms
    Ran 10 Modal commands
    1. modal app list --json -e can-dev
    2. modal app logs demo-gpu-burn --since 12h -e can-dev --timestamps
    3. modal app logs demo-batch-embed --since 12h -e can-dev --timestamps
    4. modal app logs demo-llm-api --since 12h -e can-dev --timestamps
    5. modal app info demo-llm-api -e can-dev
    6. modal app logs demo-llm-api --since 20m -e can-dev --timestamps
    7. modal function autoscaler settings demo-llm-api/generate -e can-dev
    8. modal function info demo-llm-api/generate -e can-dev
    9. modal function stats demo-llm-api/generate --since 12h --json -e can-dev
    10. modal app logs demo-llm-api --since 2026-11-09T20:19:00Z --until 2026-11-09T20:27:00Z -e can-dev --timestamps
  4. Spend less on Modal

    “Looking only at the demo-* apps: what does each one cost per day, and what concrete changes would cut that? Give estimated monthly savings.”

    About $440 a month today, and about $325 of it can go without stopping anything.

    Cost Watcher’s chart of each demo app’s daily cost, Nov 3 to 9Cost Watcher’s chart of each demo app’s monthly cost today against after its suggested changes
    Ran 13 Modal commands
    1. modal app info demo-gpu-burn -e can-dev
    2. modal app info demo-batch-embed -e can-dev
    3. modal app info demo-llm-api -e can-dev
    4. modal app info demo-flaky-ingest -e can-dev
    5. modal function info demo-llm-api/complete -e can-dev
    6. modal function info demo-batch-embed/embed_batch -e can-dev
    7. modal function info demo-gpu-burn/burn -e can-dev
    8. modal environment billing report can-dev --start 2026-11-09 --resolution h --json
    9. modal function stats demo-gpu-burn/burn --since 6h --json -e can-dev
    10. modal function stats demo-llm-api/traffic --since 6h --json -e can-dev
    11. modal function stats demo-batch-embed/embed_batch --since 6h --json -e can-dev
    12. modal function stats demo-batch-embed/resize_thumbnails --since 6h --json -e can-dev
    13. modal function stats demo-llm-api/generate --since 6h --json -e can-dev
  5. Right-size every Function

    “Which functions here have slow cold starts or more GPU or memory than they use? Tell me what you would change and why.”

    embed_batch asks for 4 CPUs and 8 GiB, and uses 0.02 cores and 0.33 GiB.

    Throughput Doctor’s chart of requested against used CPU for four demo functions; embed_batch requests 4 cores and uses almost noneThe same for memory: embed_batch requests 8 GiB and uses 0.33
    Ran 15 Modal commands
    1. modal app list --json
    2. modal app info demo-batch-embed
    3. modal app info demo-gpu-burn
    4. modal function info demo-llm-api/generate --json
    5. modal function stats demo-llm-api/generate --since 12h --json
    6. modal function info demo-llm-api/complete --json
    7. modal function stats demo-llm-api/complete --since 12h --json
    8. modal function info demo-llm-api/traffic --json
    9. modal function stats demo-llm-api/traffic --since 12h --json
    10. modal function info demo-batch-embed/embed_batch --json
    11. modal function stats demo-batch-embed/embed_batch --since 12h --json
    12. modal function info demo-batch-embed/resize_thumbnails --json
    13. modal function stats demo-batch-embed/resize_thumbnails --since 12h --json
    14. modal function info demo-gpu-burn/burn --json
    15. modal function stats demo-gpu-burn/burn --since 12h --json
  6. Catch problems while you’re away

    “Set up a routine that checks demo-gpu-burn every 15 minutes: write a check script … that fails when the GPU sat under 20% utilization for most of its last run.”

    It wrote the check, proved it on a real run, and has passed 50 runs since. A failure wakes it to tell you.

    The routine page for demo-gpu-burn’s GPU utilization check: every 15 minutes, last run passed, 50 runs, all green
    Ran 1 Modal command
    1. modal function metrics demo-gpu-burn/burn --since 60m --bucket-size 1m --jsonin check.py, every 15 min

    The bot wrote check.py around this command. A pass is recorded quietly and calls no model.

Watch the full film.

Three and a half minutes in the product: routines, a chief of staff, memory, approvals, and a new bot shipping an app.