Research: how modern terminals display images inline — and what should DropTerm adopt?

Date: 2026-11-05 Question: iTerm2’s OSC 1337 inline images, kitty’s APC graphics protocol, and Sixel’s DCS bitmaps are the three ways a program puts pixels in a terminal — which of them should DropTerm implement, through which seams, and with which grid-integration model?

TL;DR

Implement the iTerm2 inline-images protocol first, as image cells living in the scrollback grid, riding the exact OSC fork seam that already carries OSC 133-D / 7 / 9;4 — then add the query-capable subset of the kitty graphics protocol (the one new parser surface the fork needs: vte 0.15 drops APC payload bytes on the floor today); defer Sixel. The iTerm2 protocol is the only one of the three that arrives as OSC — bytes vte already delivers to DropTerm’s forks — and it is the lingua franca of emitters: imgcat emits it unconditionally with no detection, WezTerm/Konsole/mintty/rio/VS Code/xterm.js all accept it, and timg prefers it even on WezTerm and Konsole. The grid-integration model with the strongest Rust precedent is WezTerm’s: every image (all three protocols) is sliced into one ImageCell per covered cell, stored inside the cell’s attributes — precisely the CellExtra seam (zerowidth/underline_color/hyperlink) that alacritty_terminal 0.25.1 already carries and that DropTerm’s fork can extend. Kitty’s protocol then reuses the same cells for its cursor-anchored placements. Sixel buys xterm/foot/mlterm-class emitters only, costs a DCS palette+RLE decoder plus the ansi-adapter hook/put/unhook forwarding the fork would have to add, and its DA1 ;4 detection is the one signal DropTerm could answer “no” to honestly for years — every surveyed terminal that ships images ships either iTerm2 or kitty first.

What DropTerm has today

Findings

1. The iTerm2 inline-images protocol (OSC 1337 File)

Primary sources: iTerm2 images docs, the shipped imgcat script, and iTerm2 source at master 40a75e3b32bbd697802135d13dc3cb78d2c65c1a (fetched 2026-11-05).

2. The kitty graphics protocol (APC G)

Primary source: docs/graphics-protocol.rst in kovidgoyal/kitty @ master (last commit touching it 3dc4f8d5aa0092b77c839145aff5602ca8664cd4, 2026-10-29; master head 3e4ffc109891, fetched 2026-11-05).

3. Sixel (DCS q)

Primary sources: xterm’s ctlseqs (https://invisible-island.net/xterm/ctlseqs/ctlseqs.html, fetched 2026-11-05) and parser sources (WezTerm, mintty, Windows Terminal); the classic vt100.net mirror was unreachable (404) on the fetch date.

4. The adoption matrix (each cell from that project’s own docs/source)

Terminal iTerm2 1337 kitty APC Sixel DCS Source for the row
kitty — ✓ (author) — graphics-protocol.rst; no sixel parser anywhere in the repo (grep, master 3e4ffc10)
WezTerm ✓ ✓ ✓ docs/imgcat.md; term/src/terminalstate/{iterm,kitty,sixel}.rs
iTerm2 ✓ (author) ✓ ✓ (3.7 beta line, 2026) VT100Terminal.m L4178+; KittyImageCommand.swift; VT100SixelParser.m
Konsole ✓ ✓ ✓ src/Vt102Emulation.cpp L643 (1337;File=:), L2411 processGraphicsToken, L693 processSixel; src/Screen.h L38-49 TerminalGraphicsPlacement_t with enum source { Sixel, iTerm, Kitty }
foot — — ✓ README “Sixel image support” (codeberg dnkl/foot master)
ghostty — ✓ — include/ghostty/vt/kitty_graphics.h; no sixel parser in tree; DA1 tests answer ?62;22c without ;4 (src/terminal/device_attributes.zig:178-185)
mintty ✓ — ✓ src/termout.c:5091 when 1337: { // iTerm2 image protocol; src/sixel.c
xterm — — ✓ (opt-in via decTerminalID/decGraphicsID) ctlseqs Sixel Graphics section
VS Code terminal ✓ ✓ (partial) ✓ xterm.js addons/addon-image README (“Supports SIXEL, iTerm’s inline image protocol (IIP) and partially Kitty’s”); wired behind terminal.integrated.enableImages and the WebGL renderer (xtermTerminal.ts:1046-1050, vscode main ae231882)
rio ✓ ✓ ✓ rio-vt/src/ansi/{iterm2_image_protocol,kitty_graphics_protocol,sixel}.rs + graphics.rs
contour ? ? ✓ docs/features.md “Sixel Image support” (iTerm2/kitty cells unverified)
GNOME Terminal / VTE — — build-time opt-in, upstream default off src/image.hh (saitoha/hpj attribution), vte.cc:3664 #if WITH_SIXEL; meson.options sixel: value: false
Ptyxis — — (follows the system VTE build) no image code in its own repo (search)
cosmic-term (closest peer) — — — no sixel/image/kitty code anywhere in pop-os/cosmic-term @ 8a05e9194881 (tree grep + code search, 2026-11-05)
Alacritty — — — issue #910 “Add support for libsixel” open since 2017-11-21 (238 comments, last activity 2026-10-06); PR #4763 “Support for graphics in the terminal” open and described by regulars as “most likely never be merged”; maintainer chrisduerr (2026-07-07): “The way this ‘should’ be implemented is not.” Note: alacritty/alacritty #3037 is not the sixel issue — it is a closed OSC52/tmux set-clipboard bug
tmux no image support of its own — passes bytes through DCS tmux; … ST behind allow-passthrough [on|off|all] (tmux(1), man7.org, fetched 2026-11-05); timg auto-runs tmux set -p allow-passthrough on (needs tmux ≥ 3.3)      

Two matrix facts that shape the verdict: iTerm2+kitty is the modern pair (every 2024+ terminal above ships at least one, and VS Code/rio/Konsole ship all three), and the closest peers ship none — cosmic-term has no image code, and VTE keeps sixel behind an off-by-default build flag.

5. How emitters pick a protocol (this is what buys working software)

Consequences for DropTerm: the iTerm2 protocol is detected through channels a new terminal can actually join (TERM_PROGRAM string, XTVERSION answer, or no detection at all with imgcat), while the kitty protocol is detected by a live query only DropTerm itself can answer. Shipping 1337 first means imgcat, wezterm imgcat, forced-mode timg/viu/chafa all work on day one; shipping kitty means kitten icat works the day the a=q round-trip lands.

6. Rust prior art: WezTerm’s one-model-for-three-protocols design

wez/wezterm @ main e68a8ec27727fbbda15e07f0a0737cd5e16481f6 (2026-10-26; file/line citations from that tree):

7. Size limits and security posture across implementors

Implementation Documented/measured ceilings
kitty 320 MB image data per buffer; animation frames on disk at 5×; temp-file path must contain tty-graphics-protocol; identical EBADF error for all read failures (anti-probing)
WezTerm 320 MiB budget in KittyImageState::prune_unreferenced (kitty.rs:33); check_image_dimensions rejects images whose “required RAM > max allowed”
xterm.js addon-image (defaults) pixelLimit 16,777,216 px per image; sixelSizeLimit/iipSizeLimit/kittySizeLimit 33,554,432 bytes per sequence; storageLimit 128 MB FIFO with placeholder-on-eviction
iTerm2 sequence cap 1 MiB (documented); no published pixel quota found
imgcat chunk mode 200-byte chunks (its own choice, tmux-safe)

Security gradient (kitty spec, its own words + matrix reality): in-band base64 over the PTY (iTerm2; kitty t=d) costs only PTY bandwidth and parser memory; file paths and shared memory (t=f/t/t=s) make the terminal a filesystem/shm reader on behalf of possibly-untrusted programs — hence kitty’s mandated refusals and uniform error. DropTerm can ship kitty t=d-only and answer a=q,e=T honestly with only t=d supported; kitten icat’s DetectSupport then picks direct mode.

Recommendation

Ship in this order:

  1. Phase 1 — iTerm2 inline images (OSC 1337 File) through the existing OSC fork arm. Parser cost is zero (vte already delivers OSC to the fork’s Handler::osc_dispatch); the arm follows the PromptExit/ CurrentDirectory/ProgressReport pattern exactly. Decode base64, decode PNG/JPEG/GIF/WebP (the image crate, as WezTerm does), resolve the documented unit grammar (N/Npx/N%/auto, preserveAspectRatio, inline), enforce ceilings (WezTerm-style dimension check + a per-image byte cap at the documented 1 MiB sequence size and a WezTerm/xterm.js-style total store, evicting oldest).
  2. Store images as cells in the scrollback grid — WezTerm’s model, through alacritty_terminal’s own CellExtra precedent: the fork grows an image field on CellExtra (like hyperlink), the placement is sliced per covered cell, and the cursor advances below the image. This makes scrollback, selection-exclusion, reflow, alt-screen clearing, and search “just work” the way text does — every protocol normalizes into the same cells, and the widget draws each covered cell with the renderer.draw_image primitive that already exists at the pinned libcosmic (iced/wgpu/src/lib.rs:863). Konsole’s placement enum and rio’s split prove the overlay model is viable too, but for DropTerm the cell model has strictly less new state to keep coherent with alacritty_terminal’s grid.
  3. Phase 2 — the kitty graphics subset through a new APC arm in the vte fork. The fork’s one genuinely new parser surface: accumulate SosPmApcString payload and dispatch it. Implement the subset the tooling actually uses and kitty’s own clients negotiate: f=32/f=100 PNG, t=d only, m=1 chunking, a=t/T/p/q/d, i/p ids, c/r sizing, z, and the a=q/e=T reply (plus the DA1 fence). Reject t=f/t/s with the spec’s uniform error — no filesystem or shm reads, the security posture iTerm2-mode already has. The 320 MB-class budget from Phase 1 is reused; placements render through the same cells. Unicode placeholders (U=1) come later if tmux/vim users ask.
  4. Phase 3 — Sixel, deferred with named triggers. Cost: forward hook/put/unhook through the ansi adapter (src/ansi.rs:1324-1340) into new Handler methods, plus a palette/RLE decoder — none of it reusable with Phases 1–2 except the final cell write. Trigger it only when a DropTerm user base shows up with sixel-only tools (gnuplot sixel, lsix) that matter to them; until then DropTerm’s DA1 simply does not report ;4, which every emitter treats as “not sixel, try something else”. iTerm2’s 3.7 and Windows Terminal’s 1.22 show the legacy channel still moves, but neither is the first protocol a modern terminal ships.
  5. Be findable. Export a TERM_PROGRAM value of DropTerm’s own, answer the CSI >q XTVERSION query with it, and answer the OSC 1337 Capabilities feature-report query (mintty’s termout.c:4753-4759 shows the reply shape). That is how timg/viuer’s string-matching detection lists grow — the same road ghostty (TERM=xterm-ghostty is special-cased in timg) and Warp (TERM_PROGRAM=WarpTerminal) took. Do not spoof another terminal’s TERM_PROGRAM.

Rejected, with reasons:

Unverified / open questions

Sources