Input

Pointer and keyboard input: clicks, drags and the wheel on any node, hover, scrolling containers, keyboard activation, and typing into a textfield, including password fields whose keys never reach Lua. A handler usually writes a named state, and the next pass shows the result.

local clicks = state("clicks", 0)

return panel {
    id = "bar",
    layer = "top",
    keyboard_interactivity = "on_demand",
    anchor = { top = true },
    child = rect {
        padding = 8,
        background = "#313244",
        accessible_name = "Increase clicks",
        on_click = function(rect, which)
            if which == "left" then clicks:set(clicks:get() + 1) end
        end,
        children = { text { content = clicks:map(function(n) return "Clicked " .. n end) } },
    },
}

Hit testing

Every pointer event asks which nodes lie under the pointer, from the surface down.

Pointer

Any node takes clicks, drags and the wheel, hit-tested by its own box. For each event the innermost node with a handler for that event wins; a node without one is transparent, so a handle inside a draggable track leaves the track draggable, and a text with no handler passes the click to the row around it. A node with a handler or submits_secure = true shows the "pointer" cursor unless it sets its own cursor.

Click. A press arms the click and the release fires it. Leaving the node and coming back before release still clicks; the pointer leaving the surface cancels. The click also cancels if the node’s laid-out box moved between press and release, so give press feedback with scale or translate rather than width or margin. A textfield clicks only as the on_click row says. A link in a text (on_link) takes the click before any on_click, the text’s own included.

Double click. on_click fires for both clicks, then on_double_click for the second; a third click starts a new pair, so a triple click is one double and one single. The innermost node with on_double_click takes it, apart from on_click’s target: an icon with only on_double_click inside a row with on_click still lets each single click reach the row. The pair is the same node even if the first click moved or resized it. A left click on a textfield goes to the field and never doubles; its non-left clicks double on its own handler. Enter, Space and screen-reader activation never double-click, and a node with only on_double_click takes no keyboard focus. Wayland has no portable double-click setting, so the 400 ms and 4 px are fixed.

Modifiers. Each pointer handler takes modifiers after its own arguments (only on_drag’s velocity follows it), the { ctrl, shift, alt, super } booleans on_key reports, so a list can toggle on Ctrl-click or extend on Shift-click. Enter and Space activation report the keyboard’s modifiers too. Wayland tells a client the modifiers only while one of its surfaces has keyboard focus; otherwise all four read false.

local picked = state("picked", false)
return panel {
    id = "bar",
    layer = "top",
    anchor = { top = true },
    child = rect {
        padding = 8,
        background = picked:map(function(on) return on and "#89b4fa" or "#313244" end),
        on_click = function(_, _, _, modifiers)
            picked:set(modifiers.ctrl and not picked:get() or not modifiers.ctrl)
        end,
        children = { text { content = "Ctrl-click toggles" } },
    },
}

Press. on_press is the one place a callback can start a window move, resize or menu with toplevel(id): the compositor checks those requests against the press, and on_click fires on release, too late. An on_drag "start" works too. After the compositor takes the pointer it may send no release, so the engine ends the drag and the armed click; on_drag gets no "end" and on_click does not fire.

Drag. A left press on an on_drag node calls "start" at once, so clicking a slider track also seeks. Every pointer motion on that surface then calls "move", wherever the pointer is. "end" comes on the left release, when the pointer leaves the surface, or when the surface closes. rect stays the box from the press for the whole drag, but pointer is measured against the node’s current box, so a node that moves mid-drag shifts it. On release, "end" fires first and the click (if the node also has on_click) after it; a leave ends the drag and cancels the click.

velocity is how fast the pointer moved over the last 100 ms, in surface logical px per second, timed by the compositor’s event clock rather than when the shell got round to the event. It is { x = 0, y = 0 } on "start", and on "end" when the pointer was still for 100 ms before the release. Motion spanning under 8 ms keeps the last measured value, so a 1 px twitch after a rest does not read as a flick. A drag that ends without a release (the pointer leaving the surface, the surface closing, a compositor move) reports the last motion’s velocity, so a flick off the edge keeps its speed. The velocity on "end" is what a flick scroll keeps going with. mantle input drag reports zero.

Wheel. steps is positive away from the user (scroll up) and negative toward. One notch is 1; high-resolution wheels send fractions of a notch, and touchpads send distance divided by one notch’s 39 px. Horizontal motion never reaches on_wheel. The innermost on_wheel node or scroll container under the pointer takes the whole event, with no chaining to a parent; on a node that is both, the scroll wins.

local level = state("brightness", 0.5)
local function clamp(value) return math.max(0, math.min(1, value)) end
local percent = level:map(function(value) return string.format("%d%%", math.floor(value * 100 + 0.5)) end)

return panel {
    id = "bar",
    layer = "top",
    anchor = { top = true },
    child = row {
        padding = 14,
        spacing = 12,
        radius = 12,
        background = "#1e1e2e",
        children = {
            icon { name = "display-brightness-symbolic", size = 18, foreground = "#f9e2af", align_v = "center" },
            rect {
                width = 200,
                height = 10,
                radius = 5,
                align_v = "center",
                clip = "rounded",
                background = "#45475a",
                -- A press is "start", so clicking the track also seeks.
                on_drag = function(rect, pointer, phase) level:set(clamp(pointer.x / rect.width)) end,
                on_wheel = function(_, steps) level:set(clamp(level:get() + steps * 0.05)) end,
                children = { rect { height = "fill", background = "#f9e2af", width = percent } },
            },
            text { content = percent, width = 36, foreground = "#cdd6f4", align_v = "center" },
        },
    },
}

Hover

hover(name) returns a read-only boolean signal, false until the pointer first arrives. Bind it to a node’s hover and the engine writes it as the pointer moves; read the same signal anywhere else to react. The name is the identity: every hover("wifi") call returns the same signal, and it survives reloads.

local over = hover("wifi")

return panel {
    id = "bar",
    layer = "top",
    anchor = { top = true },
    child = rect {
        padding = 6,
        radius = 6,
        hover = over,
        background = over:map(function(on) return on and "#45475a" or "#313244" end),
        on_hover = function(inside) log.debug("wifi hovered:", inside) end,
        on_click = function() process.detach("nm-connection-editor", {}) end,
        children = { icon { name = "network-wireless-symbolic", size = 16 } },
    },
}

Pointer position

pointer(name) returns a read-only signal of the pointer’s { x, y } in logical pixels from the top-left corner of one node, or nil while the pointer is outside it. Bind it to a node’s pointer; the name is the identity, as for hover.

A pointer read only inside a callback (on_click, on_press, on_key) is always nil: nothing read it, so motion never wrote it. Bind it to a node’s pointer, or read it in a computed that a node reads.

local at = pointer("canvas")

return panel {
    id = "bar",
    layer = "top",
    anchor = { top = true },
    child = rect {
        width = 200,
        height = 60,
        pointer = at,
        children = {
            rect {
                width = 8,
                height = 8,
                radius = 4,
                background = "#f5c2e7",
                visible = at:map(function(p) return p ~= nil end),
                translate = at:map(function(p) return { x = p and p.x - 4 or 0, y = p and p.y - 4 or 0 } end),
            },
        },
    },
}

Scroll

scroll(name) returns a read-only signal holding a scroll offset in px, 0 at first. Bind it to the scroll property of a row, column or list and the wheel moves that container’s children. Like hover, the name is the identity and survives reloads.

Arrow buttons for a carousel; strip:scroll_to(0) would rewind it:

local strip = scroll("carousel")
local cards = {}
for i = 1, 12 do cards[i] = rect { width = 120, height = 80, radius = 8, background = "#313244" } end
local function arrow(label, step)
    return rect {
        padding = 6,
        accessible_name = label,
        on_click = function() strip:scroll_by(step) end,
        children = { text { content = label } },
    }
end

return panel {
    id = "carousel",
    layer = "top",
    child = row { spacing = 8, children = {
        arrow("<", -128),
        row { width = 376, spacing = 8, scroll = strip, animate = { scroll = 200 }, children = cards },
        arrow(">", 128),
    } },
}

Keyboard controls and accessibility

A node with on_click, submits_secure = true, on_key or on_key_release becomes a keyboard control when it has a nonempty accessible_name. The name is announced through AT-SPI. Give every textfield an accessible_name too; fields take focus through their existing callbacks or secure_submit. Keys reach controls only on the surface with keyboard focus. Panels use keyboard_interactivity = "on_demand" or "exclusive"; shown popups under that surface share its focus scope.

Tab moves to the next control in document order across the focused surface and its shown popups. Shift+Tab moves backward, and both wrap. Hidden, leaving and zero-size controls are skipped. Tab on the only control, while it holds focus, moves nothing, shows the outline and reaches on_key. Enter or Space activates a named control once per press, calling on_click(rect, "left", pointer) with pointer at the node’s centre, and running submits_secure when set. Pointer presses also focus named controls.

A panel, window or popup takes on_escape() for dismissing without a field. It fires once per Escape press (not on repeat) on the innermost shown popup under the focused surface that declares it (no order is promised among sibling popups), else the surface itself, and never on a surface without keyboard focus. A focused field keeps its own Escape while it has text to clear (or a composition) or declares on_cancel; otherwise Escape reaches on_escape, so a launcher can clear on the first Escape and close on the second.

local open = state("menu_open", true)

return panel {
    id = "menu",
    layer = "overlay",
    anchor = { top = true },
    keyboard_interactivity = "on_demand",
    visible = open,
    on_escape = function() open:set(false) end,
    child = text { content = "Menu" },
}

Key handlers

on_key(key) on any node or surface hears the keys of a custom control: a grid, a menu, a slider. key is { name, text, modifiers = { ctrl, shift, alt, super }, repeat }.

A key goes to the focused node first, then each ancestor up to the surface, which hears it last whether or not it holds a control. A handler that returns true stops it; returning nothing or false passes it up. A node without on_key is skipped. Without focus on a control, only the surface’s own on_key hears keys, while the surface has the keyboard. A control’s on_key runs before its Enter/Space activation, and a handled Escape never reaches on_escape.

A node with on_key takes Tab focus, a press and the outline like any named control, but only with an accessible_name, so a screen reader can announce it. Without one it is not focusable, and hears just what its descendants pass up. That is how a container sees the keys of the field inside it.

A focused plain textfield takes the keys it edits with (characters, Backspace, Delete, Enter, caret motion, Ctrl+A/Z/Y, and Escape, unless escape = "pass" or there is nothing to clear and no on_cancel) and passes the rest up: Up, Down, paging, Tab when the field is the only control, an arrow at the caret’s edge, F-keys and other Ctrl chords (Ctrl+C, Ctrl+X and Ctrl+V copy, cut and paste while a field takes them, and reach on_key otherwise). Tab moves control focus when it can, so on_key hears Tab only where nothing moves. Repeats arrive with repeat = true.

A key typed into a secure_submit field never reaches on_key, and while one is armed no on_key on that surface is called.

on_key_release(key) hears a key going up, routed the same way: focused node first, then each ancestor, then the surface; true stops it. key is the press’s name and text with the modifiers held at release, and no repeat. A release is heard only when its press was offered to on_key, whether or not a handler took it, and goes to the node that press was routed from even if the press moved focus: never for a key a field typed or pasted with, a Tab that moved focus, a key typed into an armed secure_submit field, a bare modifier, a key already held when the keyboard entered, or after the keyboard left. Like on_key, it makes a node focusable only with an accessible_name. mantle input key sends the release right after the press.

local picked = state("picked", 1)

return panel {
    id = "grid",
    layer = "top",
    keyboard_interactivity = "on_demand",
    child = row {
        accessible_name = "Picker",
        on_key = function(key)
            if key.name == "Right" and not key.modifiers.ctrl then
                picked:set(picked:get() + 1)
                return true
            elseif key.name == "Left" then
                picked:set(math.max(1, picked:get() - 1))
                return true
            end
        end,
        children = { text { content = picked:map(tostring) } },
    },
}

The engine draws a black and white outline around the focused control only when Tab, Shift+Tab or an assistive-technology action moved focus there. Focus from a press, autofocus or focus_target(name):request() from a click draws none, and a press hides an outline Tab drew. focus_ring = false keeps the outline off a node. The outline follows the node’s radius, corner_smoothing and shape.

focused(name) returns a read-only boolean signal, false until the bound node first holds focus. Bind it to a node’s focused and the engine sets it true while that node or any node inside it holds control focus, however focus got there. Like hover(name), the name is the identity and survives reloads. It is not focus_target(name), the handle :request() uses to move focus.

focus_visible(name) is the same signal under the rule the engine’s outline follows (CSS :focus-visible): true only while the bound node or one inside it holds control focus that Tab, Shift+Tab, an assistive-technology action or a :request() from a key callback or an Enter or Space activation moved there, and false after a press, autofocus or a :request() from a click, while focused(name) stays true. It ignores focus_ring. Bind it to a node’s focus_visible, and draw a ring that shows for keyboard users only:

local search = focused("search")

return panel {
    id = "launcher",
    layer = "top",
    keyboard_interactivity = "on_demand",
    child = rect {
        padding = 6,
        radius = 8,
        border_width = 2,
        focused = search,
        border_color = search:map(function(on) return on and "#89b4fa" or "#45475a" end),
        children = {
            textfield {
                width = 240,
                height = 28,
                autofocus = true,
                focus_ring = false,
                accessible_name = "Search",
                on_change = function(text) end,
            },
        },
    },
}

local ring = focus_visible("save")

return panel {
    id = "dialog",
    layer = "top",
    keyboard_interactivity = "on_demand",
    child = rect {
        padding = 4,
        radius = 8,
        border_width = 2,
        focus_visible = ring,
        border_color = ring:map(function(on) return on and "#89b4fa" or "#00000000" end),
        children = {
            rect {
                padding = 6,
                background = "#45475a",
                accessible_name = "Save",
                focus_ring = false,
                on_click = function() end,
                children = { text { content = "Save" } },
            },
        },
    },
}

Mantle publishes the resolved node tree through AT-SPI. Text nodes expose their displayed text; plain fields expose their draft. Secure fields expose a password role but no value or length. Assistive-technology focus and click actions use the same control path as keyboard input. A screen reader can also click a button on a surface without keyboard focus; the keyboard stays where it is.

Text fields

A textfield is a text input, one line unless multiline. The engine holds what the user types (the draft); Lua sees it through callbacks, and sets it only with :set_text. The field with focus is the one keys go to. It has no intrinsic width, so give it width; height defaults to one line (nodes).

A field takes the keyboard only when both hold:

A press on the field focuses it and puts the caret under the pointer. autofocus focuses it without a press.

Selection and clipboard. Dragging or Shift+clicking with the pointer selects too. A double-click selects the word under the pointer (a space run or a single punctuation mark when it lands on one), and a triple-click the line: newline to newline in a multiline field, all the text in a single-line one. Presses within 400 ms and 4 logical pixels of the last on the same field form a series, and a fourth starts over as a single click. Dragging or Shift+clicking after a double or triple click grows the selection by whole words or lines. Paste accepts up to 64 KiB of valid UTF-8 without control characters (a multiline field also takes newlines; \r\n and \r become \n). A paste is dropped if the selection, field, or keyboard focus changes before the read ends. Copy and cut work only with a plain-field selection. has_selection(name) is a read-only boolean signal, false until the plain field whose focus_target is name holds the keyboard and has text selected. It survives reloads. Editing keys repeat while held; Escape, undo and redo do not. The undo stack retains at most 100 snapshots and 1 MiB of saved text. Submit, leaving the field, and fresh autofocus clear that history. When text-input-v3 enters the field’s own surface, composition appears underlined. Preedit alone does not call on_change; committed text and surrounding deletions do. Raw typing resumes when composition ends. A pointer press or Escape discards uncommitted composition. Secure fields do not use an IME.

Draft lifetime. Each field keeps its own draft. Clicking elsewhere, moving to another field, or the surface losing the keyboard stops typing but keeps the draft; focusing the field again resumes it, though a click that returns to a field puts the caret where it was left. Undo history does not survive leaving a field. Enter and Escape clear it. An autofocus arm starts it empty. It is dropped when the field’s node leaves the tree or its surface closes. Restoring a draft is silent: on_change does not fire, so reset a field with set_text.

To return typing to a field after a click changes the view, give the field the handle and call :request() from the on_click. focus_target("") and a focus_target property that is not a handle raise. on_key, and edits typed or committed into a field (on_change, on_submit, on_cancel), can request too; the on_change an autofocus arm or set_text fires cannot. Requests from anywhere else, to a hidden or masked field, or to a surface other than the focused one and its popups do nothing.

local search_focus = focus_target("search")

return panel {
    id = "search",
    layer = "top",
    keyboard_interactivity = "on_demand",
    child = row { children = {
        textfield {
            focus_target = search_focus,
            autofocus = true,
            width = 180,
            height = 32,
            on_change = function() end,
        },
        rect {
            on_click = function() search_focus:request() end,
            children = { text { content = "Return to search" } },
        },
    } },
}

A Cut button that shows whether it can act:

local note = focus_target("note")
local selected = has_selection("note")

return panel {
    id = "note",
    layer = "top",
    keyboard_interactivity = "on_demand",
    child = row { children = {
        textfield { focus_target = note, width = 180, height = 32, on_change = function() end },
        rect {
            on_click = function()
                note:request()
                note:cut()
            end,
            children = { text { content = selected:map(function(on) return on and "Cut" or "Nothing selected" end) } },
        },
    } },
}

A right-click menu on a field. The press focuses the field without clearing a selection it lands in, so :cut() acts on it:

local note = focus_target("note")
local menu = state("note_menu", false)
return panel {
    id = "note",
    layer = "top",
    keyboard_interactivity = "on_demand",
    child = column { children = {
        textfield {
            focus_target = note,
            width = 180,
            height = 32,
            on_change = function() end,
            on_click = function(_, button)
                if button == "right" then menu:set(true) end
            end,
        },
        row {
            visible = menu,
            children = {
                rect {
                    on_click = function() note:request(); note:cut(); menu:set(false) end,
                    children = { text { content = "Cut" } },
                },
            },
        },
    } },
}

focus_target and autofocus also work on any focusable control. A request from a key callback shows the outline, so Lua can rove focus with the arrow keys. On a node that takes no focus they do nothing, and set_text does nothing on a control.

local next_button = focus_target("next")

return panel {
    id = "toolbar",
    layer = "top",
    keyboard_interactivity = "on_demand",
    child = row { children = {
        rect {
            accessible_name = "First",
            autofocus = true,
            width = 40,
            height = 32,
            on_key = function(key)
                if key.name == "Right" then next_button:request() return true end
            end,
        },
        rect { accessible_name = "Next", focus_target = next_button, width = 40, height = 32, on_key = function() end },
    } },
}

A search field that picks from a list moves the selection with on_key and closes on a second Escape. The app launcher recipe is the full version.

local query = state("query", "")
local selected = state("selected", 1)
local open = state("launcher_open", true)

return textfield {
    width = 240,
    height = 32,
    placeholder = "Search apps",
    autofocus = true,
    on_change = function(text)
        query:set(text)
        selected:set(1)
    end,
    on_key = function(key)
        local step = ({ Up = -1, Down = 1 })[key.name]
        if step then selected:set(math.max(1, selected:get() + step)) end
    end,
    -- First Escape clears the text; a second one, on an empty field, closes.
    on_cancel = function(cleared)
        if not cleared then open:set(false) end
    end,
}

Secure fields

secure_submit = { capability, action } turns a textfield into a password field. Its keys go to a native buffer and leave the Renderer as one message to that target; no Lua value ever holds the secret or its length.

return lock {
    id = "lock",
    child = column {
        width = "fill",
        height = "fill",
        align_h = "center",
        align_v = "center",
        spacing = 12,
        background = "#11111b",
        children = {
            textfield {
                width = 280,
                height = 40,
                placeholder = "Password",
                accessible_name = "Password",
                mask_character = "•",
                secure_submit = { capability = "lock", action = "authenticate" },
            },
            rect {
                padding = 8,
                radius = 8,
                background = "#89b4fa",
                submits_secure = true,
                accessible_name = "Unlock",
                children = { text { content = "Unlock", foreground = "#11111b" } },
            },
        },
    },
}

How do I…

Right-click menu. on_click reports the mouse button and the node’s rect, which is what a popup’s anchor_rect wants. The click is a pointer release, so the popup may take its grab. The menu hangs from the node, not the click point: no handler reports the pointer position of a click.

local menu_open = state("context_open", false)
local menu_at = state("context_at", { x = 0, y = 0, width = 1, height = 1 })

local function item(label, run)
    return rect {
        width = "fill",
        padding = 8,
        radius = 6,
        on_click = function()
            menu_open:set(false)
            run()
        end,
        children = { text { content = label, foreground = "#cdd6f4" } },
    }
end

local files = row {
    padding = 8,
    spacing = 8,
    radius = 8,
    background = "#1e1e2e",
    on_click = function(rect, which)
        if which == "right" then
            menu_at:set(rect)
            menu_open:set(true)
        else
            process.detach("nautilus", {})
        end
    end,
    children = {
        icon { name = "folder", size = 18, align_v = "center" },
        text { content = "Files", foreground = "#cdd6f4", align_v = "center" },
    },
}

return {
    panel { id = "bar", layer = "top", anchor = { top = true }, child = files },
    popup {
        id = "files_menu",
        parent = "bar",
        anchor_rect = menu_at,
        anchor = "bottom",
        gravity = "bottom_right",
        offset = { y = 4 },
        visible = menu_open,
        on_dismiss = function() menu_open:set(false) end,
        child = column {
            width = 160,
            padding = 4,
            radius = 10,
            background = "#1e1e2e",
            border_width = 1,
            border_color = "#45475a",
            children = {
                item("New window", function() process.detach("nautilus", { "--new-window" }) end),
                item("Downloads", function() process.detach("nautilus", { os.getenv("HOME") .. "/Downloads" }) end),
            },
        },
    },
}

Drag to reorder. Each row is an on_drag node in a keyed list. pointer is measured against the row’s current slot, so pointer.y divided by the row pitch counts the rows to move from where it is now; offsetting from the slot at the press would undo each move on the next motion. The dragged row jumps slot by slot rather than following the pointer; to make it follow, also bind its translate to the drag offset. There is no drag-and-drop between surfaces or applications, and no drag image.

local items = state("order", { "Music", "Mail", "Files", "Terminal" })
local STEP = 32 -- row height 28 + spacing 4

local function index_of(name)
    for index, other in ipairs(items:get()) do
        if other == name then return index end
    end
end

local function row_for(name)
    return rect {
        width = 200,
        height = 28,
        padding = 6,
        background = "#313244",
        cursor = "grab",
        on_drag = function(_, pointer, phase)
            if phase ~= "move" then return end
            -- `pointer` is relative to the row's current slot, so the offset counts rows from there.
            local order = { table.unpack(items:get()) }
            local now = index_of(name)
            local target = math.max(1, math.min(#order, now + math.floor(pointer.y / STEP)))
            if target ~= now then
                table.insert(order, target, table.remove(order, now))
                items:set(order)
            end
        end,
        children = { text { content = name } },
    }
end

return panel {
    id = "bar",
    layer = "top",
    anchor = { top = true },
    child = list { spacing = 4, items = items, each = row_for, key = function(name) return name end },
}

Gotchas

See also: nodes (textfield, list), surfaces (keyboard_interactivity, popups, lock), signals (state the handlers write), animation (press and hover motion), capabilities (lock, polkit, network).

Source: pointer, press and click, callbacks, wheel, keyboard, text fields, secure fields, hit testing, hover, scroll.