Input & Motion

Respond to touches and key presses with functions such as touched(touch) and keyPressed(press). You can also check what is happening in draw(): key.pressing(key.space) tells you whether Space is held, and mouse.x gives the pointer position.

Touch and keyboard events are delivered before draw() runs, so the state you read in draw() is up to date for that frame.

Touches

Define a global touched(touch) function to receive touches. It is called each time a touch changes: when a finger goes down, every time it moves, and when it lifts. On a Mac, or an iPad with a mouse or trackpad, a click arrives as a touch of type touch.pointer.

local touches = {}

function touched(touch)
    if touch.ended or touch.cancelled then
        touches[touch.id] = nil
    else
        touches[touch.id] = touch.pos
    end
end

function draw()
    background(40)
    fill(255, 100, 100)
    for id, pos in pairs(touches) do
        ellipse(pos.x, pos.y, 60, 60)
    end
end

Each touch provides:

• touch.id, a number that stays the same for the life of the touch
• touch.x, touch.y and touch.pos, the current position
• touch.prevX, touch.prevY and touch.prevPos, the position at the previous event
• touch.deltaX, touch.deltaY and touch.delta, the movement since the previous event
• touch.began, touch.changed, touch.ended and touch.cancelled, which are true in the matching phase (touch.moving is the same as touch.changed)
• touch.state, the phase as one of BEGAN, CHANGED, ENDED or CANCELLED (MOVING is the same value as CHANGED)
• touch.tapCount, the number of quick taps in a row
• touch.type, one of touch.direct, touch.indirect, touch.pencil or touch.pointer

Positions are in screen coordinates with the origin at the bottom left. Apple Pencil touches also report force, maxForce, azimuth, altitude, precisePos and precisePrevPos.

To poll instead of using the callback, read touch.current. It holds the first touch that is down, and keeps its last state after it ends.

function draw()
    background(40)
    local t = touch.current
    if t.began or t.changed then
        fill(255)
        ellipse(t.x, t.y, 80, 80)
    end
end

Touching Entities

Entities in scene.main receive touches through a touched method. Return true when the touch begins to keep receiving it until it ends. A touch kept by an entity is not passed to the global touched function.

function setup()
    scn = scene()
    scene.main = scn

    local e = scn:entity()
    function e:touched(touch)
        if touch.began then
            print("began", touch.x, touch.y)
        elseif touch.ended then
            print("ended")
        end
        return true
    end
end

Without hit testing, every entity with a touched method is offered every new touch. Set entity.hitTest = true to offer it only touches that land on its 2D or 3D physics body, as seen by the scene’s camera. For a 3D body the method also receives the raycast hit as a second argument.

Inside a callback, touch:makeExclusive(entity) gives a touch to that entity alone, and touch:cancel(scene) ends it for every entity in the scene that holds it.

Gestures

Gesture recognizers use the system’s tap, pan, pinch and rotation recognizers and call a function with a gesture when they fire. Keep a reference to each recognizer you create. It stops working once it is garbage collected.

function setup()
    zoom = 1
    offset = vec2(0, 0)

    pinch = gesture.pinch(function(g)
        zoom = zoom * g.pinchDelta
    end)

    pan = gesture.pan(function(g)
        offset = offset + g.delta
    end, 2, 2)

    doubleTap = gesture.tap(function(g)
        zoom = 1
        offset = vec2(0, 0)
    end, 2)
end

function draw()
    background(40)
    fill(100, 180, 255)
    local size = 120 * zoom
    rect(WIDTH/2 + offset.x - size/2, HEIGHT/2 + offset.y - size/2, size, size)
end

A two finger pan moves the square, a pinch scales it, and a double tap resets it. By default a touch is handled by one recognizer at a time. Give recognizers the same non-zero group to let them run together, for example pinch.group = 1 and pan.group = 1.

Keyboard

Hardware keyboards are supported on iPhone, iPad and Mac. The on-screen keyboard is described below.

Read key state from draw() with the key functions. key.pressing is true while a key is held. key.wasPressed and key.wasReleased are true only in the frame where the key went down or came up.

function setup()
    pos = vec2(WIDTH/2, HEIGHT/2)
end

function draw()
    background(40)
    local speed = 300 * time.delta
    if key.pressing(key.left) then pos.x = pos.x - speed end
    if key.pressing(key.right) then pos.x = pos.x + speed end
    if key.pressing(key.up) then pos.y = pos.y + speed end
    if key.pressing(key.down) then pos.y = pos.y - speed end
    if key.wasPressed(key.space) then print("jump") end

    fill(255)
    ellipse(pos.x, pos.y, 60, 60)
end

To react to each key as it happens, define keyPressed(press) and keyReleased(press). The key.press passed in holds the keyCode, the modifierFlags held at the time, and the characters the key typed. Use characters for shortcuts and individual key actions. For editable text, use keyboard.text as shown below: it handles Unicode, autocorrection, pasting, and composed characters without slicing UTF-8 strings yourself.

keyPressed and keyReleased are not called while a user interface text field has focus.

Modifier keys are reported as a bit field. key.modifiersPressed(key.cmd) is true while either Command key is held. Combine masks with | to require several at once:

function keyPressed(press)
    if press.keyCode == key.s and key.modifiersPressed(key.cmd | key.shift) then
        print("save as")
    end
end

On-Screen Keyboard

Call keyboard.show() to bring up the on-screen keyboard and keyboard.hide() to put it away. keyboard.visible is true while it is on screen. Touching the screen does not dismiss it.

What the player types collects in keyboard.text, including autocorrected and pasted text and the characters of languages such as Korean, which are composed as they are typed. Each insertion also calls keyPressed, so code written for a hardware keyboard keeps working.

function setup()
    keyboard.show()
end

function keyPressed(press)
    if press.keyCode == key.enter then
        print("You typed " .. keyboard.text)
        keyboard.text = ""
    end
end

function draw()
    background(40)
    fill(255)
    text(keyboard.text, WIDTH/2, HEIGHT - 100)
end

Mouse and Trackpad

On a Mac, or an iPad with a mouse or trackpad, the global mouse holds the pointer state. mouse.active is true while a mouse or trackpad is connected.

function draw()
    background(40)
    if mouse.left.pressed then
        fill(255, 200, 0)
    else
        fill(255)
    end
    ellipse(mouse.x, mouse.y, 40, 40)
end

mouse.x and mouse.y follow the pointer while it hovers and while a button is held. mouse.delta and mouse.scroll hold the movement and scrolling since the last frame. Set mouse.visible = false to hide the system pointer over the viewer.

Two callbacks report pointer activity. hover(gesture) is called as the pointer moves over the viewer with no button held, and scroll(gesture) is called for trackpad and scroll wheel scrolling. Both receive a gesture whose location is the pointer position. For scroll, gesture.delta is the scroll movement since the previous call, with positive y meaning up.

local scrollY = 0
local hoverX, hoverY = 0, 0

function scroll(gesture)
    scrollY = scrollY + gesture.delta.y
end

function hover(gesture)
    hoverX, hoverY = gesture.location.x, gesture.location.y
end

function draw()
    background(40)
    fill(255)
    ellipse(hoverX, hoverY + scrollY, 40, 40)
end

Gamepads

Game controllers connected to the device are listed in gamepad.all. gamepad.current is the controller used most recently, or nil if none is connected.

function setup()
    pos = vec2(WIDTH/2, HEIGHT/2)
    gamepad.connected = function(pad)
        print("connected", pad.vendorName)
    end
end

function draw()
    background(40)
    local pad = gamepad.current
    if pad then
        pos = pos + pad.leftStick.dir * 300 * time.delta
        if pad.a.pressed then print("A") end
    end
    fill(255)
    ellipse(pos.x, pos.y, 60, 60)
end

Controllers already connected when the project starts are in gamepad.all but do not trigger gamepad.connected.

Each button has pressing, which is true while it is held, pressed and released, which are true only in the frame it changed, and an analog value. Sticks, the directional pad and the touchpad surface report a dir vector along with x, y, left, right, up and down.

gamepad.virtual returns an on-screen gamepad with a stick at the bottom left and an A button at the bottom right, controlled by touch. While a controller is connected it is hidden and mirrors that controller, so a project can read the same buttons either way:

function setup()
    pad = gamepad.virtual
    pos = vec2(WIDTH/2, HEIGHT/2)
end

function draw()
    background(40)
    pos = pos + pad.leftStick.dir * 300 * time.delta
    if pad.a.pressed then print("A") end
    fill(255)
    ellipse(pos.x, pos.y, 60, 60)
end

Controlling a scene with parameters

Use Controls to rename, move, recolor, or hide the sphere, or make it squash and stretch. Try 1.5 + math.sin(time.elapsed) in the expression field to make its label move up and down.

The label uses style.textRenderer(style.atlas) because its font size is one world unit and the 3D camera magnifies it. Distance-field glyphs stay sharp as the label grows on screen. The native renderer rasterizes at the requested font size, so a one-point native label becomes blurry when enlarged. The camera uses camera.depthAndColor to clear both its color and depth buffers.

-- Use this function to perform your initial setup
function setup()
    print("Hello Parameters!")

    -- Create a simple scene containing a model to play with
    scn = scene.default3d()
    rig = scn.camera:add(camera.rigs.orbit)
    scn.camera:get(camera).clearMode = camera.depthAndColor

    -- Basic entity with a sphere mesh
    ent = scn:entity()
    ent:add(mesh.sphere())

    mat = material.lit()
    mat.color = color.red
    ent.material = mat
    ent.expr = function() return 1.5 end

    -- Draw some text above the entity
    function ent:draw()
        -- Distance-field text stays sharp at this small world-space size.
        style.depth(true).textRenderer(style.atlas).fill(255).fontSize(1)
        shapeMode(CENTER)
        textAlign(CENTER | MIDDLE)
        matrix.translate(0, self.expr(), 0)
        text(self.name)
    end

    -- Auto-update/draw the scene
    scene.main = scn


    parameter.text("Name", ent.name, function(value)
        ent.name = value
    end)

    -- Expression idiom (use multiline as well)
    parameter.text("Expression", "1.5", function(value)
        ent.expr = load("return "..value) or function() return 0 end
    end, true)

    -- Y value for the entity
    parameter.number("Y", 0, 10, 0, function(value)
        ent.y = value
    end)

    -- (De)activate entity
    parameter.boolean("Active", true, function(value)
        print(value and "On" or "Off")
        ent.active = value
    end)

    -- Adjust entity material color
    parameter.color("Color", color.yellow, function(value)
        mat.color = value
    end)

    -- Do a simple tween to squash and stretch
    parameter.action("Action", function()
        scn:tween(ent):punch{sx=2, sy=0.5}:time(1)
    end)

end

Tracking more than one finger

Drag several fingers to move their circles independently. Each position is stored using touch.id and removed when the touch ends or is cancelled.

-- Track simultaneous fingers by their stable touch IDs.
local fingers = {}
function setup() end
function draw()
    background(30, 35, 45)
    style.fill(255).fontSize(20).textAlign(CENTER | MIDDLE)
    text("Touch and drag to draw circles", WIDTH/2, HEIGHT - 40)
    for id, position in pairs(fingers) do
        style.fill(100, 220, 200).noStroke()
        ellipse(position.x, position.y, 70)
    end
end
function touched(t)
    if t.ended or t.cancelled then
        fingers[t.id] = nil
    else
        fingers[t.id] = vec2(t.x, t.y)
    end
end

Functions and constants for reading a hardware keyboard. Key events also call the global keyPressed(press) and keyReleased(press) functions, except while a user interface text field has focus

Use key.pressing for continuous movement and key.wasPressed for a one-off action. Multiply movement speed by time.delta so it does not depend on the frame rate. Modifier masks such as key.cmd and key.shift can be combined with |. For entered text, use keyboard.text so composition, autocorrection, and Unicode editing are handled by the system.

function setup()
    x = WIDTH / 2
end

function draw()
    background(30)
    if key.pressing(key.left) then x = x - 200 * time.delta end
    if key.pressing(key.right) then x = x + 200 * time.delta end
    if key.wasPressed(key.space) then x = WIDTH / 2 end
    fill(255)
    ellipse(x, HEIGHT / 2, 40)
end

See Input for key callbacks and shortcut examples.

The on-screen keyboard. While it is shown, typing sets keyboard.text and also calls the global keyPressed(press) and keyReleased(press) functions, one after the other, for each insertion. press.characters holds the inserted text and press.keyCode is set for letters, digits, space, return and backspace. A hardware keyboard types into the same text while the on-screen keyboard is shown.

Use the buffer as the source of truth for text entry; do not append press.characters to it yourself, which would duplicate insertions. Clear or prefill the buffer before showing the keyboard. With a hardware keyboard attached, text entry can work even when keyboard.visible is false.

function setup()
    keyboard.text = ""
    keyboard.show()
end

function keyPressed(press)
    if press.keyCode == key.enter then
        print("Entered:", keyboard.text)
        keyboard.hide()
    end
end

function draw()
    background(30)
    fill(255)
    shapeMode(CENTER)
    text(keyboard.text, WIDTH / 2, HEIGHT / 2)
end

The pointer state on a Mac, or an iPad with a mouse or trackpad. The global hover(gesture) and scroll(gesture) functions are also called for pointer movement and scrolling

Positions use drawing coordinates: the origin is at the lower-left of the viewer. mouse.delta and mouse.scroll are changes for the current frame, not accumulated positions. mouse.left.pressed stays true while dragging; compare it with its previous value to detect a new click. Check mouse.active before showing pointer-only UI, and keep a touch alternative for devices without a mouse.

function setup()
    position = vec2(WIDTH / 2, HEIGHT / 2)
end

function draw()
    background(30)
    if mouse.active and mouse.left.pressed then
        position = mouse.pos
    end
    fill(255, 190, 70)
    ellipse(position.x, position.y, 60)
end

See Input for hover and scroll callbacks and pointer locking.

These are bit masks for key.modifiers(), key.modifiersPressed() and press.modifierFlags. Each one covers both the left and right key. They are not key codes, so use key.leftAlt and similar with key.pressing()

Read the device’s acceleration, rotation, and orientation to respond to tilts and movement. Motion tracking uses battery power; call motion.stop() when you no longer need it.