Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

External Keyboard, Mouse + Gamepad API?

Any chance of a key down/up API for external keyboards?
Either using GCKeyboard or UIResponder::pressesBegan(). pressesBegan is supported in iOS 13.4.

That and potentially APIs making use of GCGamepad & GCMouse (& prefersPointerLocked).
I know the hover gesture should also provide the mouse location but mouse locking would also be great.

There’s a fair bit of work involved here I know but just wanted to throw the suggestion out there.



  • JohnJohn Admin Mod
    Posts: 639

    We have some (possibly undocumented) stuff that was added to support the Shade UI:

    -- Mouse/trackpad was moved (no button held down)
    function hover(gesture)
    -- Mouse/trackpad was scrolled (i.e. 2 finger scroll gesture)
    function scroll(gesture)

    Where gesture has the following stuff (ignore the binding code):

            .addPropertyReadOnly("location", &Gesture::getLocation)
            .addPropertyReadOnly("translation", &Gesture::getTranslation)
            .addPropertyReadOnly("delta", &Gesture::getDeltaTranslation)
            .addPropertyReadOnly("touchCount", &Gesture::getNumberOfTouches)
            .addPropertyReadOnly("state", &Gesture::getState)
            .addPropertyReadOnly("shift", &Gesture::shiftEnabled)
            .addPropertyReadOnly("capsLock", &Gesture::capsLockEnabled)
            .addPropertyReadOnly("alt", &Gesture::altEnabled)
            .addPropertyReadOnly("control", &Gesture::controlEnabled)
            .addPropertyReadOnly("command", &Gesture::commandEnabled)
            .addPropertyReadOnly("numPad", &Gesture::numPadEnabled)

    With location, translation, delta all being vec2 types and state using the obligatory BEGAN, MOVING, ENDED, CANCELLED constants.

    The alt, control, etc. are just booleans indicating if those keys were held during the gesture.

    Since iOS 14 supports proper access to mouse stuff it might be worth adding an API for it. Maybe it would look like this?

    local mouse = input.mouse
    -- or mouse = input.mice[1]
    mouse.buttonChanged = function(mouse, button, isPressed)
      print("Pressed mouse button:", button)
    mouse.moved = function(mouse, position, delta)
    mouse:release() -- not sure if an API for this exists yet

    Codea 4 (our new Metal runtime built from the ground up) will have support for keyboard callbacks as global functions, but could also be done similar to the mouse if we use GCKeyboard.

    I also want to have support for gamepads, so we can use PS4, Xbox, etc... controllers.

  • Posts: 12


    I had seen the scroll & hover globals which should work well with the mouse in general but once the cursor is ‘captured’ the hover function will no longer receive the events as the gesture handler no longer detects gestures from the locked mouse. In that case, I think GCMouse will be the way to go.

    On the other hand, I’d definitely suggest not locking the mouse by default as if the user wanted to draw a cursor, the system cursor is much more responsive as it isn’t tied to the app’s frame rate.

    As for the suggested mouse API, it looks like it could work well, though we’d also need mouseConnected & mouseDisconnected callbacks for obvious reasons. Possibly even calling these (if implemented by the user) for each mouse connected at app startup rather than have the user poll for availability directly.

    The mouse:release() function should be fine though (although you won’t be able to do this & capture per mouse) as you can just change the return value in prefersPointerLocked and call setNeedsUpdateOfPrefersPointerLocked

    I’m glad to hear keyboard support is on the way though, can’t wait to give it a go.


  • JohnJohn Admin Mod
    Posts: 639

    That all sounds fine, would be cool to play around with multiple mice too

Sign In or Register to comment.