The new OSC capability in 2.7 is wonderful

DavidLeibs

The new OSC capability in 2.7 is as wonderful as I had hoped it would be. To test the implementation and api I built a multi iPad application.

One iPad, the client, has a graphical interface which does the control of the camera position, the direction of viewing, and manages an insertion point. The control app also has a palette of models to insert. It also has an interface that lets you choose between different cameras to interact with. It turns ui actions into OSC messages to the server.

The other IPad app, the server, just displays the 3D scene and takes the incoming OSC messages and then adds models, updates the camera position, direction of view, or camera changes.

The really nice part is having the server use airplay mirroring to my big TV. It is all very performant.

The only tedious part was making it be just one app that worked whether it was split between two iPads or not. It works just fine to send OSC messages to itself or just know it’s local and avoid OSC entirely.

Simeon did an incredible job to get this capability in so quickly as I only suggested it a few days ago.

Simeon

@DavidLeibs that sounds like a really cool project, any chance of a video to see it in action?

So glad the feature worked out for you. Once I looked into it, it seemed like a very "Codea" feature to add so I was excited to include it.

DavidLeibs

Let me scrape a few visual barnacles of as I did it so fast and then I will post a video.

John

@DavidLeibs Sounds really cool! It should be possible to also use OSC to push code updates from one iPad to the other by using readProjectTab() / saveProjectTab() and such to keep the client and server in-sync.

iam3o5am

Ha, “Codea” as an adjective...I’ll take this moment to ask what I’ve always wanted to ask, but haven’t since it’s just not that important... @Simeon , do you pronounce it COdea, or coDEa, or?

Thanks, now back to programming!...

DavidLeibs

@John
Thanks, I will look into that.

Simeon

@iam3o5am I say Co-dee-ah (like "Idea") but I like hearing it said other ways.

iam3o5am

@Simeon Like “Idea” - I’m sure that was the intention all along. How did I miss that... Thanks.

enzyme69

I manage to send message to my Mac listening. I am curious on how to listen and whether Codea has way to control let say 3D asset with blendshape in such a way that:
- iPhone X app like FaceApp can send OSC into iPad that is hosting a 3D head + blendshapes

Bri_G

@DavidLeibs - when you posted this initially I looked up OSC apps for Android and wind-ooohs. There seems to be quite a few but mainly focussed on audio applications. Have you used it on any other platform?

DavidLeibs

@Bri_G I have used OSC with Quartz Composer, Pure Data, SuperCollider, TUIO, and Processing. An iOS app that I have also used is TouchOSC which is pure interface widgets.

DavidLeibs

@enzyme69 I don’t know FaceApp but if it sends captured information by OSC where you can configure the IP address and port then you can configure Codea.
All you need to do is:

myIpAddr = osc.start(myPort)

Then implement a function that Codea will call

function osc.listen(path, ...)
local data = {...}
— path is the key your Face App uses to identify
— the transmitted data
End

DavidLeibs

@John as for keeping my code synced. I was quite happy when I found that I can export my project and choose to air drop right to my other iPad which lets me directly choose Codea and then the right thing just happens. I do delete the secondary iPad’s project first as I have no expectation of proper merging :-)

dave1707

@John Here’s some code using OSC that allows you to send a project from one iPad to another. To use this, put this code on both iPads. The code will get its own iPad address, but you have to put the iPad address of the other iPad in osc.host. Do that on both iPads. Run the code on both iPads then enter the project you want to send in ProjectToSend on the sending iPad. If you want to save it as a different name on the other iPad, enter a different name in NameToSave. If a project by that name already exists and you want to overwrite it, slide the Overwrite slider on. Then press Send. I wrote something similar to this using sockets, but OSC looks a little easier to use. I haven’t tried this on large projects and I don’t know the max size of messages that can be sent, so its just an example and use at your own risk.

function setup()
    func=showMsg
    msg="Enter ProjectToSend name"
    myIp = osc.start()  -- address of this ipad
    if myIp==nil then
        msg="iPad is offline"
    else
        print("My ip address "..myIp)
        osc.host = "192.168.254.15"  -- put address of other ipad here
        parameter.text("ProjectToSend")
        parameter.text("NameToSave")
        parameter.boolean("Overwrite",false)
        parameter.action("Send",function() func=sendProjectName end)
    end
end

function draw()
    background(0)
    if Overwrite then
        fill(255,0,0)
        fontSize(30)
        text("Overwrite is on! ",WIDTH/2,HEIGHT/2+50)
    end
    fill(255)    
    fontSize(20)
    func()
end

function sendProjectName()
    cnt=0
    if ProjectToSend=="" or ProjectToSend==nil then
        msg="ProjectToSend is blank"
        func=showMsg  
        return      
    end
    if NameToSave=="" or NameToSave==nil then
        NameToSave=ProjectToSend
    end
    lst=listProjectTabs(ProjectToSend)
    sendMsg("/projectName",NameToSave,Overwrite) 
    msg="Receiving iPad is offline"
    func=showMsg   
end

function sendProjectTab()
    cnt=cnt+1
    if cnt>#lst then
        msg="Project send complete"
        func=showMsg
        sendMsg("/Complete","OK")
        Overwrite=false
        return
    end
    str=readProjectTab(ProjectToSend..":"..lst[cnt])
    sendMsg("/projectTab",NameToSave,lst[cnt],str)
    func=showMsg
end

function showMsg()
    text(msg,WIDTH/2,HEIGHT/2)
end

function sendMsg(id,...)
    osc.send(id,...)        
end

function osc.listen(id,...) 
    arg={...}
    -- messages for receiving ipad
    if id=="/projectName" then
        if arg[2]==1 then
            deleteProject(arg[1])
        end
        if hasProject(arg[1]) then
            sendMsg("/ProjectCreated","NotOK")
            func=showMsg
        else
            projectReceived=arg[1]
            createProject(arg[1])
            sendMsg("/ProjectCreated","OK")
            func=showMsg  
        end
    end
    if id=="/projectTab" then  
        saveProjectTab(arg[1]..":"..arg[2],arg[3])
        sendMsg("/TabSaved","OK")
        msg="Tab saved"
        func=showMsg
    end
    if id=="/Complete" and arg[1]=="OK" then
        msg="Project received "..projectReceived
        func=showMsg
    end

    -- messages for sending ipad
    if id=="/ProjectCreated" then
        if arg[1]=="OK" then
            func=sendProjectTab
        else
            msg="Project already exists"
            func=showMsg
        end
    end
    if id=="/TabSaved" and arg[1]=="OK" then
        func=sendProjectTab
    end
end

John

@dave1707 nice!

Simeon

@dave1707 I really like that, beaming projects around via UDP is pretty low level but cool.

dave1707

@Simeon I was playing around with message size and what I found was 65,491 bytes. That was a single message. I don’t know if multiple entries in the message affect the total size.