Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Having trouble coordinating between immediate mode drawing and Craft in the AR Face Demo

edited January 15 in General Posts: 25

I have continued hacking the AR Face Demo. I am trying to draw a point cloud on the face based on walking the positions of the AR faceModel. I want it to track the face just like the textures in the demo do. Those textures stick just like the face grabbers in the Alien movies. :-)

I can get frustratingly close. The triangle mesh I use wraps around nicely. My problem is that I can’t seem to set up the camera correctly and my point cloud drifts all around yet stays “kinda” close. The rotation does look good. I don’t see how to make my viewing setup jive with what is happening down in the Craft used in the demo. I am hoping there is an api that lets me get the setup values I need.

I am certain I am just suffering the confusion that goes with being a new user. I am hoping someone in the community can guide me.

Below is the “hopefully” revealing subset of my code. Also attached is a picture of how close I can get when I position my iPad by hand just right.

function draw()
    scene:update(DeltaTime)
    scene:draw(WIDTH, HEIGHT)
    pushMatrix()
    setupCamera()
    drawFacePointCloud(mainEntity,triMesh)
    popMatrix()
    drawTarget(mainAnchor)

end

function setupCamera()
    local cz = 0.15    
    perspective()
    camera(0,0,cz,0,0,0,0,1,0)  
end

function drawFacePointCloud(face, mesh)
    if face == nil then return end
    local wp = face.worldPosition
    local wr = face.eulerAngles
    local positions = face.model.positions
    pushMatrix()
    translate(wp.x,wp.y,wp.z)
    rotate(wr.x,1,0,0)
    rotate(wr.y,0,1,0)
    rotate(wr.z,0,0,1)
    mesh:setColors(color(255, 255, 0, 128))
    for k,v in pairs(positions) do
        local p = v
        pushMatrix()
        translate(p.x,p.y,p.z)
        mesh:draw()
        popMatrix()
    end
    popMatrix()   
end

Comments

  • dave1707dave1707 Mod
    Posts: 7,553

    @DavidLeibs When you post code, put ~~~ on a line by itself before and after the code so it shows correctly. I added them to your code above.

  • @dave1707,
    Thank you so much for making the code readable. I will now add the needed marks in the future.

  • JohnJohn Admin Mod
    Posts: 572

    @DavidLeibs I'm not exactly sure why it's out of sync but it may have something to do with your camera settings and the way ARKit updates in a multithreaded manner.
    That said you can also use craft to make a model that visualises each point using a quad and place that in the face transform hierarchy. I've made an example project that does this and it seems to work pretty well.

  • @John Thank you so much for pointing me in the right direction. Your solution is so beautifully direct it restored my childlike sense of wonder. :-) I had not realized that you could use position with an index and do surgery on the vertices inside a model. I had imagined having to do much worse and that drove me down the rathole of direct drawing and trying to deal with dueling cameras and a world of magic numbers.

    I am still left with the question of just how I would I go about coordinating with the Craft camera after the scene:draw() has returned but since I am typically in control of that camera it isn’t really an issue that will keep me up at night.

    Once again, thank you so much!

Sign In or Register to comment.