Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

3D dress-up game?

13

Comments

  • Posts: 2,020

    I guess not, just make sure you don't call that function and then try to do more 3D drawing afterwards. Just in terms of keeping code tidy though, I prefer to have a clear distinction in the draw loop: 1. Draw game world (3D) 2. Do 2D drawing.

  • Posts: 2,020

    If the draw texture function is only going to be called from the touched routine, then you could just put the ortho/viewMatrix calls right at the very end of you draw loop, as all touched calls happen after draw.

  • Posts: 557

    That makes total sense to me, at least in the context of a self-contained project. I'm building the class as a library, though. So I want to put as much functionality as I can inside the class itself.

    @Ignatz, can you break down what's happening in this part of the shader:

        highp vec2 T = vTexCoord * 4096.0;
        highp float x1 = mod(T.x, 16.0);
        highp float x2 = (T.x - x1) / 16.0;
        highp float y1 = mod(T.y, 16.0);
        highp float y2 = (T.y - y1) / 16.0;
        gl_FragColor = vec4(x2 / 255.0, y2 / 255.0, (x1 * 16.0 + y1) / 255.0, 1.0);
    

    I think I understand what's going on conceptually, but the rationale behind the specific mathematic operations, as well as some of the C syntax, is fuzzy to me.

  • IgnatzIgnatz Mod
    Posts: 5,396

    @UberGoober - I usually do my 2D drawing at the end of the draw function, but the beginning should be fine too.

    The reason you need those extra lines of code is that they control whether you are looking at a flat 2D scene or a 3D scene with z depth, while pushMatrix and popMatrix only store/restore translations and rotations within your scene.

  • edited August 2015 Posts: 557

    @Ignatz, in the shader code, can you tell me specifically:

    1. Where does the number 4096.0 come from?
    2. All the individual coordinates are either divided by or mod()'d by 16.0--why?
  • IgnatzIgnatz Mod
    edited August 2015 Posts: 5,396

    The shader code takes the texture x,y position (a fraction 0-1) and encodes it in the r,g,b colours of the pixel.

    The x position is encoded in the red colour and half of the blue colour. The y value is encoded in the green colour and half of the blue colour.

    Practically, this means that the x and y position can each be encoded to an accuracy of 1/(256x16).

    So the code takes the x texture position (0-1), multiplies by 4096 (=256x16), divides the result by 16 and puts the remainder in blue, and the rest of it in red. Similar for y.

    If you don't need more accuracy than 1/256, then simply use this instead - which also frees up the blue colour so you can store the id of the texture that was touched.

        //this replaces all the lines above
        gl_FragColor = vec4(vTexCoord,0.0, 1.0);
    

    if you want to store the id of the touched item, use this instead

        //convert ID to fraction 0-1, Codea will multiply it by 255
        gl_FragColor = vec4(vTexCoord, id/255.0, 1.0);
    
        //and put this line with the other "uniform" items above the main function
        uniform float id;   
    
        --and back in Codea, set the ID when you create the mesh for each part 
        --of the body, eg
        Pants=mesh()
        --your code to create mesh
        Pants.shader = ..... (your code)
        Pants.shader.id=2
    

    And when you touch the pants, the blue colour will be 2.

  • Posts: 557

    @Ignatz, thanks for clarifying, I think I almost get it.

    Is there a reason you don't use the alpha value as well? Why use r, g, and b but not a?

  • IgnatzIgnatz Mod
    edited August 2015 Posts: 5,396

    alpha gets messed up by Codea's anti aliasing, which interpolates the alpha around colour transitions, and there's no way to turn it off. So alpha is unreliable.

    EDIT - see minor correction to code in my previous post to add decimal point to 255 (shader doesn't like integers without decimals)

  • Posts: 557

    @Ignatz: so, here's an odd question. The answer might be obvious to someone who understood everything better, so please forgive me if it's dumb.

    My question is based on the fact that a shader can't directly output information to the Codea environment, but Codea can directly put information in, right? The clipping window, for example, is a parameter directly set by Codea.

    So would it be possible to encode even more information by putting it in more than one pixel?

    The process I'm imagining goes like this: Codea tells the shader to render a 3 x 3 pixel region, centered around the touch. Now, in the shader, we're currently encoding individual coordinate information in every one of those pixels. But since we know that the only one we are going to need is the middle one, is there some way to tell the shader to use, for instance, all of the first three pixels for the x value, and all of the last three pixels for the y? Basically we would only use the first two places of each r, g, and b of the color values, so that each used digit could go from one to nine, but since we would be using three sets of colors that would give us accuracy down to 18 decimal places for both x and y. We could represent fairly precisely the actual decimal number that the shader is using.

    Would that be possible?

  • Posts: 557

    @Ignatz, the code at this link also no longer runs: https://gist.githubusercontent.com/dermotbalson/7443057/raw/cc47c8c0a7bfaced35feb90e824aced904cba8dd/gistfile1.txt
    When you tap play, every tab lights up red, complaining about calling some nil value or other. Sorry! Would you rather not know?

  • IgnatzIgnatz Mod
    Posts: 5,396

    I'll fix the code. It's another software update breaking something.

    Wrt your question, my understanding is that the shader still has to process all the pixels even if they will be clipped, because at the time the fragment shader runs, it doesn't know where on the screen the pixel is going to end up - the camera angle hasn't been applied yet.

    So the shader isn't just processing 9 pixels.

    But I don't see a problem with an accuracy of 1/256, bearing in mind your finger has an accuracy of about 1/50! So isn't one pixel colour good enough?

  • Posts: 557

    @Ignatz: to be embarrassingly honest, I don't fully understand the math behind the current calculations. My attempts to repurpose the code aren't working, and in trying to debug it I'm running up against my own math limitations. So I was trying to think of a way to do the same thing but have it be more parseable to me. In the end I think I just have to force myself to understand the existing code.

    But as an aside: if the shader has to process all the pixels anyway, why do we clip the buffered image at all?

  • Posts: 557

    @Ignatz, you may find these results interesting.

    I was working with your touch demo, trying to dissect it.

    First, to make the math simpler for me to understand, I changed the fragment shader to this:

    void main()
    {
        gl_FragColor = vec4(vTexCoord.x, vTexCoord.y, 0, 1.0);
    }
    

    ...what surprised me is that, without changing any other code at all, it still seemed to work just as well. So I guess your point about necessary precision is a good one.

    Next, keeping this simplified shader, I applied a texture to the plane and tried to use this code to draw to it. Inside GetPlaneTouchPoint I added this:

        local codedX,codedY =shaderImg:get(touchX,touchY)
        local proportionalX = codedX / 255
        local proportionalY = codedY / 255
        local convertedX = proportionalX * shaderImg.width
        local convertedY = proportionalY * shaderImg.height
        textureTouched = vec2(convertedX, convertedY)
    

    ...and at the end of draw I added:

        if textureTouched then
            ortho()
            viewMatrix(matrix())
            setContext(plane.mesh.texture)
            fill(0, 255, 255, 255)
            ellipse(textureTouched.x,textureTouched.y,10)
            textureTouched = nil
            setContext()
        end
    

    ...which works okay, but oddly slightly differently the other code.

    If you try it out, you'll see that the textureTouched point is drawn slightly above and off-center from the p point. Which seems odd. Both are using the simplified shader, so any loss of accuracy should extend to both of them, shouldn't it?

    And between the two, the p point feels more right--it seems to appear closer to where I actually touch.

  • IgnatzIgnatz Mod
    Posts: 5,396

    It may come down to the fact that your code multiplies the shader result by 255, while mine uses 4096 (256*16). Mine may be slightly incorrect because the colours go from 0 to 255, not 256. But I haven't checked to be sure.

    Anyway, what you've done is a great way of understanding what is happening, and I have done that many times myself. So it is well worthwhile!

  • IgnatzIgnatz Mod
    Posts: 5,396

    @UberGoober - I fixed that broken code you mentioned above,
    here

  • Posts: 289

    it is for advanced Codea user, and for female player, i always dont know how to play this type of game

  • edited August 2015 Posts: 557

    @Ignatz, @Yojimbo2000 - drawing to texture (with visible buffer image & pseudo-unit-tests):


    --# Main -- MakePlane function setup() textMode(CORNER) setUpTests() makePlane = MakePlane() texture = makePlane.plane.mesh.texture colorCoordinates = {} overlayImage = image(WIDTH, HEIGHT) setUpBufferDisplay() end function setUpBufferDisplay() bufferImage = image(WIDTH, HEIGHT) local bufferFract = 1 / 6 cameoSize = vec2(WIDTH * bufferFract, HEIGHT * bufferFract) camoPos = vec2((cameoSize.x/2)+20, (cameoSize.y/2)+20) setContext(bufferImage) --fill image with yellow for visibility strokeWidth(0) fill(220, 210, 122, 255) rect(0,0,WIDTH,HEIGHT) setContext() end function setUpTests() local parameterTable = {} --series of buttons to confirm visuals local assignableAction = function() end local nextParameter = function () parameter.clear() if #parameterTable == 0 then return end local testDefinition = parameterTable[1] local passAction = function() print(testDefinition..": PASS") assignableAction() end local failAction = function() print(testDefinition..": FAIL") assignableAction() end parameter.action("PASS if "..testDefinition, passAction) parameter.action("FAIL", failAction) table.remove(parameterTable, 1) end assignableAction = nextParameter parameterTable = {"plane is showing", "touching draws red dots", "dragging leaves trail", "touch activates shader", "buffer image in corner", "clipping drawn in buffer", "mesh clips in buffer", "buffer perspective matches", "buffer coordinates read", "touch draws to texture", "draws in right place", "all touches draw"} nextParameter() end function draw() backingMode(STANDARD) background(69, 119, 104, 255) drawPlane() draw2D() end function drawPlane() pushMatrix() makePlane:draw() popMatrix() end function draw2D() ortho() viewMatrix(matrix()) sprite(overlayImage, WIDTH/2, HEIGHT/2, WIDTH, HEIGHT) sprite(bufferImage, camoPos.x, camoPos.y, cameoSize.x, cameoSize.y) if #colorCoordinates > 0 then drawTouchesToTexture() end end function drawTouchesToTexture() local textureX, textureY, baseX, baseY, marker, intCoord, fX, fY textureX, textureY = 0, 0 for i in ipairs(colorCoordinates) do coord = colorCoordinates[i] intCoord = vec2(math.floor(coord.x), math.floor(coord.y)) baseX, baseY, marker = bufferImage:get(intCoord.x, intCoord.y) if marker == 1 then fX, fY = baseX / 255, baseY / 255 textureX = texture.width * fX textureY = texture.height * fY setContext(texture) fill(0, 43, 255, 255) ellipse(textureX, textureY, 25) setContext() end end text("x: "..textureX..", y: "..math.floor(textureY, 10, 10)) colorCoordinates = {} end function touched(touch) setContext(overlayImage) fill(255, 4, 0, 255) ellipse(touch.x, touch.y, 10) setContext() if touch.state == MOVING then toggleShader(true) drawClipToBufferPosition(touch) table.insert(colorCoordinates, vec2(touch.x, touch.y)) else toggleShader(false) end end function drawClipToBufferPosition(touch) setContext(bufferImage) pushMatrix() clip(touch.x - 10, touch.y - 10, 20, 20) makePlane:draw() popMatrix() setContext() end function toggleShader(onOrOff) if onOrOff == true then local planeShader = shader(PlaneShader.v, PlaneShader.f) makePlane.plane.mesh.shader = planeShader else makePlane.plane.mesh.shader = nil end end --# MakePlane MakePlane = class() function MakePlane:init() self:createSimpleMesh() end function MakePlane:draw() self:setupPerspective() self.plane.mesh:draw() end function MakePlane:createSimpleMesh() --settings for our rectangle (named "plane" below) self.plane={} self.plane.centre=vec3(0,0,0) self.plane.rotate=vec3(45,0,20) self.plane.size=vec2(500,500,0) strokeWidth(10) stroke(255, 0, 0, 255) --set up mesh, needed for shader self.plane.mesh=mesh() self.plane.mesh.texture = readImage("SpaceCute:Background") local s=self.plane.size local x1,y1,x2,y2,z=-s.x/2,-s.y/2,s.x/2,s.y/2,s.z self.plane.mesh.vertices={vec3(x1,y1,z),vec3(x2,y1,z),vec3(x2,y2,z),vec3(x2,y2,z),vec3(x1,y2,z),vec3(x1,y1,z)} self.plane.mesh.texCoords={vec2(0,0),vec2(1,0),vec2(1,1),vec2(1,1),vec2(0,1),vec2(0,0)} self.plane.mesh:setColors(color(255,255,0)) self.img=image(WIDTH, HEIGHT) self.shaderImg=image(WIDTH, HEIGHT) end function MakePlane:setupPerspective() perspective() camera(0,0,900,0,0,-1) translate(self.plane.centre.x, self.plane.centre.y, self.plane.centre.z) rotate(self.plane.rotate.x,1,0,0) rotate(self.plane.rotate.y,0,1,0) rotate(self.plane.rotate.z,0,0,1) end PlaneShader = { v = [[ uniform mat4 modelViewProjection; attribute vec4 position; attribute vec2 texCoord; varying highp vec2 vTexCoord; void main() { vTexCoord = texCoord; gl_Position = modelViewProjection * position; } ]], f = [[ precision highp float; varying highp vec2 vTexCoord; void main() { lowp float marker = 1.0 /255.0; gl_FragColor = vec4(vTexCoord.x,vTexCoord.y,marker,1.0); } ]]}

    Progress!

  • Posts: 557

    For extra fun add this to the end of setup:

        newX = 400
        tween( 10, makePlane.plane.rotate, { x = newX }, { easing = tween.easing.linear, loop = tween.loop.pingpong } )
    

    ...a testament to Ignatz's code that it works so well during rotation, no extra programming needed!

  • IgnatzIgnatz Mod
    Posts: 5,396

    Ah, rotations, my favourite nightmare! @-)

  • Posts: 557

    @Ignatz it's your rotation code rotating your mesh and using your texture-drawing code, and it works great, so I'd say you got some things right!

  • edited August 2015 Posts: 557

    Update: still working on this, making incremental improvements. Thanks to @Ignatz, I am displaying multiple meshes that I can draw on with my finger. Frame rate is really low, but I'm working on it.

  • IgnatzIgnatz Mod
    Posts: 5,396

    I'm surprised your frame rate is low, unless it's been low all along. Is it related to the number of meshes, or anything you can identify?

  • edited August 2015 Posts: 2,020

    This is why I was asking about the number of vertices. You can check it quite easily, just find the part of the code where the vertices table is assigned m.vertices = verts or whatever and add print("vertices:", #verts).

    If you have too many, you can lower the poly count in Blender with a decimate modifier, or by removing parts of the mesh that you can't see (eg sometimes you see character rigs with inner mouths, teeth etc, for doing facial animation, which you probably don't need)

  • IgnatzIgnatz Mod
    Posts: 5,396

    Or m.size

  • Posts: 557

    @Ignatz, @Yojimbo: there has to be something more fundamental I'm doing wrong, because I'm seeing frame rates around 15fps while simply displaying two textured squares. Which is, what, eight vertices total?

    I'm doing a bit of ground-up re-inspection of my code to try and find the problem.

    I will say this though: math.floor() seems to realllly slow stuff down.

  • Posts: 2,020

    I doubt it's math.floor. You could try integer divide //1

  • IgnatzIgnatz Mod
    edited August 2015 Posts: 5,396

    We need more detail. Can you post some code, or pseudo code, to give us an idea of the work Codea is doing?

    PS a square is 6 vertices (2 triangles), so two of them is 12 vertices. You should have no problem drawing thousands at 60 FPS, so something else is causing it (and I doubt it's math.floor).

  • Posts: 557

    The code is pretty much the same as the code I already posted above, but I do want to do a little more digging of my own before I post the current state of it. I think, after all, @Yojimbo2000 may have hit the nail on the head in warning me about mixing up 2-D and 3-D rendering orders

  • IgnatzIgnatz Mod
    Posts: 5,396

    Yes, you should do your 2D drawing before or after 3D

  • Posts: 557

    @Ignatz I may be measuring FPS wrong. I grabbed the code I'm using from your tilt-racer project. I may be deploying it wrong.

    --globals.FPS defined as 60 in setup
    
    function draw()  
        backingMode(STANDARD)
        background(69, 119, 104, 255)
        globals.FPS=globals.FPS*0.9+.1/DeltaTime
        text("FPS: "..math.floor(globals.FPS),25,HEIGHT-25)
    end
    

    With nothing else going on at all, it shows FPS 15 or 16 most of the time. I must be doing it wrong.

  • Posts: 2,020

    I added this profiler to the drawing to texture code above:

    profiler={}
    
    function profiler.init(silent)    
        profiler.del=0
        profiler.c=0
        profiler.fps=0
        profiler.mem=0
        if not silent then
            parameter.watch("profiler.fps")
            parameter.watch("profiler.mem")
        end
    end
    
    function profiler.draw()
        profiler.del = profiler.del +  DeltaTime
        profiler.c = profiler.c + 1
        if profiler.c==10 then
            profiler.fps=profiler.c/profiler.del
            profiler.del=0
            profiler.c=0
            profiler.mem=collectgarbage("count", 2)
        end
    end
    

    I got 45 fps when not touching, dropping to around 22-25 fps when touching. I'm on an iPad Air. That does seem way, way lower than it should be. You really want this running at 60 fps.

  • Posts: 2,020

    Also, in the code above, it seems as if you have separated the 2d and 3d drawing?

  • Posts: 2,020

    I haven't fully pulled this apart, but based on what I've done previously with drawing to offscreen buffers (eg the Gaussian blur shader I ported), I would make the offscreen buffer image as small as you can make it. Given that the touch event only has an accuracy of 0-255, overlayImage does not need to be 2000x1400 pixels, does it? (remember that Codea automatically converts all pixels to retina coordinates). It should just be 255 by 255. So when you draw ellipses to it, divide the coords of the touch by 4. (Or, given the retina-ization, 128x128, and divide by 8)?

  • Posts: 2,020

    Also, why is the buffer image WIDTH x HEIGHT if you only draw to a 10x10 section of it? It could just be 10x10 pixels.

  • Posts: 2,020

    Is there any need to clip? Just make the image the size you need and translate.

  • Posts: 2,020

    With the Gaussian blur shader, quartering the size of the offscreen buffer made a big difference to the speed

  • Posts: 2,020

    Here's a version that quarters the size of the two buffers. When not touching, it's still 45 fps, but it doesn't drop quite so much when you do touch. Not as big a saving as I'd hoped for. The overlay image can probably go down to /8, although the overlay sprite would look a bit weird.

    -- MakePlane
    
    function setup()
        textMode(CORNER)
      --  noStroke()
        setUpTests()
        makePlane = MakePlane()
        texture = makePlane.plane.mesh.texture
        colorCoordinates = {}
        overlayImage = image(WIDTH/4, HEIGHT/4)
        setUpBufferDisplay()
        profiler.init()
    end
    
    function setUpBufferDisplay()
        bufferImage = image(WIDTH/4, HEIGHT/4)
        local bufferFract = 1 / 6
        cameoSize = vec2(WIDTH * bufferFract, HEIGHT * bufferFract)
        camoPos = vec2((cameoSize.x/2)+20, (cameoSize.y/2)+20)
        setContext(bufferImage) --fill image with yellow for visibility
        strokeWidth(0)
        fill(220, 210, 122, 255)
        rect(0,0,WIDTH,HEIGHT)
        setContext()
    end
    
    function setUpTests()
        local parameterTable = {} --series of buttons to confirm visuals
        local assignableAction = function() end
        local nextParameter = function ()
            parameter.clear()
            if #parameterTable == 0 then
                return
            end
            local testDefinition = parameterTable[1]
            local passAction = function()
                print(testDefinition..": PASS")
                assignableAction()
            end
            local failAction = function()
                print(testDefinition..": FAIL")
                assignableAction()
            end
            parameter.action("PASS if "..testDefinition, passAction)
            parameter.action("FAIL", failAction)
            table.remove(parameterTable, 1)
        end
        assignableAction = nextParameter
        parameterTable = {"plane is showing", "touching draws red dots", "dragging leaves trail", "touch activates shader", "buffer image in corner", "clipping drawn in buffer", "mesh clips in buffer", "buffer perspective matches", "buffer coordinates read", "touch draws to texture", "draws in right place", "all touches draw"}
        nextParameter()
    end
    
    function draw()
        backingMode(STANDARD)
        background(69, 119, 104, 255)
        drawPlane()
        draw2D()
        profiler.draw()
    end
    
    function drawPlane()
        pushMatrix()
        makePlane:draw()
        popMatrix() 
    end
    
    function draw2D()
        ortho()
        viewMatrix(matrix())
     --   sprite(overlayImage, WIDTH/2, HEIGHT/2, WIDTH, HEIGHT)
        sprite(bufferImage, camoPos.x, camoPos.y, cameoSize.x, cameoSize.y)
        if #colorCoordinates > 0 then
            drawTouchesToTexture()
        end
    end
    
    function drawTouchesToTexture()
        local textureX, textureY, baseX, baseY, marker, intCoord, fX, fY
        textureX, textureY = 0, 0
        for i in ipairs(colorCoordinates) do
            coord = colorCoordinates[i] * 0.25
            intCoord = vec2(math.floor(coord.x), math.floor(coord.y))
            baseX, baseY, marker = bufferImage:get(intCoord.x, intCoord.y)
            if marker ==  1 then
                fX, fY = baseX / 255, baseY / 255
                textureX = texture.width * fX
                textureY = texture.height * fY
                setContext(texture)
                fill(0, 43, 255, 255)
                ellipse(textureX, textureY, 25)
                setContext()
            end
        end
        text("x: "..textureX..", y: "..math.floor(textureY, 10, 10))
        colorCoordinates = {}
    end
    
    function touched(touch)
        setContext(overlayImage)
        fill(255, 4, 0, 255)
        ellipse(touch.x/4, touch.y/4, 3)
        setContext()
        if touch.state == MOVING then
            toggleShader(true)
            drawClipToBufferPosition(touch)
            table.insert(colorCoordinates, vec2(touch.x, touch.y))
        else
            toggleShader(false)
        end
    end
    
    function drawClipToBufferPosition(touch)
            setContext(bufferImage)
            pushMatrix()
            clip((touch.x*0.25) - 10, (touch.y*0.25) - 10, 20, 20)
            makePlane:draw()
            popMatrix()
            setContext()
    end
    
    function toggleShader(onOrOff)
        if onOrOff == true then
            local planeShader = shader(PlaneShader.v, PlaneShader.f)
            makePlane.plane.mesh.shader = planeShader
        else 
            makePlane.plane.mesh.shader = nil
        end
    end
    
    profiler={}
    
    function profiler.init(silent)    
        profiler.del=0
        profiler.c=0
        profiler.fps=0
        profiler.mem=0
        if not silent then
            parameter.watch("profiler.fps")
            parameter.watch("profiler.mem")
        end
    end
    
    function profiler.draw()
        profiler.del = profiler.del +  DeltaTime
        profiler.c = profiler.c + 1
        if profiler.c==10 then
            profiler.fps=profiler.c/profiler.del
            profiler.del=0
            profiler.c=0
            profiler.mem=collectgarbage("count", 2)
        end
    end
    
  • Posts: 2,020

    Ok, I see what you mean about not separating 2d and 3d draws. So when you touch, you actually set up the camera twice, draw the plane twice etc, hence the frame rate dropping. I guess it's tricky because you need continuous touch events as your finger moves

  • edited August 2015 Posts: 2,020

    Bingo, I got it. For some reason, it is backingMode(STANDARD) in the draw loop. Comment that out, and for some reason, fps jumps up to 60. Here's a version that tries to do all the 3d drawing in one go. I don't think it makes much difference though, as you need to re-establish the camera when you setContext to a different image anyway. It also gets rid of the colorCoordinates table loop (as it only ever contains one value? Were you planning on implementing multi-touch later?), to try to speed things up a bit. The fps stays at 60 fps though, regardless of whether you're touching. So, quarter-size buffers (smaller is possible) and not setting the backing mode each frame triple the running speed, and only allowing for a single touch seems to help too.

    And the overlay image is just there for testing isn't it? So you could get rid of that in the final version.

    Edit (tidied up the line drawing). Sorry I left the code in such a messy state. But feel that speed!


    --# Main -- MakePlane function setup() textMode(CORNER) planeShader = shader(PlaneShader.v, PlaneShader.f) -- noStroke() setUpTests() makePlane = MakePlane() texture = makePlane.plane.mesh.texture colorCoordinates = {} overlayImage = image(WIDTH/4, HEIGHT/4) setUpBufferDisplay() profiler.init() end function setUpBufferDisplay() bufferImage = image(WIDTH/4, HEIGHT/4) local bufferFract = 1 / 6 cameoSize = vec2(WIDTH * bufferFract, HEIGHT * bufferFract) camoPos = vec2((cameoSize.x/2)+20, (cameoSize.y/2)+20) setContext(bufferImage) --fill image with yellow for visibility strokeWidth(0) fill(220, 210, 122, 255) rect(0,0,WIDTH,HEIGHT) setContext() end function setUpTests() local parameterTable = {} --series of buttons to confirm visuals local assignableAction = function() end local nextParameter = function () parameter.clear() if #parameterTable == 0 then return end local testDefinition = parameterTable[1] local passAction = function() print(testDefinition..": PASS") assignableAction() end local failAction = function() print(testDefinition..": FAIL") assignableAction() end parameter.action("PASS if "..testDefinition, passAction) parameter.action("FAIL", failAction) table.remove(parameterTable, 1) end assignableAction = nextParameter parameterTable = {"plane is showing", "touching draws red dots", "dragging leaves trail", "touch activates shader", "buffer image in corner", "clipping drawn in buffer", "mesh clips in buffer", "buffer perspective matches", "buffer coordinates read", "touch draws to texture", "draws in right place", "all touches draw"} nextParameter() end function draw() -- backingMode(STANDARD) background(69, 119, 104, 255) drawPlane() draw2D() profiler.draw() end function drawPlane() pushMatrix() -- makePlane:setupPerspective() if touching then toggleShader(true) drawClipToBufferPosition(touching) toggleShader(false) -- touching = nil end -- pushMatrix() makePlane:draw() popMatrix() end function draw2D() ortho() viewMatrix(matrix()) sprite(overlayImage, WIDTH/2, HEIGHT/2, WIDTH, HEIGHT) sprite(bufferImage, camoPos.x, camoPos.y, cameoSize.x, cameoSize.y) -- if #colorCoordinates > 0 then if touching then drawTouchesToTexture() touching = nil end end function drawTouchesToTexture() local textureX, textureY, baseX, baseY, marker, intCoord, fX, fY textureX, textureY = 0, 0 -- for i in ipairs(colorCoordinates) do -- coord = colorCoordinates[i] * 0.25 coord = touching * 0.25 intCoord = vec2(math.floor(coord.x), math.floor(coord.y)) baseX, baseY, marker = bufferImage:get(intCoord.x, intCoord.y) if marker == 1 then fX, fY = baseX / 255, baseY / 255 textureX = texture.width * fX textureY = texture.height * fY lastTexX = lastTexX or textureX lastTexY = lastTexY or textureY setContext(texture) stroke(0, 43, 255, 255) strokeWidth(25) line(textureX, textureY, lastTexX, lastTexY) -- ellipse(textureX, textureY, 25) setContext() lastTexX, lastTexY = textureX, textureY end -- end -- text("x: "..textureX..", y: "..math.floor(textureY, 10, 10)) -- end -- colorCoordinates = {} end function touched(touch) setContext(overlayImage) fill(255, 4, 0, 255) ellipse(touch.x/4, touch.y/4, 3) setContext() touching = vec2(touch.x, touch.y) if touch.state == ENDED then touching, lastTexX, lastTexY = nil,nil, nil end --[[ if touch.state == MOVING then touching = vec2(touch.x, touch.y) -- table.insert(colorCoordinates, vec2(touch.x, touch.y)) else toggleShader(false) end ]] end function drawClipToBufferPosition(touch) setContext(bufferImage) pushMatrix() clip((touch.x*0.25) - 10, (touch.y*0.25) - 10, 20, 20) makePlane:draw() clip() popMatrix() setContext() end function toggleShader(onOrOff) if onOrOff == true then makePlane.plane.mesh.shader = planeShader else makePlane.plane.mesh.shader = nil end end profiler={} function profiler.init(silent) profiler.del=0 profiler.c=0 profiler.fps=0 profiler.mem=0 if not silent then parameter.watch("profiler.fps") parameter.watch("profiler.mem") end end function profiler.draw() profiler.del = profiler.del + DeltaTime profiler.c = profiler.c + 1 if profiler.c==10 then profiler.fps=profiler.c/profiler.del profiler.del=0 profiler.c=0 profiler.mem=collectgarbage("count", 2) end end --# MakePlane MakePlane = class() function MakePlane:init() self:createSimpleMesh() end function MakePlane:draw() self:setupPerspective() self.plane.mesh:draw() end function MakePlane:createSimpleMesh() --settings for our rectangle (named "plane" below) self.plane={} self.plane.centre=vec3(0,0,0) self.plane.rotate=vec3(45,0,20) self.plane.size=vec2(500,500,0) strokeWidth(10) stroke(255, 0, 0, 255) --set up mesh, needed for shader self.plane.mesh=mesh() self.plane.mesh.texture = readImage("SpaceCute:Background") local s=self.plane.size local x1,y1,x2,y2,z=-s.x/2,-s.y/2,s.x/2,s.y/2,s.z self.plane.mesh.vertices={vec3(x1,y1,z),vec3(x2,y1,z),vec3(x2,y2,z),vec3(x2,y2,z),vec3(x1,y2,z),vec3(x1,y1,z)} self.plane.mesh.texCoords={vec2(0,0),vec2(1,0),vec2(1,1),vec2(1,1),vec2(0,1),vec2(0,0)} self.plane.mesh:setColors(color(255,255,0)) self.img=image(WIDTH, HEIGHT) self.shaderImg=image(WIDTH, HEIGHT) end function MakePlane:setupPerspective() perspective() camera(0,0,900,0,0,-1) translate(self.plane.centre.x, self.plane.centre.y, self.plane.centre.z) rotate(self.plane.rotate.x,1,0,0) rotate(self.plane.rotate.y,0,1,0) rotate(self.plane.rotate.z,0,0,1) end PlaneShader = { v = [[ uniform mat4 modelViewProjection; attribute vec4 position; attribute vec2 texCoord; varying highp vec2 vTexCoord; void main() { vTexCoord = texCoord; gl_Position = modelViewProjection * position; } ]], f = [[ precision highp float; varying highp vec2 vTexCoord; void main() { lowp float marker = 1.0 /255.0; gl_FragColor = vec4(vTexCoord.x,vTexCoord.y,marker,1.0); } ]]}
  • Posts: 557

    @Yojimbo thanks for snooping my code! Wow!

  • edited August 2015 Posts: 557

    @Yojimbo I'm not sure I follow you on commenting out the colorCoordinates loop. The amount of calls to touched() per call to draw() can be around 20-1. By capturing every moving touch in colorCoordinates, and then drawing them all at once in draw(), I got way better results than I had before. I was basically able to doodle on the texture, with very natural looking curves and everything.

    Maybe, with your framerate boost, the ratio of touches to draws gets a lot smaller, so the difference isn't noticable. I can't say--haven't had the chance to run the code yet. Eager to as soon as I can though.

  • IgnatzIgnatz Mod
    Posts: 5,396

    B-)

  • Posts: 557

    Er--am I missing something? I just ran it and it only draws touches to the 2D overlay image, not to the texture at all. When I comment out line 123 there's no drawing to the screen at all. Is there something else I'm supposed to change?

  • Posts: 2,020

    Well, maybe that's an option if the frame rate drops. But at 60fps, touching with a single finger, you mostly see just 1 touch event per draw cycle, sometimes 2. I connect the dots with a line in the code above. Well, I don't think I actually deleted anything, so if you remove the comments, you can restore the colorCoordinates table loop. I doubt that it makes much difference to the performance, and you're right that it probably does create a more fluid line.

  • Posts: 2,020

    @UberGoober I repasted the code in my comment above, this time with the makePlane class (I didn't make any changes to it, I don't think), just in case I pasted it wrong, or you changed that class since you last posted it. Let me know if it's not working still.

  • Posts: 557

    @Yojimbo2000
    There we go! Awesome. Very cool!

  • Posts: 557

    And try adding

        local newX = 400
        tween( 10, makePlane.plane.rotate, { x = newX }, { easing = tween.easing.linear, loop = tween.loop.pingpong } )
    

    At the end of setup--it is so fun to see it working at the great smooth rate.

  • Posts: 2,020

    Nice! And good work putting this together. Wait, wasn't a finger fatness parameter added? You should use that to vary the thickness of the line

  • Posts: 557

    @Yojimbo, I think I ought to work, first, on abstracting this to accept multiple meshes.

    The end purpose, after all, is to let my girls be able to draw directly on the clothes of a 3D model. I have to be able to handle the pants, shirt, shoes, etc, separately.

    This is hard work for me but it's rewarding to work through it. You guys have been so much help!

  • @yojimbo2000, @Ignatz, I now have Yojimbo's refinement implemented with one of the dice models from Ignatz's 3D demos. I must say I'm really pleased with the results.

    Still very much a work in progress, but even at this stage it's fun to draw right on a 3D model.

    Gist:
    https://gist.github.com/GlueBalloon/9d81558f78f984ca30b1

    You'll notice a couple glitches:

    • the texture for the dice is mapped to the wrong vertexes somehow, making it look very strange as it rotates. I just slapped a standard Codea texture on the original dice model. I don't know how to fix this, and I'm not sure I even should; it won't be a problem with the .obj files.
    • since Yojimbo switched from drawing ellipses to drawing lines, the plus is that the drawing feels much more natural and pleasing. The minus is that occasionally a big straight line will get drawn where it shouldn't be. Also not sure if this is worth trying to fix before moving on.

    I think next I need to figure out how to get this working with an .obj file.

Sign In or Register to comment.