Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Shader return values

edited June 2013 in Shaders Posts: 372

Can I edit a shader so that it gives me the information of a pixel(which satisfies a condition). Like, say I want it to give me the x,y coordinates of a pixel whose colour is (1.,1.,1,1.) or the colour of a pixel whose x,y coordinates are width/2 and height/2 . Can we set values for global variables using the shader lab?


  • edited June 2013 Posts: 454

    I think the answer is no. The only output you can get from a shader is what it renders. So you can come up with some scenarios by rendering to a buffer (using setContext()) and using that output as a way of getting info out.

    For your second scenario, give me the color at a particular coordinate, that could be done, you just pass it the x,y and get it to do a texture lookup and render the color out. But it would be easier to just use image:read() or whatever the Codea function is.

  • IgnatzIgnatz Mod
    Posts: 5,396

    I've been exploring motion detection and trying to find ways to get information out of an image efficiently without querying every pixel value. As spacemonkey says, you could draw your mesh to an image instead of the screen, with setContext, but then you have to read all the pixels in the image anyway.

    Here is code for a shader that draws an image only for pixels that are changing (ie there is movement). You get a blue image on the screen.

    However, the best way I found (and I think the most common approach generally) was to sample points with image:get(x,y), eg only look at every 20th pixel, but this will only work where you are looking for chunks of colour.

  • edited June 2013 Posts: 372

    Thanks @Spacemonkey. @Ignatz I've been reading through your tutorial and then started to think about alternative ways of detecting motion. I wrote a code which detects me using colour of my skin. But that was very simple, what I couldn't understand was how to get the position of the hand from the picture. One more thing @Ignatz I realiased right now that the camera in your program should be flipped acroos the Y-axis that is left should be right and right should be left. Else it works perfectly.

  • IgnatzIgnatz Mod
    Posts: 5,396

    I did some more work on it, and calculated the average x,y position of all the pixels in my hand, and stored it. I can then see how that position changes at the next redraw. Usually it will change in both x and y, so you need to set a minimum threshold so you ignore small differences.

  • Posts: 454

    @Ignatz very nice. It crashed mine though... I think the issue is that you create the mesh every draw cycle, and I presume this blows out the memory somehow...

    Below modified to declare the mesh once up front, I also added a debug view of the camera in bottom right. Had to do a little messing around because the camera doesn't start working for a few frames....

    -- Motion shader --shows only moving pixels on screen function setup() cameraSource(CAMERA_FRONT) parameter.integer("FPS",0,60,6) img0=nil print("move the ipad around, wave your hand") m2=mesh() m2.shader = shader(MoveShader.vertexShader, MoveShader.fragmentShader) debugMesh = mesh() debugMesh:addRect(WIDTH-200, 100,400,200) debugMesh:setRectTex(1,0,0,1,1) end function draw() background(200) local img=image(CAMERA) for i=1,10000 do end if img0~=nil then --take picture if m2.size == 0 then u=m2:addRect(WIDTH/2,HEIGHT/2,img0.width,img0.height) m2:setRectTex(u,0,0,1,1) end m2.texture=img0 m2.shader.texture2=img m2:draw() debugMesh.texture =img debugMesh:draw() end img0=img FPS=FPS*.9+.1/DeltaTime end MoveShader = { vertexShader = [[ // // A basic vertex shader // //This is the current model * view * projection matrix // Codea sets it automatically uniform mat4 modelViewProjection; //This is the current mesh vertex position, color and tex coord // Set automatically attribute vec4 position; attribute vec4 color; attribute vec2 texCoord; //This is an output variable that will be passed to the fragment shader varying lowp vec4 vColor; varying highp vec2 vTexCoord; void main() { //Pass the mesh color to the fragment shader vColor = color; vTexCoord = texCoord; //Multiply the vertex position by our combined transform gl_Position = modelViewProjection * position; } ]], fragmentShader = [[ // // A basic fragment shader // //Default precision qualifier precision highp float; //This represents the current texture on the mesh uniform lowp sampler2D texture; uniform lowp sampler2D texture2; //The interpolated vertex color for this fragment varying lowp vec4 vColor; //The interpolated texture coordinate for this fragment varying highp vec2 vTexCoord; void main() { //Sample the texture at the interpolated coordinate lowp vec4 col = texture2D(texture,vTexCoord) ; lowp vec4 col2 = texture2D(texture2,vTexCoord) ; lowp float v1=col.r*.71+col.g*.21+col.b*.08; lowp float v2=col2.r*.71+col.g*.21+col.b*.08; //Set the output color to the texture color if (abs(v1-v2)>0.2) gl_FragColor = vec4(0.,0.,1.,0.5)* vColor; //blue else discard; //gl_FragColor = col* vColor; } ]]}
  • IgnatzIgnatz Mod
    Posts: 5,396

    Yeah, I had trouble with the camera not working for a couple of frames, had to put a while loop in to deal with that

    Maybe we should report that as a bug

  • Posts: 372

    @Ignatz, tried your new code the FPS always stays above 50, and that's awesome =D>. How'd you do that. If I just create a sprite using the camera my FPS drops to 30. Why does that happen??

  • edited June 2013 Posts: 372

    I'm facing this weird problem! While working on getting a solution to increase the FPS in the motion detection program i found out that If I increase the size of the sprite using CAMERA then the frame rate drops significantly. Am I doing something wrong?? Just can't understand.
    Here's the code.

    --# Main cameraSource(CAMERA_FRONT) function setup() print("keep sensitvity below camwidth") timer = 0 X,Y,reds,greens,blues = {},{},{},{},{} MoveX={} XX=100 parameter.number("FPS",0,60,60) parameter.integer("camwidth",10,500,10) parameter.integer("sensitivity",5,50,5,changeSense) end function CreateImage() img = image(camwidth,camwidth) local w,h=img.width,img.height setContext(img) sprite(CAMERA,w/2,h/2,w,h) setContext() end function draw() background(40,40,50) CreateImage() if img~=nil then sprite(img,WIDTH/2,HEIGHT/2,img.width,img.height)--in motion detection X,Y MotionAndDirection(Left,Right) end pushStyle() strokeWidth(5) stroke(255, 255, 255, 255) noFill() ellipse(XX,HEIGHT-100,60) popStyle() FPS = .9*FPS+.1/DeltaTime end function Left() XX=100 end function Right() XX=WIDTH-100 end function changeSense() X,Y,reds,greens,blues = {},{},{},{},{} moving=false end --# MotionAndDirection function MotionAndDirection(leftMove,rightMove) local sense = sensitivity if #reds==2*sense^2 then --pixels change reset X,Y,reds,greens,blues = {},{},{},{},{} moving=false end if not moving then if timer<ElapsedTime-.5 then if #MoveX>=2 then if MoveX[1]>MoveX[#MoveX] then rightMove() MoveX={} elseif MoveX[1]<MoveX[#MoveX] then leftMove() MoveX={} else MoveX={} end end timer=ElapsedTime sound(SOUND_PICKUP, 26546) end end local accuracy = 40 --half the range of value for which colour will change for j = 1,sense do for i=1,sense do red,green,blue = img:get(img.width/sense*i,img.height/sense*j) table.insert(X,img.width/sense*i) table.insert(Y,img.height/sense*j) table.insert(reds,red) table.insert(greens,green) table.insert(blues,blue) end end if #reds>sense^2 then for i=1,sense^2 do if reds[sense^2+i]>reds[i]+accuracy and greens[sense^2+i]>greens[i]+accuracy and --pixel change blues[sense^2+i]>blues[i]+accuracy or reds[sense^2+i]<reds[i]-accuracy and greens[sense^2+i]<greens[i]-accuracy and blues[sense^2+i]<blues[i]-accuracy then pushStyle() fill(255,0, 0, 255) ellipse(WIDTH/2-img.width/2+X[i],HEIGHT/2-img.height/2+Y[i],5)--make dots, from main X,Y popStyle() table.insert(MoveX,X[i]) moving=true end end end end
  • IgnatzIgnatz Mod
    Posts: 5,396

    Presumably it's because you're drawing more pixels.

    In answer to your previous question, I think the way I increased the frame rate was to analyse movement every N frames instead of every frame.

  • Posts: 372

    by drawing more pixels do you mean more pixels whose colour i heck has changed or not, then i think not. cause i used only 25 such pixels but yeah it could be cause i used to update the camera every frame not every X frames.

Sign In or Register to comment.