Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Passing an array to a shader?

in Questions Posts: 147

Can I somehow pass an array to a shader, or get acces to its contents?

Its an array of up to maybe 60 images and I want to read their pixel colours.

Comments

  • dave1707dave1707 Mod
    Posts: 6,397

    @Kirl Instead of passing 60 images to the shader, pass 1 at a time. If I'm not mistaken, you can't get any info back from a shader. So if you read the pixel colors, what info are you after.

  • Posts: 213

    @Kirl To read information and retrieve data from shaders, look at the buffer in the reference book. Basically saying myMesh:buffer("variableName"), you can get a variable from a shader.

  • Posts: 147

    The array saves the last 60 frames from the camera and I make a new image by combining parts from those images. I have it working without a shader by copying regions and/or individual pixels from those images, but that's terribly slow as expected.

    So every frame I make one new image from all of those 60 images.

    I was hoping I could maybe do everything inside of a shader to speed things up?

  • dave1707dave1707 Mod
    Posts: 6,397

    @CamelCoder Do you have a small example of using buffer to retrieve info from a shader. @Kirl What are you trying to do by making a new image from 60 images.

  • Posts: 147

    You can make some trippy effects by drawing parts of the screen later than others. I use a noise where white is the most recent frame and black 60 frames back, or from top to bottom where the top is now and the bottom 60 frames back.

    It works, but it's too slow.

    So I'm not trying to read stuff out from the shader (although that sounds useful) but rather get that array of 60 images into the shader. If that's at all possible.

  • dave1707dave1707 Mod
    Posts: 6,397

    @Kirl If I understand shaders correctly, which I may not, they work on one image at a time. You don't send an array of images to it, you send 1 image to it. That doesn't mean you can't send 60 images to it one after the other. I'll have to look thru my shader examples, but I might have something where i take an image and create a different image seperate from the first. If that's the case, the 60 images can be sent one after the other and all of them can be use to create another seperate image. I'll let you know what I find.

  • dave1707dave1707 Mod
    Posts: 6,397

    @Kirl I found the shader I was thinking of. I'm passing 2 images to it and any pixels that are different by an amount I choose, I turn them yellow. I use the camera and every 2 images gets sent to the shader to compare them. So if I can pass 2 images to the shader, maybe you can do 60. I'll have to play with it more to see what happens if I try more images.

  • dave1707dave1707 Mod
    edited July 27 Posts: 6,397

    @Kirl Are you trying to do something like this. This uses the camera and each frame part of the image is moved down a section of the screen. You can change nbr to change the number of sections shown.

    displayMode(FULLSCREEN)
    
    function setup()
        nbr=30
        h=HEIGHT//nbr
        imgTab={}
        for z=1,nbr do
            table.insert(imgTab,image(WIDTH,h))
        end
    end
    
    function draw()
        background(40, 40, 50)
        collectgarbage()
        for a,b in pairs(imgTab) do
            sprite(b,WIDTH/2,HEIGHT-(h*a)+h/2)
        end
        img=image(CAMERA) 
        if img~=nil then
            imgTab[1]=img:copy(0,HEIGHT-h,WIDTH,h)
            for z=nbr,2,-1 do
                imgTab[z]=imgTab[z-1]:copy(0,0,WIDTH,h)   
            end  
        end
        fill(255)
        text(1/DeltaTime//1,WIDTH/2,HEIGHT-30)
    end
    
  • dave1707dave1707 Mod
    Posts: 6,397

    @Kirl Here's an example of passing multiple images to a shader. The only problem is an error on the line m.shader.tbl trying to create a table. I don't know shaders well enough to know why, so maybe someone else can fix it. The second version sends 3 seperate images to the shader. That one works.

    This version sends a table to the shader, but causes an error.

    function setup() 
        img1=readImage("Planet Cute:Enemy Bug")
        img2=readImage("Planet Cute:Character Horn Girl")
        img3=readImage("Planet Cute:Star")
        m=mesh()
        m:addRect(300,300,img1.width,img1.height)
        m.shader=shader(Shader.vs,Shader.fs)
    
        m.shader.tbl={img1,img2,img3}    -- this line causes an error
    end
    
    function draw() 
        background(40, 40, 50) 
        m:draw()
    end
    
    Shader = 
        {   vs = [[
            uniform mat4 modelViewProjection;
            attribute vec4 position; 
            attribute vec2 texCoord;
            varying highp vec2 vTexCoord;    
            void main() 
            {   vTexCoord = texCoord;
                gl_Position = modelViewProjection * position;
            }]],
    
            fs = [[
            uniform lowp sampler2D tbl[3];              // 3 = number of images
            varying highp vec2 vTexCoord; 
            mediump vec4 col1;
            void main() 
            {   
                col1=vec4(0.,0.,0.,0.);
                for (lowp int z=0;z<3;z++)              // 3 = number of images
                    col1 = col1 + texture2D( tbl[z], vTexCoord);
                gl_FragColor = col1/3.;                 // 3 = number of images
            }]]
        }
    

    This version sends 3 different images to the shader and combines them.


    function setup() img1=readImage("Planet Cute:Enemy Bug") img2=readImage("Planet Cute:Character Horn Girl") img3=readImage("Planet Cute:Star") m=mesh() m:addRect(300,300,img1.width,img1.height) m.shader=shader(Shader.vs,Shader.fs) m.shader.t1=img1 m.shader.t2=img2 m.shader.t3=img3 end function draw() background(40, 40, 50) m:draw() end Shader = { vs = [[ uniform mat4 modelViewProjection; attribute vec4 position; attribute vec2 texCoord; varying highp vec2 vTexCoord; void main() { vTexCoord = texCoord; gl_Position = modelViewProjection * position; }]], fs = [[ uniform lowp sampler2D t1; uniform lowp sampler2D t2; uniform lowp sampler2D t3; varying highp vec2 vTexCoord; mediump vec4 col1; void main() { mediump vec4 col1 = texture2D( t1, vTexCoord); mediump vec4 col2 = texture2D( t2, vTexCoord); mediump vec4 col3 = texture2D( t3, vTexCoord); gl_FragColor = (col1+col2+col3) / 3.; }]] }
  • Posts: 147

    thanks dave! I'll look into your examples! =)

    Below is an example of the kind of effects I'd like to do with shaders, the effect doen't really work at these speeds. It can be a great effect with a static camera especially, anything that moves will deform while the rest of the scene appears normal.

    -- Cam
    
    -- Use this function to perform your initial setup
    function setup()
        cameraSource(CAMERA_BACK)
    
        frames = {}
        frameBuffNr = 30
    
        parameter.boolean("toggleMode", true)
        parameter.integer("frameBuffNr", 1, 60, frameBuffNr)
        parameter.number("waves", 0, .1, .01)
    
        pic = image(400,400)    -- eventual resolution
    end
    
    function p2pNoise(pic1, pic2)
        rx,ry = pic1.width/pic2.width, pic1.height/pic2.height
        for i=1, pic2.width do
            for j=1, pic2.height do
                fi = (noise(i*waves, j*waves)+1)/2 * #frames //1 +1   -- frame index
    
                c = color( frames[fi]:get(i*rx//1, j*ry//1) )
                pic2:set(i,j, c)
            end
        end
        sprite(pic, WIDTH/2, HEIGHT/2 ,WIDTH, HEIGHT)
    end
    
    function p2pScan(pic1, pic2)
        steps = frameBuffNr
        step = HEIGHT/steps
        for i=1, HEIGHT, step do
            local r = i/HEIGHT
            fi = math.ceil(r * #frames) -- (math.sin(r * math.pi) +1)/2 * #frames //1 +1 --
            il = frames[fi]:copy(1,math.ceil(r*pic1.height), pic1.width, math.ceil(pic1.height/steps))
            sprite(il, WIDTH/2, i+math.ceil(pic1.height/steps)/2, WIDTH, step)
        end
    end
    
    -- This function gets called once every frame
    function draw()
        -- This sets a dark background color 
        background(40, 40, 50)
    
        -- Do your drawing here
        collectgarbage()
    
        while #frames > frameBuffNr do
            table.remove(frames)
        end
    
        img = image(CAMERA)
        if img ~= nil then
            table.insert(frames, 1, img)
    
            if toggleMode == true then   p2pScan(img, pic)
            else                         p2pNoise(img, pic) end
    
            sprite(img, 100,100, 200,200)
        end
    
        text(1//DeltaTime, WIDTH/2,HEIGHT-50)
    end
    
    
  • dave1707dave1707 Mod
    edited July 28 Posts: 6,397

    @Kirl Tried your latest code. Keeps crashing after a few seconds. I thought you needed collectgarbage(), but you already have it. Added a collectgarbage() inside the while loop and that seemed to fix the crashes. This runs at 0 to 5 frames/sec on my iPad Air.

    EDIT: Still crashes, just takes longer for it to happen.

  • dave1707dave1707 Mod
    Posts: 6,397

    @Kirl Here an example using a shader. I take an image and randomly shift lines of 10 pixels back and forth. You can add code to move the lines more smoothly if you want. This runs a 6 FPS on my iPad Air.

    supportedOrientations(PORTRAIT_ANY)
    displayMode(FULLSCREEN)
    
    function setup() 
        offset={}
        m=mesh()
        m:addRect(WIDTH/2,HEIGHT/2,WIDTH,HEIGHT)
        m.texture=readImage("Cargo Bot:Startup Screen") 
        m.shader=shader(vShader,fShader)
    end
    
    function draw() 
        background(40, 40, 50) 
        for z=1,100 do
            offset[z]=math.random()/50
        end
        m.shader.offset=offset
        m:draw()
        fill(255,0,0)
        text(1/DeltaTime//1,WIDTH/2,HEIGHT-50)
    end
    
    vShader = [[
        uniform mat4 modelViewProjection;
        attribute vec4 position; 
        attribute vec2 texCoord;
        varying highp vec2 vTexCoord;
        void main() 
        {   vTexCoord = texCoord;
            gl_Position = modelViewProjection * position;
        }    ]]
    
    fShader = [[
        uniform lowp sampler2D texture;
        varying highp vec2 vTexCoord;
        uniform lowp float offset[100];
        lowp float xx;
        void main() 
        {   lowp int d=0;
            {   for (highp float z=0.; z<=1.; z=z+.01)
                {   if (vTexCoord.y>z && vTexCoord.y<z+.01)
                        xx=offset[d]; 
                    d=d+1;               
                }
                if (vTexCoord.x+xx>=1.0)
                    xx=xx-1.0;
            }
            gl_FragColor=texture2D( texture,vec2(vTexCoord.x+xx,vTexCoord.y));
        }    ]]
    
  • Posts: 147

    Thanks dave, I'll play around with it.

    BTW, if it crashes you have to lower the framebuffnr before running (this is the nr of camera frames it saves), I was surprised I managed save up to 30-60 images myself.

    This reminds me of a related question I had; is it possible to set the resolution of the actual camera image? So that I can lower the size of the frames that I store in the array?

  • dave1707dave1707 Mod
    Posts: 6,397

    @Kirl As far as I know, you can't change anything with the camera. But you can copy a section of the screen to a smaller size.

    function setup()
    end
    
    function draw()
        background(0)
        collectgarbage()
        img1=image(CAMERA)
        if img1~=nil then
            img=img1:copy(100,100,400,400)
            sprite(img,WIDTH/2,HEIGHT/2)
        end
    end
    
Sign In or Register to comment.