Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

ShaderToy API : access the biggest library of amazing shaders on the planet

in Shaders Posts: 101

I am working on implimenting the ShaderToy.com API, if you aren't familiar with ShaderToy.com, go there now, you are in for a treat.

So, the first thing you need to do is get an account there, and in your account you make an API key.
You do this in your profile page, I will gather more info if needed. There is info on the API if you hit the link at the very bottom of the page labeled "API"

Once you have your key, you can add it in the code I am working on, and I hope to polish this up and add full parsing along with my ShaderALL / GLSL library I have been working on and get a proper installer going on my git as soon as I can.

for now here is my class that will download a list of shaders, and parse the JSON of a shader:
main tab for testing:

function setup()
    ShaderToy:init()
    ShaderToy.key = '' -- INSERT YOUR API KEY
    ShaderToy:getList()
    ShaderToy:getShader('MdX3Rr') 
end

ShaderToy tab for class:

ShaderToy = class()

function ShaderToy:init()
    self.decoded = {}
    self.list = {}
    self.key = ''
end

function ShaderToy:getList(search, key)
    http.request('https://www.shadertoy.com/api/v1/shaders?key='..self.key,
    function(data, status, headers)
        ShaderToy:gotList(data, status, headers)
    end )
end

function ShaderToy:gotList(data, status, headers)
    if data ~= nil then
        print(data)
        saveText("Documents:ShaderToyListFull",data)
        local list = json.decode(data)
        tablePrint(list)
    end
end

function ShaderToy:getShader(sss, key)
    http.request( "https://www.shadertoy.com/api/v1/shaders/"..sss.."?key="..self.key,
    function(data, status, headers) 
        ShaderToy:decoder(data, status, headers) 
    end )
end

function ShaderToy:decoder(data, status, headers)
    if data ~= nil then 
        local decoded = json.decode(data) 
        tablePrint(decoded)

    elseif data == nil then
        return false
    end
end

function tablePrint(t)
    if t ~= nil then
        for k,v in pairs(t) do
            if type(v) == 'table' then
                tablePrint(v)
            else
                print(k..':'..v)
            end
        end
    end
end

Comments

  • Posts: 101

    Worth noting, tablePrint(list) takes a long time, you may want to comment it out.

  • Posts: 101

    actually it may be the json.decode(list) as well.

  • dave1707dave1707 Mod
    Posts: 6,230

    @AxiomCrux You don't need tablePrint calling itself. Try this routine to print the names. The json.decode routine isn't slowing thing down, it's the print(k..':'..v) that's doing it. You're trying to print 5838 names one line at a time. The output window wasn't meant to print a lot of lines at a time.

    function tablePrint(t)
        local tab={}
        if t ~= nil then
            for k,v in pairs(t) do
                if type(v) == 'table' then
                    for a,b in pairs(v) do
                        table.insert(tab,b)
                    end
                end
            end
        end    
        print(table.concat(tab,",")) 
    end
    
  • Posts: 101

    @dave1707 it appears that the text editor in the document browser also gets absurdly slow when I try to select parts of the text.

  • Posts: 101

    Would you be willing to explain a little of why this way of printing the table is better? It seems like you are creating another table out of something that's already a table? I know you've been doing this a lot longer than I have so I want to learn and improve my skill and understanding, and I am not currently clear just by looking at the code.

  • Posts: 35

    It makes a single (very long) string and prints it as a single (very large) line rather than printing hundreds or thousands of smaller lines.

  • dave1707dave1707 Mod
    Posts: 6,230

    @AxiomCrux I don't think any editor used by Codea is meant to work with large amounts of text. That's why large projects have a lot of tabs, that's to keep the amount of text in the editor small. I don't know the exact workings of the output window for printing, but I think it might work the same way strings work. Each time you do a print, the text is added to the text that's already there. It has to create another print area, copy the existing text and add the new text to it. When you were doing your print, you're printing 5838 names. That means the print routine has to create a new print area, copy the existing text to it, and then copy the new text to the end 5838 times. Each time the amount of text gets larger and larger. By creating another table with just the information that I want to print, the print(table.concat) routine just creates 1 large string that get sent to the print area once instead of 5838 times. As for your print routine calling itself, that would work great when you have a lot of tables within tables. In this case, it's only 1 table within a table. Just in case you didn't notice, I hid your ShaderToy key that you posted in the other discussion. I didn't think you wanted everyone using your key.

  • Posts: 101

    Oh @dave1707, you maybe didn't see the shader json that the tablePrint is meant to parse. There are several nested structures. The list is not what I designed it for, the shader is.

  • Posts: 101

    Where did I post my key??!! I did not mean to do that and I thought I had removed it??!!

  • Posts: 101

    @dave1707 I am super confused, what is the thread you are mentioning, I can't seem to find it and I really thought I had removed it entirely beforehand. I also don't remember posting that code anywhere else, so maybe some kind of accident that I want to look into what I might have done.

    RE: the list / print, it seems like everything works just fine and fast if I print one by one to the screen rather than the console. Now its on to parsing the shaders for display! :smiley:

  • Posts: 101

    Also, to reitterate, the printTable function I did was meant to print nested tables parsed from the json.decode of the actual shader, but your version is better for printing the list. I am wondering if there is a good way to merge the two, to make the best optimized merger for proposing in my other thread of adding a built in metamethod for printing tables.

    https://codea.io/talk/discussion/8588/print-table-example-add-to-api-metamethods#latest

  • Posts: 101

    Is there a way to delete or edit my own forum reply posts?

  • dave1707dave1707 Mod
    Posts: 6,230

    @AxiomCrux If you want to delete or re-edit your own posts, tap the gear icon at the upper right of your post. It gives you some options.

  • Posts: 101

    @dave1707 that only seems to let me edit the original top level post, and not my replies. The replys are what I am wondering about.

  • dave1707dave1707 Mod
    Posts: 6,230

    I signed in as a non MOD and posted a reply. There was a gear icon at the top right of my post. When I tapped on it, there was an EDIT option. Apparently you can only edit your posts and not delete them. That means you can completely backspace over your post and leave it blank, or change something you don't like. I guess a MOD can only delete posts.

  • Posts: 101

    Is it possible that that doesn't show up on the browser I am using ? Or that it is only available to moderators? I've got the one on the main post but nothing on any of my replies

  • Posts: 101

    I'm using the current Safari on iOS with an iPad Pro

  • dave1707dave1707 Mod
    Posts: 6,230

    I'm not sure what's up with the iPad Pro, but it doesn't act like a normal iPad. To get the gear to show, tap on your name on the post you want the gear to show on.

  • Posts: 101

    hahahaha good one! what is it with apple and making life hard for devs. :P`

  • edited September 5 Posts: 196

    Some of the shadertoy shaders seem to use more than one buffer, is it possible to make those ones work within the Codea implementation of shaders?

  • Posts: 449

    @piinthesky what do you mean by "more than one buffer"? Could you provide a link to an example?

  • Posts: 196

    for example: https://www.shadertoy.com/view/Ms3XWN

    there is a buf a, buf b, image, each with their own main()

  • edited September 7 Posts: 449

    @piinthesky I'm not a shader pro, but here's what my brain thinks about this.

    Roughly speaking, shaders only act on images - the fragments (pixels) and the vertices (of its mesh). This means that you can make all kind of crazy computations inside the shader code (like in "Buf A" from your example) but sooner or later that code has to affect either a pixel or a vertex (otherwise its worthless).

    Buf A: the author seems to handle all the controls here (buttons, map setup, etc) but you also see that the data is stored in fragment color (pixels)

    Buf B: handles the object drawings (ghosts, pacman, etc)

    Image: combines everything

    The result of all shader-steps is always an image. Codea can only run one shader per mesh. This means, that this example code will not run inside Codea without further modifications.

    From what I see, I would conclude that you should try use multiple render passes. Meaning, apply the shader code from "Buf A" onto an empty image. Than use that resulting image and apply another shader onto it (code from "Buf B"). Finally take that image and apply shader code from "Image" tab onto it. The final outcome is the combination of all the steps.. which is what you want.

  • Posts: 101

    I did some more work on this last night, I set up the remainder of the encoder / decoder to save a file to a text document, including all of the JSON which contains all the needed info for setting up multiple render buffers.

    @piinthesky the buffers will each be rendered as an image in memory using setContext(bufferName). I have a few rather complex Codea projects going, and in a few of them I have already tested / solved all of the things needed to accomplish this.

    Also, check out InteractiveShaderFormat.com which I am working on supporting, it's rather brilliant way to create complex multi-pass shaders with mapped controls. I have also set up my own means to accomplish all of this using regex to parse uniforms and setup UI objects, with optional short form comments after each uniform to set the range while still retaining full GLSL_ES compatibility.


    @se24vad for the most part you are correct, for the sake of clarity, here are a few specifics about the nature of ShaderToy:

    shadertoy is based entirely around the fragment (aka pixel) shader, and does not use any vertex shader (there is a cool vertexShaderArt.com for a fully vertex shader based online creative tool)

    the wizard - creator of ShaderToy Iñigo Quilez goes by the handles "iq" or "rgba", he explains the concept in his seminal paper "Rendering Worlds with Two Triangles" - http://www.iquilezles.org/www/material/nvscene2008/rwwtt.pdf
    ( if I recall correctly, one of my rather amazing friends / collaborators -Taylor, used to work with at Pixar and now makes an awesome modular synthesis app for iOS called Audulus 3 )

    The general idea being to create a quad (two triangles) that make up the size of the desired render area and apply a fragment shader that does all of the actual drawing/rendering.

    This idea historically comes from the "Demo Scene" world.

    As ShaderToy grew, they added the ability to load textures, render to offscreen buffers (for complex feedback / layering / post FX options that are extremely difficult if not impossible without)

    The 3d render technique commonly used in shaderToy is called "Ray Marching" and is rather mind blowing in its simplicity. If anyone would like, I can make a tutorial or explain further. It is one of the most facinating, powerful, and fun techniques for creating 3d images I have ever seen. If you are eager to learn more, search "Ray Marching Tutorial" or something like that on google / and for that matter there are loads of amazing tutorials right on shadertoy.

  • Posts: 196

    @se24vad ok thanks i get it

  • Posts: 522

    I'm wondering if you could outline the steps for using this. I'm curious to try it. It does seem rather a lot of work before you see anything though, if I understand correctly.

    It seems like the things you have to do are:

    • Get a key to ShaderToy
    • Grab the code at the top of the forum and run it
    • Wait a long time for a file to be created and filled with a bunch of JSon
    • Make a project that uses a shader for something and include ShaderToy in it
    • Always run your project while online because ShaderToy dynamically loads the shaders off the web and you don't actually get the shaders as files of your own.
    • Now you can experiment with different shaders

    I'm almost positive I have a lot of that wrong. If you'd be so kind, can you correct me where I'm wrong?

  • Posts: 101

    Almost done with the new version, just need to finish the function for viewing one immediately. Currently have 3 main functions in the API that are:

        ShaderToy:init(" '') -- INSERT YOUR API KEY string, make one by creating an account on shadertoy.com and in your account making a key, copy paste the string into these quotes.
        ShaderToy:saveShader('lsjBWG') -- saves a shaderToy json to a txt file in documents
        ShaderToy:tabShader('XtXyDN')  -- makes a tab with this shader
    
    

    Here is the current version, the above functions work and I may be able to finish the immediate viewer for it tonight.

    ShaderToy = class()
    
    function ShaderToy:init(API_key, SHADER)
        self.decoded = {}
        self.encoded = ''
        self.key = API_key or ''
    
        if SHADER ~= nil and self.key ~= nil then
            self:getShader(SHADER)
        end
    
        --[[
        self.list = readText("Documents:ShaderToyListFull")
        if self.list == nil then
            ShaderToy:getList()
        end
        -- ]]
    end
    
    
    function ShaderToy:setKey(key)
        if key ~= nil and type(key) == string then
            self.key = key
        end
    end
    
    --------------- LIST ---------------------
    
    function ShaderToy:getList(search, key)
        http.request('https://www.shadertoy.com/api/v1/shaders?key='..self.key,
        function(data, status, headers)
            ShaderToy:gotList(data, status, headers)
        end )
    end
    
    function ShaderToy:gotList(data, status, headers)
        if data ~= nil then
            --print(data)
            --local list = json.decode(data)
            --tablePrint(list)
            saveText("Documents:ShaderToyListFull",data)        
        end
    end
    
    function ShaderToy:loadList(file)
        if file ~= nil then
    
        end
    end
    
    ------------------ SHADER ---------------------
    
    function ShaderToy:getShader(sss)
        if sss ~= nil then
            http.request( "https://www.shadertoy.com/api/v1/shaders/"..sss.."?key="..self.key,
            function(data, status, headers) 
                ShaderToy:decoder(data, status, headers) 
            end )
        else
            print('shader == nil')
        end
    end
    
    function ShaderToy:saveShader(sss, destination)
        if sss ~= nil then
            http.request( "https://www.shadertoy.com/api/v1/shaders/"..sss.."?key="..self.key,
            function(data, status, headers) 
                ShaderToy:decoder(data, status, headers)
                ShaderToy:encoder(self.decoded)
                local destFile = destination or "Dropbox:ShaderToyTest"
                saveText(destFile, self.encoded)
            end )
        else
            print('shader == nil')
        end
    end
    
    function ShaderToy:tabShader(sss, destination)
        if sss ~= nil then
            http.request( "https://www.shadertoy.com/api/v1/shaders/"..sss.."?key="..self.key,
            function(data, status, headers) 
                ShaderToy:decoder(data, status, headers)
                ShaderToy:encoder(self.decoded)
                local destTab = destination or self.decoded.Shader.info.id or sss or 'tab'
                --tableTab(destTab)
                saveProjectTab(destTab, sss.. '=' .. '[['..self.encoded..']]')
            end )
        else
            print('shader == nil')
        end
    end
    
    ------------------ JSON ---------------------
    
    function ShaderToy:encoder(t)
        local stateTable = {
            indent = true,
            keyorder = {},
            level = 4,
            --buffer    =   -- array to store strings for the result,
            --bufferlen =   -- index of last element of buffer,
            --exception =   -- function called when encoder can not encode a given value 
                            -- will be given the parameters, reason, value, state, and defaultmessage.
        }
        if t ~= nil then
            self.encoded = json.encode(t, stateTable)
        end
    end
    
    function ShaderToy:decoder(data, status, headers)
        if data ~= nil then 
            self.decoded = json.decode(data) 
            tablePrint(self.decoded)
        else -- if data == nil then
            print('data == nil')
            return false
        end
    end
    
    -------------------- TABLE ------------------------
    
    function tableTab(t)
        if t ~= nil then
            for k,v in pairs(t) do
                if type(v) == 'table' then
                    tableTab(v)
                else
                    -- pp = pp..k..":"..v
                    print(k..':'..v)
                end
            end
        end
    end
    
    function tablePrint(t)
        if t ~= nil then
            for k,v in pairs(t) do
                if type(v) == 'table' then
                    tablePrint(v)
                else
                    -- pp = pp..k..":"..v
                    print(k..':'..v)
                end
            end
        end
    end
    
    --------------------------------------------
    
    function ShaderToy:draw()
        -- Codea does not automatically call this method
    end
    
    function ShaderToy:touched(touch)
        -- Codea does not automatically call this method
    end
    
    --------------------------------------------
    
    function drawList()
        if st~=nil then
            idx = math.floor(math.fmod(ElapsedTime*60, 5000)+2)
            sss = sss..st[idx]
            text(sss, WIDTH/2, HEIGHT/2)
        end
    end
    
    function initText()
        textMode(CENTER)
        stroke(100,100,100)
        fill(100,100,100)
        strokeWidth(1)
        fontSize(8)
        textAlign(CENTER)
        textWrapWidth(WIDTH*.5)
        noSmooth()
    end
    
    ------------------------------------------------------------------------------------------------------------------------------------
    --https://raw.githubusercontent.com/beautypi/shadertoy-iOS-v2/master/shadertoy/shaders/fragment_base_uniforms.glsl
    fragment_base_uniforms = [[
    #version 300 es
    
    precision highp float;
    precision highp int;
    precision highp sampler2D;
    
    uniform vec3      iResolution;                  // viewport resolution (in pixels)
    uniform float     iGlobalTime;                  // shader playback time (in seconds)
    uniform vec4      iMouse;                       // mouse pixel coords
    uniform vec4      iDate;                        // (year, month, day, time in seconds)
    uniform float     iSampleRate;                  // sound sample rate (i.e., 44100)
    uniform vec3      iChannelResolution[4];        // channel resolution (in pixels)
    uniform float     iChannelTime[4];              // channel playback time (in sec)
    
    uniform vec2      ifFragCoordOffsetUniform;     // used for tiled based hq rendering
    uniform float     iTimeDelta;                   // render time (in seconds)
    uniform int       iFrame;                       // shader playback frame
    ]]
    
    
    
    --https://raw.githubusercontent.com/beautypi/shadertoy-iOS-v2/master/shadertoy/shaders/fragment_main_sound.glsl
    fragment_main_sound = [[
    out vec4 glFragColor;
    
    void main()  {
        float t = ifFragCoordOffsetUniform.x + (((iResolution.x-0.5+gl_FragCoord.x)/11025.) + (iResolution.y-.5-gl_FragCoord.y)*(iResolution.x/11025.));
        vec2 y = mainSound( t );
        vec2 v  = floor((0.5+0.5*y)*65536.0);
        vec2 vl = mod(v,256.0)/255.0;
        vec2 vh = floor(v/256.0)/255.0;
        glFragColor = vec4(vl.x,vh.x,vl.y,vh.y);
    }
    ]]
    
    --https://raw.githubusercontent.com/beautypi/shadertoy-iOS-v2/master/shadertoy/shaders/fragment_main_copy.glsl
    fragment_main_copy = [[
    #version 300 es
    
    precision highp float;
    precision highp int;
    
    uniform highp sampler2D sourceTexture;
    uniform vec2 sourceResolution;
    uniform vec2 targetResolution;
    
    out vec4 glFragColor;
    
    void main()  {
        vec2 fragCoordScaled = gl_FragCoord.xy / targetResolution;
        fragCoordScaled *= targetResolution / sourceResolution;
    
        if( fragCoordScaled.x >= 1. || fragCoordScaled.y >= 1. ) discard;
    
        // gl_FragColor = vec4( fragCoordScaled, 0,1); //
        glFragColor = texture( sourceTexture, fragCoordScaled );
    }
    ]]
    
    --https://raw.githubusercontent.com/beautypi/shadertoy-iOS-v2/master/shadertoy/shaders/vertex_main.glsl
    vertex_main = [[
    #version 300 es
    
    precision highp float;
    precision highp int;
    
    in vec3 position;
    
    void main() {
        gl_Position.xyz = position;
        gl_Position.w = 1.0;
    }
    ]]
    
    --https://raw.githubusercontent.com/beautypi/shadertoy-iOS-v2/master/shadertoy/shaders/fragment_main_image.glsl
    fragment_main_image = [[
    out vec4 glFragColor;
    
    void main()  {
        mainImage(glFragColor, gl_FragCoord.xy + ifFragCoordOffsetUniform );
    }
    ]]
    
Sign In or Register to comment.