Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Json encode / decode questions

in Questions Posts: 1,254

Apologies for such a basic question. This is the first time I've used the json.encode / json.decode functions, and I've clearly missed something basic. Though the help document indicates that the encode statement will take any object that implements the metamethod, it gives no example of how to do so. I've searched Jason on the forum, and also did not see an example.

I tried implementing as I have simple things like a change in _index, but it doesn't seem to be effective as I'm still getting an error stating encode won't that "user data."

So,,, someone give me a smack and explain the blindingly obvious thing I missed. Thanks.

Tagged:

Comments

  • Posts: 2,020

    My entry to the gamejam uses json for saving levels. Have a look at the Level IO tab in this:

    https://gist.github.com/Utsira/8a364a35f397f26aefad

  • Posts: 1,254

    @yojimbo2000 Thanks, but it looks as if you're encoding simple tables. That works fine for me, as well. My question is implementing the __tojson method within a complex object so that it can properly encoded. I can turn the object into the save string I've been using to file them away, or a set of key-value pairs and encode that, but there must be a neater way. After all, that seems like the work I expect out of encode / decode.

  • Posts: 2,020

    Oh I see. You mean adding json encode/decode support to, say, a vec2? Can you be more specific about which objects you want to encode?

  • Posts: 1,254

    These are classes I created in the current application (in this case, a class called "Location" which is actually a work location within an office / work site and contains a number of fields, including a series of coordinates).

    However, just an example of how to implement __tojson so that it supports json.encode would be fine. Even on something like a Vec2. All I can find is the notation in the help file that indicates .encode supports any object that implements the metamethod __json encode, but I can't find any examples here or elsewhere showing what that implementation looks like.

    Thanks.

  • edited June 2015 Posts: 2,020

    So you're trying to encode the instances of the location class? If they're held in a table, you can encode that table, unless it contains "user data" (ie vectors, matrices etc). I'd also like to know how to implement json encoding for vectors etc. Do you know specifically within the location class, what part of it is causing the "user data" error? What I'm saying is, I don't think you need to implement a custom json encode for your location class as such, you just need json encoding for the objects contained within it (from your description, vectors).

  • Posts: 2,020

    Eg. Instances of this class can be encoded, as long as they don't contain "user data" (vectors).


    --# Main -- Json Test -- Use this function to perform your initial setup function setup() --cant use vec2s objects={ MyClass(17), MyClass(23), MyClass(9) } tabSave() tabLoad() end function tabLoad() str=readProjectTab("Test") tab=json.decode(string.sub(str,3,-3)) for k,v in pairs(tab) do print(v) end -- tab[1]:draw() --doesnt work end function tabSave() str=json.encode(objects) saveProjectTab("Test", "--"..str.."--") end --# MyClass MyClass = class() function MyClass:init(x) -- you can accept and set parameters here self.foo = "bar" self.x=x -- self.pos=vec2(50,50) --vec2 cant be used with json end function MyClass:draw() print (self.foo) end function MyClass:touched(touch) -- Codea does not automatically call this method end --# Test --[{"x":17,"foo":"bar"},{"x":23,"foo":"bar"},{"x":9,"foo":"bar"}]--
  • dave1707dave1707 Mod
    Posts: 7,522

    I'm not sure the encode, decode works properly, or else I'm wrong in what I'm expecting. In the example below, table tab1 is created. I encode it and print the result. I then decode it into table tab2. I encode it and print the result so I can compare it to the original table. The results are the same, but maybe out of order which shouldn't matter. I can print values from tab1, but I get errors when I try to print it from tab2.

    function setup()
        tab1={a=123,b=456,{3,4,5}}
        z=json.encode(tab1)
        print(z)
    
        tab2=json.decode(z)
        z=json.encode(tab2)
        print(z)
    
        print("\ntab1 --------")
        for a,b in pairs(tab1) do
            print(a,b)
        end
    
        print("\ntab2 --------")
        for a,b in pairs(tab2) do
            print(a,b)
        end
    
        print("\ntab1 --------")
        print(tab1[1])
        print(tab1[1][1],tab1[1][2],tab1[1][3])
    
        print("\ntab2 --------")
        print(tab2[1])
        print(tab2[1][1],tab2[1][2],tab2[1][3])    
    end    
    
  • Posts: 154

    @dave1707, tab2 doesn't have an integer key 1, but a string "1". That's because JSON only allows string keys, so when you encode/decode tab1 into tab2, the [1] = {3,4,5} becomes: ["1"] = {3,4,5}. You can see that in the first print of z.

  • Posts: 2,020

    @dave1707 mixing key-values with an array is a terrible idea IMO. You shouldn't expect to be able to index it by number.

  • Posts: 2,020

    @Mark instead of adding the tojson method to whatever object you need encoding, how about using the "exception" field to catch the parts of the class that are causing json to fall over. If it's just vec2s that should be quite straight forward. Can you show us the code for the class whose instances you're trying to encode?

  • edited June 2015 Posts: 2,020

    @Mark here's my example from above, but using the "exception" field to catch the vec2s and convert them into a table/json object: "pos":{"x":9.0,"y":50.0}. I haven't worked out how to do the equivalent decode though yet. I need to work out how to indictate that this is a type vec2.

    Edit: v1.2 more elegant jsonException class, now indicates that table-object is a vec2 "pos":{"vec2":{"x":9.0,"y":50.0}}. No decoding of vec2s yet.

    --# Main
    -- Json Test
    
    -- Use this function to perform your initial setup
    function setup() 
    
        objects={
        MyClass(17), MyClass(23), MyClass(9)
        }
    
        tabSave()
        tabLoad()
    end
    
    function tabLoad()
        local str=readProjectTab("Test")
        local tab=json.decode(string.sub(str,3,-3))
        tabIterate(tab, "")
       -- tab[1]:draw() --doesnt work
    end
    
    function tabIterate(tab, indent)
        for k,v in pairs(tab) do
            if type(v)=="table" then print(indent..k.."=") tabIterate(v, indent.."  ") else print(indent..k,v) end
        end
    end
    
    function tabSave()
         str=json.encode(objects, {exception = jsonException})
        saveProjectTab("Test", "--"..str.."--")
    end
    
    function jsonException(reason, v, state, message)
        if v.x and v.y and not v.z then --vec2
            return json.encode({vec2={x=v.x,y=v.y}})
        end
    end
    
    --# MyClass
    MyClass = class()
    
    function MyClass:init(x)
        -- you can accept and set parameters here
        self.foo = "bar"
        self.x=x
        self.pos=vec2(x,50) --vec2 cant be used with json without an exception class
    end
    
    function MyClass:draw()
        print (self.foo)
    end
    
    function MyClass:touched(touch)
        -- Codea does not automatically call this method
    end
    
    --# Test
    --[{"pos":{"vec2":{"x":17.0,"y":50.0}},"foo":"bar","x":17},{"pos":{"vec2":{"x":23.0,"y":50.0}},"foo":"bar","x":23},{"pos":{"vec2":{"x":9.0,"y":50.0}},"foo":"bar","x":9}]--
    
  • edited June 2015 Posts: 2,020

    Here's a version that attempts to decode the vec2s. It's not particularly elegant, as there's no corresponding equivalent to the exception function in json.decode (this is where implementing the __tojson meta method might be the way to go, as @Mark said in his original enquiry).

    Edit: here's a working version of the code

    --# Main
    -- Json Test
    
    -- Use this function to perform your initial setup
    function setup()
    
        objects={
        MyClass(17), MyClass(23), MyClass(9)
        }
    
        tabSave()
        tabLoad()
    end
    
    function tabLoad()
        local str=readProjectTab("Test")
        local result=json.decode(string.sub(str,4,-2))
        result=tabIterate(result)
        result=tabIterate(result) --re-iterate to see if decoding has stuck
    end
    
    function tabSave()
        str=json.encode(objects, {exception = jsonException, indent = true})
        saveProjectTab("Test", "--["..str.."]")
    end
    
    local underscore = {"=","~","-","."} --underscore characters indicate table depth
    
    function tabIterate(tab, i)
        local new = {}
        local i = i or 0
        local indent = string.rep(" ", i*3) --depth of indentation
        for k,v in pairs(tab) do
            if type(v)=="table" then
                if v.vec2 then --convert back to vec2
                    new[k] = vec2(v.x, v.y)
                    print (indent..k..": decoding vec2"..tostring(new[k]))
                else
                    local under = string.rep(underscore[math.min(#underscore, i+1)], string.len(k))
                    print(indent..k.."\n"..indent..under)
                    new[k]=tabIterate(v, i+1) --increase indent upon recursion
                end
            else
                if type(v)=="userdata" and v.x and v.y then --check whether vec2 decoding has worked
                    print(indent..k..": vec2"..tostring(v))
                else
                    print(indent..k..": "..v)
                end
                new[k]=v
            end
        end
        return new
    end
    
    function jsonException(reason, v, state, message)
        if v.x and v.y and not v.z then --vec2
            return json.encode({x=v.x,y=v.y,vec2=true}) --set a vec2 flag
        end
    end
    
    --# MyClass
    MyClass = class()
    
    function MyClass:init(x)
        -- you can accept and set parameters here
        self.foo = "bar"
        self.x=x
        self.pos=vec2(x,50) --vec2 cant be used with json without an exception class
    end
    
    function MyClass:draw()
        print (self.foo)
    end
    
    function MyClass:touched(touch)
        -- Codea does not automatically call this method
    end
    
    --# Test
    --[[{
        "foo":"bar",
        "pos":{"vec2":true,"y":50.0,"x":17.0},
        "x":17
      },{
        "foo":"bar",
        "pos":{"vec2":true,"y":50.0,"x":23.0},
        "x":23
      },{
        "foo":"bar",
        "pos":{"vec2":true,"y":50.0,"x":9.0},
        "x":9
      }]]
    
  • Jmv38Jmv38 Mod
    Posts: 3,295

    here is how to implement _toJjson to vec2


    --# Main -- Json Test -- Use this function to perform your initial setup function setup() --cant use vec2s objects={ MyClass(17), MyClass(23), MyClass(9) } Vec2ToJson() tabSave() out = tabLoad() setmetatable(out,{__tostring = function(obj) return json.encode(obj,{indent=true}) end}) print(out) end function Vec2ToJson() local meta = getmetatable(vec2()) meta.__tojson = function(t) local a = {x=t.x,y=t.y} return json.encode(a) end end function tabLoad() local str=readProjectTab("Test") str = string.sub(str,4,-4) return json.decode(str) end function tabSave() local str=json.encode(objects,{indent=true}) saveProjectTab("Test", "--["..str.."]--") end --# MyClass MyClass = class() function MyClass:init(x) -- you can accept and set parameters here self.foo = "bar" self.x=x self.pos=vec2(50,50) --vec2 cant be used with json end function MyClass:__tostring() return json.encode(self,{indent=true}) end --# Test --[[{ "foo":"bar", "pos":{"x":50.0,"y":50.0}, "x":17 },{ "foo":"bar", "pos":{"x":50.0,"y":50.0}, "x":23 },{ "foo":"bar", "pos":{"x":50.0,"y":50.0}, "x":9 }]]--
  • Posts: 2,020

    @Jmv38 that's awesome, thanks. Though might I suggest adding a vec2=true flag to the json-vec2 so that the decode process can convert it back to a genuine vec2 (rather than leaving it as a pseudo-vec2 table)? Unless someone can think of a more elegant way to handle the decode?

  • Jmv38Jmv38 Mod
    Posts: 3,295

    @yojimbo2000 i agree. I had not enough time to rework and post your last version, so i just posted this one to show you how to use __toJason (as you asked for it earlier). Thanks.

  • Posts: 2,020

    Ok, here's my version using @Jmv38 's meta methods. It now supports vec2, vec3, vec4, matrix, with support for conversion back to the proper types. In the code below I show it recreating the instances of the class in the objects table from the saved code.


    --# Main -- Json Test -- Use this function to perform your initial setup function setup() VecToJson() --establish metamethods for encoding vec2, vec3, vec4, matrix as json objects objects={ MyClass{pos=vec3(17,18,19), angle=45, foo="bar"}, MyClass{pos=vec4(123,124,127,1), angle=0, marzipan="tasty", foo="boo"}, MyClass{pos=vec3(257,263,278), angle=-90, boolean=true, foo="bah"} } for i=1,2 do --do this twice to check back-conversion tabSave(objects, "Test") result=tabLoad("Test") result=tabIterate(result) --decode json objects back to vec2, vec3, vec4 result=tabIterate(result) --re-iterate to see if decoding has stuck objects = {} --clear objects for i,v in ipairs(result) do objects[i]=MyClass(v) --recreate them from loaded table end end --rinse and repeat end function VecToJson() local meta = getmetatable(vec2()) meta.__tojson = function(t) return json.encode({x=t.x,y=t.y,vec2=true}) end meta = getmetatable(vec3()) meta.__tojson = function(t) return json.encode({x=t.x,y=t.y,z=t.z,vec3=true}) end meta = getmetatable(vec4()) meta.__tojson = function(t) return json.encode({x=t.x,y=t.y,z=t.z,w=t.w,vec4=true}) end meta = getmetatable(matrix()) meta.__tojson = function(t) return json.encode({values={t[1],t[2],t[3],t[4], t[5],t[6],t[7],t[8], t[9],t[10],t[11],t[12], t[13],t[14],t[15],t[16]},mat=true}) --a matrix.unpack function would've been handy here! end end function tabSave(tab, loc) local str=json.encode(tab, {indent = true}) --exception = jsonException, saveProjectTab(loc, "--["..str.."]") end function tabLoad(loc) local str=readProjectTab(loc) return json.decode(string.sub(str,4,-2)) end local underscore = {"=","~","-","."} --underscore characters indicate table depth function tabIterate(tab, i) local new = {} local i = i or 0 local indent = string.rep(" ", i*3) --depth of indentation for k,v in pairs(tab) do if type(v)=="table" then if v.vec2 then --convert back to vec2 new[k] = vec2(v.x, v.y) print (indent..k..": decoding vec2"..tostring(new[k])) elseif v.vec3 then --convert back to vec3 new[k] = vec3(v.x, v.y, v.z) print (indent..k..": decoding vec3"..tostring(new[k])) elseif v.vec4 then --convert back to vec4 new[k] = vec4(v.x, v.y, v.z, v.w) print (indent..k..": decoding vec4"..tostring(new[k])) elseif v.mat then --convert back to matrix new[k] = matrix(table.unpack(v.values)) print (indent..k..": decoding matrix"..tostring(new[k])) else --genuine table local under = string.rep(underscore[math.min(#underscore, i+1)], string.len(k)) print(indent..k.."\n"..indent..under) new[k]=tabIterate(v, i+1) --increase indent upon recursion end else print(indent..k..": "..tostring(v)) new[k]=v end end return new end --[[ --not needed when using meta-methods function jsonException(reason, v, state, message) if v.x and v.y and not v.z then --vec2 return json.encode({x=v.x,y=v.y,vec2=true}) --set a vec2 flag end end ]] --# MyClass MyClass = class() function MyClass:init(t) for k,v in pairs(t) do self[k]=v end self:draw() end function MyClass:draw() --some pseudo drawing translate(self.pos.x, self.pos.y, self.pos.z) rotate(self.angle) self.matrix=modelMatrix() print(self.foo) resetMatrix() end --# Test --[[{ "angle":45, "foo":"bar", "pos":{"y":18.0,"x":17.0,"vec3":true,"z":19.0}, "matrix":{"values":[0.70710676908493,0.70710676908493,0.0,0.0,-0.70710676908493,0.70710676908493,0.0,0.0,0.0,0.0,1.0,0.0,17.0,18.0,19.0,1.0],"mat":true} },{ "foo":"boo", "angle":0, "matrix":{"values":[1.0,0.0,0.0,0.0,0.0,1.0,0.0,0.0,0.0,0.0,1.0,0.0,123.0,124.0,127.0,1.0],"mat":true}, "pos":{"y":124.0,"x":123.0,"w":1.0,"vec4":true,"z":127.0}, "marzipan":"tasty" },{ "pos":{"y":263.0,"x":257.0,"vec3":true,"z":278.0}, "angle":-90, "foo":"bah", "boolean":true, "matrix":{"values":[-4.3711388286738e-08,-1.0,0.0,0.0,1.0,-4.3711388286738e-08,0.0,0.0,0.0,0.0,0.99999994039536,0.0,257.0,263.0,278.0,1.0],"mat":true} }]]
  • Jmv38Jmv38 Mod
    Posts: 3,295

    you are faster than me! I was preparing this: a version to manage any class with "Class" keyword. How do you like it?


    --# Main -- Json Test -- Use this function to perform your initial setup function setup() --cant use vec2s objects={ MyClass(17), MyClass(23), MyClass(9) } Vec2ToJson() Vec3ToJson() tabSave() out = tabLoad() setmetatable(out,{__tostring = function(obj) return json.encode(obj,{indent=true}) end}) print(out[1].pos) print(out[1].mis) end function Vec2ToJson() local meta = getmetatable(vec2()) meta.__tojson = function(t) local a = {Class="vec2",x=t.x,y=t.y} return json.encode(a) end end function Vec3ToJson() local meta = getmetatable(vec3()) meta.__tojson = function(t) local a = {Class="vec3",x=t.x,y=t.y,z=t.z} return json.encode(a) end end function tabLoad() local str=readProjectTab("Test") str = string.sub(str,4,-4) local t = json.decode(str) t = checkForClass(t) return t end function checkForClass(t) if type(t) ~= "table" then return t end for key,val in pairs(t) do if type(val) == "table" then t[key] = checkForClass(val) end local Class = t["Class"] if Class then local obj = _G[Class]() for k,v in pairs(t) do if k~="Class" then obj[k] = v end end t = obj end end return t end function tabSave() local str=json.encode(objects,{indent=true}) saveProjectTab("Test", "--["..str.."]--") end --# MyClass MyClass = class() function MyClass:init(x) -- you can accept and set parameters here self.foo = "bar" self.x=x self.pos=vec2(50,50) self.mis =vec3(1,2,3) end function MyClass:__tostring() return json.encode(self,{indent=true}) end --# Test --[[{ "x":17, "mis":{"x":1.0,"y":2.0,"z":3.0,"Class":"vec3"}, "pos":{"x":50.0,"y":50.0,"Class":"vec2"}, "foo":"bar" },{ "x":23, "mis":{"x":1.0,"y":2.0,"z":3.0,"Class":"vec3"}, "pos":{"x":50.0,"y":50.0,"Class":"vec2"}, "foo":"bar" },{ "x":9, "mis":{"x":1.0,"y":2.0,"z":3.0,"Class":"vec3"}, "pos":{"x":50.0,"y":50.0,"Class":"vec2"}, "foo":"bar" }]]--
  • Jmv38Jmv38 Mod
    Posts: 3,295

    i found this more general than vec2=true.

  • edited June 2015 Posts: 2,020

    @Jmv38 Ok. Actually, I think my preferred method would be, rather than a keyword, nesting the data in a sub table with an appropriate key, eg a vec3 called pos looks like this: "pos":{"vec3":{"y":18.0,"x":17.0,"z":19.0}}. Edit, added exception method, has feature parity with meta-method:



    --# Main -- Json Test -- Use this function to perform your initial setup function setup() -- VecToJson() --establish metamethods for encoding vec2, vec3, vec4, matrix as json objects objects={ MyClass{pos=vec3(17,18,19), angle=0, foo="bar"}, MyClass{pos=vec2(123,124), angle=-45, marzipan="tasty", foo="boo"}, MyClass{pos=vec3(257,263,278), angle=-90, boolean=true, foo="bah"} } for i=1,2 do --do this twice to check back-conversion tabSave(objects, "Test") local loadBack=tabLoad("Test") loadBack=tabIterate(loadBack) --decode json objects back to vec2, vec3, vec4 print("---------------------\nDECODING COMPLETE, CHECKING\n---------------------") loadBack=tabIterate(loadBack) --re-iterate to see if decoding has stuck print("=====================\nRECREATING OBJECTS FROM DECODED DATA, CHECKING\n=====================") objects = {} --clear objects for i,v in ipairs(loadBack) do objects[i]=MyClass(v) --recreate them from loaded table end end --rinse and repeat end --meta-methods function VecToJson() local meta = getmetatable(vec2()) meta.__tojson = function(t) return json.encode({vec2={x=t.x,y=t.y}}) end meta = getmetatable(vec3()) meta.__tojson = function(t) return json.encode({vec3={x=t.x,y=t.y,z=t.z}}) end meta = getmetatable(vec4()) meta.__tojson = function(t) return json.encode({vec4={x=t.x,y=t.y,z=t.z,w=t.w}}) end meta = getmetatable(matrix()) meta.__tojson = function(t) return json.encode({mat4={t[1],t[2],t[3],t[4], t[5],t[6],t[7],t[8], t[9],t[10],t[11],t[12], t[13],t[14],t[15],t[16]}}) --a matrix.unpack function would've been handy here! end end --encode exception func can be used instead of meta-methods function jsonException(reason, t, state, message) if t.x and t.y and t.z and t.w then --vec4 return json.encode({vec4={x=t.x,y=t.y,z=t.z,w=t.w}}) elseif t.x and t.y and t.z then --vec3 return json.encode({vec3={x=t.x,y=t.y,z=t.z}}) elseif t.x and t.y then --vec2 return json.encode({vec2={x=t.x,y=t.y}}) elseif t[16] then --matrix, presumably. nb #t not allowed. what other "userdata" types are there? return json.encode({mat4={t[1],t[2],t[3],t[4], t[5],t[6],t[7],t[8], t[9],t[10],t[11],t[12], t[13],t[14],t[15],t[16]}}) --a matrix.unpack function would've been handy here! end end function tabSave(tab, loc) local str=json.encode(tab, {indent = true, exception = jsonException}) saveProjectTab(loc, "--["..str.."]\n") end function tabLoad(loc) local str=readProjectTab(loc) return json.decode(string.sub(str,4,-3)) end local underscore = {"=","~","-","."} --underscore characters indicate table depth function tabIterate(tab, i) local new = {} local i = i or 0 local indent = string.rep(" ", i*3) --depth of indentation for k,v in pairs(tab) do if type(v)=="table" then --check first whether its a pseudo-table representing a vec2, vec3, vec4, mat4 if v.vec2 then --convert back to vec2 local vec = v.vec2 new[k] = vec2(vec.x, vec.y) print (indent..k..": decoding vec2"..tostring(new[k])) elseif v.vec3 then --convert back to vec3 local vec = v.vec3 new[k] = vec3(vec.x, vec.y, vec.z) print (indent..k..": decoding vec3"..tostring(new[k])) elseif v.vec4 then --convert back to vec4 local vec = v.vec4 new[k] = vec4(vec.x, vec.y, vec.z, vec.w) print (indent..k..": decoding vec4"..tostring(new[k])) elseif v.mat4 then --convert back to matrix new[k] = matrix(table.unpack(v.mat4)) print (indent..k..": decoding matrix"..tostring(new[k])) else --genuine table local under = string.rep(underscore[math.min(#underscore, i+1)], string.len(k)) print(indent..k.."\n"..indent..under) new[k]=tabIterate(v, i+1) --increase indent upon recursion end else print(indent..k..": "..tostring(v)) new[k]=v end end return new end --# MyClass MyClass = class() function MyClass:init(t) for k,v in pairs(t) do self[k]=v end self:draw() end function MyClass:draw() --some pseudo drawing translate(self.pos:unpack()) --(self.pos.x, self.pos.y, self.pos.z) rotate(self.angle) self.matrix=modelMatrix() print(self.foo) resetMatrix() end --# Test --[[{ "pos":{"vec3":{"y":18.0,"z":19.0,"x":17.0}}, "angle":0, "matrix":{"mat4":[1.0,0.0,0.0,0.0,0.0,1.0,0.0,0.0,0.0,0.0,1.0,0.0,17.0,18.0,19.0,1.0]}, "foo":"bar" },{ "foo":"boo", "matrix":{"mat4":[0.70710676908493,-0.70710676908493,0.0,0.0,0.70710676908493,0.70710676908493,0.0,0.0,0.0,0.0,1.0,0.0,123.0,124.0,0.0,1.0]}, "pos":{"vec2":{"y":124.0,"x":123.0}}, "angle":-45, "marzipan":"tasty" },{ "foo":"bah", "angle":-90, "boolean":true, "pos":{"vec3":{"y":263.0,"z":278.0,"x":257.0}}, "matrix":{"mat4":[-4.3711388286738e-08,-1.0,0.0,0.0,1.0,-4.3711388286738e-08,0.0,0.0,0.0,0.0,0.99999994039536,0.0,257.0,263.0,278.0,1.0]} }]]
  • Jmv38Jmv38 Mod
    Posts: 3,295

    Good idea. However, it means you cant have a field in a table with the same name as an existing userdata. This forbids many keywords!

  • edited June 2015 Posts: 2,020

    Ok, I see what you mean. How about I preface each keyword ie "uservec4", "usermat4". Edit: this method is handy with matrices where adding the keyword ruins the array (table.unpack stops working).

    Is there a comprehensive list somewhere of all the kinds of "userdata" ? I guess it's all the extra non-Lua APIs that Codea provides, eg Mesh, physics.body etc. I wonder how far this could be taken? Would be quite cool to be able to save a physics body or a mesh in this way. Will need to check whether you can iterate through all the values of a physics body or a mesh using pairs...

  • Posts: 1,254

    Wow. I looked a way for a bit, and when I looked back, you two had beaten the problem into submission.

    Thanks, @jmv38 and @yojimbo2000. Exactly what I wanted to see.

  • Posts: 2,020

    You can't use pairs on a mesh or a physics body, so you'd have to go through all the keys by hand. That's not too bad I guess, a cut-and-paste job...

  • Posts: 2,020

    @Mark you're welcome. I learned a lot. I'm new to meta-methods, don't really use them in my own code. Depending on the complexity of the object, I think an encode exception function might be simpler (ie my last post above). If you have lots of different kinds of objects though, meshes, physics bodies etc, then meta methods is probably the way to go

  • Posts: 2,020

    @Jmv38 or how about 2 underscores before each special key, eg __mat4?

  • Jmv38Jmv38 Mod
    Posts: 3,295

    great idea! I think you've nailed it.

  • Jmv38Jmv38 Mod
    Posts: 3,295

    here is a possible solution. It is not 'finished' but you get the idea.


    --# Main -- Json Test -- Use this function to perform your initial setup function setup() --cant use vec2s objects={ MyClass(100,500), MyClass(200,500), MyClass(400,500) } Vec2ToJson() Vec3ToJson() body2ToJson() tabSave() print("tap screen to load saved data") end function draw() background(0) for i,o in pairs(objects) do o:draw() end end function touched(t) if t.state==BEGAN then for i,o in pairs(objects) do o.body:destroy() end objects = tabLoad() end end function body2ToJson() local b = physics.body(CIRCLE,10) local meta = getmetatable(b) b:destroy() meta.__tojson = function(t) local a = {Class="Body",x=t.x,y=t.y} return json.encode(a) end Body = function() return physics.body(CIRCLE,10) end end function Vec2ToJson() local meta = getmetatable(vec2()) meta.__tojson = function(t) local a = {Class="vec2",x=t.x,y=t.y} return json.encode(a) end end function Vec3ToJson() local meta = getmetatable(vec3()) meta.__tojson = function(t) local a = {Class="vec3",x=t.x,y=t.y,z=t.z} return json.encode(a) end end function tabLoad() local str=readProjectTab("Test") str = string.sub(str,4,-4) local t = json.decode(str) t = checkForClass(t) return t end function checkForClass(t) if type(t) ~= "table" then return t end for key,val in pairs(t) do if type(val) == "table" then t[key] = checkForClass(val) end local Class = t["Class"] if Class then local obj if Class == "MyClass" then obj = MyClass(t.body.x,t.body.y) else obj = _G[Class]() end for k,v in pairs(t) do if k~="Class" then obj[k] = v end end t = obj end end return t end function tabSave() local str=json.encode(objects,{indent=true}) saveProjectTab("Test", "--["..str.."]--") end --# MyClass MyClass = class() function MyClass:init(x,y) self.Class = "MyClass" self.foo = "bar" self.pos=vec2(50,50) self.mis =vec3(1,2,3) self.body = physics.body(CIRCLE,10) self.body.x = x self.body.y = y end function MyClass:__tostring() return json.encode(self,{indent=true}) end function MyClass:draw() ellipse(self.body.x,self.body.y,self.body.radius) end --# Test --[[{ "body":{"y":500.0,"x":100.0,"Class":"Body"}, "Class":"MyClass", "pos":{"y":50.0,"x":50.0,"Class":"vec2"}, "mis":{"z":3.0,"y":2.0,"x":1.0,"Class":"vec3"}, "foo":"bar" },{ "body":{"y":500.0,"x":200.0,"Class":"Body"}, "Class":"MyClass", "pos":{"y":50.0,"x":50.0,"Class":"vec2"}, "mis":{"z":3.0,"y":2.0,"x":1.0,"Class":"vec3"}, "foo":"bar" },{ "body":{"y":500.0,"x":400.0,"Class":"Body"}, "Class":"MyClass", "pos":{"y":50.0,"x":50.0,"Class":"vec2"}, "mis":{"z":3.0,"y":2.0,"x":1.0,"Class":"vec3"}, "foo":"bar" }]]--
  • dave1707dave1707 Mod
    Posts: 7,522

    @juce That's because JSON only allows string keys. JSON works with numeric keys, see the example below. If I add a string key to tab1, it doesn't. So it looks like you can have numeric keys or string keys, but not both in the same table. I haven't needed to use json for anything, but just wanted to see how it worked.

    function setup()
        tab1={1,2,{7,8,9},3,4}
        z=json.encode(tab1)
        print(z)
    
        tab2=json.decode(z)
        z=json.encode(tab2)
        print(z)
    
        print("\ntab1 --------")
        for a,b in pairs(tab1) do
            print(a,b)
        end
    
        print("\ntab2 --------")
        for a,b in pairs(tab2) do
            print(a,b)
        end
    
        print("\ntab1 --------")
        print(tab1[1])
        print(tab1[3][1],tab1[3][2],tab1[3][3])
    
        print("\ntab2 --------")
        print(tab2[1])
        print(tab2[3][1],tab2[3][2],tab2[3][3])    
    end
    
  • Posts: 2,020

    @Jmv38 this is my version of json encoding of a physics scene. You can save a snapshot of the scene, and then load it in again. Could be useful for a level editor in an Angry Birds style physics game:

    https://gist.github.com/Utsira/da4fda9b95e9112d705a

    Here's the class with the various meta-methods:

    --meta-methods
    function AddToJson()
        local meta = getmetatable(vec2())
        meta.__tojson = function(t)
            return json.encode({__vec2={x=t.x,y=t.y}})
        end
        meta = getmetatable(vec3())
        meta.__tojson = function(t)
            return json.encode({__vec3={x=t.x,y=t.y,z=t.z}})
        end
        meta = getmetatable(vec4())
        meta.__tojson = function(t)
            return json.encode({__vec4={x=t.x,y=t.y,z=t.z,w=t.w}})
        end
        meta = getmetatable(matrix())
        meta.__tojson = function(t)
            return json.encode({__mat4={t[1],t[2],t[3],t[4], t[5],t[6],t[7],t[8], t[9],t[10],t[11],t[12], t[13],t[14],t[15],t[16]}}) --a matrix.unpack function would've been handy here!
        end
        meta = getmetatable(color())
        meta.__tojson = function(t)
            return json.encode({__color={r=t.r, g=t.g, b=t.b, a=t.a}})
        end
        meta = getmetatable(mesh())
        meta.__tojson = function(t)
            return json.encode({nil}) --({__mesh=t.id})
        end
    
        local dummy = physics.body(CIRCLE, 10)
        meta = getmetatable(dummy)
        dummy:destroy()
    
        meta.__tojson = function(t)
        return json.encode({
        __rigidbody={shapeType=t.shapeType, radius=t.radius, points=t.points},
        __rigidbodyProperties={type=t.type, position=t.position, angle=t.angle, linearVelocity=t.linearVelocity, angularVelocity=t.angularVelocity, restitution=t.restitution, friction=t.friction, linearDamping=t.linearDamping, angularDamping=t.angularDamping
        }})
           end 
        --add extra categories from below according to needs
        --gravityScale=t.gravityScale, info=t.info, density=t.density, sensor=t.sensor, categories=t.categories, mask=t.mask, fixedRotation=t.fixedRotation, active=t.active, awake=t.awake, sleepingAllowed=t.sleepingAllowed, interpolate=t.interpolate, joints=t.joints, bullet=t.bullet, 
    
    end
    
  • Posts: 2,020

    @Jmv38 thanks for showing us how to set the toJson metamethod (for physics bodies too!)

  • Posts: 154

    @dave1707, you are right: i should've been more specific - json only allows string keys in dictionaries. The edge cases ultimately come from data structure differences: Lua's table can have keys of arbitrary types and also has the notion of having an array part and a hash (dictionary) part, while json has two different container types: dictionary and list.

    I think what happens is that json.encode looks for special case when encoding tables: if a table only has integer keys, with the smallest one being 1, then it converts such table to a list, putting nulls into missing slots. Otherwise, it converts the table into a dictionary, collating numeric keys into string keys.

    --# Main
    
    function setup()
        t = {[1]="foo",[10]="bar"}
        print(json.encode(t))
    
        t = {[1]="foo",[20]="bar"}
        print(json.encode(t))
    
        t = {[-1]="foo",[0]="bar", [1]="baz"}
        print(json.encode(t))
    
        -- print array part of the table
        for i,v in ipairs(t) do
            print(i,v)
        end
    end
    

    But even that description is not entirely accurate: notice how second table has "too many" nil slots, compared to first one - and gets converted into a dictionary, not a list.

  • Posts: 2,020

    @juce that's interesting how it decides whether to go with dictionary or list. I avoid mixed dictionary-lists in Lua, it just seems like too much potential for unexpected behaviour

  • dave1707dave1707 Mod
    Posts: 7,522

    @juce Interesting. I guess you really need to know how the decode is going to handle a table that's encoded. You could write a program that uses a table without any problems only to have it crash when you try to recreate a new table with decode.

  • edited June 2015 Posts: 154

    @yojimbo2000, i agree that mixing numeric keys with keys of other types is dangerous, and best avoided if possible. However, even if we use only numeric keys, this table-to-list conversion algorithm can play unexpected tricks: say you have a sparse table with integer keys - it can probably change back and forth between list and dictionary, as you add more values to it, depending on the exact distribution of keys.

  • Posts: 154

    Like that:

    --# Main
    
    function setup()
        t = {[1]="a", [4]="b", [9]="c"}
        t[10]="d"
        t[5]="e"
        print(json.encode(t))
        t[13]="f"
        print(json.encode(t))
        t[11]="g"
        t[12]="h"
        print(json.encode(t))
    end
    

    On my iPad 2 it goes from list to dictionary and back to list:

    ["a",null,null,"b","e",null,null,null,"c","d"]
    {"1":"a","9":"c","10":"d","4":"b","5":"e","13":"f"}
    ["a",null,null,"b","e",null,null,null,"c","d","g","h","f"]
    
  • Posts: 2,020

    I also try to avoid sparse lists, by using table.remove to make sure there's no holes. @dave1707 try my code at the gist. I'll put my neck on the block and say it's reasonably stable at recreating a physics scene. :-SS

    https://gist.github.com/Utsira/da4fda9b95e9112d705a

  • Posts: 154

    @dave1707, i think decode is ok, because it is fully deterministic. It's the encoding step that can have unexpected results. But yes, the end result of encode + decode will not necessarily recreate the table as it was before.

  • dave1707dave1707 Mod
    Posts: 7,522

    @yojimbo2000 I'm not saying it won't work and I'm sure yours does, but for someone who doesn't understand how encode/decode works could be in for a surprise if they use it. Just like me, I thought encode/decode would take a table, convert it to a string that could be saved, then recreate the table exactly like the original. But it turns out there's limits, even with simple tables. I don't have any code that uses encode/decode and probably never will. It's just that when it was brought up in this discussion, I thought I'd try it to see how it works and ran into some surprises that I pointed out above. If someone uses this in an App Store program without understanding everything about it, their program could crash.

  • Posts: 2,020

    I guess I tend to think of dictionaries as being somewhat unpredictable anyway - they have no order as such, in that you can't predict what order pairs will return the items in.

  • Posts: 109

    @yojimbo2000 @dave1707 I am perpetually baffled by the fact that pairs() seems to be so inconsistent in its traversal of a table.

    @Simeon @John I am curious how it is possible for pairs() based for loops to occur in different order each run?

    I was using them to parse and layout GUI objects, and it was practically random order each run. This goes against everything I have ever seen in programming, usually the same scenario produces the exact same result..

    I alwasy was assuming that pairs() would work alphabetically, or in the order that the values were created.. Is this related to available memory locations / garbage collection / frame rate? It may be worth adding a note in the built in documentation, I was going bannanas trying to figure out why things were always showing up in a different order and ended up having to switch to ipairs to keep from losing my mind :P`

    Super useful thread btw, some of this seems worthy of inclusion into a built in demo / example of some sort.

  • dave1707dave1707 Mod
    Posts: 7,522

    @AxiomCrux I think one of the reasons pairs works the way it does is because pairs will iterate thru a table even if it contains empty entries. For example if tab[5]=100 and tab[1000]=345, pairs will show both entries, but ipairs wont show either because it's not consecutive starting at 1. You're probably correct when you say that the tables are probably created based on available memory locations. I did a google search for pairs and found this:

    Description

    Returns an iterator function(next) for a for loop that will return the values of the specified table in an arbitrary order.

  • Posts: 109

    heheh yeah! @dave1707 "arbitrary order" thats the thing I can't seem to fully wrap my head around..

    I've been using computers since I could form memories, and I have recently come to understand cryptography and how 'random' number generation works (which opened my mind to profound philisophical insight into chaos theory and the nature of the universe).

    "arbitrary order" and programming are opposing concepts IMHO. The notion that a key component of this awesome Lua programming language functions that way.. . scrambles my brain into "arbitrary order" a bit.

  • dave1707dave1707 Mod
    Posts: 7,522

    @AxiomCrux Maybe you'll find this interesting. It contains something about tables but I don't know if it really answers anything about arbitrary order.

    https://www.lua.org/gems/sample.pdf
    
  • Posts: 2,020

    Don't think of it as "arbitrary order", just unordered. JSON dictionaries are unordered as well. If order is important, you need to put the elements into an array.

  • dave1707dave1707 Mod
    Posts: 7,522

    @yojimbo2000 @AxiomCrux The comment I mentioned in the above post and what I show below is wrong for pairs. It won't return the values in arbitrary order, but will create the table in arbitrary order. If you run the example code below and keep tapping the screen to read the table, it will show the same results in some order each time. If you restart the program, it will create the table in a different order.

    Description
    
    Returns an iterator function(next) for a for loop that will return the values of the specified table in an arbitrary order.
    
    function setup()
        tab={a=1,b=2,c=3,d=4,e=5}
        for a,b in pairs(tab) do
            print(a,b)
        end
    end
    
    function touched(t)
        if t.state==BEGAN then
            output.clear()
            for a,b in pairs(tab) do
                print(a,b)
            end
        end
    end
    
Sign In or Register to comment.