Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Save/Load a Table

edited July 2013 in General Posts: 2,042

I was wondering if anyone knew a way to save and load tables? Since they are not supported by saveLocalData, etc.

Comments

  • IgnatzIgnatz Mod
    edited July 2013 Posts: 5,396

    Yep, you need to flatten ("serialise") them into a string, then you can save them into local data.

    I do it in this game tutorial to create an undo function (see "multiple undo" heading)

    http://coolcodea.wordpress.com/2013/03/31/19-lessons-from-a-simple-board-game-part-2/

  • BriarfoxBriarfox Mod
    edited July 2013 Posts: 1,542

    I do it the lazy way and use json but thats overkill. Check out ignatz's tutorial on debugging there is a dump class that i bet could be setup to decode a string then write an encoder.

  • Posts: 2,820

    I literally just finished the code for this in my game and was thinking about posting it. I'll post it tomorrow.

  • Posts: 2,042

    @Zoyt, awesome

  • Posts: 2,820

    @JakAttak - All right, Here we go:
    Here is my function I altered from the Programming in Lua book for converting a table to an executable string:

    function tToS (name, value, saved)
        local function basicSerialize (o)
            if type(o) == "number" then
                return tostring(o)
            else -- assume it is a string
                return string.format("%q", o)
            end
        end
        saved = saved or {}
        local returnStr = name.." = " 
        if type(value) == "number" or type(value) == "string" then
            returnStr = returnStr..basicSerialize(value).."\n"
        elseif type(value) == "table" then
            if saved[value] then
                returnStr = returnStr..saved[value].."\n"
            else
                saved[value] = name
                returnStr = returnStr.."{}\n"
                for k,v in pairs(value) do 
                    local fieldname = string.format("%s[%s]", name, basicSerialize(k))
                    returnStr = returnStr..tToS(fieldname, v, saved)
                end
            end
        else
            error("Cannot save a " .. type(value))
        end
        return returnStr
    end

    Then, to store the table, I say:

    table.insert(table,value)
    saveProjectData("table",tToS("table",table))

    Finally, to read and set the table, I say:

    table = readProjectData("table","table = {}")
    assert(loadstring(table))()

    Threre you have it. It uses the horrible loadString function, but it works well. I've looked into adding more data storage types like nils, vec2, touches, collisions, etc, but I haven't had the need for it yet. If anyone wants me to do that, just let me know.
    Thanks!
    P.S. @Ignatz - You migh suggest including this method somewhere in your tutorials. Lots of people use more than 2D tables. I use tables of all shapes and sizes.

  • Posts: 2,042

    @Zoyt thanks! This should be perfect for what I want

  • Posts: 2,820

    @JakAttak - No problem...

  • Posts: 2,042

    @Zoyt, having a little trouble getting it to work:

    Here is the code I'm using to test it:


    --# Main -- String Table -- Use this function to perform your initial setup function setup() jo = {{1,2,3},{2,3,5}} saveProjectData("table",tToS("table",jo)) end -- This function gets called once every frame function draw() -- This sets a dark background color background(40, 40, 50) -- This sets the line thickness strokeWidth(5) -- Do your drawing here table = readProjectData("table") assert(loadstring(table))() if table == jo then print("It Worked") end end function tToS(name, value, saved) local function basicSerialize (o) if type(o) == "number" then return tostring(o) else -- assume it is a string return string.format("%q", o) end end saved = saved or {} local returnStr = name.." = " if type(value) == "number" or type(value) == "string" then returnStr = returnStr..basicSerialize(value).."\n" elseif type(value) == "table" then if saved[value] then returnStr = returnStr..saved[value].."\n" else saved[value] = name returnStr = returnStr.."{}\n" for k,v in pairs(value) do local fieldname = string.format("%s[%s]", name, basicSerialize(k)) returnStr = returnStr..tToS(fieldname, v, saved) end end else error("Cannot save a " .. type(value)) end return returnStr end
  • Posts: 2,820

    The first argument in tToS is the name of the table you're putting in. So it should be "saveProjectData("jo",tToS("jo",jo))". Hope that helps.

  • edited July 2013 Posts: 2,042

    @Zoyt It still doesn't work... Does it ever change the string back into a table?

  • Posts: 2,820

    Did you change the key to read the data to "Jo"? If so, can you post your code?

  • Posts: 2,042

    @Zoyt sure here it is:


    --# Main -- String Table -- Use this function to perform your initial setup function setup() jo = {{1,2,3},{2,3,5}} saveProjectData("jo",tToS("jo",jo)) end -- This function gets called once every frame function draw() -- This sets a dark background color background(40, 40, 50) -- This sets the line thickness strokeWidth(5) -- Do your drawing here table = readProjectData("jo", "jo = {}") assert(loadstring(table))() if table == jo then print("It Worked") end end function tToS(name, value, saved) local function basicSerialize (o) if type(o) == "number" then return tostring(o) else -- assume it is a string return string.format("%q", o) end end saved = saved or {} local returnStr = name.." = " if type(value) == "number" or type(value) == "string" then returnStr = returnStr..basicSerialize(value).."\n" elseif type(value) == "table" then if saved[value] then returnStr = returnStr..saved[value].."\n" else saved[value] = name returnStr = returnStr.."{}\n" for k,v in pairs(value) do local fieldname = string.format("%s[%s]", name, basicSerialize(k)) returnStr = returnStr..tToS(fieldname, v, saved) end end else error("Cannot save a " .. type(value)) end return returnStr end
  • Posts: 2,820

    @JakAttak - Right... My bad. I forgot that the tables have different identifiers. If you look at the individual values, you'll see that they're the same.
    Hope that helps!

  • Posts: 2,042

    @Zoyt, actually I tried that

    if table[1][1] == jo[1][1] 
    

    But I get an error: attempting to index a nil value

  • Posts: 2,820

    I finally had the time to copy over your code. So sorry about that. The first field you pass into tTooS should be "table", not "jo". Also, it's not a good technique to override Lua libraries, like the table one. Hope that works.

  • Posts: 2,042

    @Zoyt, so saveProjectData("jo", tToS("table", "jo")?

  • Posts: 2,820
    function setup()
        jo = {{1,2,3},{2,3,5}}
        saveProjectData("jo",tToS("t",jo))
    end
     
    function draw()
        background(40, 40, 50)
     
        t = readProjectData("jo", "t = {}")
        assert(loadstring(t))()
     
        if t[1][1] == jo[1][1] then
            print("It Worked")
        end
    end
     
    function tToS(name, value, saved)
        local function basicSerialize (o)
            if type(o) == "number" then
                return tostring(o)
            else -- assume it is a string
                return string.format("%q", o)
            end
        end
        saved = saved or {}
        local returnStr = name.." = " 
        if type(value) == "number" or type(value) == "string" then
            returnStr = returnStr..basicSerialize(value).."\n"
        elseif type(value) == "table" then
            if saved[value] then
                returnStr = returnStr..saved[value].."\n"
            else
                saved[value] = name
                returnStr = returnStr.."{}\n"
                for k,v in pairs(value) do 
                    local fieldname = string.format("%s[%s]", name, basicSerialize(k))
                    returnStr = returnStr..tToS(fieldname, v, saved)
                end
            end
        else
            error("Cannot save a " .. type(value))
        end
        return returnStr
    end
  • Posts: 2,042

    I got it working. The "t = {}" doesn't appear to be necessary

  • Posts: 2,820

    @JakAttak - That is correct, but when I store project data, I read it before I write anything to it.

  • Posts: 2,820

    Here's another link I found on saving/loading tables using a similar method, but contains more code:
    http://lua-users.org/wiki/SaveTableToFile

  • Posts: 2,042

    Your method worked perfectly

  • Posts: 2,820

    Thanks. I recently added booleans and am working on nils. Let me know if you need those.

  • edited July 2013 Posts: 2,042

    @Zoyt No thanks. Just needed to be able to save and load tables, I wrote a program that lets me design levels and then I save them and load them in other games

  • edited July 2013 Posts: 2,042

    Edit: Double post

  • If I'm reading this correctly, you're saving variables to storage by basically creating a Lua statement...then reading the data back in by storing the saved Lua statement.

    What happens when someone replaces the data with code that does something malicious, like use your computer to send spam or format a hard drive?

  • IgnatzIgnatz Mod
    Posts: 5,396

    Fortunately, there aren't enough Codea users to make it worthwhile to try to break into our data and use us for spam.

  • Posts: 2,820

    @tomxp411 - It's easy (I've tried it), not to mention to can access the source code of every app itself, but as@Ignatz said, Codea doesn't have that many users that want to hack the apps. Simeon did mention earlier that he'd look into restricting users from accessing our code in our apps.

  • Posts: 2,042

    Yea, when you publish an app, any Jailbroken iPad (or someone using iExplorer) can modify the code easily. So if, for example, your game gets relatively popular, lots of user might simply add a line of code to save the highscore as very high

  • I'm just pointing out that it's a bad - a VERY bad programming practice in the real world; we shouldn't be promoting this as a good practice at all, especially when serializing (using something like JSON or XML) is not any more difficult. Yes, deserializing is a little bit more complex, but that's the tradeoff with Lua: since code and data are basically the same thing, you have to work harder to write secure code.

    Sorry if I seem overbearing. I write police software for a living. =)

  • Posts: 2,820

    @tomxp411 - Nice. I agree. I do hope for this feature soon.

  • I've got to do this at some point for my own project anyway... So I will try to throw something together this week.

  • I started looking at JSON as a serialization format, and found a bunch of implementations in Lua:

    http://lua-users.org/wiki/JsonModules

    I'm still going to roll something, just for learning and as a sample for other users. When it's ready, I'll post it in the code sharing section.

  • I use JSON, though it is bloated, simply because all this stuff that you go through to fix and optimize a serializer is a pain, and I want to write other things.

    That said, I (and oters) wrote serializers that persisted to am image and wrote that to wither Dropbox: or Documents:, so that's another option.

  • edited September 2013 Posts: 580

    @tomxp411: In Lua 5.1 there are ways to do sandboxed loading, primarily through use of the setfenv() function, which allows you to specify the global environment that a function has access to (yeah, in Lua 5.1, every function actually has it's own "global" environment..it just happens to be _G by default).

    In cmodule's load() function I use this facility to load files that are intended to be data only, exposing only what is necessary for the data files to compile by allowing you to pass in an optional "environment" table; when the data file is compiled it's only access to the outside world is what is defined in this table (for example, in my UI kit prototype, I load view templates that are Lua tables, and provide an environment that only has the Codea data types available, like color(), vec2(), etc, for setting the view properties).

    I think JSON is great, if you're writing javascript, or writing server comm code that needs to send serialized data or parse JSON responses...but for game data? It should be fine to use Lua. Since it's already a data description language (and a very good and fast one at that) I wouldn't preclude it's use if it's possible to use it without creating a security concern. I believe in most cases it is. Anyway, even if you did release an app on the app store, and someone hacked the bundle and changed data, if you are properly sandboxing your input the worst they could really do is hose their own app/device I think. Codea itself is already sandboxed so there's really only so much someone with malicious intent could get away with.

  • edited September 2013 Posts: 157

    Thanks for the info, @toadkick. I'm still new to Lua in general, so I've got a lot to learn.

    Speaking of serialization libraries:

    Here's something I whipped up. It's not JSON. It's not XML. It's a psuedo-INI file format that I use with my projects on various platforms. It's Human readable, and the actual format is very simple to understand.

    http://gist.github.com/compiledtom/6443652

    you only need to include the Serialization tab in your project. The main tab is just a test rig, and the Output tab just shows a sample of what the data looks like.

    The test rig actually creates an object, serializes it, deserializes it to a new object, then serializes it again. It writes both serializations out to the Output tab so that you can see a full round trip.

    It's worth noting that I haven't gone to the trouble to instantiate classes or userdata objects (like vec2.) If you look at lines 4 and 26 of output.lua, you will see that I've got special key names for objects.

    If you want to deserialize in to objects, you'll need to instantiate your objects, then copy the fields from the class: fields over to the object in question. Using a for loop, that should be fairly straightforward.

    If you want to deserialize in to tables, or if your objects don't have child objects, there's no extra work required.

        -- save table data to a string
        data=getString(myTable)
        -- then write the string to storage however you like
    
        -- loading a table requires 2 steps
        data=[load from storage]
        newTable={} 
        loadTable(newTable,data)   
    
        -- loading an object is easy, as long as there's only 1 level of data
        data=[load from storage]
        newObject=MyClass()
        loadTable(newObject,data)
    

    Have fun!

  • Posts: 2,042

    @Zoyt, I am getting an error when using this with a table of images. Is that normal?

  • Posts: 2,820

    @JakAttak - Yes. You need to find a way to encode the images in a string. So read the image and use image:set() to encode it intor easble code.

  • Posts: 2,042

    Thanks.

  • Posts: 557

    @tomxp411, your serializer works great, thanks for it! One thing: in making a string from an indexed table, it doesn't separate the values with commas, so it can't, for instance, write usable Lua tables to a Codea tab.

  • Posts: 2,020

    @UberGoober Codea now has json.encode and json.decode which for certain uses is a very handy way of converting a table to a string and back. It takes a bit of work if the table contains Codea data types (vec, matrices, color etc), and can be unpredictable if you feed it tables that mix array and dictionary types in a single space, or ambiguous cases such as arrays that have large gaps between the numbers (which is a bad idea anyway IMO). With huge tables (a million entries) json encode/decode can be slow. If the table is, say, a single dimension array, then table.concat is fastest. But if you need a simple way to convert a deep table containing standard Lua variable types into a string, then json is very simple and handy.

  • Posts: 557

    @yojimbo2000, thanks for the tip. @tomxp411's code is perfect for the case at hand, except for the comma thing, which seems like it should only take a little tweak to fix. It actually should be easy for me to see how to fix it, but unfortunately it isn't. I'll look into those others if he doesn't respond.

Sign In or Register to comment.