It looks like you're new here. If you want to get involved, click one of these buttons!
Apologies for such a basic question. This is the first time I've used the json.encode / json.decode functions, and I've clearly missed something basic. Though the help document indicates that the encode statement will take any object that implements the metamethod, it gives no example of how to do so. I've searched Jason on the forum, and also did not see an example.
I tried implementing as I have simple things like a change in _index, but it doesn't seem to be effective as I'm still getting an error stating encode won't that "user data."
So,,, someone give me a smack and explain the blindingly obvious thing I missed. Thanks.
Comments
My entry to the gamejam uses json for saving levels. Have a look at the Level IO tab in this:
https://gist.github.com/Utsira/8a364a35f397f26aefad
@yojimbo2000 Thanks, but it looks as if you're encoding simple tables. That works fine for me, as well. My question is implementing the __tojson method within a complex object so that it can properly encoded. I can turn the object into the save string I've been using to file them away, or a set of key-value pairs and encode that, but there must be a neater way. After all, that seems like the work I expect out of encode / decode.
Oh I see. You mean adding json encode/decode support to, say, a vec2? Can you be more specific about which objects you want to encode?
These are classes I created in the current application (in this case, a class called "Location" which is actually a work location within an office / work site and contains a number of fields, including a series of coordinates).
However, just an example of how to implement __tojson so that it supports json.encode would be fine. Even on something like a Vec2. All I can find is the notation in the help file that indicates .encode supports any object that implements the metamethod __json encode, but I can't find any examples here or elsewhere showing what that implementation looks like.
Thanks.
So you're trying to encode the instances of the location class? If they're held in a table, you can encode that table, unless it contains "user data" (ie vectors, matrices etc). I'd also like to know how to implement json encoding for vectors etc. Do you know specifically within the location class, what part of it is causing the "user data" error? What I'm saying is, I don't think you need to implement a custom json encode for your location class as such, you just need json encoding for the objects contained within it (from your description, vectors).
Eg. Instances of this class can be encoded, as long as they don't contain "user data" (vectors).
I'm not sure the encode, decode works properly, or else I'm wrong in what I'm expecting. In the example below, table
tab1
is created. I encode it and print the result. I then decode it into tabletab2
. I encode it and print the result so I can compare it to the original table. The results are the same, but maybe out of order which shouldn't matter. I can print values fromtab1
, but I get errors when I try to print it fromtab2
.@dave1707, tab2 doesn't have an integer key 1, but a string "1". That's because JSON only allows string keys, so when you encode/decode tab1 into tab2, the [1] = {3,4,5} becomes: ["1"] = {3,4,5}. You can see that in the first print of z.
@dave1707 mixing key-values with an array is a terrible idea IMO. You shouldn't expect to be able to index it by number.
@Mark instead of adding the tojson method to whatever object you need encoding, how about using the "exception" field to catch the parts of the class that are causing json to fall over. If it's just vec2s that should be quite straight forward. Can you show us the code for the class whose instances you're trying to encode?
@Mark here's my example from above, but using the "exception" field to catch the vec2s and convert them into a table/json object:
"pos":{"x":9.0,"y":50.0}
. I haven't worked out how to do the equivalent decode though yet. I need to work out how to indictate that this is a type vec2.Edit: v1.2 more elegant jsonException class, now indicates that table-object is a vec2
"pos":{"vec2":{"x":9.0,"y":50.0}}
. No decoding of vec2s yet.Here's a version that attempts to decode the vec2s. It's not particularly elegant, as there's no corresponding equivalent to the
exception
function injson.decode
(this is where implementing the__tojson
meta method might be the way to go, as @Mark said in his original enquiry).Edit: here's a working version of the code
here is how to implement _toJjson to vec2
@Jmv38 that's awesome, thanks. Though might I suggest adding a
vec2=true
flag to the json-vec2 so that the decode process can convert it back to a genuine vec2 (rather than leaving it as a pseudo-vec2 table)? Unless someone can think of a more elegant way to handle the decode?@yojimbo2000 i agree. I had not enough time to rework and post your last version, so i just posted this one to show you how to use __toJason (as you asked for it earlier). Thanks.
Ok, here's my version using @Jmv38 's meta methods. It now supports vec2, vec3, vec4, matrix, with support for conversion back to the proper types. In the code below I show it recreating the instances of the class in the objects table from the saved code.
you are faster than me! I was preparing this: a version to manage any class with "Class" keyword. How do you like it?
i found this more general than vec2=true.
@Jmv38 Ok. Actually, I think my preferred method would be, rather than a keyword, nesting the data in a sub table with an appropriate key, eg a vec3 called pos looks like this:
"pos":{"vec3":{"y":18.0,"x":17.0,"z":19.0}}
. Edit, added exception method, has feature parity with meta-method:Good idea. However, it means you cant have a field in a table with the same name as an existing userdata. This forbids many keywords!
Ok, I see what you mean. How about I preface each keyword ie "uservec4", "usermat4". Edit: this method is handy with matrices where adding the keyword ruins the array (table.unpack stops working).
Is there a comprehensive list somewhere of all the kinds of "userdata" ? I guess it's all the extra non-Lua APIs that Codea provides, eg Mesh, physics.body etc. I wonder how far this could be taken? Would be quite cool to be able to save a physics body or a mesh in this way. Will need to check whether you can iterate through all the values of a physics body or a mesh using
pairs
...Wow. I looked a way for a bit, and when I looked back, you two had beaten the problem into submission.
Thanks, @jmv38 and @yojimbo2000. Exactly what I wanted to see.
You can't use
pairs
on a mesh or a physics body, so you'd have to go through all the keys by hand. That's not too bad I guess, a cut-and-paste job...@Mark you're welcome. I learned a lot. I'm new to meta-methods, don't really use them in my own code. Depending on the complexity of the object, I think an encode exception function might be simpler (ie my last post above). If you have lots of different kinds of objects though, meshes, physics bodies etc, then meta methods is probably the way to go
@Jmv38 or how about 2 underscores before each special key, eg
__mat4
?great idea! I think you've nailed it.
On a related note, I'm having issues here: http://codea.io/talk/discussion/6679/how-do-i-get-the-meta-table-of-a-physics-body#latest
@Jmv38 any ideas?
here is a possible solution. It is not 'finished' but you get the idea.
@juce
That's because JSON only allows string keys
. JSON works with numeric keys, see the example below. If I add a string key to tab1, it doesn't. So it looks like you can have numeric keys or string keys, but not both in the same table. I haven't needed to use json for anything, but just wanted to see how it worked.@Jmv38 this is my version of json encoding of a physics scene. You can save a snapshot of the scene, and then load it in again. Could be useful for a level editor in an Angry Birds style physics game:
https://gist.github.com/Utsira/da4fda9b95e9112d705a
Here's the class with the various meta-methods:
@Jmv38 thanks for showing us how to set the toJson metamethod (for physics bodies too!)
@dave1707, you are right: i should've been more specific - json only allows string keys in dictionaries. The edge cases ultimately come from data structure differences: Lua's table can have keys of arbitrary types and also has the notion of having an array part and a hash (dictionary) part, while json has two different container types: dictionary and list.
I think what happens is that json.encode looks for special case when encoding tables: if a table only has integer keys, with the smallest one being 1, then it converts such table to a list, putting nulls into missing slots. Otherwise, it converts the table into a dictionary, collating numeric keys into string keys.
But even that description is not entirely accurate: notice how second table has "too many" nil slots, compared to first one - and gets converted into a dictionary, not a list.
@juce that's interesting how it decides whether to go with dictionary or list. I avoid mixed dictionary-lists in Lua, it just seems like too much potential for unexpected behaviour
@juce Interesting. I guess you really need to know how the decode is going to handle a table that's encoded. You could write a program that uses a table without any problems only to have it crash when you try to recreate a new table with decode.
@yojimbo2000, i agree that mixing numeric keys with keys of other types is dangerous, and best avoided if possible. However, even if we use only numeric keys, this table-to-list conversion algorithm can play unexpected tricks: say you have a sparse table with integer keys - it can probably change back and forth between list and dictionary, as you add more values to it, depending on the exact distribution of keys.
Like that:
On my iPad 2 it goes from list to dictionary and back to list:
I also try to avoid sparse lists, by using
table.remove
to make sure there's no holes. @dave1707 try my code at the gist. I'll put my neck on the block and say it's reasonably stable at recreating a physics scene. :-SShttps://gist.github.com/Utsira/da4fda9b95e9112d705a
@dave1707, i think decode is ok, because it is fully deterministic. It's the encoding step that can have unexpected results. But yes, the end result of encode + decode will not necessarily recreate the table as it was before.
@yojimbo2000 I'm not saying it won't work and I'm sure yours does, but for someone who doesn't understand how encode/decode works could be in for a surprise if they use it. Just like me, I thought encode/decode would take a table, convert it to a string that could be saved, then recreate the table exactly like the original. But it turns out there's limits, even with simple tables. I don't have any code that uses encode/decode and probably never will. It's just that when it was brought up in this discussion, I thought I'd try it to see how it works and ran into some surprises that I pointed out above. If someone uses this in an App Store program without understanding everything about it, their program could crash.
I guess I tend to think of dictionaries as being somewhat unpredictable anyway - they have no order as such, in that you can't predict what order
pairs
will return the items in.@yojimbo2000 @dave1707 I am perpetually baffled by the fact that pairs() seems to be so inconsistent in its traversal of a table.
@Simeon @John I am curious how it is possible for pairs() based for loops to occur in different order each run?
I was using them to parse and layout GUI objects, and it was practically random order each run. This goes against everything I have ever seen in programming, usually the same scenario produces the exact same result..
I alwasy was assuming that pairs() would work alphabetically, or in the order that the values were created.. Is this related to available memory locations / garbage collection / frame rate? It may be worth adding a note in the built in documentation, I was going bannanas trying to figure out why things were always showing up in a different order and ended up having to switch to ipairs to keep from losing my mind :P`
Super useful thread btw, some of this seems worthy of inclusion into a built in demo / example of some sort.
@AxiomCrux I think one of the reasons pairs works the way it does is because pairs will iterate thru a table even if it contains empty entries. For example if tab[5]=100 and tab[1000]=345, pairs will show both entries, but ipairs wont show either because it's not consecutive starting at 1. You're probably correct when you say that the tables are probably created based on available memory locations. I did a google search for pairs and found this:
Description
Returns an iterator function(next) for a for loop that will return the values of the specified table in an
arbitrary order
.heheh yeah! @dave1707 "arbitrary order" thats the thing I can't seem to fully wrap my head around..
I've been using computers since I could form memories, and I have recently come to understand cryptography and how 'random' number generation works (which opened my mind to profound philisophical insight into chaos theory and the nature of the universe).
"arbitrary order" and programming are opposing concepts IMHO. The notion that a key component of this awesome Lua programming language functions that way.. . scrambles my brain into "arbitrary order" a bit.
@AxiomCrux Maybe you'll find this interesting. It contains something about tables but I don't know if it really answers anything about arbitrary order.
Don't think of it as "arbitrary order", just unordered. JSON dictionaries are unordered as well. If order is important, you need to put the elements into an array.
@yojimbo2000 @AxiomCrux The comment I mentioned in the above post and what I show below is wrong for pairs. It won't
return
the values in arbitrary order, but willcreate
the table in arbitrary order. If you run the example code below and keep tapping the screen to read the table, it will show the same results in some order each time. If you restart the program, it will create the table in a different order.