Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

__call() metamethod in class() function

edited February 10 in General Posts: 10

I'm new to Codea, but not new to Lua, and have a question about the implementation of the class() function in Codea, specifically the __call() metamethod. According to the Codea Wiki (which of course may be out of date), this function (declared inside class()) is as follows:

mt.__call = function(class_tbl, ...)
        local obj = {}
        setmetatable(obj,c)
        if class_tbl.init then
            class_tbl.init(obj,...)
        else 
            -- make sure that any stuff from the base class is initialized!
            if base and base.init then
                base.init(obj, ...)
            end
        end

        return obj
    end

This all makes sense to me (though I'd have liked to see the way init() is handled documented more clearly in the docs), EXCEPT for the way setmetatable() is used. Why is the second argument "c" and not "class_tbl" ?

__call() will be invoked when I create an object instance using (e.g.) "Button()", and in this case the class_tbl argument will be "Button", which is of course "c" anyway, so why use "c" here? Using "c" has a side-effect of the __call() function capturing "c" as an upvalue, which is really not needed (and makes __call() slightly less efficient) so far as I can see.

I suspect I'm missing something subtle here, probably to do with class inheritance, but I've thought hard about this and can't come up with anything. Anyone want to comment on this?

--Tim

Comments

  • Posts: 195

    @drtimhill I read the class function, and you can also see it in xcode in the runtime resource bundle when you export your project? Can you explain your question, I don't seem to understand it. The setmetatable will have a method for when you first instantiate it, for example:

    Button=class()
    function Button:init()print"hello world"end
    function setup()
          b=Button()
    end
    

    this will of course print the hello world because it is the default in the class() function as the mt:__call(), if your asking how to call a different function, then you could set up the meta table manually or say:

    function Button:init()self:callFunction()end
    

    If I didn't answer your question, can you please restate it?

  • Posts: 131

    @drtimhill, i see what you mean. I think this is a question for @Simeon.
    (My wild guess is that "c" was shorter to type :) )

  • edited February 10 Posts: 157

    @drtimhill One thing that you may not have considered is that using c instead of class_tbl prevents someone attempting (and succeeding at) something like this:


    MyClass = class() MyOtherClass = class() local invalidInstance = MyClass.__call(MyOtherClass)

    Obviously it is probably quite unlikely that somone would actually try that but otherwise I don't think that there is really much else to miss :). Inheritance is being handled by the by the shallow copy from the base argument at the top of the class function.

    I would expect that the performance impact/overhead is minimal within a typical use of Codea. Perhaps it would make a difference if you were trying to work with an excessively large amount of classes and instances but you'd probably be struggling to keep your sanity in that circumstance anyway ;)

  • @xanddemox Yes that's a good point, though in that case I'd go one step further and argue that ALL the uses of class_tbl are suspect in __call(). Better yet, I would prevent your invalid use case by doing "if class_tbl ~= c then error() end" as the first line of the __call() metamethod. This would of course still capture c, but as you note there are not typically enough classes for this to be much of concern.

  • edited February 12 Posts: 157

    @drtimhill Can't fault your logic there. In fact my case would still create a potentially invalid mix-in like instance as a result of the init function being called on the passed class_tbl.

    So in reality all that it is actaully preventing, possibly unintentionally, is the __call metamethod returing an instance with a meta table other than the original class table returned by the class function.

    Personally I'd probably just take the chance with class_tbl and assume that my users would not call meta methods directly and not worry. Simply because they have no reason to, not for any purpose that I can think of anyway, invoking them is handled by the runtime and thats how it probably should remain.

  • Posts: 1,979

    @drtimhill

    You could experiment by overriding Codea's class implementation. If you wanted to do comparative benchmarking, you could retain Codea's implementation (eg by putting codeaClass = class before you override class).

    It seems there are lots of variations on class implementations in Lua

    http://lua-users.org/wiki/YetAnotherClassImplementation

    ^ This one for example supports calls to the superclass with the super keyword (eg super:init(args), more elegant than the Codea version of passing in self as the first parameter with dot notation, Animal.init(self, args)) and retrieving the name of the class.

    It'd be interesting to see if there are any performance differences or other issues between the various implementations

  • SimeonSimeon Admin Mod
    Posts: 4,352

    @yojimbo2000 @drtimhill @XanDDemoX should we migrate to a different class implementation in Codea? I'm open to the idea.

    It would be nice to keep compatibility with existing code, but I'm not too concerned about that if the ultimate result is an API improvement.

  • Posts: 157

    @Simeon I would certainly be hesitant about migrating to a totally different class implementation. Because generally I find Codea's current implementation is simple, pretty elegant and largely fit for most purposes. I think If I needed something more sophisticated then I'd just extend it to meet that specific case's requirements.

    Having said that, I still couldn't resist a little bit of experimenting with the current implementation from here and I think I have come up with a couple of improvements and made it slightly shorter whilst I was there. They should all be non breaking changes but please do scrutinise :)

    Changes
    * Removed upvalues from mt.__call.
    * Changed c in mt.__call to class_tbl as suggested by @drtimhill
    * Removed calling base init because class_tbl will already be a shallow copy of base if it has one. (You can test this by commenting out the init function from TestClass3 in example below)
    * Renamed _base to base to imply that its public instead of private. So it can then be used to reduce the potential amount of renaming required when refactoring class names within inheritance hierarchies by using ClassName.base.func instead of BaseClassName.func. It's not quite the same as a super keyword but its a simple way of achieving the same result. Example below implementation.

    function class(base)
        local c = {}    -- a new class instance
        if type(base) == 'table' then
            -- our new class is a shallow copy of the base class!
            for i,v in pairs(base) do
                c[i] = v
            end
            c.base = base
        end
    
        -- the class will be the metatable for all its objects,
        -- and they will look up their methods in it.
        c.__index = c
    
        c.is_a = function(self, klass)
            local m = getmetatable(self)
            while m do 
                if m == klass then return true end
                m = m.base
            end
            return false
        end
    
        -- expose a constructor which can be called by <classname>(<args>)
        local mt = {}
        mt.__call = function(class_tbl, ...)
            local obj = setmetatable({},class_tbl)
            -- Calls class_tbl.init or base.init, no need to check for a base init 
            -- explicitly because class_tbl is already a shallow copy of base.
            if class_tbl.init then
                class_tbl.init(obj,...)
            end
    
            return obj
        end
        return setmetatable(c, mt)
    end
    

    Example

    TestClass = class()
    function TestClass:init()
        print("Test class init")
    end
    
    TestClass2 = class(TestClass)
    function TestClass2:init()
        TestClass2.base.init(self)
        print("Test class 2 init")
    end
    
    
    TestClass3 = class(TestClass2)
    function TestClass3:init()
        TestClass3.base.init(self)
        print("Test class 3 init")
    end
    
    function setup()
        obj = TestClass3()
    end
    
    -- Output
    -- Test class init
    -- Test class 2 init
    -- Test class 3 init
    
  • SimeonSimeon Admin Mod
    Posts: 4,352

    @XanDDemoX I like your Class.base.func syntax better than BaseClass.func as it decouples your class code from the type name of the base class.

    I look into updating the class function post-2.3.3 (which is still in review for some time now, sadly).

  • edited February 14 Posts: 10

    @simeon if we are thinking of re-working this I have a couple of suggestions...

    -- There is no reason why all classes should not share the exact same metatable "mt" and __call() function. You can handle this by moving the declaration outside of the class function, but inside a "do" block to make the metatable private to class(). For example:

    do
        local mt = {}
        mt.__call = function(class_tbl, ...)
            ...
        end
        function class(base)
            ...
            setmetatable(c, mt)
            ...
        end
    end
    

    -- As @xanddemox noted, "base" is better than "_base", and in fact "self.base" will work as well from any method, which is pretty close to a "super" construct.

    -- Is shallow copy a good way of doing inheritance? This model means that later additions/alterations to a base class are not inherited by any existing derived classes. A more common way in Lua (see PiL chapter 16) is to use the recursive nature of [] and __index to follow the inheritance chain when a member/method is resolved. Admittedly this is perhaps more advanced than a typical Codea user might be comfortable with, but it does have advantages when it comes to mix-ins.

  • edited February 14 Posts: 157

    @drtimhill Careful with self.base it works for a single level of inheritance but if you attempt to use it across two or more levels of inheritance it will result in infinite recursion and crash Codea.

    Class1 = class()
    
    function Class1:init()
    end
    
    Class2 = class(Class1)
    function Class2:init()
       self.base.init(self) -- base is still Class2 not Class1
    end
    
    Class3 = class(Class2)
    function Class3:init()
       self.base.init(self) -- Class2 as expected 
    end
    
    
    function setup()
       --local i = Class3()
       -- uncomment above to test. Warning, testing will cause Codea to crash.
    end
    
  • @xanddemox Agreed, but that's quite easy to fix with a proper inheritance chain .. I'll post my alternate for class() later today, would be interested in your feedback.

  • edited February 15 Posts: 10

    @XanDDemoX @Simeon Well this is what I came up with as a replacement for the existing class() in Codea. Advantages are:

    -- Same syntax and functions as existing class() model
    -- Each object has '_class' member to locate it's class
    -- Each class has '_base' member to locate it's immediate superclass
    -- Shallow copy of class replaced with dynamic binding allowing superclass mixins to be inherited by subclasses even after they are created
    -- Can use 'MyClass._base.method(self, ...)' to access superclass, thus keeping class hierarchy elastic and easy to refactor (no hard-wired class names in subclass methods)

    Note that code that relies on the old shallow copy might break in some rather unusual cases (which are probably bad code anyway imho).

    As noted by @XanDDemoX , it might be better usage to rename '_base' to 'base', though this would break some existing code. I would however leave '_class' alone as this value is placed in the object table and using 'class' might collide too easily with user-defined object fields (which might mean retaining '_base' makes more sense as it mirrors '_class').

    Note that, within a method, there are now three ways to call another method (e.g. 'foo()'). Each way scans the class hierarchy from a starting point to the _Root class until the 'foo()' method is found, but each way differs in the starting point for the scan (this also applies to class variables). The three ways are:

    self:foo(...)
    -- This scans the class chain for 'foo()' starting with the actual class of the self object. This is the most typical way to call methods from other methods as it follows the classic virtual method model.

    MyClass.foo(self, ...)
    -- This scans the class chain for 'foo()' starting at MyClass (presumably the same class as the current method). This is used for non-virtual functions that may be found in the current class or a base class.

    MyClass._base.foo(self, ...)
    -- This scans the class chain for 'foo()' starting at the superclass of MyClass, and is used to call "down" the class chain to an over-ridden method.

    (Note that all these scans are handled directly by the Lua VM and so are fast.)

    Code and example usage follows:

    --//// Replacement for existing Codea class() functionality (@drtimhill)
    
    do    -- Hide locals private to implementation
        --//// factory(): Construct new object of specified class
        -- (Called indirectly using class constructor e.g. "Foo(...)")
        -- (As a convenience, we set obj:_class to the class)
        -- cls      Class of object to create
        -- ...      Arguments to forward to class:init() initializer
        -- returns  Newly constructed object
        local function factory(cls, ...)
            local obj = setmetatable({}, cls)
            obj._class = cls
            obj:init(...)
            return obj
        end
    
        --//// _Root: The root class for the class hierarchy
        -- Implements basic class functions, but has no __call, so
        -- that it is not possible to create instances of _Root
        local _Root = {}
        _Root.__index = _Root
        _Root.__call = factory
    
        --//// obj:init(): Initialize a new object (class instance)
        -- self     Object to initialize
        -- ...      Arguments forwarded from constructor (e.g. Ball(...))   
        function _Root:init(...)
            -- Dummy init for _Root
        end
    
        --//// obj:is_a(): Check to see if object is instance of class
        -- self     Object to check
        -- cls      The class to check (this class and all its superclasses)
        -- returns  true if the object is an instance of the class/superclass
        function _Root:is_a(cls)
            local m = getmetatable(self)
            while m do
                if m == cls then return true end
                m = getmetatable(m)
            end
            return false
        end
    
        --//// class(): Create a new class, with optional base class
        -- (As a convenience, we set cls:_base to the base class of the class)
        -- base     Base class, default is _Root class if none specified
        -- returns  New class (create objects using class constructor e.g. "Foo()")
        function class(base)
            base = base or _Root
            local cls = setmetatable({}, base)
            cls.__index = cls
            cls.__call = factory
            cls._base = base
            return cls
        end
    end
    
    -- Some simple examples
    MyClass = class()
    MyDerivedClass = class(MyClass)
    MyThirdClass = class(MyDerivedClass)
    
    -- Note that we can create base class methods after setting inheritance above
    function MyClass:init(x, y)
        print("MyClass:init(): x=" .. tostring(x) .. ", y=" .. tostring(y))
        MyClass._base.init(self)        -- Not strictly needed for class derived from _Root
        self.x, self.y = x, y
    end
    
    function MyDerivedClass:init(x, y, r)
        print("MyDerivedClass:init(): x=" .. tostring(x) .. ", y=" .. tostring(y))
        MyDerivedClass._base.init(self, x, y)
        self.r = r
    end
    
    function MyDerivedClass:draw()
        print("MyDerivedClass:draw(): x=" .. tostring(self.x) .. ", y=" .. tostring(self.y))
        MyDerivedClass._base.draw(self)
    end
    
    function MyClass:draw()
        print("MyClass:draw(): x=" .. tostring(self.x) .. ", y=" .. tostring(self.y))
    end
    
    o1 = MyClass(10, 100)
    o2 = MyDerivedClass(20, 200, 1.0)
    o3 = MyThirdClass(30, 300, 2.0)     -- Will use MyDerivedClass:init()
    
    print("o3:is_a(MyDerivedClass)=" .. tostring(o3:is_a(MyDerivedClass)))
    print("o2:is_a(MyThirdClass)=" .. tostring(o2:is_a(MyThirdClass)))
    
    o1:draw()   -- Calls MyClass:draw()
    o2:draw()   -- Calls MyDerivedClass:draw()
    o3:draw()   -- Calls MyDerivedClass:draw()
    
  • Posts: 157

    @drtimhill Interesting implementation, I would have perhaps named the _Root class Object and also kept it public so that it would be available to use with the is_a function.

    But I quite like the idea of having a .class member it would be useful for situations where you would want to avoid checking is_a for combinations of classes by replacing it for something like instance:is_a(instance2.class).

    I suspect it may actually be just a little bit cheaper than using getmetatable(instance2) also. Although admittedly situations where I have found that I have needed to do that myself have only been once or twice perhaps but it's inexpensive, one extra pointer per class table, so I wouldn't say it's completely unjustified.

    I can certainly see it would be useful for classes where their members could potentially mutate anywhere within their hierarchy at runtime. I expect it would be very good for things like binding to COM objects, complicated deserialisation situations like a custom script parser or as you mentioned, mixins.

    However I think that is also what makes up part of my concern, I'm not sure that classes whose members have the ability to mutate at runtime is truely standard class-like behaviour.

    Comparing to similar ideas in other languages for example, It feels closer to concepts like the .net framework's dynamic objects than its classes. Of course overlooking the fact that obviously Lua and c# are very different languages indeed. Lua already has much more dynamic-ness baked in to start with than c# but thats not really what I'm getting at.

    Dynamic class member mechanisms in other languages, that I know of, tend to be provided as specialised objects, modules or frameworks instead of being available as part of the general class mechanism. Although they're not necessarily dynamic languages like Lua so this makes sense to some degree.

    However the term class is more universal. Although your implementation is probably closer to how class tables are actually arranged behind the scenes in other languages. Users are likely to have a certain expectation of what a class is in Codea and what it does and doesn't do. Considering also that there will already be a good number of existing users who will be used to the current mechanism. Some may find it hard to understand the difference, recursion and trees can be tricky topics for beginners.

    Still, of course I couldn't resist a brief play around with the code anyway. As you mentioned that the searches are fast because they are handled by the Lua vm. I thought I would try and get an idea of just how fast the various implementations are by writing some tests. The source code for them is here.

    They're not completely comprehensive and they have been run on an iPad2 so expect different results on other devices. In fact they can vary every time they are run, probably the resolution of os.clock() or something internal to Codea or ARM wizardry, who knows. Timing things completely accurately is not necessarily as easy as it sounds but os.clock() is still good enough to get a rough idea of how they compare.

    I threw them together a bit quick, so they just focus on a few small common operations. Creating instances, accessing a member in a base class, calling a function in the base class and doing an is_a lookup using a base class. Then to get some sort of measurment the operation is simply executed repeatedly over hundreds, thousands, or millions of iterations and the difference of the starting and ending os.clock is taken to obtain the rough execution time in seconds as a float.

    100 iterations
    * CodeaClass3 init: 0.003641999999999
    * NoUpValuesClass3 init: 0.000684999999990
    * IndexClass3 init: 0.001076999999995
    * ShallowCopyClass3 init: 0.001724999999993
    * IndexClass3 veryBase lookup: 0.000000000000000
    * ShallowCopyClass3 veryBase lookup: 0.000000000000000
    * IndexClass3 veryBase call: 0.000349999999997
    * ShallowCopyClass3 veryBase call: 0.000316999999995
    * IndexClass3 is_a: 0.005448999999999
    * ShallowCopyClass3 is_a: 0.000728000000009

    1,000 iterations
    * CodeaClass3 init: 0.006657999999987
    * NoUpValuesClass3 init: 0.006633999999991
    * IndexClass3 init: 0.016254000000004
    * ShallowCopyClass3 init: 0.012048000000021
    * IndexClass3 veryBase lookup: 0.001968000000005
    * ShallowCopyClass3 veryBase lookup: 0.000000000000000
    * IndexClass3 veryBase call: 0.003249000000011
    * ShallowCopyClass3 veryBase call: 0.002688000000006
    * IndexClass3 is_a: 0.010435000000001
    * ShallowCopyClass3 is_a: 0.019332999999989

    10,000 iterations
    * CodeaClass3 init: 0.067147000000006
    * NoUpValuesClass3 init: 0.056746000000004
    * IndexClass3 init: 0.062687000000011
    * ShallowCopyClass3 init: 0.063163000000003
    * IndexClass3 veryBase lookup: 0.022717999999998
    * ShallowCopyClass3 veryBase lookup: 0.015279999999990
    * IndexClass3 veryBase call: 0.022357000000000
    * ShallowCopyClass3 veryBase call: 0.026342000000000
    * IndexClass3 is_a: 0.081246999999991
    * ShallowCopyClass3 is_a: 0.051556000000005

    100,000 iterations
    * CodeaClass3 init: 0.588872000000009
    * NoUpValuesClass3 init: 0.595162000000016
    * IndexClass3 init: 0.843783000000002
    * ShallowCopyClass3 init: 0.633445999999992
    * IndexClass3 veryBase lookup: 0.111572999999993
    * ShallowCopyClass3 veryBase lookup: 0.090078000000005
    * IndexClass3 veryBase call: 0.149608000000001
    * ShallowCopyClass3 veryBase call: 0.116005000000001
    * IndexClass3 is_a: 0.604467999999997
    * ShallowCopyClass3 is_a: 0.381372999999996

    1,000,000 iterations
    * CodeaClass3 init: 6.174376999999993
    * NoUpValuesClass3 init: 7.417577999999992
    * IndexClass3 init: 12.753726999999969
    * ShallowCopyClass3 init: 10.579114000000004
    * IndexClass3 veryBase lookup: 1.679075000000012
    * ShallowCopyClass3 veryBase lookup: 1.163724999999999
    * IndexClass3 veryBase call: 2.188662999999963
    * ShallowCopyClass3 veryBase call: 1.688381999999990
    * IndexClass3 is_a: 9.564642999999990
    * ShallowCopyClass3 is_a: 4.918670000000020

    To be honest its not easy to draw a conclusion from the results above because of the inconsistency across repeated runs. However I will make an observation that under normal use conditions there does not appear to be a significant amount of variance so there would probably be no noticeable difference between using the different implementations ordinarily.

    However still I think the results are better off speaking for themselves accompanied with the following disclaimer.

    The results above are taken after a number of runs at the point where the numbers feel sensible-ish. The lower iteration tests took a number of runs to even get a measurement. So there is no point taking any of them too seriously and you should probably expect a good degree of error within them. Try the tests out for yourself, you should see what I mean :).

  • @XanDDemoX Wow thanks for the great feedback. The benchmarks are particularly interesting, and I think I agree that in practice they tend to suggest that there is no significant overhead to the dynamic inheritance mechanism.

    I admit i flipped back and forth on making _Root public, and what it should be called. However, I was worried about using Object as I suspect it would collide with some existing code. Also, "someobj:is_a(_Root)" will always be true, since _Root is always in the hierarchy of any class, so calling this isn't really useful. Given that I don't allow instances of _Root, and I felt that it would be bad for someone to tinker with the base class (adding or changing a root method), I felt that hiding it added to the robustness of the system (quite apart from avoiding pollution of the global namespace).

    With regards to how closely this maps to other class models, as you note Lua is a dynamic language so it's nice to take advantage of this. In fact, the model I suggest is very similar to the one described in PiL, though with a clearer distinction between classes and instances.

    The dynamic binding model essentially allows mixins to existing classes, which is very much like protocols in Objective-C and other late binding languages like Scheme. There are a couple of reasons I like this in Codea:

    -- I think it's more intuitive for beginners (yes, really). My feeling is that a user who changes a base-class method will be surprised when his derived class doesn't use that method (which is what happens with the current shallow copy model). In my dynamic model there is only ever one copy of a given class method, and changes to it are instantly picked up everywhere (existing instances and existing derived classes).

    -- It allows better reuse of existing classes. This is particularly true when you use dependencies to pick up existing classes. Say you want to use an existing class but with a minor tweak to a single method. At the moment, you could clone the entire class and modify it (bad). Or take a dependency and sub-class it to add your new method. But this doesn't work for other classes that already derive from this class. With dynamic binding all you do is take a dependency on the class, then just mix-in your modified method. Voila! Everyone starts using the new method, even existing classes derived from that base class, without you touching the code (so you can import more dependencies and use them unchanged).

    Here's a real example. You are using a class, "Widget", which you added to your project through a dependency. You have lots of calls to Widget.doit() scattered about your code, but have a bug you cannot track down. To help track the bug down, you decide to add a temporary "print()" statement in Widget.doit() so you can track down what is going on. How do you do this?

    With the existing model, you have to modify the external Widget dependency class (and thus have debug code injected into other projects as a side-effect). But with the dynamic model, you can inject a private Widget.doit() into the Widget class only for this project:

    Widget.olddoit = Widget.doit
    function Widget.doit(x, y)
        print(x, y)
        return Widget.olddoit(x,y)
    end
    

    Add that code to "Main" and you're all set to go. When you find the bug, delete it and you are done.

  • edited February 21 Posts: 157

    @drtimhill I agree that colliding names could produce undesirable results but hiding _Root wouldn't prevent it being overridden.

    SomeClass = class()
    _Root = getmetatable(SomeClass)
    

    However adding another reserved name can still be avoided by switching class to be a table with a __call metamethod instead of a function. It would be more consistent with other parts of the Codea API by adding the possibility of exposing class related static functions in Codea's API.

    For example

    class.is_a(instance,MyClass)
    

    Is shorter and clearer to write than:

    type(obj) == "table" and obj.is_a and obj:is_a(MyClass)
    

    Another benefit is that it could also introduce a possible extensibility mechanism such as below where the function that creates new classes, the base class table and metatable that is assigned to class tables (which enables access to the __call metamethod that creates instances) can all be overidden naturally via the public api.

    class = {}
    class.metatable = {
        __call = function(class_tbl, ...)
            local obj = setmetatable({},class_tbl)
            if class_tbl.init then
                class_tbl.init(obj,...)
            end
            return obj
        end
    }
    class.is_a = function(self, klass)
        local m = getmetatable(self)
        while m do 
            if m == klass then return true end
            m = m._base
        end
        return false
    end
    class.new = function(base)
        base = base or class.base
        local c = setmetatable({}, class.metatable)
        for i,v in pairs(base) do
            c[i] = v
        end
        c.base = base
        c.__index = c
        return c
    end
    class.base = {}
    class.base.is_a = class.is_a
    
    setmetatable(class,{ __call = function(self,...) return class.new(...) end })
    
    TestClass = class()
    function TestClass:init()
        print("TestClass.init")
    end
    
    TestClass1 = class(TestClass)
    function TestClass1:init()
        TestClass1.base.init(self)
        print("TestClass1.init")
    end
    
    function setup()
        TestClass1()
    end
    
    

    Regarding your Widget class example, I can certainly see that it could be convenient to write in some circumstances whilst debugging and I agree that it may be easier for a beginner who has a basic grasp of dependencies and overriding to understand, although I suspect most would probably just modify the original function anyway.

    However I would solve the problem differently using function hooking and a few peices of known information to my advantage:

    • Codea executes tabs in the order, dependencies tabs in project tab order first to last excluding Main tabs then the project tabs first to last including the main tab and then it calls setup()
    • Objects that inherit and override the function to debug will call it directly with BaseClass.func so they can be safely ignored (unless their implentation contains the problem, but if thats the case you are debugging a different function)
    • The object(s) to be debugged are known.

    So from the above we can deduce that anything which holds on to a copy of the function pointer which will be replaced at SomeClass.func is the problem:

    • Any objects that inherit but don't override the function to be debugged because they shallow copy over the pointer from the base class table at the time of creation.
    • Function hooks. - Obviously function hooks would have to be debugged directly so they can be safely ignored.

    So that just leaves objects which inherit but don't override the function and objects that also inherit those objects but I would do some narrowing down somehow. Then in the last tab of the project or the setup function I would use something similar to the following example.

    A further benefit of this approach is that the hook function can also be used on specific instances so it could be used for more targeted debugging to some degree if that was necessary.


    function hook(base,func) return function(...) return func(base,...) end end SomeClass.func = hook(SomeClass.func,function(base,self,x,y) print(x,y) return base(self,x,y) end) SomeDerrivedClass.func = hook(SomeDerrivedClass.func,function(base,self,x,y) print(x,y) return base(self,x,y) end)
  • Posts: 157

    @Simeon Continuing on the theme of improving Codea's API around class I thought I'd put togther an implementation which extends my thoughts above for using a table with a __call metamethod to create classes instead of a function, which is closer to Codea's current definition of class but still includes some of the previously discussed improvements without changing the class model completely.

    Whilst writing the code I also found myself considering how it could also apply to userdata types such as vec2, vec3, matrix as there can be times where it can be necessary or convenient to be able to identify a specific userdata type.

    After all, they are essentially just classes except they have a special status because of their dual coexistance in Codea's backend. However even though they are technically classes, because of their specialness I felt it would still be clearer if working with them was handled separately but implemented so that the public interface is consistent with class's public interface. Some more in depth thoughts and example code example representing the public interface follows after the class implementation.

    Class
    Features/Changes

    • Change class from a function to an equivalent table with __call metamethod.
    • Removed upvalues from class table __call metamethod.
    • Renamed ._base to .base on class tables.
    • Renamed .is_a to .is to shorten and avoid typing a special character for on-screen keyboard users (an is_a alias is retained for backwards compatibility).
    • Exposed a class.is function to enable easier class instance checking when dealing with multiple types e.g class.is(obj, MyClass).
    • Use class as base for root class tables so that a class instance can be identified by class.is(obj,class) but class.is({},class) == false
    • Added a .class member to class tables to enable instance to instance type comparision e.g instance:is(otherinstance.class)
    • Added parameter assert validations to give clearer errors when misuse occurs.
    class = {}
    class.is = function(instance, klass)
        if type(instance) ~= "table" then return false end
        if klass ~= nil and type(klass) ~= "table" then
            assert(false,string.format("bad argument #2 to 'class.is' (class table expected, got %s)",type(klass)))
        elseif klass == nil then
            klass = class
        end
        local m = getmetatable(instance)
        while m do 
            if m == klass then return true end
            m = m.base
        end
        return false
    end
    
    setmetatable(class,{ __call = function(self,base) 
        if base ~= nil and type(base) ~= "table" then
            --avoid string format unless needed
            assert(false,string.format("bad argument #1 to 'class' (class table expected, got %s)",type(base)))
        end
        local c = setmetatable({}, {
            __call = function(klass, ...)
                local obj = setmetatable({},klass)
                if klass.init then
                    klass.init(obj,...)
                end
                return obj
            end
        })
        if base then
            for i,v in pairs(base) do
                c[i] = v
            end
        end
        c.is_a = class.is -- backwards compatibility
        c.is = class.is
        c.class = c
        c.base = base or class
        c.__index = c
        return c
    end 
    })
    

    Example usage

    function setup()
       local instance = TestClass1()
        local instance2 = TestClass1()
    
        print(string.format("class.is({},class) = %s", class.is({},class)))
        print(string.format("class.is(1) = %s", class.is(1)))
        print(string.format("class.is(instance,TestClass1) = %s",class.is(instance,TestClass1)))
        print(string.format("class.is(instance,TestClass) = %s",class.is(instance,TestClass)))
        print(string.format("class.is(instance,class) = %s",class.is(instance,class)))
        print(string.format("class.is(instance) = %s",class.is(instance)))
        print(string.format("class.is(instance.class, instance2.class) = %s",
        class.is(instance, instance2.class)))
    end
    --[[
    class.is({},class) = false
    class.is(1) = false
    class.is(instance,TestClass1) = true
    class.is(instance,TestClass) = true
    class.is(instance,class) = true
    class.is(instance) = true
    class.is(instance.class, instance2.class) = true
    ]]--
    

    Userdata
    Although I realise that identifying userdata types has been discussed quite a bit and there are a number of solutions available. It seems logical that Codea's API should provide some help with this because of the obscurity of the current solution for identifying them directly which usually works something like this.

    • obtain instances of userdata types which need to be identified later.
    • call getmetatable on the instances to obtain there metatable
    • create some form of lookup with the metatables that is keyed with a string name or the function used to create instances.
    • expose a function which does something like getmetatable(instance) == lookup[key] which is equivalent of class.is_a(SomeType) or overide type for example type(vec3) == "userdata", "vec3".

    Despite its slightly hacky feel the getmetatable solution is actually pretty effective, up to a point. But it all starts to get a bit messy and less containable to a single tab when you start trying to include metatables from types that can only be obtained via functions called by Codea such as touch or physics.contact.

    Also considering the fact that CodeaCraft is somewhere on the horizion it seems reasonable to assume that its release will probably increase the number of userdata types that are going to be available. So logically it could potentially also increase the likelihood of users programming themselves into situations where identifying the specific userdata type that they are working with would help them meet their requirments or just genrally be convenient and/or shorter to write.

    A simple example is easy to consider. Imagine a function which works with several different userdata types but only userdata types exclusively. Also assume that it performs slighlty different operations depending on the type.
    You can validate that a userdata type was passed easily, assert(type(obj)=="userdata","message"). But to validate which type you have several options:

    • use a function with named parameters
    • wrapping the values in classes so is_a becomes possible
    • resort to comparing metatables

    All of which obviously just generally increases the overhead (a little) and the amount of code for a user to write or tap out if they don't use AirCode or a bluetooth keyboard. With the comparing metatables solution probably being the longest and most complex.

    Features

    • Exposes a userdata.is global function which behaves like class.is
    • Exposes a userdata.type global function which behaves like typefor userdata types only
    • Parameter assert validations to give clear errors when misuse occurs.

    Note: Implementation only supports vec2,vec3, vec4.

    userdata = {}
    do
        -- Ideally userdata should, if possible, use tables with __call metatables to return instances so that 
        -- userdata.is could effectivly be getmetatable(vec3(0,1,0)) == vec3
        -- another alternative could be to use luaL_newmetatable for userdata metatables in Codea's backend
        -- it appears that in lua 5.3 it adds a __name key which can be used to lookup the name of the userdata type.
        -- e.g getmetatable(vec2(0,0,0)).__name would be "vec2".
        -- http://stackoverflow.com/questions/38932374/lua-querying-the-name-of-the-metatable-of-a-userdata-object
        -- https://github.com/keplerproject/lua-compat-5.3/issues/13
        -- https://www.lua.org/source/5.3/lauxlib.c.html#luaL_newmetatable
    
        local userdataInfo = { 
            {name="vec2", func = vec2},
            {name="vec3", func = vec3},
            {name="vec4", func = vec4},
        }
        for i,item in ipairs(userdataInfo) do
            local metatable = item.metatable
            if metatable == nil then
                metatable = getmetatable(item.func())
            end
            metatable.__descriptor = {
                name = item.name, 
                func = item.func
            }
        end
    end
    userdata.is = function(instance, func)
        if type(instance) ~= "userdata" then return false end
        if func ~= nil and func ~= userdata and type(func) ~= "function" then
            --avoid string format unless needed
            assert(false,
            string.format("bad argument #2 to 'userdata.is' (userdata function expected, got %s)",type(func)))
        elseif func == nil or func == userdata then
            return true
        end
        local descriptor = getmetatable(instance).__descriptor
        return descriptor and descriptor.func == func
    end
    
    userdata.type = function(instance)
        local instanceType = type(instance)
        --avoid string format unless needed
        if instanceType ~= "userdata" then
            assert(false,string.format("bad argument #1 to 'userdata.type' (userdata expected, got %s)",instanceType))
        end
        local descriptor = getmetatable(instance).__descriptor
        return descriptor and descriptor.name or "userdata"
    end
    

    Usage Example

    function setup()
        local vec = vec3(0,1,0)
        print(string.format("userdata.is({},vec2) = %s",userdata.is({},vec2)))
        print(string.format("userdata.is(1) = %s",userdata.is(1)))
        print(string.format("userdata.is(vec, vec2) = %s",userdata.is(vec, vec2)))
        print(string.format("userdata.is(vec, vec3) = %s",userdata.is(vec, vec3)))
        print(string.format("userdata.is(vec, userdata) = %s",userdata.is(vec, userdata)))
        print(string.format("userdata.is(vec) = %s",userdata.is(vec)))
        print(string.format("userdata.type(vec) = %s",userdata.type(vec)))
    end
    --[[
    userdata.is({},vec2) = false
    userdata.is(1) = false
    userdata.is(vec, vec2) = false
    userdata.is(vec, vec3) = true
    userdata.is(vec, userdata) = true
    userdata.is(vec) = true
    userdata.type(vec) = vec3
    ]]--
    
  • @XanDDemoX @Simeon Yes I was aware I wasn't fully hiding _Root, but just making it hard enough that you have to be REALLY keen to change it :)

    I did think about making class() a callable, but didn't feel the complexity bought enough in the way of added value. Being able to have a custom "new" might be interesting but I'm not sure that aspect of the object/class model really can be tinkered with much without creating strange bugs.

    Your alternative is interesting, but I think it still lacks the clarity (and conventionality) of the dynamic binding I was suggesting; shallow copy creates all sorts of odd ordering requirements that dynamic binding avoids, imho.

    Good point about "isa()" rather than "is_a()", though I wonder how often this function will get called in practice?

    As far as things like vec2() are concerned, yes it would be nice if they followed the same model, though I think they have other issues, such as lacking a copy constructor (important if you are using mutable vectors).

    As far as reflective access to class names goes, are you aware of the newer Lua convention of using "__name" in a metatable for the type of a userdata/table? This is respected by Lua 5.3.3 onwards for things like tostring() and print().

  • edited February 25 Posts: 157

    @drtimhill Thanks for the feedback. I agree that the first class example in reality probably wouldnt be that extensible without total replacement. It was more just to show adding a new reserved word could be avoided.

    The seccond example of class is more about the public interface. The implementation detail isn't really relevant to it. It is intended to work identically no matter what class model it is actually using, but I probably should have been clearer on that point :).

    I think the amount of calls to is_a is probably subjective and dependent on the user and what they are writing. Even perhaps on their individual style of writing code in general. Another consideration is that it is currently the only way of generically identifying a class instance.

    I think the easiest way to consider its usefulness is function parameter validation because it can shorten this quite drastically eg. assert(class.is(obj, MyClass), "Invalid type passed") vs assert(type(obj)=="table" and obj.is_a and obj:is_a(MyClass),"Invalid type passed").

    I did stumble across the __name convention in Lua 5.3 which is the version of Lua that Codea uses however it appears that currently Codea does not implement this convension. There are a few links in the userdata example that I found whist researching it.

    Finally just quickly you can copy vecs like this, but its not quite a copy ctor :)

    local vec = vec3(0,1,0)
    local veccopy = vec3(vec:unpack())
    
  • @XanDDemoX Interesting thoughts, I'm continuing to iterate on the whole "class() is a callable" thing and see where I end up. I admit atm I tend to like a distinct _Root just for clarity .. my brain reaches a point where too many self-referencing metatables simply shuts it down :)

    And don't forget that obj:is_a() is just syntactic sugar, there is nothing to stop you calling it directly passing the object as the first argument (and then adding your additional type validations to is_a() of course).

    And yes, agreed about the vec2() work-around, though I tend to use: (v * 1.0) as a fake copy constructor as it seems cleaner, though I hanker for the cleaner vec2(v) construct.

    I also thought hard about why it is I like dynamic binding more than the shallow copy. I think what it really comes down to is I think the tab-ordering model used by Codea is fine to a point, but also fragile. With dynamic binding, declaration of methods and fields inside a class or pretty much any order you like, the only exception being that the classes themselves must be declared in order (super to sub). What this means is that the ideal way to declare a class in this model is to have a tab with:

    If not Button then Button = class(...) end
    (... class methods etc go here)

    This allow you to use the class normally as-is, but also, if you want, to gather all the class declarations into one tab and thus expose all the class relations next to each other, which I like.

    I'll post my attempt at a "clean" class() as callable here soon, though of course we could keep tinkering away with this till the universe ends :)

    --Tim

  • Posts: 157

    @drtimhill I'm not sure that I'd declare all my classes on a single tab unless I had a good reason. I generally prefer keeping a class nicely contained to a single tab/file unless I'm using a language which requires me to do otherwise, like C/C++ for example, but everyone has their own different preferences :)

    You do make a good point about statically calling of is_a, which I had definitely overlooked. However there is still unfortunately still a couple of minor usability problems with using it directly 'out of the box' which are still likely give the unwary some headaches and/or unnecessary typing.

    Firstly, assuming that there is no special knowledge of the implementation, using is_a statically in a consistent way would likely require a user to create some type of base object.

    Otherwise they would have to resort to a either checking for the presence of an is_a function which is less clear and obviously wrong in the unlikely circumstance of a table which has an is_a function but isnt a class instance. I suspect a user would probably trivially avoid this.

    Or checking by a particular type, which is definitely going to be the long way around for doing something like generically checking whether a table is just a table or is actually a class instance. Assuming that there is potentially multiple types where there isn't a common base class. It would require every possible type of interest to be checked to be absolutely sure.

    Admittedly its trivial to work around, but its not necessarily as intuative as it would be if it was just available natively from Codea's API. I expect this would be especially true for somone who is new to Codea, Lua or programming in general.

    Object = class()
    
    SomeObject = class(Object)
    
    OtherObject = class()
    
    SomeOtherObject = class(OtherObject)
    
    local instance = SomeObject()
    local otherInstance = SomeOtherObject()
    Object.is_a(instance,Object) -- true
    Object.is_a({},Object) -- false
    Object.is_a(1,Object) -- false
    Object.is_a(otherInstance, Object) -- false 
    Object.is_a(otherInstance, OtherObject) -- true
    
    

    The other problem is another usability issue. Consider the following ignoring the implementation detail of is_a.

    Object = class()
    local instance = Object()
    
    if Object.is_a(instance) then
       -- do something
    end
    

    Taking the code at face value it looks pretty clear, but there is a not so subtle bug when you know how is_a is implemented. But it would be reasonable to expect Object.is_a(instance)==true but its not, and actually its something that all of our proposed improvements so far have still missed.

    To get is_a to work how it reads in this circumstance it needs to be implemented specifically for each class table substituting in the class table when the klass parameter is null. For example setting c.is and c.is_a in __call from my example above would be done as follows.

    local is = function(instance,klass) return class.is(instance,klass or c) end
    c.is = is
    c.is_a = is
    

    The equivalent is equally as trivial to in Codea's current class function.

    c.is_a = function(self, klass)
         klass = klass or c
         local m = getmetatable(self) 
    ...
    
  • Posts: 10

    @XanDDemoX @Simeon So after thinking about your suggestions this is the final model I came up with. It pretty much combines all the things we have been discussing, and is still nearly 100% backward compatible with the existing model. The only time it would break existing code is if someone was doing some odd hacking on inherited methods, which is pretty unlikely (and probably a dubious thing to do anyway).

    -- Implements class() function as callable that also encapsulates the root class. The implementation uses no upvalues, nor direct references to "class", allowing the entire model to be enhanced/revised/replaced at run-time for advanced uses. Even class() can be replaced (with care!).

    -- Dynamic binding to inherited methods at runtime, allowing mixins and flexible ordering for method creation. Base classes can be created that are later augmented in individual projects without the need to copy+paste the base class (works well with the Codea model of sharing classes between projects).

    -- Objects can access their class via "_class" field in each object. Similarly, classes can access their base class via "_base" field in each class. This allows robust calling to superclass methods without wiring in knowledge of class inheritance.

    -- Optional class names at creation time, with name accessible in "__name" field of class (useful for debugging). With Lua 5.3.4 this means tostring() and debug output will use the class name of object instead of "table".

    -- Direct syntax for calling class methods within a class method: intra-class: "MyClass.foo(self, ...)", superclass: "MyClass._base.foo(self, ...)" and virtual: "self:foo(...)".

    -- "is_a()" can be called conventionally ("self:is_a(cls)") or as a static method ("class.is_a(obj, cls)"), the latter being safe for any value of "obj" (even nil). The default for "cls" is "class", thus allowing "class.is_a(obj)" to be used as an object check (is this an object?).

    Here is the code:

    --//// CodeaClass.lua: Enhanced Codea class model (@drtimhill)
    do  -- Hide locals private to implementation
        local classmt = {}
        class = setmetatable({}, classmt)
        class.__name = "class"
        class.__index = class
    
        --//// class(): Class factory - create new class (with optional base class)
        -- (As a convenience, we set cls:_base to the base class)
        -- base     Base class, default is class if none specified
        -- name     Optional class name (for debug help)
        -- returns  New class (create objects using class constructor e.g. "Ball(...)")
        classmt.__call = function(root, base, name)
            base = base or root
            local cls = setmetatable({}, base)
            cls.__index = cls
            cls.__call = root.__call
            cls.__name = name or "unknown"
            cls._base = base
            return cls
        end
    
        --//// someclass(): Object factory - Create new object of this class
        -- (As a convenience, we set obj:_class to the class)
        -- cls      Class of object to create
        -- ...      Arguments to forward to class:init() initializer
        -- returns  Newly constructed object
        class.__call = function(cls, ...)
            local obj = setmetatable({}, cls)
            obj._class = cls
            obj:init(...)
            return obj
        end
    
        --//// obj:init(): Initialize a new object (class instance)
        -- self     Object to initialize
        -- ...      Arguments forwarded from constructor (e.g. "Ball(...)") 
        function class:init(...)
            -- Dummy init() for root class
        end
    
        --//// obj:is_a(): Check to see if object is instance of class
        -- self     Object to check
        -- cls      The class to check (this class and all its superclasses)
        -- returns  true if the object is an instance of the class/superclass
        function class:is_a(cls)
            cls = cls or class
            if type(self) ~= "table" then return false end
            local m = getmetatable(self)
            while m do
                if m == cls then return true end
                m = getmetatable(m)
            end
            return false
        end
    end
    

    Here is some demo code this highlights the features. Note that the final tostring() demo will only with with Lua 5.3.4 or later.

    -- Create some classes, note the use of optional class name (for debugging)
    MyClass = class()
    MyDerivedClass = class(MyClass, "MyDerivedClass")
    MyThirdClass = class(MyDerivedClass)
    
    -- Note that we can create base class methods *after* creating derived classes
    -- and they will still be inherited by them, thus allowing base class mix-ins
    function MyClass:init(x, y)
        print("MyClass:init(): x,y         = " .. tostring(x) .. "," .. tostring(y))
        MyClass._base.init(self)        -- Not strictly needed for class derived from root
        self.x, self.y = x, y
    end
    
    function MyClass:draw()
        print("MyClass:draw(): x,y         = " .. tostring(self.x) .. "," .. tostring(self.y))
    end
    
    -- The construct "MyClass._base" is the superclass and so can be used to robustly
    -- call the base class even if the inheritance model changes (e.g. to call base
    -- class init() or draw() as shown here).
    function MyDerivedClass:init(x, y, r)
        print("MyDerivedClass:init(): x,y  = " .. tostring(x) .. "," .. tostring(y))
        MyDerivedClass._base.init(self, x, y)
        self.r = r
    end
    
    function MyDerivedClass:draw()
        print("MyDerivedClass:draw(): x,y  = " .. tostring(self.x) .. "," .. tostring(self.y))
        MyDerivedClass._base.draw(self)
    end
    
    -- Create some objects for each class
    -- Note that MyThirdClass() has no "init()" and so the superclass init() is used
    o1 = MyClass(10, 100)
    o2 = MyDerivedClass(20, 200, 1.0)
    o3 = MyThirdClass(30, 300, 2.0)
    
    -- Do some drawing. Again, MyThirdClass will use the superclass draw() function
    o1:draw()   -- Calls MyClass:draw()
    o2:draw()   -- Calls MyDerivedClass:draw()
    o3:draw()   -- Calls MyDerivedClass:draw()
    
    -- Check object classes
    -- Note that we can use the form "class.is_a()" to check possible non-objects safely
    print("o3:is_a(MyDerivedClass)     = " .. tostring(o3:is_a(MyDerivedClass)))
    print("o2:is_a(MyThirdClass)       = " .. tostring(o2:is_a(MyThirdClass)))
    print("class.is_a('fred', MyClass) = " .. tostring(class.is_a("fred", MyClass)))
    
    -- tostring() for objects includes class with Lua 5.3.4 or later
    print("tostring(o2)                = " .. tostring(o2))
    
  • edited March 6 Posts: 10

    @XanDDemoX As an example of mixins, consider a classic Button class that has a draw() method:

    Button = class()
    function Button:init(x,y)
        self.x, self.y = x, y
    end
    function Button:draw()
        -- Draw the button
    end
    

    Simple. Now consider a project where we want to draw lots of buttons via some drawall_buttons() function. Clearly we need to keep track of all the buttons to draw (e.g. in a Lua table) and then iterate the table drawing each button. Not very hard. But It would be more elegant to let the Button class keep this list of buttons and move drawall_buttons() into Button.drawall().

    Again, not very hard, and one way to do this is to modify the Button class to add this. But if that class is being used elsewhere, this might cause problems. The buttons are added to a table which means in existing projects they "leak" since the table holds a reference to them (that existing projects know nothing about). A weak table of course solves this, but then that means you still need to track the buttons outside of the class to stop them being garbage collected. In other words, you want to customize the Button class only for the current project, but not break any of the others that use the class.

    Which means you need to copy the class and modify it. Hmm, not good for code reuse, and a bug fixing mess as the variations proliferate over time.

    So, instead, we don't change the Button class at all, but instead add a mix-in outside of the Button tab (e.g. inside Main tab, but could be elsewhere):

    Button.buttons = {}
    Button.oinit = Button.init
    function Button:init(...)
        Button.buttons[self] = true
        self:oinit(...)
    end
    function Button.drawall()
        for b in pairs(Button.buttons) do
            b:draw()
        end
    end
    function Button:remove()
        Button.buttons[self] = nil
    end
    

    Now we can draw all the buttons created using Button.drawall() and let the Button class track all those buttons. And without changing a line of code in the Buttons class.

    An alternative to this would of course be to create a new derived class that handled the buttons collection. However, the huge advantage of the mixin is that any other class that already is derived from Button also inherits the mixin without needing to hack that class to change the inheritance. So you can use NumberButton etc etc and still gain access to the Button.drawall() function.

Sign In or Register to comment.