Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

3D OBJ model importer (for potential future inclusion with Codea. Now with wireframe mode)

edited October 2015 in Code Sharing Posts: 2,020

As discussed ( http://codea.io/talk/discussion/comment/63536/#Comment_63536 ), here is an obj importer, plus a test model. I'll add some more models over time. Feedback welcome (see especially the note below about how to split models).

EDIT the repo link: https://github.com/Utsira/Codea-OBJ-Importer

From the readme:

Codea OBJ Importer

An importer to bring 3D models in the popular .obj format into Codea

To test:

  1. Install the Codea OBJ Importer in Codea (copy the entire contents of the /CodeaOBJImporter.lua file to the clipboard. In the Codea projects page, long press "+ add new project" and select "paste into project")
  2. Save the files in /models, and any other obj files you want to import, into /Dropbox/Apps/Codea
  3. In the Codea asset picker, open Dropbox, and press sync in the topright corner. You'll see a message saying files are being synced, though you won't actually see the files themselves appear in the asset picker, because the extensions .obj and .mtl are not supported by the asset picker. Though this will probably change soon!

Notes

  • Only supports triangular faces. Select "Triangulate Faces" in the Blender .obj exporter (see image below) or apply a triangulate modifier.
    export setting
  • Expects to find the texture image in the Dropbox. This can be automated by setting "Path Mode" to "Copy" in the Blender .obj exporter (see image above). Note that you can save export presets, so that you can save with consistent settings.
  • Currently, the importer only supports models with a single image texture, but models could have multiple textures. Previous implementations have addressed this by splitting the imported model according to its material. This is not a satisfactory solution however, as models could have multiple materials with the same (or no) image texture (as in the Tank.obj example). 2 potential approaches:

    1. Have the user split and join the blender model into separate objects for each image texture. These separate objects would still be contained in a single obj file, and are flagged with the "o" code. The importer would start a new object when it sees "o". This would mean the user would have to split models that contain multiple textures (in Blender edit mode, tap "a" to deselect all, select the material, press "select" to select the geometry that have that material, press "p" for separate, click "selection"), and join objects that have no texture or share the same texture (select the two objects, hit "j" for join). This would also allow the user to split objects for animation purposes (eg separate the wheels from the body of a vehicle)
    2. Have the importer analyse the material list, and only split the object if a material uses a different texture.
    3. Both of the above.

Comments

  • Posts: 2,020

    I added much better drag to rotate controls, now when you twist and swipe to rotate, the model always rotates in a consistent direction, no matter what its orientation is. I do this using the inverse of the model matrix to convert global touches into a consistent local rotation.

  • Posts: 2,020

    @Ignatz just saw your world of tanks post. Why not use this tank model (it's a cc free licence)? Could do with being slightly lower poly... By coincidence I'm working on something tank-y myself....

  • Posts: 905

    Hi,

    Been a while since I've posted, still keep an eye on forum but have little time now to code. Also, updated my iPad2 to 9.02 and feel I need a machine upgrade.

    As always interested in 3D so picked up on this thread - noted with interest on .obj model loader from @yojimbo2000 but can't find link. Been playing with the one from @ignatz. Really need to get my head around how the system approaches the display of 3D. Some good references within the forum but none outlining how the OS approaches 3D.

    Is it something like:

    1. Defines environment with camera() and perspective.
    2. Reads in world array
    3. Generates world image (is this as a mesh?)
    4. Displays world image
    5. Waits for input which describes positional change (user driven)
    6. Recalculates world array based on input then loops to 3

    World image taking into consideration hidden polygons, rotation, field of vision and transposition in movement of camera and objects.

    Objects reconstructed in a specific order to ensure finished image of object intact.

    Have I got the right approach? Any other aspects to consider?

    Cheers,

    Bri_G

  • Posts: 2,020

    @Bri_G oh yeah, I forgot to post the link. 8-} Thanks for spotting that.

    Here it is:

    https://github.com/Utsira/Codea-OBJ-Importer

    I added a couple more models to the repo.

    character

    island

  • Posts: 905

    Hi,

    Thanks @yojimbo2000, very smooth. Just started loading up to dig into the code. Impressive model displayed.

    Thanks again.

    Bri_G

  • edited October 2015 Posts: 2,020

    One more model added. I really like this guy. Cute/ threatening, bit of a Miyazaki influence I think.

    robot

  • edited October 2015 Posts: 2,020

    @Bri_G definitely check out @Ignatz 's 3D ebook, and the 3d blog posts, but basically you're correct

    Is it something like:

    >
    1. Defines environment with camera() and perspective.
    2. Reads in world array
    3. Generates world image (is this as a mesh?)
    4. Displays world image
    5. Waits for input which describes positional change (user driven)
    6. Recalculates world array based on input then loops to 3

    Except no 1, camera and perspective, go just before 4, as part of your draw cycle. A mesh is a unique set of geometry. A single mesh can be drawn in many different places in your scene. eg for an army of tanks, you'd have a single tank mesh, and then translate and rotate to various locations corresponding to each instance of the tank object to :draw() it. You could if you wanted further subdivide each tank into smaller meshes, say if you wanted the turret to move independently of the body. But that would slow things down a little, as generally speaking, you should try to minimise the number of draw() calls

  • IgnatzIgnatz Mod
    Posts: 5,396

    @yojimbo2000 - yes, the problem with importing objects is that they often have too many vertices, and animating parts of them is difficult. I don't want to spend dozens of hours just on the model.

    Anyway, I do this for enjoyment, so I'll end up doing whatever is the most fun :D

  • IgnatzIgnatz Mod
    Posts: 5,396

    @Bri_G - I would list the steps this way

    1. In setup, read in the model mesh. If it is going to move about, rotate or be used to draw several objects, center it on 0,0,0, whereas if it is fixed scenery, you can just define the vertex positions where you want them

    2. In draw, set up 3D with perspective

    3. Set the camera position and where it is looking

    4. Update your object positions (for movable objects, this doesn't mean changing the vertex positions. Instead, you will translate to the position you want, so you only have to update that position).

    5. Draw the objects in the scene, translating and rotating if required. This apparently complex process is managed very efficiently using 4x4 matrices. Generally, it doesn't matter which order you do this in, because OpenGL figures out which pixels are in front.

    And yes, a lot of reading helps. I've written a lot about it as my own understanding has developed, as well as trying a lot of different projects.

  • Posts: 905

    Hi,

    Thanks guys - read most of what is presently in the forum. Taking a while to get my head round it. Think I'm close to being able to apply it now.

    Thanks again,

    Bri_G

  • edited October 2015 Posts: 2,020

    I added a wireframe mode! (Press the wireframe mode button on the panel to activate)

    Adapted from here:
    http://codea.io/talk/discussion/3170/render-a-mesh-as-wireframe

    I tweaked it to draw the struts with the colour of the materials, and to use discard for the inner part of each face, for more consistent transparency.

    I'd never heard of GLES standard derivatives before, they're not even mentioned in the Kronos GLES spec! Amazing what you can learn on Codea Talk.

    An image:

    wireframe

  • IgnatzIgnatz Mod
    Posts: 5,396

    that is VERY cool B-)

  • edited October 2015 Posts: 2,020

    @Ignatz thanks!

    My version of the wireframe code (I use the normal attribute, rather than color). The repository linked to at the top of the thread now has this code included:

    edit: added stroke width

    wireframe = {}
    
    function wireframe.set(m)
        local cc = {}
        for i = 1, m.size/3 do
            table.insert(cc, vec3(1,0,0))
            table.insert(cc, vec3(0,1,0))
            table.insert(cc, vec3(0,0,1))
        end
        m.normals = cc
        m.shader = shader(wireframe.vert, wireframe.frag)
        m.shader.strokeWidth = strokeWidth()
    end
    
    wireframe.vert = [[
    uniform mat4 modelViewProjection;
    
    attribute vec4 position;
    attribute vec4 color;
    attribute vec3 normal;
    
    varying highp vec4 vColor;
    varying highp vec3 vNormal;
    
    void main(void) {
        vColor = color;
        vNormal = normal;
        gl_Position = modelViewProjection * position;
    }]]
    
    wireframe.frag = [[
    #extension GL_OES_standard_derivatives : enable
    
    uniform highp float strokeWidth;
    
    varying highp vec4 vColor;
    varying highp vec3 vNormal;
    
    void main(void) {
        highp vec4 col = vColor;
        if (!gl_FrontFacing) col.rgb *= 0.5; //darken rear-facing struts
        highp vec3 d = fwidth(vNormal);    
        highp vec3 tdist = smoothstep(vec3(0.0), d * strokeWidth, vNormal); 
    
        //2 methods: 1. discard method: best way of ensuring back facing struts show through
        if (min(min(tdist.x, tdist.y), tdist.z) > 0.5) discard; 
        else gl_FragColor = mix(col, vec4(col.rgb, 0.), -0.5 + 2. * min(min(tdist.x, tdist.y), tdist.z)); // anti-aliasing
    
        //2. alpha method means some rear faces wont show. Would be good for a "solid" mode though
        //gl_FragColor = mix(col, vec4(0.), min(min(tdist.x, tdist.y), tdist.z)); 
    }]]
    
  • Posts: 2,020

    I'm thinking I could use this standard derivative extension to further refine the stroke and anti-aliasing in the Soda roundedRectangle class, as it seems to have something to do with anti-aliasing.

    ... if I could find some decent documentation on what it actually does.

  • edited October 2015 Posts: 2,020

    I've added antialiasing to the wireframe mode by combining mixing and discarding. Changes are in the repo

  • Posts: 2,020

    Added an option and a slider for wireframe stroke width. Should stroke width be proportional to distance do you think?

    thick wireframe

  • edited October 2015 Posts: 2,020

    I've found an awesome paper online that describes these kinds of procedural textures, GLES derivatives, anti - aliasing etc:

    http://liu.diva-portal.org/smash/get/diva2:618262/FULLTEXT02.pdf

    The section on derivatives is pretty complex though

    :-O

    Edit: found another post describing the wireframe technique:

    http://codeflow.org/entries/2012/aug/02/easy-wireframe-display-with-barycentric-coordinates/

  • IgnatzIgnatz Mod
    Posts: 5,396

    Haha, I'm not going to do any testing until the dust settles, but I'm enjoying watching =D>

  • IgnatzIgnatz Mod
    Posts: 5,396

    @yojimbo2000 - I've tried the latest code, and while I don't get errors, when I load any model, it reports the number of bytes in the mtl and object files, pauses while it loads the files, then prints "0 bytes processed", followed by the time in seconds.

    But nothing on the main screen.

    Any ideas?

  • IgnatzIgnatz Mod
    edited October 2015 Posts: 5,396

    @yojimbo2000 - strike that. It seems I downloaded an HTML mess by "saving as" from the file links.

    The wireframe is great for seeing how the vertices are arranged.

    I suggest there should be an option to download model files automatically, rather than having to copy and paste mtl and obj files and then sync manually.

    wrt model selection, I suggest that vehicles would be a good starting point, because they require little or no animation, as opposed to people figures.

    I like the tank, but there are so many vertices in the tracks, which is something I am trying to get around, for example by using an image for the wheels and most of the track, instead of vertices.

    wrt anti-aliasing, given that Codea automatically does this, do you need to program it yourself?

  • Posts: 2,020

    I suggest there should be an option to download model files automatically, rather than having to copy and paste mtl and obj files and then sync manually.

    Download from where? These 4 models were downloaded as blender files, I did the .obj export myself (I didn't make any other changes to the models), and this is how I imagine this kind of code being used, people processing their own files (or, I would urge, creating their own models). So it's a lot easier just to export it straight into your Dropbox and hit sync, then have to upload it to some server, type in a long and complex URL to your Codea project, and then have a class that waits for the various files needed to asynchronously downloaded. If there was a massive library somewhere of good models ready to go in .obj format, then I'd see the point of adding a download function, but there isn't, as far as I'm aware. So I'm assuming that a blender object importer does involve some blender usage. Also, if you export to Xcode, you want the model files to be included in one of the asset folders.

    wrt model selection, I suggest that vehicles would be a good starting point, because they require little or no animation, as opposed to people figures.

    The models are mainly to test the loader. I browsed through the "low poly" tag in blendswap, and downloaded models that were a megabyte or less. Nowadays though it seems that anything that isn't absolutely photo-realistic can be considered "low poly". The tank does have too much detail in the undercarriage (which you'll hardly see). This is where the "decimate" modifier comes in handy in Blender.

    If models were to be bundled with Blender, I guess the best thing would be a themed set, a 3d equivalent to "Planet Cute" or whatever. I'll keep looking.

    wrt anti-aliasing, given that Codea automatically does this, do you need to program it yourself?

    Codea doesn't alias the edges of meshes (though it does alias any textures on the mesh), so yes you do (the built-in primitives alias the edges, though they appear to use different techniques, the thickness of the stroke is not quite consistent between rectangle and ellipse) . Given that the wireframe shader is already calling fwidth, it's almost free in this case.

  • IgnatzIgnatz Mod
    Posts: 5,396

    I was thinking of having a download option for situations where you share code on the forum, it makes it easier for other people to try out. The models could be stored anywhere in the cloud.

    The most common models likely to be used in Codea are likely to be vehicles, so if we build a common library, that's what I would start with. (Anyone who can animate humanoid figures doesn't need our help finding models!).

    Anyway, if you've finished tweaking the loader, I'll put some models through it.

  • Posts: 289

    pretty awesome

  • edited October 2015 Posts: 2,020

    If anyone's using this who's a Codea beta-tester, note that in 2.3.2 build 51, matrix.rotate command takes radians, instead of degrees, so the models will spin round way too fast when you swipe the screen. Add math.rad to the 3 rot:rotate commands ie rot:rotate(math.rad(dx), x:unpack()). Or you could just wait, as apparently it will revert back to degrees in a future build.

  • Posts: 905

    Hi @yojimbo2000,

    Been in circles with your app, my iPad problem with Dropbox (now resolved) and the change with the Dropbox.asset... Nomenclature. Now up and running - very impressed.

    One suggestion, I have a number of models without materials, detecting this in the obj file and setting up a default material would avoid error generation and allow display of models. There is also the instances where the obj file incorporates the mtl file. Just trying to think of a way to load most files without error.

    Thanks for your work/code.

    Bri_G

  • Posts: 2,020

    @Bri_G

    OK, I didn't realise the mtl file could be part of the obj file. Do you know what software exports that way? If you could link to a source file example of an object that incorporates the mtl file, that would be really helpful.

    Checking for the absence of mtl files shouldn't be too hard to add.

  • edited November 2015 Posts: 905

    Hi @yojimbo2000,

    See the following link, it appears this is incorporating vertex colouring in the vertices data and is not legal obj.

    Vertex colouring

    @Ignatz included the text of the mtl file in some of his earlier 3D modelling files. Also I think the full spec on obj models includes a usemtl feature.

    obj spec

    The other option is the ply model which does incorporate vertex colouring.

    Question is, if we are looking for a model loading package in Codea how far do you take it?

    Bri_G

  • IgnatzIgnatz Mod
    Posts: 5,396

    I would just stick to standard obj/mtl files, and not get too fancy

    Only a handful of users will need it anyway

  • Posts: 2,020

    Adding vertex colouring to OBJ files is not standard, and not something I plan on adding.

    I have written a PLY importer, but haven't updated it in a while, mainly because the Blender PLY exporter is not as fully-featured as the OBJ one, so it's not as useful a format. PLY though does support vertex colouring as standard (if you search the forum, I posted the code and some images).

    I don't think that vertex painting is as useful a technique as just defining colours for various materials.

    I believe @Ignatz concatenated the files manually in a text editor, I'm not keen on that approach, because it slows the workflow too much, eg if you just want to make a small change to a model.

    The usemtl field is the name of the mtl file, and not a way to include the mtl data directly in the obj file.

  • Posts: 905

    Hi @yojimbo2000, @ignatz,

    Thanks for the feedback, agree that any Codea model loader should cover the dominant obj format. Other formats could be covered by user code for specific needs.

    Bri_G

  • edited November 2015 Posts: 286

    embarassing question!...how do i copy the model files from the github into the dropbox?

  • IgnatzIgnatz Mod
    Posts: 5,396

    Save them to a file in the Codea folder of your Dropbox account, then go into Codea and sync your Dropbox.

  • i can copy into the pasteboard but i dont see the way to save the file from the github?

  • Posts: 2,020

    @piinthesky you can download the entire repository as a zip file (should be a download zip button on the right, might not appear if you use a content blocker)

    Or, I have just uploaded a new version that automatically downloads the models for you from the repository. It's also designed to address a few changes coming in Codea 2.3.2. You can install it with the installer called Codea 2_3_2 OBJ Importer Installer.lua. If you're not a beta-tester, then in the readAssets tab you'll have to change the Dropbox.assets string back to Dropbox.assetpack but other than that, it should work.

    @Ignatz adding asynchronous download of remote files does increase the code base quite a bit. File loading is now separate from OBJ parsing, which is probably a good thing. It doesn't yet support remote download of textures though, that's next.

  • thanks @yojimbo2000 the automatic download works great.

  • @yojimbo2000 i would like to use this .obj model of a submarine...

    http://tf3dm.com/download-page.php?url=deep-sea-sub-28037

    i managed to read in the files, but it processes 0 vertices. Any ideas to fix this?

  • Posts: 2,020

    @piinthesky I'm not sure what this means, but when I examine the file for the sub, the faces are defined as 3 sets of 3 values separated with slashes. eg f 270/270/69 217/217/70 271/271/108. Usually, the .obj files created by Blender just have three values per face, separated by spaces, eg f 120 121 127, and this is the only .obj format that the importer knows about. My guess is that the the three sets of values for the face refer to the vertices, texCoords, and normals, perhaps it's v/tc/n v/tc/n v/tc/n for each face. That's just my guess though. What I suggest you do is this:

    • try to find some documentation online about different forms of the .obj format (this is one of the most frustrating things about 3d work, that many of the formats are so poorly specced), to confirm this alternative format for faces.
    • fork the Codea obj importer code on GitHub
    • modify it so that it works with this format
    • submit a pull request (if you like) and I can merge your changes back in. Or you can keep your version as a separate fork, and just let us know on the forum.
  • @yojimbo2000 the obj specs seems to be quite well documented in wikipedia and it confirms your guess.

    I had the idea to load the model into blender and then export in the preferred codea format. That worked well and i can now load it into codea.

    Thanks for your help.

  • Posts: 2,020

    That's interesting. I tried loading the model into blender and re-exporting as an .obj, but it still had the same format for the faces. Did you do anything to the model or change the export settings?

    Glad you got it working.

  • No, i just checked the boxes as you indicated in the export graphic you loaded earlier in this thread.

    At the moment the model is grey and does not have any colour. I declare the rough.jpg as a texture in the models table of the assets function, but doesn't seem to work?

  • Ahhh, i had to specify the diffusetexshader, now the texture is present

  • Wow, just wow! This is one of the BEST codea projects I have used! Great job!

  • Hey I don't get why's there a star war game and then this like on the same page both are cool but don't understand how to work this thing

  • Posts: 2,020

    @EvanDavis which part don't you understand how to work?

  • Sorry my bad. I am talking about how to add models. I don't have a folder in Dropbox called Codea, do I need to make one?

  • Posts: 2,020

    @EvanDavis no. In Codea, when you link Codea to your Dropbox, it will create a folder for you. All Dropbox integrations are in a folder called Apps, so it wil be Dropbox/Apps/Codea. But don't create the folder yourself manually.

  • How do I add the file from github into Dropbox, I can't save the files or anything.

  • edited September 2017 Posts: 553

    @yojimbo2000 this is just superlative work.

    This, together with SODA, are incredible leaps that seriously address weaknesses in Codea's environment: the difficulty of doing complex things in 3D, and the "reinventing the wheel" situation that every new Codea user faces when they want to make a button.

    The heavy lifting you've done here is immense, and I truly hope more people start taking advantage of it.

    [btw, two of the model links no longer work: the girl and the island.]

Sign In or Register to comment.