Loading 3d Objects from OBJ

[matthewkaney]

Is anyone working on loading 3d objects from files? If not, then I'd like to tackle it.

In Processing, this is part of PShape, but based on #458, it seems like the consensus is against directly porting that API to P5. Does it make sense to treat loading/drawing of 3d shapes as a separate feature? Any thoughts on what the API for this should look like?

[indefinit]

Go for it! It's definitely a feature we would love to include, but it got pushed down in priority.

As for api style, well this is a little tricky but I'll offer my 2 cents (pinging for weigh-in: @lmccart @codeanticode @shiffman @karenpeng ), especially as per #458 as you mention:

Note: I'm kind of putting on the svg blinders right now and just focussing on loading as it relates to webgl.

Although the Processing api includes svg | obj in a single loadShape(), I think it would be more intuitive for new users and experienced 3D programmers alike to have a function called loadModel(). It's more specific to 3D, however it still supports more filetypes than just .obj (even if you start with implementing an obj loader first).

You might be familiar with the THREE.js model converter util (for offline file conversion): https://github.com/mrdoob/three.js/tree/master/utils/converters/fbx

Conceptually what I like about this approach is that it supports a wider variety of 3d model formats and converts to javascript's best friend, JSON.

• Fbx (.fbx)
• Collada (.dae)
• 3D Studio Max (.3ds)

If we had a loadModel function, perhaps it would have arguments of either paths to .obj files or model -> JSON objects, which would enable indirect compatibility with the above 3D model formats.

As for rendering the model after loading, there are a lot of different ways we can go, but perhaps here's one option- Behind the scenes, the performative way to implement would be to add the model vertices to the retainedMode hash map. To render the model, a user would then:

// in preload or setup
var model = loadModel('path/to/model.obj');
...
// in draw
model.draw(); 

What do others think of this approach? It might be worth noting, I've recently been playing around with toxiclibs.js and p5js to get them to work well together in webgl mode. Toxiclibs currently has an objwriter object, so consider checking out as inspiration for your work on an obj (or generic model) loader.

[shiffman]

I like the approach of having a loadModel() function, I think this makes sense in terms of staying true to what's happening with the webgl renderer.

[matthewkaney]

Thanks for the advice. I'm making pretty good progress here.

A technical question for @indefinit or @karenpeng, (or whoever has an opinion): Do you see any potential issue with storing three vertex normals per face (like how UVs are stored) rather than as one vertex normal per vertex? This is how the geometry is stored in OBJ (and THREE.js JSON format, and probably others), and is pretty necessary if we have models with sharp edges. Unless there are any objections, I can go ahead and make this change first, assuming I figure out how to update the primitive classes and shaders.

[lmccart]

hey @mindofmatthew I'll defer to @indefinit and @karenpeng on the specifics here, but great that you're working on this, and the overall plan sounds like the right track!

[indefinit]

hey @mindofmatthew, Storing 3 vertexNormals per face sounds right, but in comparison with THREE.js, we don't really have separate Geometry and Face classes so we've been keeping the vectors in arrays inside the Geometry3D Object. Here's where that info is defined. Note, we have both a vertexNormals array as well as faceNormals:
https://github.com/processing/p5.js/blob/master/src/3d/p5.Geometry3D.js#L8-L25

Do you think maybe there's a way to be more efficient with this?

[matthewkaney]

@indefinit: This is what I was thinking as far as 3d geometry goes...

//assuming we've got some vertices, create some faces
geometry3d.faces = [[0, 1, 2], [0, 2, 3], [0, 1, 3]];

//faceNormals will hold one vector per face (I think this is how it already works)
geometry3d.faceNormals = [new p5.Vector(...), new p5.Vector(...), new p5.Vector(...)];

//vertexNormals will hold three vectors per face
geometry3d.vertexNormals = [[new p5.Vector(...), new p5.Vector(...), new p5.Vector(...)],
                            [new p5.Vector(...), new p5.Vector(...), new p5.Vector(...)],
                            [new p5.Vector(...), new p5.Vector(...), new p5.Vector(...)]];

Do you think this would work? If you're worried about size, we could maybe do something where we have a pool of all the possible normal vectors. Each face would then store pointers to some of the normal vectors in the pool. I don't really understand what happens to the vectors when they're turned into WebGL buffers. Once I understand that, I might have better suggestions here.

Also, here's how my functions currently work:

var myModel = loadModel('some-model.obj'); // This returns a p5.Geometry3d object

model(myModel); //This draws the model to the canvas

I know that the p5.Geometry3d seems to be mostly for internal use. Is it okay to give the user a reference to it? Or should we wrap it in some more user-friendly class (like, p5.Model() or something).

Thanks for helping me figure this out.

[kadamwhite]

This isn't strictly relating to the implementation, but is it safe to assume that this will be an add-on module, like p5.sound? I ask because this sounds like an awesome piece of code, but not necessarily a small one (in terms of lines of code), and loading 3d objects feels like more of a niche use-case than sound.

[indefinit]

@mindofmatthew Yes, I think making use of p5.Geometry3d seems the right way to go and most extensible. It would be great if we could solve this by extending the parametricGeometry() method, passing in our OBJ vertices and normals in a callback. Curious about @karenpeng 's weight-in here since came up with the elegant solution for parametric geometry. Regarding the size of the Geometry3D object, I'd love to run tests on a few different models to see how heavy it gets. It might not be so terrible for small to medium sized models.

@kadamwhite Yea, it might be valuable to keep the core lightweight and have an add-on called p5.Model3D (or something similar), especially if in the future we not only want to read from but also write to an OBJ filetype. There's also precedence for the library approach in the Processing obj lib.

Just a high level overview/gut check, there are 3 pieces with regards to the OBJ file loading process:

1. Loader/file parser
2. Geometry batch processor
3. webgl buffer management

My inclination is to have solid core support for 3, leave 2 open for user extension both within draw() and an external module, and 1 could be managed by an external module/lib.

[karenpeng]

Hi @mindofmatthew!
Actually there're both face normals and vertex normals for each geometry. Just face normal is being used for the lighting so you might not know.
You could access p5.Geometry3D by creating an instance, so that you could have all the functionalities like generating vertex normals and face normals.

Here's an example how we create a sphere by instantiating it.

Like what @indefinit mentioned, the structure of each geometry is like this now:

  //an array holding every vertice
  //each vertex is a p5.Vector
  this.vertices = [];
  //an array holding each normals for each vertice
  //each normal is a p5.Vector
  this.vertexNormals = [];
  //an array holding each three indecies of vertices that form a face
  //[[0, 1, 2], [1, 2, 3], ...]
  this.faces = [];
  //an array holding every noraml for each face
  //each faceNormal is a p5.Vector
  //[[p5.Vector, p5.Vector, p5.Vector],[p5.Vector, p5.Vector, p5.Vector],...]
  this.faceNormals = [];
  //an array of array holding uvs (group according to faces)
  //[[[0, 0], [1, 0], [1, 0]],...]
  this.uvs = [];

Yes it's pretty minimum, you are welcomed to adjust it in order to fit more 3d objects formats, but please let us know if we need to refactor the whole structure.

Hope it helps! And sorry for my late response. 😉

[indefinit]

closing this issue since it is implemented in #1285 . 👍