Asset Pipeline: Optimizing 3D Models for AR/VR

Guide to properly setting up your 3D assets for use in AR/VR

Here’s a handy guide on how to set up production-ready 3D objects and import them successfully into your AR/VR projects. I’ll show you some best practices and go over my 3D asset production pipeline.

Software

This is a list of what I currently use to produce 3D models. If you prefer other software packages, by all means use what you’re comfortable with. Just follow the guide to make sure you’re exporting the 3D model correctly.

Autodesk Maya (Free Trial) — Industry standard 3D modeling software for many years but no longer the king of the hill. There are many other software packages I use depending on the type of model I make, but this has a great exporting tool (Game Exporter) which has made it easy for me to export complex FBX models that work well with rendering platforms.

Pixelogic Zbrush ($795) — Industry standard 3D sculpting tool I use to create a high-poly base mesh for characters or complex objects.

FBX Review (Free) — A lightweight standalone tool that lets you view your exported FBX files. If it shows up properly here, it will show up properly in your rendering platform.

Unfold 3D VS (Free Trial) — Great UV unwrapping tool that saves a lot of time creating UV maps. Define seams, cut, then unwrap. Very simple.

Mighty Bake (Free Trial) — Fast 64-bit texture baking tool that bakes a wide variety of essential map types including ambient occlusion, normal, etc.

Photoshop CC (Free Trial) — Industry standard image editing tool that I use to paint/edit textures.

Viro Media (Free) — developer platform for building AR/VR applications. Great tool for visualizing assets in AR or VR and for iterative AR/VR design. (Detailed instructions below)

Best Practices

It doesn’t matter what you use to create or get your 3D model, it just needs to be optimized for real-time use and exported to an OBJ or FBX format, the most widely supported formats. I prefer exporting to FBX, because you can embed multiple animations and yields the most consistent and predictable results.

3D Resources

If you’re not creating a 3D model from scratch, there are several great sites that have quality realtime models to download:

www.sketchfab.com | www.turbosquid.com | www.poly.google.com |www.free3d.com | www.cgtrader.com

I also frequently post free 3D assets for use in AR/VR apps here -> Viro 3D Assets

Real-Time Specs

• Polycount — There is no absolute answer for this for any platform. Polycount limitation is dependent on the overall complexity (lighting, animation, # of objects, etc.) of your scene. But as a general rule, make sure the model is optimized for real-time use. This means the model is not overly tessellated and smoothed. When buying a 3D model off Turbosquid, I usually check “Real-Time” under the More dropdown and “Up to 10K” box under the Poly Count dropdown. This will filter results optimized for real-time use. Other sites like Google Poly is all real-time so this won’t be an issue.
• Efficient Modeling — Be clever when it comes to efficiency. Consider how close the user can get to the 3D object. If the user won’t ever get close to your 3D object, a higher polycount will be unnecessary. Delete faces the user will never see. Your GPU will use unnecessary resources trying to render faces in areas that don’t need to be rendered. Optimize and keep polys as low as possible per 3D model. Try to make sure the model has only quads and triangles for faces. This makes it easier to unwrap and will create more predictable results when brought into the platform.
• Texture Maps — AR/VR rendering platforms use traditional 3D shaders and materials, which includes diffuse, normal, and specular maps, and can handle multiple large textures and sizes. But try to use the least amount of textures and keep dimensions as small as possible. For instance, a baseball with 6 2048x2048 textures is probably not optimized and could use another pass.
• Compatible Texture Formats: .JPG, .PNG. Make sure your textures are using one of these formats before export.

Create 3D Model

Clean, simple model using quads and triangles

For this guide, I created a simple modern lamp in Maya. You can grab it here. (Want the Maya file? Grab it here.)

I made sure polycount is as low as possible using only quads and triangles with just enough polys to look decent up close. For efficiency, I deleted faces which aren’t seen by the user.

Detail in the hi-res mesh in Zbrush

To add more surface texture and to round out the edges through normal maps, I brought the lampshade into Zbrush. I subdivided it a few times and added surface noise to get the look I wanted. For the legs, I just used Maya. I added some extra edge loops at the edges and used the Mesh>Smooth modifier in Maya to round out the edges.

UV Unwrap

As mentioned above, try to use the least amount of textures as possible. The less textures to load, the more performant your app experience will be. Because of this principle, I will layout the entire lamp UV’s onto 1 texture.

I bring the lowpoly lamp mesh into Unfold3D where I place seams that I want cut.

• Once seams are placed, I hit “C” to cut,
• Hit “Unfold” to automatically unfold the mesh into pelts.
• I tap “Optimize” a couple times to relax the pelts a little
• Then I manually move and lay out the pelts in the right window making sure there’s some padding between each.

Bake Textures

Mightybake setup UI

Once I’m happy with both my low poly and high poly meshes, I bring it into Mightybake to bake ambient occlusion and normal maps. There’s quite a few settings to play with depending on the type of mesh you want to bake to, so I’ll only suggest a few recommended settings:

• I reference both meshes under “High Poly Meshes” and Low Poly Mesh” and make sure “Normal Map” and “AO Map” is selected under “Maps.”
• For Normal Map, make sure “Unity 5.3” is selected for Tangents.
• Output is wherever you’d like the maps saved to and make sure you set the width and height.
• Hit “BAKE IT” once you’re ready to bake.

Paint Textures

Diffuse, Normal, Specular Maps

After both textures are baked, I load them up into Photoshop to create the Diffuse, Normal, and Specular Maps. There are other great ways of creating textures, but for the sake of this guide, I’ll use some tried and true methods. Here’s what I produced:

• Diffuse — I use the ambient occlusion map as a “Multiply” layer in Photoshop and paint underneath it. I bring in a photo texture of wood for the legs and lay it out.
• Normal — the normal map Mightybake produced may have empty spaces that are transparent. I like to flatten the map and export it as a .PNG making sure “Transparency” is not checked.
• Specular — I brighten up the diffuse texture wherever needed, and make sure areas like the lampshade is darker so it will not appear as shiny as the legs.

Apply Materials to 3D Object

Maya’s Hypershade with Blinn material

Now I need to apply the materials to my object. Within your 3D modeling app, open its Material Editor and point all the textures to the model.

Since I’m using Maya, I go to Window>Rendering Editors>Hypershade.

• Create a new Material Ball by clicking on “Blinn.”
• Under the “Attribute Editor,” click the checkerboard button next to “Color “and assign the diffuse map.
• Click the checkerboard button next to “Specular Color” and assign the specular map.
• Click the checkerboard button next to Bump Mapping” and assign the normal map.

Export 3D Model

Now that the model is set up, I need to export it to .FBX. To do this, I use Maya’s Game Exporter. This is a useful plugin provided by Maya to aid in exporting real-time models to .FBX. If you don’t see it under File>Game Exporter then go here for help.

• If I have other objects in the scene, I make sure the lamp is selected and “Export Selection” is selected in the top dropdown.
• I keep “Smoothing Groups” checked and check “Smooth Mesh.” I’ll keep the other default options the way they are.
• I set the “Path” and define where I want the .FBX to be saved to.
• Name the .FBX next to “Filename.”
• Click “Export.” If it’s successful, it’ll give you a Success prompt that you can close.

Note: If your object is rigged and has multiple animations, you can define the animation clips under the “Animation Clips” tab. Since the lamp has no animations, I’ll leave this untouched. I’ll be glad to help if you have any questions on how to export animated objects. Download the final Lamp FBX file and textures HERE.

Test in Autodesk FBX Review

Load your object into Autodesk’s FBX Review. FBX Review doesn’t display normal maps, so don’t worry that it doesn’t show up. If your object appears correctly in the viewer with no anomalies, it should display correctly in Viro.

Visualizing 3D Assets AR/VR

This next section shows you how to view your assets in AR using the Viro platform (To view your assets in VR using Viro, follow the steps below started with HelloWorldScene.js). While Viro is a platform for developers, it can also be used by designers and creatives to visualize 3D assets and iterate on ideas. Set up is simple and the steps listed below are the same one’s I use to view 3D assets in AR/VR as a designer.

Setting up Viro

Use our Quick Start guides to get setup with Viro in minutes. The Viro platform is free with no limits on distribution. Sign up to get your API key and start developing AR apps today:

• Mac/Linux Quick Start
• Windows Quick Start

You will also need either an iOS ARKit enabled device or an Android ARCore enabled device.

Java developer? Get started here.

Once you complete the setup step and see Hello World on your device, you should move onto the next section to display your 3D assets in AR.

Displaying 3D Assets in Viro

We will be using our lamp model for this next section to simplify the steps. (Download Lampviro Model)

First you need to convert our FBX files in VRX format by using the ViroFBX script. The ViroFBX script can found in your project’s bin directory.

After download the lamp model, unzip the file. Copy the folder (viro_object_lamp) into your bin folder which is located within the ViroSample folder.

Open Terminal on your Mac and go to your bin directory. I setup Viro in a folder called “WorkSpace” so I would enter the command below (replace WorkSpace with your root folder in the command below):

cd WorkSpace/ViroSample/bin

Once you are at your bin directory, enter the command below.

./ViroFBX viro_object_lamp/object_lamp.fbx viro_object_lamp/object_lamp.vrx

Once the script is completed, you should see a file called “object_lamp.vrx” in your viro_object_lamp folder in your bin directory.

Once the VRX file is created, it can be loaded into an application with <Viro3DObject>. Copy the files within your viro_object_lamp folder and paste them into the res folder located at /ViroSample/js.

To see your 3D assets in AR, open your HelloWorldSceneAR.js file and replace the text in your HellowWorldSceneAR.js with the code below.

Save your HelloWorldSceneAR.js file. Shake your device to bring up the menu and tap “Reload”. You should see the lamp in front of you below “Hello World”. We added a shadow to the lamp and enabled drag so you can move the lamp around with your finger.

To see your own assets in AR, modify <Viro3DObject> (lines 46–54):

<Viro3DObject source={require('./res/object_lamp.vrx')}
             resources={[require('./res/object_lamp_diffuse.png'),
                         require('./res/object_lamp_normal.png'),
                         require('./res/object_lamp_specular.png'),
                       ]}
             position={[0, 0, 0]}
             scale={[0.3, 0.3, 0.3]}
             type="VRX"
/>

1. Convert your FBX file into VRX format.
2. Place your VRX assets in the correct folder/path.
3. Update your code for the new asset. In the example above, you would change “object_lamp.vrx” to your vrx file name and change the texture names also.
4. Save your file and reload to see your 3D asset in AR.

All Done! Stay tuned! Next, I’ll be covering 3D models with skeletal animations.

Questions? Problems? Need extra help importing your 3D object? Send me an email. I’ll be glad to help.