- Messages
- 10,158
- Country
TUTORIAL FOR CREATING AMBIENT OCCLUSION MAPS IN MAX 2012 32BIT
When Max 2012 came out, many changes occured with how one does AO Maps (Ambient Occlusion rendered shading). This explains how to do these in Max 2012 with Mental Ray as the Rendering engine. If you move up from Max 9 to Max 2012, you will need to know this if you do AO maps.
Follow these in order.......................... Must...
Boot up your scene
[RENDERER SETUP]
Open your Renderer Control Settings Panel; F10
Go to Commons Tab/Page
Setup Mental Ray as your Renderer
Go to Processing, tick 'Enable' at Material Override
[AMBIENT OCCLUSION MATERIAL SETUP]
* Next, open Material Editor (M key)
* Create a new Material, name it something like Mental Ray AO Material
* Tick on the Bitmap small square at Diffuse
* On the Textures Navigator that pops up, go down to the Mental Ray materials section, find AO, it should be a bright white dot on its icon, select it
* IN your Material Editor, you will find that the texture will have your Ambient Occlusion settings (yes, in the Material in the 'texture' section for
Diffuse). There you want to set up your settings, though later, you will also set this up identically in the Render to Texture control panel.
Settings for the AO texture.
NOTE: You will have many settings, but only a few count, worry only about those few for now.
Samples........Refinement of the shadows; 220 is very smooth, 50 is grainy like mud, great for mud effects by the way
Max Distance...Size of the shadow. Test to see how big it needs to be. Usually 2.0 on some aircraft
Spread.........Darkness. Low numbers are dark, high numbers are brighter, lighter. 0.2 to 2.0. I use 0.8 for now.
* Next, drag the Sphere of the 'Mental Ray AO Material' in the Spheres section of the Material Editor onto the 'material' button of the 'Material
Override' on the Renderer Setup page (F10) that is on the 'Processing' section. (Drag and drop). This will cause the AO Material to show up on 'all'
Materials/Objects when doing a Render 'only'. Otherwise, the AO material is 'not' seen. You can leave this on. Make sure you ticked on the 'Enable'
tick on the Material Override on the Processing Page in the Renderer Setup center.
[RENDER TO TEXTURE SETUP]
Now open your 'Render to Texture' control panel page, Keyboard shortcut; 0 (zero) on the Keyboard side (not NumPad).
* Select a part in the scene, one you will probably create an AO map for.
* Output, Path, set this to where you want your maps to export to. I use Desktop as I delete the AO maps once I have placed them in the PSD layers.
They are basically no longer needed. (You will only need to do this once, this will be stored in Max memory settings, exporting of Render to Texture target location).
* Tick 'on' Rendered Frame Window.
* Render Settings flyout; 'do not touch', should look like this; ------------
* Object and Output settings; 'do not touch, should also look like this; -----------
* Name; Here you will see your Part you will be exporting. You should see the part name, like Fuselage.
* Selected Object Settings; 'Enabled' (tick on). Padding = 2. Padding is next to this.
* Project Mapping; 'not' ticked
* Mapping Coordinates; 'Use existing mapping channel' 'on'. Channel 1 set.
* Subobjects is locked off
* OUTPUT SECTION: Click Add, find Ambient Occlusion (MR), click Select (This is only done once for doing a group of parts. This sets up the format of the map as well, such as Targa or JGP).
* Selected Common Settings:
-- Enable (tick on), Ambient Occlusion should be in this slot
-- File Name and Type; click on the ... button and select JPG or TGA or what ever you like. Most Maxxers use Targa's, so TGA is standard for AO Maps
-- Use Default Gamma (thats what I use, others may have their own Gamma settings). Click Save.
* Element Background. Use pure black or pure white. I use white, as white is transparent in Photoshop/Layer/Blend=Multiply Only gray is used in the
shading in Photoshop in Multiply layers.
* Use Automatic Map Settings; leave ticked 'off' (do not select)
* Set size of your texture. Note; make a small one and check that its the one you want, right settings in rendering, then go to full size. Scale is
important, as you can then fit your map to your texture perfectly. If you work in 2048 pixels, then make your good AO Map to 2048. For a test, try
512, its quicker to Render.
* SELECT ELEMENT UNIQUE SETTINGS
-- Samples............Quality of Shadow; smooth or grainy, 220 for good, 50 or speckled, 120 for fair.
-- Spread.............0.8 (Test and change if necessary for your model)
-- Max Distance.......2.0 (Test and change if necessary for your model)
-- Falloff............Usually set to 1.0, I never touch it.
***NOTE: The above settings 'should' match your settings in the Material Editor/Mental Ray AO Material/Diffuse texture settings center.
* BAKED MATERIAL SETTINGS
-- Save Source ticked 'on'
-- Duplicate Source to Baked ticked 'on'
-- Render to Files Only ticked 'on' <<< IMPORTANT
-- Keep Source Material ticked 'on' <<< IMPORTANT
* AUTOMATIC MAPPING
(I do not touch these, but here are my settings, just incase)
-- Rotate Clusters ticked 'on'
-- Fill Holes ticked 'on'
-- Threshold Angle....45.0
-- Spacing............0.02
* AUTOMATIC MAP SIZE
(I do not touch these settings, but here are my settings just in case)
-- Scale = 0.01
-- Min....45.0
-- Max............0.02
* VIEWS RENDER (Very bottom, right side of the Render to Texture control panel)
-- Original.....Render ticked
-- Baked........Views ticked
NOTE ABOVE 1: Spread Setting shows 0.2. It should be 0.8.
NOTE ABOVE 2: Be sure, on the bottom of that screenshot above, that you select (tick) on 'Save Source (Create Shell) ', then 'Duplicate Source to Bake', and then especially this one below that; 'Render to Files Only'. VERY IMPORTANT. If you do not have this 'active', then every time you create a new Render to Texture AO map, you create a new Shell Material in your Library. You want to Export the AO Map, not add it into your textures. (That is Baking Textures, we are not doing that here).
[BAKING (RENDERING) AN AO MAP]
* Now, select a part, on your Render to Texture control panel, at the bottom, click Render.
* You should get a warning popup: Warning; There are no photon emmitters in the scene. This is because you have no Lights. Carry on, ignore this.
* If this is your full size unit, it may take a short while to export, like 3 min's plus. A popup window of the Rendering will appear showing progress,
along with another Mental Ray progress 'green bars' mini window.
* When it is done, turn off the popup window. Save your scene, Minimize Max,
* Go to your AO map that should be on your Desktop or where ever you wanted it to export to. Open that with Photoshop. If you have already created
your basic Texture in Photoshop, then open that too and add your new AO Map into the Layer stacks, high in the stacks so the shading is visible and not
hidden. Double click on the Layer and select Blend/Multiply. The image should immediately turn into a soft shading effect in Photoshop.
* Done. Keep doing these, adding them into your Texture. I do 'not' combine these, as the white might override other parts shadings, so I keep these all in separate layers. I also name them; AO Map for each one. Great and much needed for organization of a good PSD file.
* Alt-Tab back over to Max and check your shading on your object(s). Redo if necessary. You want them to be as real as possible. They should be very
subtle. If they are radically dark, they will look horrible later. You want to effect the subconcious of the 'simmer' with the realism of these
shadings, not to overwhelm the visuals.
* You are good to delete the AO Targa maps (on your Desktop) after use. You have a copy of them in the Layers of your PSD files/graphics and you can
make new ones fast enough.
[NOTES]
NOTE 1: You can only make 'one' Render per 'part' or Object. To save time of stacking all the AO Map Layers in Photoshop, you can make 1 or just a few
by Attaching parts in a 'version' of your model. Save a version of your model, Attach parts that share the same Material/Texture. Run an AO Render on
them, done. Discard that version of model later, or detach the parts to restore the model.
NOTE 2: You 'can' do Multiple AO maps at once. Max will generate these one at a time. Each one will be for each part though. Best to test your
settings first to make sure you have the right AO scale and darkness settings for your shadows. To do multiple renders, select several parts, then click
on Render on the Render to Texture button.
NOTE 3: To stop a render, click the Cancel button on the Mental Ray 'bars' progress window. Usually in a 'scene' (3D window, not a render to texture)
you would hit escape, but this doesnt work in the Mental Ray version 'Render to Texture'.
NOTE 4: Supposedly you use 'no' lights in an Ambient Occlusion scene, from what 'some' say at CGSociety. I found that the Renders really do not use
them that much, oddly. I am fairly new to this, and I do know that the pros do use various Lights for this and that appearences. Look for tutorials on
this and experiment. Always experiment and take notes in Notepad on your findings. Store your notes in good places you will find them at later.
NOTE 5: Rendering times and Samples settings. Samples set at 200+ and a map set at 2048 pixels big, with a large part/obejct like a wing, will take a
short while to Render, such as 3 min's, etc. Slower the computer, much longer the Render time. To do test runs, lower the Sample rate and size of the
output texture.
NOTE 6: The Render Window for Render to Texture will 'not' look like the actual Targa map you export. Do 'not' go by the Render Window.
NOTE 7: The Render button on the Render to Texture is the one you need to click on when making a Render to Texture. Making a Rendering (regular
rendering from the Render Setup Control Panel 'will not' create your Render to Texture AO map. You must use the button in the Render to Texture Control
Panel.
NOTE 8: Troubleshooting: Odd things can happen in creating the Ambient Occlusion maps.
* Having multiple objects on the same 'spot' of a Graphics, if the parts are in various locations (like bolts on a canopy or screws on a panel) then
shadings for ALL of those parts will occur, probably making the parts black or dark gray. To repair, make as many bolt texture 'spots' as needed and
render them that way,or just do one part, and place that around the cockpit where needed. Note that the second option is a poor one. You purchased Max
to make these things look real, so use it. Make a bunch of very small spots and take advantage of the system. You can duplicate 'sides' to cut down on
texture 'spots'.
* Dual sided parts (like cockpit structure struts) should be mapped on both sides instead of one Plannar side, or you will have a dark part again in the
AO Map.
* Black triangles; Make sure you have all Polygon Normals facing outward properly. Turn on Cull, make sure nothing is reversed. Another thing that can
cause goofy looking 'render damage' is a part sticking out of the side of the one being rendered, but is reversed, so you do not see it, like the cabin
wall sticking through the fuselage. This will cause a big shading thing to appear on a smooth fuselage surface. So check your model well when these
little flaws occur.
NOTE 9: Render to Texture Settings; Auto Changes. The settings you put in for one part, will sometimes change with the next in the Render to Texture window. The auto-set for Render to Texture settings appears to link to the Mental Ray AO Material / Texture in the Material Editor. Thus, each time you render a AO map on a 'new' part, the RTT (Render to Texture) settings will reset to the Material settings. Thus, if you reset things for doing seats, you will need to remember the seat settings. The RTT Setting will stay with each part (I have found) so you can find the settings on the previous part you did by selecting it, looking at the Render settings in the RTT control panel and duplicating those settings into the next part/selection. So, each time you Render to Texture, check your settings in the Render to Texture
Control Panel.
NOTE 10: When setting up the 'Diffuse' texture in your Mental Ray Material in Material Editor, note that if you do not have Mental Ray 'active' (setup
and on) as your Renderer, then you will not see the Mental Ray Materials or the Mental Ray AO texture that you need for 'Diffuse'. You 'must' do these
in order, you Must have Mental Ray 'active' before setting up your AO rendering Material.
NOTE 11: Perimeter Shading: If you want 'shadows' cast from other objects intermixed on the part you are rendering (answer is yes, that is what this is all about), then you 'must' have other parts around the part being rendered. What ever you leave out of the scene 'during' a render, those shadows will not interact on the rendering when it is generated. Likewise, if you do not want the shadow of the pilot on the seat, then hide the pilot during seat renderings and his shadow will not be on it. Also, if you have the fuselage 'in' the scene and you are rendering seat textures or things in the cockpit, the fuselage polygons have a tendency to block out light, so that might make parts 'darker' then usual, so you you might hide the fuselage to allow more ambient light in to bounce around on the objects. Experiment, experiment, experiment. Do not be afraid to try 5 versions of AO renderings. You want it perfect.
NOTE 12: I find that smaller parts may sometimes use a lower Max Distance setting. If the parts come out too darkly shaded, raise the Spread setting and see how that works. For leather seats and things, use a really grainy AO effect by using a super low Samples setting.
Bill
LHC
When Max 2012 came out, many changes occured with how one does AO Maps (Ambient Occlusion rendered shading). This explains how to do these in Max 2012 with Mental Ray as the Rendering engine. If you move up from Max 9 to Max 2012, you will need to know this if you do AO maps.
Follow these in order.......................... Must...
Boot up your scene
[RENDERER SETUP]
Open your Renderer Control Settings Panel; F10
Go to Commons Tab/Page
Setup Mental Ray as your Renderer
Go to Processing, tick 'Enable' at Material Override
[AMBIENT OCCLUSION MATERIAL SETUP]
* Next, open Material Editor (M key)
* Create a new Material, name it something like Mental Ray AO Material
* Tick on the Bitmap small square at Diffuse
* On the Textures Navigator that pops up, go down to the Mental Ray materials section, find AO, it should be a bright white dot on its icon, select it
* IN your Material Editor, you will find that the texture will have your Ambient Occlusion settings (yes, in the Material in the 'texture' section for
Diffuse). There you want to set up your settings, though later, you will also set this up identically in the Render to Texture control panel.
Settings for the AO texture.
NOTE: You will have many settings, but only a few count, worry only about those few for now.
Samples........Refinement of the shadows; 220 is very smooth, 50 is grainy like mud, great for mud effects by the way
Max Distance...Size of the shadow. Test to see how big it needs to be. Usually 2.0 on some aircraft
Spread.........Darkness. Low numbers are dark, high numbers are brighter, lighter. 0.2 to 2.0. I use 0.8 for now.
* Next, drag the Sphere of the 'Mental Ray AO Material' in the Spheres section of the Material Editor onto the 'material' button of the 'Material
Override' on the Renderer Setup page (F10) that is on the 'Processing' section. (Drag and drop). This will cause the AO Material to show up on 'all'
Materials/Objects when doing a Render 'only'. Otherwise, the AO material is 'not' seen. You can leave this on. Make sure you ticked on the 'Enable'
tick on the Material Override on the Processing Page in the Renderer Setup center.
[RENDER TO TEXTURE SETUP]
Now open your 'Render to Texture' control panel page, Keyboard shortcut; 0 (zero) on the Keyboard side (not NumPad).
* Select a part in the scene, one you will probably create an AO map for.
* Output, Path, set this to where you want your maps to export to. I use Desktop as I delete the AO maps once I have placed them in the PSD layers.
They are basically no longer needed. (You will only need to do this once, this will be stored in Max memory settings, exporting of Render to Texture target location).
* Tick 'on' Rendered Frame Window.
* Render Settings flyout; 'do not touch', should look like this; ------------
* Object and Output settings; 'do not touch, should also look like this; -----------
* Name; Here you will see your Part you will be exporting. You should see the part name, like Fuselage.
* Selected Object Settings; 'Enabled' (tick on). Padding = 2. Padding is next to this.
* Project Mapping; 'not' ticked
* Mapping Coordinates; 'Use existing mapping channel' 'on'. Channel 1 set.
* Subobjects is locked off
* OUTPUT SECTION: Click Add, find Ambient Occlusion (MR), click Select (This is only done once for doing a group of parts. This sets up the format of the map as well, such as Targa or JGP).
* Selected Common Settings:
-- Enable (tick on), Ambient Occlusion should be in this slot
-- File Name and Type; click on the ... button and select JPG or TGA or what ever you like. Most Maxxers use Targa's, so TGA is standard for AO Maps
-- Use Default Gamma (thats what I use, others may have their own Gamma settings). Click Save.
* Element Background. Use pure black or pure white. I use white, as white is transparent in Photoshop/Layer/Blend=Multiply Only gray is used in the
shading in Photoshop in Multiply layers.
* Use Automatic Map Settings; leave ticked 'off' (do not select)
* Set size of your texture. Note; make a small one and check that its the one you want, right settings in rendering, then go to full size. Scale is
important, as you can then fit your map to your texture perfectly. If you work in 2048 pixels, then make your good AO Map to 2048. For a test, try
512, its quicker to Render.
* SELECT ELEMENT UNIQUE SETTINGS
-- Samples............Quality of Shadow; smooth or grainy, 220 for good, 50 or speckled, 120 for fair.
-- Spread.............0.8 (Test and change if necessary for your model)
-- Max Distance.......2.0 (Test and change if necessary for your model)
-- Falloff............Usually set to 1.0, I never touch it.
***NOTE: The above settings 'should' match your settings in the Material Editor/Mental Ray AO Material/Diffuse texture settings center.
* BAKED MATERIAL SETTINGS
-- Save Source ticked 'on'
-- Duplicate Source to Baked ticked 'on'
-- Render to Files Only ticked 'on' <<< IMPORTANT
-- Keep Source Material ticked 'on' <<< IMPORTANT
* AUTOMATIC MAPPING
(I do not touch these, but here are my settings, just incase)
-- Rotate Clusters ticked 'on'
-- Fill Holes ticked 'on'
-- Threshold Angle....45.0
-- Spacing............0.02
* AUTOMATIC MAP SIZE
(I do not touch these settings, but here are my settings just in case)
-- Scale = 0.01
-- Min....45.0
-- Max............0.02
* VIEWS RENDER (Very bottom, right side of the Render to Texture control panel)
-- Original.....Render ticked
-- Baked........Views ticked
NOTE ABOVE 1: Spread Setting shows 0.2. It should be 0.8.
NOTE ABOVE 2: Be sure, on the bottom of that screenshot above, that you select (tick) on 'Save Source (Create Shell) ', then 'Duplicate Source to Bake', and then especially this one below that; 'Render to Files Only'. VERY IMPORTANT. If you do not have this 'active', then every time you create a new Render to Texture AO map, you create a new Shell Material in your Library. You want to Export the AO Map, not add it into your textures. (That is Baking Textures, we are not doing that here).
[BAKING (RENDERING) AN AO MAP]
* Now, select a part, on your Render to Texture control panel, at the bottom, click Render.
* You should get a warning popup: Warning; There are no photon emmitters in the scene. This is because you have no Lights. Carry on, ignore this.
* If this is your full size unit, it may take a short while to export, like 3 min's plus. A popup window of the Rendering will appear showing progress,
along with another Mental Ray progress 'green bars' mini window.
* When it is done, turn off the popup window. Save your scene, Minimize Max,
* Go to your AO map that should be on your Desktop or where ever you wanted it to export to. Open that with Photoshop. If you have already created
your basic Texture in Photoshop, then open that too and add your new AO Map into the Layer stacks, high in the stacks so the shading is visible and not
hidden. Double click on the Layer and select Blend/Multiply. The image should immediately turn into a soft shading effect in Photoshop.
* Done. Keep doing these, adding them into your Texture. I do 'not' combine these, as the white might override other parts shadings, so I keep these all in separate layers. I also name them; AO Map for each one. Great and much needed for organization of a good PSD file.
* Alt-Tab back over to Max and check your shading on your object(s). Redo if necessary. You want them to be as real as possible. They should be very
subtle. If they are radically dark, they will look horrible later. You want to effect the subconcious of the 'simmer' with the realism of these
shadings, not to overwhelm the visuals.
* You are good to delete the AO Targa maps (on your Desktop) after use. You have a copy of them in the Layers of your PSD files/graphics and you can
make new ones fast enough.
[NOTES]
NOTE 1: You can only make 'one' Render per 'part' or Object. To save time of stacking all the AO Map Layers in Photoshop, you can make 1 or just a few
by Attaching parts in a 'version' of your model. Save a version of your model, Attach parts that share the same Material/Texture. Run an AO Render on
them, done. Discard that version of model later, or detach the parts to restore the model.
NOTE 2: You 'can' do Multiple AO maps at once. Max will generate these one at a time. Each one will be for each part though. Best to test your
settings first to make sure you have the right AO scale and darkness settings for your shadows. To do multiple renders, select several parts, then click
on Render on the Render to Texture button.
NOTE 3: To stop a render, click the Cancel button on the Mental Ray 'bars' progress window. Usually in a 'scene' (3D window, not a render to texture)
you would hit escape, but this doesnt work in the Mental Ray version 'Render to Texture'.
NOTE 4: Supposedly you use 'no' lights in an Ambient Occlusion scene, from what 'some' say at CGSociety. I found that the Renders really do not use
them that much, oddly. I am fairly new to this, and I do know that the pros do use various Lights for this and that appearences. Look for tutorials on
this and experiment. Always experiment and take notes in Notepad on your findings. Store your notes in good places you will find them at later.
NOTE 5: Rendering times and Samples settings. Samples set at 200+ and a map set at 2048 pixels big, with a large part/obejct like a wing, will take a
short while to Render, such as 3 min's, etc. Slower the computer, much longer the Render time. To do test runs, lower the Sample rate and size of the
output texture.
NOTE 6: The Render Window for Render to Texture will 'not' look like the actual Targa map you export. Do 'not' go by the Render Window.
NOTE 7: The Render button on the Render to Texture is the one you need to click on when making a Render to Texture. Making a Rendering (regular
rendering from the Render Setup Control Panel 'will not' create your Render to Texture AO map. You must use the button in the Render to Texture Control
Panel.
NOTE 8: Troubleshooting: Odd things can happen in creating the Ambient Occlusion maps.
* Having multiple objects on the same 'spot' of a Graphics, if the parts are in various locations (like bolts on a canopy or screws on a panel) then
shadings for ALL of those parts will occur, probably making the parts black or dark gray. To repair, make as many bolt texture 'spots' as needed and
render them that way,or just do one part, and place that around the cockpit where needed. Note that the second option is a poor one. You purchased Max
to make these things look real, so use it. Make a bunch of very small spots and take advantage of the system. You can duplicate 'sides' to cut down on
texture 'spots'.
* Dual sided parts (like cockpit structure struts) should be mapped on both sides instead of one Plannar side, or you will have a dark part again in the
AO Map.
* Black triangles; Make sure you have all Polygon Normals facing outward properly. Turn on Cull, make sure nothing is reversed. Another thing that can
cause goofy looking 'render damage' is a part sticking out of the side of the one being rendered, but is reversed, so you do not see it, like the cabin
wall sticking through the fuselage. This will cause a big shading thing to appear on a smooth fuselage surface. So check your model well when these
little flaws occur.
NOTE 9: Render to Texture Settings; Auto Changes. The settings you put in for one part, will sometimes change with the next in the Render to Texture window. The auto-set for Render to Texture settings appears to link to the Mental Ray AO Material / Texture in the Material Editor. Thus, each time you render a AO map on a 'new' part, the RTT (Render to Texture) settings will reset to the Material settings. Thus, if you reset things for doing seats, you will need to remember the seat settings. The RTT Setting will stay with each part (I have found) so you can find the settings on the previous part you did by selecting it, looking at the Render settings in the RTT control panel and duplicating those settings into the next part/selection. So, each time you Render to Texture, check your settings in the Render to Texture
Control Panel.
NOTE 10: When setting up the 'Diffuse' texture in your Mental Ray Material in Material Editor, note that if you do not have Mental Ray 'active' (setup
and on) as your Renderer, then you will not see the Mental Ray Materials or the Mental Ray AO texture that you need for 'Diffuse'. You 'must' do these
in order, you Must have Mental Ray 'active' before setting up your AO rendering Material.
NOTE 11: Perimeter Shading: If you want 'shadows' cast from other objects intermixed on the part you are rendering (answer is yes, that is what this is all about), then you 'must' have other parts around the part being rendered. What ever you leave out of the scene 'during' a render, those shadows will not interact on the rendering when it is generated. Likewise, if you do not want the shadow of the pilot on the seat, then hide the pilot during seat renderings and his shadow will not be on it. Also, if you have the fuselage 'in' the scene and you are rendering seat textures or things in the cockpit, the fuselage polygons have a tendency to block out light, so that might make parts 'darker' then usual, so you you might hide the fuselage to allow more ambient light in to bounce around on the objects. Experiment, experiment, experiment. Do not be afraid to try 5 versions of AO renderings. You want it perfect.
NOTE 12: I find that smaller parts may sometimes use a lower Max Distance setting. If the parts come out too darkly shaded, raise the Spread setting and see how that works. For leather seats and things, use a really grainy AO effect by using a super low Samples setting.
Bill
LHC
Last edited:
