Frustum-Traced Raster Shadows: Revisiting - Research

Frustum-Traced Raster Shadows: Revisiting Irregular Z-Buffers
Chris Wyman∗
Rama Hoetzlein
NVIDIA
Aaron Lefohn
Figure 1: (Left) Our 32 sample per pixel hard shadows in the 613k triangle Chalmers Citadel (16 ms at 1080p). (Center) Compared with
a filtered 80922 shadow map, we avoid temporal and spatial aliasing and eliminate light leaking. (Right) 32 sample per pixel shadows from
the intricate, 3.8M triangle tentacles model (38 ms at 1080p).
Abstract
We present a real-time system that renders antialiased hard shadows using irregular z-buffers (IZBs). For subpixel accuracy, we
use 32 samples per pixel at roughly twice the cost of a single sample. Our system remains interactive on a variety of game assets and
CAD models while running at 1080p and 2160p and imposes no
constraints on light, camera or geometry, allowing fully dynamic
scenes without precomputation. Unlike shadow maps we introduce
no spatial or temporal aliasing, smoothly animating even subpixel
shadows from grass or wires.
Prior irregular z-buffer work relies heavily on GPU compute. Instead we leverage the graphics pipeline, including hardware conservative raster and early-z culling. We observe a duality between
irregular z-buffer performance and shadow map quality; this allows
common shadow map algorithms to reduce our cost. Compared to
state-of-the-art ray tracers, we spawn similar numbers of triangle
intersections per pixel yet completely rebuild our data structure in
under 2 ms per frame.
CR Categories: I.3.7 [Computer Graphics]: Three-Dimensional
Graphics and Realism—Color, shading, shadowing, and texture
Keywords: shadows, irregular z-buffer, frustum tracing
1
Introduction
While conceptually simple, real-time rendering of hard shadows remains an enormous challenge. Most modern applications use variants of shadow mapping [Williams 1978]. But due to discretization and sampling mismatches between eye- and light-space, robust
∗ e-mail:[email protected]
and artifact-free results remain elusive even with high quality filtering [Annen et al. 2008] and cascades [Lauritzen et al. 2011].
We describe the evolution of our system, designed with a simple
goal: interactive, sub-pixel accurate hard shadows at usable resolutions on content representative of modern workloads. We began
without any preconceptions except that shadow maps induce aliasing, potentially a more challenging problem. While desirable, extendability to soft shadows was not a design goal; our primary goal
was robust artifact-free shadows, with a secondary aim of speed.
To avoid aliasing, we only considered analytical shadow techniques that sample in eye space at or below the pixel level. This
leaves three broad algorithmic classes: ray tracing [Whitted 1980],
shadow volumes [Crow 1977], and irregular z-buffers [Johnson
et al. 2005]. Fundamentally, all analytic approaches perform raytriangle intersections; the key difference is how tests are spawned.
Ray tracers query visibility along individual rays, irregular zbuffers rasterize in light space, and shadow volumes indirectly determine ray-triangle occlusion by testing shadow boundaries.
We pursued variants of irregular z-buffers (IZBs), which appeared
the most natural fit for today’s raster pipelines. Ray tracing requires
additional acceleration structures, and shadow volumes either require object-space silhouette detection or introduce complex hierarchies [Sintorn et al. 2014] to accelerate brute force, per-triangle
volumes.
Little research has explored efficient algorithms for IZBs. Initial
work proposed major hardware changes [Johnson et al. 2005] and
alternative data structures [Aila and Laine 2004]. Later work [Arvo
2007; Sintorn et al. 2008] showed hardware implementations using
GPU computing capabilities rather than the graphics pipelines. Pan
et al. [2009] improved quality, demonstrating antialiased shadow
boundaries. To our knowledge, nobody has eliminated the key
problems with irregular z-buffering: poor scalability and large performance variations between frames.
We demonstrate that IZB shadows scale to multi-million triangle
models and HD resolutions yet remain interactive (see Figure 1).
Initial prototypes required nearly 2 seconds for complex models
like the tentacles. By simplifying and streamlining the data structure and leveraging existing strengths of the graphics pipeline we
achieved two orders of magnitude better performance. A key in-
sight: we observe a duality between irregular z-buffer performance
and shadow map quality. Regions that alias with shadow mapping
have lower performance with IZBs. This allows decades of research
improving quality and consistency of shadow maps to help improve
irregular z-buffer performance.
Additional contributions of our system include:
• Demonstration of alias-free shadows on realistic content with
consistently interactive performance. Cost is under 8 and 24
ms for game scenes with 1 and 32 samples per pixel (spp).
• An efficient IZB implementation relying on existing hardware
in the graphics pipeline to improve culling beyond that available to implementations using only GPU compute.
• An efficient extension of single sample irregular z-buffers to
render 32 spp shadows.
• An extensive performance evaluation with quantified gains for
individual optimizations.
2
Why Irregular Z-Buffers?
Decades of shadow research provide developers many algorithmic
choices [Eisemann et al. 2011; Woo and Poulin 2012]. Until recently, the choices in interactive contexts were limited to shadow
volumes [Crow 1977] and shadow maps [Williams 1978].
Shadow volumes generate pixel-accurate shadows by constructing
and testing the boundary of shadowed regions. This proves difficult to do robustly and renders invisible ”shadow quads” that consume significant fill rate. With reductions to these problems (e.g.,
McGuire et al. [2003] and Lloyd et al. [2004]) some games shipped
using shadow volumes, but most developers still avoid them.
Shadow maps are easily implemented and efficiently run on GPUs,
but regular sampling of visibility causes spatial and temporal aliasing. Filtering [Reeves et al. 1987] and using statistical models
[Donnelly and Lauritzen 2006; Annen et al. 2008] partially hide
aliasing, but often introduce other artifacts. Distorting the light
frustum [Stamminger and Drettakis 2002], adaptively refining [Fernando et al. 2001], or fitting multiple shadow maps [Lauritzen et al.
2011] improve sampling quality but can introduce overhead, reduce
robustness, and still exhibit aliasing.
On the cusp of interactivity, ray tracing [Whitted 1980] may provide another alternative. Hardware acceleration improves performance [Lee et al. 2013], but adoption rates are unclear. Ray query
costs strongly depend on existence of quality acceleration structures [Aila et al. 2013], which remain expensive to build. Concurrent work [Mittring 2014] suggests existing GPUs may suffice for
game-quality ray tracing, though their grid-of-lists structure resembles IZBs more than standard bounding volume hierarchies.
Irregular z-buffers [Johnson et al. 2005] provide another option
within existing graphics pipelines. Shadow maps use a light-space
z-buffer; IZBs instead use a light-space A-buffer [Carpenter 1984],
with each texel storing all pixels potentially occluded by geometry
in that texel. While a grid-of-lists structure increases complexity
over shadow maps, today’s GPUs construct and traverse linked-lists
efficiently [Yang et al. 2010]. Key problems with IZBs include poor
scalability and high performance variability. Prior GPU implementations [Sintorn et al. 2008; Pan et al. 2009] typically rendered at
5122 with models under 100k triangles; performance scaled linearly with triangle and pixel counts. Our initial IZB prototype exhibited 100:1 performance variations between certain frames during
even basic interactions.
P1
Li
P2
P3
Vi
P4
P5
P6
Pi
P7
P8
P9
~i , interFigure 2: (Left) A familiar mapping between view vector V
~
section point Pi , and light direction Li . Shadow maps discretize
light-space, so shadow queries return the nearest neighbor rather
~ i . (Right) Irregular z-buffer shadows
than the true visibility along L
store all sample points Pi , allowing exact shadows. Rather than
storing a single depth, as in shadow maps, IZBs store all points
that fall in a texel as a linked list.
However, their problems lie entirely in the performance domain.
While Johnson et al. [2005] explored performance characteristics
on their proposed hardware, IZB’s characteristics on modern GPUs
are largely unexplored. By carefully identifying and removing key
bottlenecks, we designed an efficient implementation achievable today. The 2.9 million triangle hairball (in Figure 7) required 12.6
seconds in Aila and Laine’s [2004] work, 1.5 seconds in our first
prototype, and 13.8 milliseconds today.
3
Irregular Z-Buffer Review
Before describing our system, we briefly review irregular z-buffer
shadows. IZBs aim to avoid shadow map aliasing from mismatches
between eye- and light-space sampling locations. Shadow maps use
a regular grid of samples in both eye- and light-space, but finding a
robust bijection between samples in these spaces remains unsolved.
By allowing light-space samples to occur irregularly, IZB shadows
easily pair samples and eliminate aliasing.
By construction, an IZB bijectively maps each pixel to one sample
~i hitting geometry at Pi
in light-space; the pixel represents ray V
~ i from Pi to the
and a corresponding light sample represents ray L
~
light (see Figure 2). Shadow map queries along Li return the nearest neighbor sample on the texel grid; irregular z-buffers store all
samples, generating a unique visibility for each pixel.
In theory, constructing an irregular z-buffer shadow map works
identically to regular shadow maps: one “rasterizes” occluders over
~ i , finding the closest triangle along
the irregular set of light rays L
~
each Li . If the depth of the closest triangle lies between the light
and Pi we know that sample is shadowed.
Since modern GPUs rasterize only over regular samples, we need
to store our irregular samples in a grid. A grid-of-lists structure
achieves this. The irregular z-buffer is a grid of light-space head
pointers, each pointing to a linked list containing irregular samples
falling within the grid cell. Intuitively, this is identical to a shadow
map except texels stores a pointer rather than a depth. Since samples can lie anywhere within a texel, conservative rasterization is
required; triangles must test samples for occlusion if they cover any
portion of a texel (not just the center, as in traditional rasterization).
Rasterizing over irregular samples requires knowing where they occur; IZBs require a prepass to identify sample locations. Game engines commonly use an eye-space z-prepass to reduce overshading.
This prepass provides exactly the needed data: locations of visible
pixels requiring shadow queries. To identify IZB samples we run
a compute pass over this z-buffer, transforming pixels into lightspace (via the shadow map transformation), and inserting them into
their corresponding light-space lists (see Figure 2).
Pseudocode describing this process follows:
High Level Pseudocode: Irregular Z-Buffer Shadows
µQuad
// Step 1: Identify locations we need to shadow
G(x, y) ← RenderGBufferFromEye()
// Step 2: Add these pixels into the light-space data structure (our IZB)
for pixel p ∈ G(x, y) do
lsTexelp ← ShadowMapTransform[ GetEyeSpacePos( p ) ]
izbNodep ← CreateIZBNode[ p ]
AddNodeToLightSpaceList[ lsTexelp , izbNodep ]
end for
// Step 3: Determine shadows; test each triangle with pixels in lists it covers
for triangle t ∈ SceneTriangles do
for fragment f ∈ ConservativelyRasterizeInLightSpace( t ) do
lsTexelf ← FragmentLocationInRasterGrid[ f ]
TraversePixelListFromStep2TestingIfTriangleShadows[ lsTexelf ]
end for
end for
Initially this seems complex, but is a relatively simple modification
to shadow mapping, essentially swapping the order of light-space
rasterization and sample projection:
High Level Pseudocode: Shadow Maps
// Step 1: Render shadow map in light-space
for triangle t ∈ SceneTriangles do
for fragment f ∈ RasterizeInLightSpace( t ) do
lsTexelf ← FragmentLocationInRasterGrid[ f ]
if depthf < Z( lsTexelf ) then Z( lsTexelf ) ← depthf
end for
end for
// Step 2: Query shadow map for each pixel
for pixel p ∈ FinalRender do
lsTexelp ← ShadowMapTransform[ GetEyeSpacePos( p ) ]
isShadowed ← ( DistanceToLight( p ) > Z( lsTexelp ) ? true : false )
end for
3.1
Performance Considerations
As with ray tracing, the key unit of work is ray-triangle intersections. These are spawned as a triangle fragment traverses its list of
potentially occluded samples (in pseudocode, step 3); each sample
~ i ) that is tested for intersection
represents a ray (from Pi along L
with the rasterized triangle. Other steps have only small performance costs: a z-prepass typically already occurs, and simple irregular z-buffer creation requires under a millisecond.
Figure 3: (Left) A standard shadow ray query. (Center) Creating
fragment shadow frusta (at black point): project pixel footprint to
the fragment tangent plane; this “micro-quad” (µQuad) becomes
the shadow frustum base. (Right) Intersection tests project each triangle edge to the tangent plane (i.e., identifying the shadow quad
intersection), and use the projected edge to index a lookup table
providing visibility for each sample. Visibility sample locations
(gray points) are developer specified during LUT construction.
ity of such a change is unclear, and prior work rarely explored such
tradeoffs.
4
Antialiasing via Frustum-Triangle Tests
Like ray tracing, IZB shadows provide pixel accurate shadows.
Naive extensions achieve subpixel accuracy via multiple sampling
(e.g., shooting additional rays or using multiple IZB samples per
pixel), but increase cost linearly with sample count.
Alternatively a beam tracing approach [Heckbert and Hanrahan
1984] can provide analytic subpixel visibility. Interactive work often replaces beams with packet tracing [Boulos et al. 2007], tiled
rasterization, or raster stamps to preserve spatial coherence while
using discrete samples. Numerous soft shadow algorithms use these
methods, including for ray traced shadows [Overbeck et al. 2007],
bitmask soft shadows [Schwarz and Stamminger 2007], and even
IZB-based soft shadows [Sintorn et al. 2008].
To achieve antialised shadows we flip these beams around, essentially tracing a frustum from the point light back to the geometry
(see Figure 3). This is similar to Pan et al.’s [2009] approach,
though we directly compute intersections in world space rather than
projecting back to the image plane.
We achieve antialised shadows by replacing the ray-triangle intersections in IZB shadows with frustum-triangle intersections. Efficient frustum intersection occurs per pixel as follows:
Simplistically, the algorithmic complexity is O(N ) for N raytriangle visibility tests. But N depends on triangle count and screen
resolution; pixels create IZB nodes, and triangles generate lightspace fragments that traverse lists of pixels. So N = tf hlizb i for
tf the number of light-space triangle fragments and hlizb i the average IZB list length traversed by each fragment.
• Construct a µQuad representing the pixel projection onto the
tangent plane (see Figure 3).
This means performance depends on total ray-triangle tests, number of light-space fragments, and the average length of IZB node
lists. Additionally, traversing IZB lists causes GPU underutilization if list lengths vary significantly between threads, so reducing
variance of lizb improves performance for GPU implementations.
These factors interact in non-intuitive ways. Increasing light-space
grid resolution reduces average list length by distributing samples
over larger numbers of texels but also increases the number of lightspace fragments generated by rasterization. At first glance the util-
• Project each shadow quad to the tangent plane, determining
which half-plane is shadowed. Use projection to lookup a 32sample visibility bitmask from a lookup table. Combine the
edge results with a binary AND to create a visibility bitmask
representing which pixel subsamples this triangle occludes.
• Treat each edge of the occluder triangle independently. An
edge plus the light define a shadow quad. The three shadow
quads plus the triangle plane bound its shadow volume.
As this intersection represents the inner loop, tight optimization is
vital for good performance. We provide an example GLSL implementation as supplementary material.
Irregular Z-buffer input
Irregular Z-buffer data structure
Eye space z-prepass + triangle soup
IZB head pointers: on
light space grid
IZB creation: apply shadow map transform
to project pixels to light space. Add each
pixel to corresponding light space list.
IZB nodes: stored on eye space grid
Irregular Z-buffer output
Visibility mask on eye space grid
Rasterize triangles in light space. Traverse list
in each covered texel. Test shadow visibility for
each listed pixel. Update shadow visibility mask.
Light-space sample positions
Figure 4: An overview of our irregular z-buffer implementation. We require an eye-space z-buffer plus triangle geometry as input; each pixel
location is transformed to light-space and added to the projected texel’s linked list. Triangles are rasterized over this light-space grid; each
fragment traverses its list and tests listed pixels for occlusion. When tests identify occlusion, the pixels’ visibility mask is updated. The final
render pass consumes these visibility masks to correctly shadow pixels. (Right) An example visualization of irregular light-space samples.
5
System Overview
As discussed in Section 3, irregular z-buffer shadows have three
distinct phases: creating the IZB data structure, spawning triangle
occlusion tests via light-space rasterization, and rendering a final
shadowed image. Figure 4 visually outlines this process, describing
inputs, outputs, and the light-space data structure. We implement
these steps with six passes: an eye-space z-prepass, bounding the
scene’s visible regions, creating the irregular z-buffer, a light-space
culling prepass, spawning triangle visibility tests, and the final render. These are outlined in greater detail below.
5.1
Eye-Space Z-Prepass
Create an irregular z-buffer requires knowing which samples to add.
Modern rendering engines often use a z-only prepass to facilitate
culling; this pass provides the required information. Our implementation simultaneously creates a G-buffer, used during our deferred
final render phase.
Single sample shadows only require fragment depths. To perform
frustum intersection for antialised shadows, we add three floats
to the G-buffer describing the µQuad projection onto the tangent
plane; this accelerates visibility tests, but could be recomputed from
the fragment normal if G-buffer space is tight.
5.2
Scene Bounds
As with shadow maps, a priori knowing the correct settings for the
light frustum is challenging. To avoid poorly bounding the scene,
we recompute the light’s projection matrix each frame to tightly
bound geometry visible in the z-prepass. We use a persistent thread
compute shader over the z-buffer from Section 5.1.
5.3
Creating the Irregular Z-Buffer
We walk through the eye-space z-buffer, transforming each pixel
into light-space and inserting it into the appropriate texel’s linked
list, as described in Section 3.
Using multiple visibility samples per pixel may also require multiple IZB nodes per pixel, in theory up to one node per visibility
sample. Unlike a point query, our µQuad may project to multiple light-space texels; triangles touching any of these texels could
shadow the pixel. Section 6.1 introduces an approximation using
at most eight (and averaging two) IZB nodes per pixel with little
quality impact.
5.4
Light-Space Culling Prepass
In Section 5.5 we spawn visibility tests via conservative rasterization. The GPU’s early-z hardware can accelerate this process by
culling triangle fragments covering empty pixel lists or falling behind the furthest list node. Intuitively, this provides frustum culling
based on actual pixel geometry.
To use early-z hardware, we need a light-space z-buffer. This could
be generated as a side effect in Section 5.3. But as that pass runs
in eye-space, standard raster operations cannot output light-space
depth. Given the GPU’s opaque depth format, we had difficulty
using global memory writes to create a z-buffer usable by the earlyz hardware. Our prepass essentially creates a stencil: setting depth
to 0 in texels with empty lists and the distance to the furthest IZB
node elsewhere. Except in our most trivial scenes, using hardware
z-cull provides a substantial speedup (between 30 and 50%).
5.5
Spawning Triangle Visibility Tests
Spawning and performing visibility tests represents our system’s
major cost. We conservatively rasterize triangles over a light-space
grid (shown in Figure 4). Each texel stores a list of pixels potentially occluded by triangles overlapping it. Conservative rasterization is required, as pixels may lie anywhere within the volume represented by a texel; triangles partially covering a texel may occlude
an arbitrary subset of its list.
Each fragment traverses the entire texel list. During traversal we
load each listed pixel, perform a visibility test, and atomically OR
the result into the pixel’s visibility mask.
A key bottleneck stems from thread divergence at this step; since
lists have variable length, some threads wait on adjacent threads.
For naive implementations, this wait can be extreme; in certain
views we observed 1000:1 variations between adjacent list lengths.
5.6
Final Render
Section 5.5 performs shadow tests and stores results in a per-pixel
visibility mask. Our final render pass loads from the G-buffer (Section 5.1) and this mask and performs Phong shading modulated by
the shadow mask.
6.2
Data Structure and Memory Layout
In any complex data structure, it is worth considering the best layout
for efficient construction and traversal. We initially prototyped a
simple 2D grid of linked lists, allocating nodes from a global node
pool (see Figure 2). Each list node contained two entries: a next
pointer and a G-buffer index (to fetch pixel data). But this structure
needs two synchronizations per insertion: a global atomic to find a
free node and a per texel atomic to update the head pointer.
Figure 5: (Left) As a fragment’s tangent plane changes orientation,
µQuads elongate along only one axis (the other dimension depends
on screen resolution). (Right) This suggests sampling them one dimensionally; we add from one to eight samples to the IZB depending on orientation.
6
Implementation Details
Section 5 provides a high-level description of our system, but key
aspects deserve greater explanation. First, we discuss how multisample shadows change our data structure. Then we explore efficient layouts for the irregular z-buffer data structure and outline
some smaller optimizations that significantly impact performance.
6.1
Simplifying IZBs for Multi-Sample Shadows
Moving from 1 to 32 spp shadows complicates our algorithm.
The key point is light-space rasterization needs to spawn frustumtriangle tests (described in Section 4) for any triangle occluding a
pixel. Turning that around, a µQuad projects to a variable number
of light-space texels; its pixel must be added to those texels’ lists.
One expensive approach rasterizes µQuads in light-space during
IZB construction. Another method adds all 32 per pixel visibility
samples to the IZB. An optimization can skip duplicate insertions
within a µQuad. We initially prototyped this sample-based insertion, though it often adds 8 or more IZB nodes per pixel.
Observing µQuad geometry suggests a simple approximation to improve performance. µQuads enlarge along eye-space silhouettes as
~ · V→0
~
N
(see Figure 5). By construction they only elongate in
depth, remaining constant width regardless of surface orientation.
We treat them as 1-dimensional samples. As µQuads elongate we
insert IZB samples along the pixel’s view ray (the µQuad center),
using between 1 and 8 samples.
This creates an approximate irregular z-buffer. As µQuads enlarge
we miss inserting some nodes, introducing light leaks for small
distant occluders falling between samples (i.e., that fail to spawn
needed frustum-triangle tests). Examples of light leaking are provided in our supplementary materials. To reduce these missed tests,
triangles can be over-conservatively rasterized. Our implementation rasterizes light-space triangles with a 1 texel dilation (rather
than a half texel in typical conservative rasterization), ensuring triangles touch more sample locations.
Given the algorithmic complexity in Section 3.1, this tradeoff initially seems questionable. Reducing the number of IZB nodes directly decreases average list length hlizb i, but enlarging conservative raster dilation increases triangle fragment count tf by a smaller
amount. Our approximate insertion averages two IZB nodes per
pixel compared to eight with the exact approach, for a 4× reduction in hlizb i; increasing triangle dilation from 0.5 to 1.0 pixels only
increases tf 6–40%, giving a large net speedup.
We tried compacting our lists, similar to Sintorn et al.’s [2008] variable length arrays. This reduces memory consumption by eliminating next pointers. But our experiments consistently showed a
performance drop of exactly the compaction cost. We also tried
sorting lists by distance to the light; this reduced performance by
roughly the sort cost. This suggests either traversal order has little
impact or we sufficiently load the GPU to hide traversal latency.
But cutting node size is appealing, so we instead eliminated the Gbuffer index. By preallocating a screen-space grid of nodes, the
node address implicitly provides its G-buffer index (see Figure 4);
each node just contains a next pointer. Besides cutting list size in
half, this provides other advantages. Inserting list nodes no longer
requires global synchronization, as we directly map pixels to node
IDs. This roughly halves build cost and removes a memory indirection during list traversal; as addresses directly map to pixels, next
pointers provide information to load the next node and G-buffer
data simultaneously.
For 32 spp shadows we continue using this method, storing up to 8
nodes per pixel. Since we allocate 8 nodes per pixel in advance, this
wastes some memory (roughly doubling the memory of a linked-list
structure). But we felt the improved performance was worth this
moderate memory increase.
6.3
Light-Space Texture Resolution
As in shadow maps, selecting the correct light-space resolution is
important. Unlike shadow maps, resolution does not impact quality
but it does affect performance.
Consider IZB’s O(tf hlizb i) complexity. Halving resolution grows
average list lengths 4× while lowering triangle fragments 4×, suggesting resolution minimally impacts performance. But conservative raster also generates more fragments, and this effect grows for
small triangles and low resolutions. Larger resolutions increase
memory consumption of the head pointer texture, though the number of IZB nodes is largely invariant with light-space resolution.
Considering our goals, we want neither high hlizb i nor lots of fragments testing empty lists. This suggests closely matching lightspace and image resolutions. Early tests showed a 20482 head texture worked well for all scenes at 1080p, though later experiments
(Figure 9) suggest the sweet spot varies from 14002 to 25002 .
6.4
Matching Sampling Rates: Cascades
Ideally, we would match eye- and light-space sampling 1:1 so each
triangle fragment spawns exactly one visibility test. Shadow map
research has explored this sampling problem for decades, suggesting various methods to approach our ideal sampling: perspective
[Stamminger and Drettakis 2002], logarithmic [Lloyd et al. 2008],
cascaded [Lloyd et al. 2006], and sample distribution shadow maps
(SDSMs) [Lauritzen et al. 2011] all improve sampling. For our purposes, perspective shadow maps have hard-to-control singularities
and logarithmic approaches require non-linear rasterization. However, cascades give better sampling with manageable overhead and
SDSMs provide automatic partitioning.
Adding cascades is straightforward, though it affects multiple steps
of our algorithm. Our extents pass not only bounds the entire scene,
but also splits it into multiple cascades with Lauritzen et al.’s [2011]
logarithmic partitioning and individually bounds each cascade.
We generate a separate IZB for each cascade. Creation of cascaded
IZBs for single sample shadows easily occurs in parallel (cascades
contain unique pixels). Cascades for multisample shadows must
overlap slightly to avoid light leaks along boundaries; we serially
generate these IZBs prior to use but expect a careful implementation
could insert samples in multiple cascades in parallel.
Light-space rasterization needs to occur over each IZB to accumulate full visibility. Currently we naively use one render pass per cascade. Culling geometry separately for each light frustum or using
a single render pass to route primitives to the appropriate cascade
would both improve performance.
Except for complex models that naturally fit in one frustum (e.g.,
the hairball), cascades’ significant reduction in GPU divergence
pays for the overhead of rasterizing geometry multiple times. Divergence data with and without cascades is provided as supplementary material.
6.5
~ ·L
~ Culling
N
~ ·L
~ ≤ 0.
Standard lighting models trivially shadow pixels with N
~ ·L
~
Not adding these pixels to the IZB reduces hlizb i. Since the N
term already shadows these pixels, their visibility mask can be left
fully lit. This avoids a common problem along light silhouettes
where geometric and shading normals provide different shadow
terms. This culling consistently improves performance 10-15%.
6.6
Early Out: IZB Node Removal
While testing visibility, a pixel often becomes fully occluded. Triangles rasterized later in the frame can have no additional impact,
so spawning additional frustum-triangle tests is wasteful. This suggests removing occluded pixels from IZB lists (analogous to ray
tracing with “any hit” rays).
Importantly, node removal requires no atomic operations. Race
conditions can occur, but at worst cause extra visibility tests on
already-occluded pixels (after which we retry removal). Node removal provides a 10-15% performance win despite additional logic
and memory operations in the inner traversal loop.
6.7
Memory Synchronization: Unchanged Masks
Visibility tests are inexpensive relative to a GPU’s maximum theoretical performance. Memory latency, throughput, and synchronization thus prove key bottlenecks. One synchronization point
is a pixel’s visibility mask; multiple triangles testing visibility at
the same pixel need to atomically combine results to avoid races.
To reduce contention, mask updates should only occur if a triangle changes the existing visibility. This requires loading the prior
mask, but the avoided contention provides up to a 14% speed boost.
6.8
Latency Hiding: Software Pipelining
When traversing a list of IZB nodes, the inner loop: loads the next
node, loads its pixel data, performs a visibility test, and updates the
visibility mask. This forms a dependency chain, with long memory
latencies between tasks. The GPU may be unable to hide all these
latencies. Fortunately we can apply software pipelining, loading
the next node and computing G-buffer coordinates in the prior loop
Figure 6: The 12.3 million triangle UNC Powerplant model running at 3840 × 2160 in under 140 ms for 32 sample per pixel shadows. We zoom twice to highlight the shadow quality.
iteration. This hides latency and improves speed 5-15%. Pushing
visibility mask updates to the next iteration may be a further win.
7
Results and Discussion
Our system uses OpenGL 4.4 and has been tested on various
NVIDIA GPUs and an AMD Radeon 290X. Timings come from
a GeForce GTX 980. Some results use two extra NVIDIA extensions: NV conservative raster and NV geometry shader pass
through. These results are identified as “GM204 optimized.”
Figures 1, 6 and 7 show our test scenes. Table 1 gives performance
breakdowns using consistent rendering parameters; these do not
provide optimal performance in all scenes, but provide reasonably
high performance across our test suite.
Table 1 shows that bounding scene extents, priming the z-buffer for
culling, and final render all have consistent costs. Computing scene
extents requires a pass over all G-buffer samples running Lauritzen
et al.’s [2011] logarithmic partitioning to delineate cascades; this
cost depends on screen resolution. Section 5.4 notes priming lightspace culling should be unnecessary; however it just requires a blit
to our light-space z-buffer. Our final render loads the G-buffer and
visibility mask and applies a simple shading model.
IZB creation costs vary depending on how many nodes are added to
our linked lists and how much atomic contention occurs. Incoherent
geometry (hairball and tentacles) and complex tessellated models
(GeeBee and powerplant) exhibit slightly lower performance. For
1 sample shadows, we add at most one IZB node per pixel. 32 sample shadows insert roughly twice as many nodes (see Section 6.1),
increasing atomic contention. Still, IZB creation costs remain remarkably consistent across all our test scenes.
Our major cost is light-space rasterization (Section 5.5), which tests
visibility via frustum- or ray-triangle intersections. Theoretically,
cost varies linearly with additional frustum-triangle tests, though
Table 1 shows this correspondence breaks in practice as GPU utilization varies between scenes. Tests of peak throughput show our
GeForce GTX 980 achieves up to 7 million frustum tests per millisecond (Table 2). While we fall short of this ideal, geometry representative of game assets achieves up to 30% this limit. Given the
extra memory latency and logic overhead in our list traversal, we
feel this is quite good. More complex models achieve a much lower
throughput; we expect this is due to reduced culling efficiency for
small triangles (full pixel occlusions occur less frequently and triangles grow by a larger percent with conservative raster).
For single sample shadows, our GM204-optimized code (using
Figure 7: Scenes used to test our system include: Chalmers Villa, GeeBee Plane, Hairball, Epic Citadel, Bungie Terrain, Bungie Building.
Timings give total frame time using four 20482 cascades, rendered at 1920 × 1080. Some scenes courtesy Epic Games and Bungie, Inc.
Scene and
Triangle Count
(sorted by complxeity)
Chalmers Villa, 89 k
Bungie Building, 255 k
Epic Citadel, 374 k
Chalmers Citadel, 613 k
Bungie Terrain, 1.5 M
Hairball, 2.9 M
Tentacles, 3.8 M
GeeBee Plane, 11.7 M
UNC Powerplant, 12.3 M
Individual Step Times for 32 spp (1 spp), in milliseconds
Z-Extent
Create
Z-Cull
Raster Tris
Final
Bounds
IZB
Setup
Over IZB
Render
0.7 (0.4)
0.8 (0.7)
1.7 (0.6)
1.1 (1.1)
2.9 (0.8)
0.3 (0.2)
1.2 (0.6)
0.8 (0.8)
1.7 (0.6)
1.1 (1.1)
7.5 (3.3)
0.4 (0.2)
1.0 (0.5)
0.7 (0.7)
1.6 (0.6)
1.1 (1.1)
7.3 (2.6)
0.3 (0.2)
1.0 (0.6)
0.7 (0.7)
1.6 (0.6)
1.1 (1.1)
11.8 (4.0)
0.4 (0.2)
1.9 (1.0)
0.8 (0.8)
1.9 (0.7)
1.1 (1.1)
19.5 (6.4)
0.4 (0.2)
2.6 (1.5)
0.7 (0.7)
1.9 (0.7)
1.1 (1.1)
51.7 (9.6)
0.3 (0.2)
1.8 (1.2)
0.7 (0.7)
1.7 (0.6)
1.1 (1.1)
55.0 (14.6)
0.3 (0.2)
6.3 (5.1)
0.7 (0.7)
1.9 (0.6)
1.1 (1.1)
152.6 (38.1)
0.3 (0.2)
3.8 (3.3)
0.8 (0.8)
2.1 (0.7)
1.1 (1.1)
67.4 (14.0)
0.4 (0.2)
G-Buf
Total Render
Time (msec)
7.6 (3.7)
12.8 (6.6)
12.0 (5.7)
16.6 (7.2)
25.6 (10.2)
58.3 (13.8)
70.5 (18.4)
162.9 (45.9)
75.6 (20.1)
Light-Space
Tri Frags,
millions
4.6 (1.3)
19.3 (9.7)
10.1 (5.8)
14.1 (7.4)
39.7 (16.0)
77.2 (16.5)
7.1 (1.6)
51.5 (11.9)
11.4 (4.7)
Tri-Pixel
Vis Tests,
millions
6.8 (1.5)
19.2 (8.8)
13.8 (7.1)
20.2 (8.8)
38.8 (14.8)
24.2 (3.9)
7.9 (1.5)
58.7 (9.4)
20.4 (4.6)
Avg. Tests Per
Tri Frag
Pixel
1.5 (1.2)
1.0 (0.9)
1.4 (1.2)
1.4 (1.2)
1.0 (0.9)
0.3 (0.2)
1.1 (0.9)
1.1 (0.8)
1.8 (1.0)
4.1 (0.9)
9.3 (4.3)
8.6 (4.4)
11.2 (4.9)
18.7 (7.1)
15.0 (2.4)
6.8 (1.4)
38.4 (6.1)
10.3 (2.3)
Table 1: Performance breakdown for individual algorithmic steps for 32 spp (and 1 spp) IZB hard shadows. To allow comparison all scenes
use identical settings: 1920 × 1080 resolution, 4 cascades each containing 20482 lists of IZB nodes, and using all our optimizations. Single
sample shadows use our GM204-optimized code path. We provide explicit measures of the work done, including light-space fragment counts
(after culling) and total frustum-triangle (or ray-triangle) tests performed. Based on those counts, we provide averages for visibility tests
performed per light-space fragment and per shaded pixel in the final rendering.
Our Best
Perf
(Tests / ms)
2.3 M
2.6 M
1.9 M
1.7 M
2.0 M
0.5 M
0.3 M
0.5 M
1.5 M
Table 2: Our “speed-of-light” tests. A prepass computes and stores
a list of all necessary frustum-triangle visibility tests. We then
launch a compute pass performing one test per thread, fully utilizing the GPU without divergence. For representative game scenes,
our system achieves throughput up to 75% the speed-of-light.
hardware conservative raster and NVIDIA’s fast geometry shader)
provides up to a 3.5× win over an equivalent software path. For
example, our software light-space rasterization takes 7.2, 52, and
115 ms for the Chalmers Citadel, Tentacles, and GeeBee models.
Hardware conservative rasterization using an extra half-pixel dila-
35
30
25
20
15
Final Render
IZB Raster
Cull Setup
IZB Creation
Scene Extents
G Buffer
10
5
0
0
0
x 216 x 144 x 1080 x 900 x 768 x 720 540 450
3840 2560 1920 1600 1366 1280 960 x 800 x
screen resolution
25
G Buffer
IZB Creation
IZB Raster
20
milliseconds
Chalmers Villa
Bungie Building
Epic Citadel
Chalmers Citadel
Bungie Terrain
Hairball
Tentacles
UNC Powerplant
Average
Speed of Light Tests at 32 spp
Tri-Frustum
Speed of
Tests / ms
Tests
Light (ms)
6.8 M
1.4
4.9 M
20.6 M
6.1
3.4 M
19.9 M
2.9
6.9 M
31.8 M
4.8
6.6 M
40.7 M
9.6
4.2 M
83.0 M
29.8
2.8 M
9.6 M
2.7
3.6 M
37.6 M
9.5
4.0 M
4.6 M
milliseconds
Scene
Scene Extents
Cull Setup
Final Render
15
10
5
0
0
1800
2700
900
frame in animation
3600
Figure 8: Performance variation in the Chalmers Citadel (left) with
varying render resolution and (right) over a 3600 frame animation.
tion would enable these optimizations for our 32-sample shadows,
and we expect a similar performance benefit. Roughly half this
performance win comes from hardware conservative raster, which
generates fewer fragments than our software implementation (using
[Hasselgren et al. 2005]).
Interestingly, Figure 8 shows non-linear performance scaling with
screen resolution. Scaling from 1080p to 2160p roughly reduces
performance by 2× rather than the expected 4×. This likely stems
from our culling, specifically the node removal in Section 6.6,
which reduces the penalty for longer linked lists. As fully occluded
1 spp Shadows
32 spp Shadows
200
compute, they implement a complex space partitioning to achieve
culling similar to our hardware-accelerated z-cull.
40
milliseconds
milliseconds
160
120
80
30
Key differences from prior irregular z-buffers include: our use
of shadow map techniques like cascades to improve performance,
culling via hardware z-cull, IZB node removals to avoid redundant
tests on shadowed pixels, and an efficient way to build and traverse
an IZB for multisample shadows.
20
10
40
0
256
1024
2048
3072
4096
0
256
light space resolution
Chalmers Villa
Bungie Terrain
Bungie Building
Tentacles
1024
2048
3072
4096
light space resolution
Epic Citadel
Powerplant
Chalmers Citadel
GeeBee Plane
Hairball
8
Conclusions and Future Work
Figure 9: Performance variation for (left) 32 spp and (right) 1 spp
shadows with varying cascade resolutions. Timings represent just
the light-space rasterization step, using 4 cascades with resolution
between 2562 and 40962 .
We introduced a system designed to render subpixel accurate shadows. This system runs in real-time for modern game content and
interactively for more complex CAD models, in both cases at HD
or higher resolutions. Adding multiple samples for subpixel visibility only costs slightly more than a single sample.
pixels are quickly eliminated, increasing resolution along epipolar
lines affects performance sublinearly.
While we designed primarily for accuracy, we introduced one optimization for multisample shadows that can introduce light leaks
~ ·V
~ near 0. We believe
when small triangles shadow pixels with N
a modified light space parameterization could eliminate this leaking, but developers needing quality could skip this optimization.
Figure 9 tracks performance under varying light-space resolution.
Larger resolutions distribute IZB nodes among more lists (reducing average list length), but also cause rasterization to emit more
fragments. For single sample shadows, these effects largely cancel
out and performance remains consistent for sufficiently large cascades. For multisample shadows, higher resolutions cause µQuads
to project to more texels (requiring additional IZB nodes); this distinctly lowers performance at higher resolutions. The sweet spot
ranges from 14002 to 25002 , and in moderately complex scenes
performance varies only slightly through this range.
Consider shadow map aliasing. A shadow map texel aliases when
it projects to multiple pixels. By construction, an irregular z-buffer
lists these same pixels in a single texel. As sampling mismatches
increase, shadow maps alias further and IZB lists lengthen. For
standard shadow map problem cases, e.g., the shadowed ground
plane at sunset, our lists can become quite large. Increasing resolution and better matching via warping or cascades addresses this
problem by splitting these long lists.
One can view an IZB as a ray-triangle acceleration structure, similar to a ray tracer’s bounding volume hierarchy. Given these intersections are the key cost, we asked how many tests our system
performs compared to a ray tracer. We implemented our frustumtracing algorithm in a beam tracer using state-of-the-art acceleration structures [Aila et al. 2013] and compared the number of
frustum-triangle tests required. Figure 10 compares these numbers
with our results from Table 1. Our system tested 0.3× to 2.6× as
many triangles per pixel as the ray tracer, surprisingly well given
our IZB construction costs only 2 ms. For validation, we compare
quality with a multisample OptiX ray tracer and show costs over an
animated sequence.
7.1
Specific Comparisons to Prior Work
Beyond ray tracers, our work shares similarities with other techniques. Lecocq et al. [2014] and Sen et al. [2003] augment a shadow
map with geometry, which can give subpixel accurate shadows. But
they fail in complex texels when multiple triangles or silhouettes intersect. Irregular z-buffers seamlessly handle these cases.
Despite the name, we share similarities with per-triangle shadow
volumes [Sintorn et al. 2011; Sintorn et al. 2014]; our light-space
rasterization and traversal essentially spawns per-triangle shadow
volumes, but Sintorn et al.’s data structure more closely resembles
Aila and Laine’s [2004] space partitioning. Because they use GPU
We hope our work spurs further exploration into real-time analytic
shadow algorithms. While we feel our performance impressive, a
number of improvements still remain: a more programmable depth
test could enable better culling during light space rasterization, applying conservative rasterization to all triangles seems wasteful (but
identifying silhouettes robustly is challenging), and extensions to
soft shadows increase culling challenges.
Acknowledgments
Thanks to Bungie, Epic Games, and Erik Sintorn for generously
sharing models and other assets. Many others at NVIDIA provided
helpful suggestions, discussion, and brainstorming throughout this
project, including: Anjul Patney, Henry Moreton, Cyril Crassin,
Dave Luebke, Marco Salvi, Eric Enderton, and Craig Kolb.
References
A ILA , T., AND L AINE , S. 2004. Alias-free shadow maps. In Proc.
Eurographics Symposium on Rendering, 161–166.
A ILA , T., K ARRAS , T., AND L AINE , S. 2013. On quality metrics of bounding volume hierarchies. In Proc. High-Performance
Graphics, 101–107.
A NNEN , T., M ERTENS , T., S EIDEL , H.-P., F LERACKERS , E.,
AND K AUTZ , J. 2008. Exponential shadow maps. In Proc.
Graphics Interface, 155–161.
A RVO , J. 2007. Alias-free shadow maps using graphics hardware.
Journal of Graphics Tools 12, 1, 47–59.
B OULOS , S., E DWARDS , D., L ACEWELL , J. D., K NISS , J.,
K AUTZ , J., WALD , I., AND S HIRLEY, P. 2007. Packet-based
Whitted and Distribution Ray Tracing. In Proc. Graphics Interface, 177–184.
C ARPENTER , L. 1984. The a-buffer, an antialiased hidden surface
method. In Proceedings of SIGGRAPH, 103–108.
C ROW, F. 1977. Shadow algorithms for computer graphics. In
Proceedings of SIGGRAPH, 242–248.
D ONNELLY, W., AND L AURITZEN , A. 2006. Variance shadow
maps. In Symposium on Interactive 3D Graphics and Games,
161–165.
500
OptiX
milliseconds
400
Scene
300
OptiX Prime
200
100
Frustum Traced IZB
Chalmers Villa
Bungie Building
Epic Citadel
Chalmers Citadel
Bungie Terrain
Ray Tracing Numbers
BVH Nodes
Tri-Frag
Avg Tests
Traversed
Vis Tests
Per Pixel
11.0 M
10.7 M
14.6
15.1 M
14.6 M
8.8
14.3 M
14.1 M
9.3
11.6 M
11.3 M
7.3
12.9 M
12.4 M
7.1
32 spp IZB
Avg Tests
Per Pixel
4.1
9.3
8.6
11.2
18.7
0
0
180
360
frame in animation
Figure 10: (Left) Compare our speed over an animation with a 32 spp OptiX ray tracer (on a Quadro K6000) and a quality comparison from
one frame. (Right) The number of frustum-triangle intersections in a beam tracer (with a state-of-the-art BVH) compared with our system.
E ISEMANN , E., S CHWARZ , M., A SSARSSON , U., AND W IMMER ,
M. 2011. Real-Time Shadows. A. K. Peters, Ltd.
F ERNANDO , R., F ERNANDEZ , S., BALA , K., AND G REENBERG ,
D. 2001. Adaptive shadow maps. In Proceedings of SIGGRAPH,
387–390.
PAN , M., WANG , R., C HEN , W., Z HOU , K., AND BAO , H. 2009.
Fast, sub-pixel antialiased shadow maps. Computer Graphics
Forum 28, 7, 1927–1934.
R EEVES , W., S ALESIN , D., AND C OOK , R. 1987. Rendering
antialiased shadows with depth maps. In Proceedings of SIGGRAPH, 283–291.
H ASSELGREN , J., A KENINE -M OLLER , T., AND O HLSSON , L.
2005. GPU Gems 2. Addison-Wesley, ch. Conservative Rasterization, 677–690.
S CHWARZ , M., AND S TAMMINGER , M. 2007. Bitmask soft shadows. Computer Graphics Forum 26, 3, 515–524.
H ECKBERT, P. S., AND H ANRAHAN , P. 1984. Beam tracing
polygonal objects. In Proceedings of SIGGRAPH, 119–127.
S EN , P., C AMMARANO , M., AND H ANRAHAN , P. 2003. Shadow
silhouette maps. ACM Trans. Graph. 22, 3 (July), 521–526.
J OHNSON , G. S., L EE , J., B URNS , C. A., AND M ARK , W. R.
2005. The irregular z-buffer: Hardware acceleration for irregular
data structures. ACM Trans. Graph. 24, 4 (Oct.), 1462–1482.
L AURITZEN , A., S ALVI , M., AND L EFOHN , A. 2011. Sample distribution shadow maps. In Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D ’11, 97–102.
L ECOCQ , P., M ARVIE , J.-E., S OURIMANT, G., AND G AUTRON ,
P. 2014. Sub-pixel shadow mapping. In Proceedings of the
Symposium on Interactive 3D Graphics and Games, 103–110.
L EE , W.-J., S HIN , Y., L EE , J., K IM , J.-W., NAH , J.-H., J UNG ,
S., L EE , S., PARK , H.-S., AND H AN , T.-D. 2013. Sgrt: A
mobile gpu architecture for real-time ray tracing. In Proc. HighPerformance Graphics, 109–119.
L LOYD , D. B., W ENDT, J., G OVINDARAJU , N., AND
M ANOCHA , D. 2004. CC shadow volumes. In Proc. Eurographics Symposium on Rendering, 197–206.
L LOYD , D. B., T UFT, D., YOON , S.- E ., AND M ANOCHA , D.
2006. Warping and partitioning for low error shadow maps. In
Proc. Eurographics Symposium on Rendering, 215–226.
L LOYD , D. B., G OVINDARAJU , N. K., Q UAMMEN , C., M OL NAR , S. E., AND M ANOCHA , D. 2008. Logarithmic perspective
shadow maps. ACM Trans. Graph. 27, 4, 106:1–106:32.
M C G UIRE , M., H UGHES , J. F., E GAN , K., K ILGARD , M., AND
E VERITT, C. 2003. Fast, practical and robust shadows. Tech.
rep., NVIDIA Corporation, Austin, TX, Nov.
M ITTRING , M., 2014. Real-time ray traced shadows. http://
kosmokleaner.wordpress.com/2014/09/26/.
OVERBECK , R., R AMAMOORTHI , R., AND M ARK , W. R. 2007.
A real-time beam tracer with application to exact soft shadows.
In Proc. Eurographics Symposium on Rendering, 85–98.
S INTORN , E., E ISEMANN , E., AND A SSARSSON , U. 2008. Sample based visibility for soft shadows using alias-free shadow
maps. Computer Graphics Forum 27, 4, 1285–1292.
S INTORN , E., O LSSON , O., AND A SSARSSON , U. 2011. An
efficient alias-free shadow algorithm for opaque and transparent
objects using per-triangle shadow volumes. ACM Trans. Graph.
30, 6, 153:1–153:10.
S INTORN , E., K ÄMPE , V., O LSSON , O., AND A SSARSSON , U.
2014. Per-triangle shadow volumes using a view-sample cluster
hierarchy. In Proceedings of the Symposium on Interactive 3D
Graphics and Games, 111–118.
S TAMMINGER , M., AND D RETTAKIS , G. 2002. Perspective
shadow maps. In Proceedings of SIGGRAPH, 557–562.
W HITTED , T. 1980. An improved illumination model for shaded
display. Communications of the ACM 23, 6 (June), 343–349.
W ILLIAMS , L. 1978. Casting curved shadows on curved surfaces.
In Proceedings of SIGGRAPH, 270–274.
W OO , A., AND P OULIN , P. 2012. Shadow Algorithms Data Miner.
A. K. Peters/CRC Press.
YANG , J. C., H ENSLEY, J., G R ÜN , H., AND T HIBIEROZ , N.
2010. Real-time concurrent linked list construction on the gpu.
Computer Graphics Forum 29, 4, 1297–1304.