<![CDATA[Self Shadow]]>http://blog.selfshadow.com/favicon.png2013-05-13T09:52:02-04:00http://blog.selfshadow.com/Octopress<![CDATA[Counting Quads]]>2012-11-12T22:59:00-05:00http://blog.selfshadow.com/2012/11/12/counting-quadsThis is a DX11 followup to an earlier article on quad ‘overshading’. If you’ve already read that, then feel free to skip to the meat of this post.
Recount
As you likely know, modern GPUs shade triangles in blocks of 2x2 pixels, or quads. Consequently, redundant processing can happen along the edges where there’s partial coverage, since only some of the pixels will end up contributing to the final image. Normally this isn’t a problem, but – depending on the complexity of the pixel shader – it can significantly increase, or even dominate, the cost of rendering meshes with lots of very small or thin triangles.
Figure 1: Quad overshading, the silent performance killer
For more information, see Fabian Giesen’s post, plus his excellent series in general.
It’s hardly surprising, then, that IHVs have been advising for years to avoid triangles smaller than a certain size, but that’s somewhat at odds with game developers – artists in particular – wanting to increase visual fidelity and believability, through greater surface detail, smoother silhouettes, more complex shading, etc. (As a 3D programmer, part of my job involves the thrill of being stuck in the middle of these kinds of arguments!)
Traditionally, mesh LODs have helped to keep triangle density in check. More recently, deferred rendering methods have sidestepped a large chunk of the redundant shading work, by writing out surface attributes and then processing lighting more coherently via volumes or tiles. However, these are by no means definitive solutions, and nascent techniques such as DX11 tessellation and tile-based forward shading not only challenge the status quo, but also bring new relevancy to the problem of quad shading overhead.
Knowing about this issue is one thing, but, as they say: seeing is believing. In a previous article, I showed how to display hi-z and quad overshading on Xbox 360, via some plaform-specific tricks. That’s all well and good, but it would be great to have the same sort of visualisation on PC, built into the game editor. It would also be helpful to have some overall stats on shading efficiency, without having to link against a library (GPUPerfAPI, PerfKit) or run a separate tool.
There are several ways of reaching these modest goals, which I’ll cover next. What I’ve settled on so far is admittedly a hack: a compromise between efficiency, memory usage, correctness and simplicity. Still, it fulfils my needs so far and I hope you find it useful as well.
Going To Eleven
First, let’s restate the problem: what we want, essentially, is to count up the number of times we shade a given screen quad. The trick is to only count each shading quad once.
The way I achieved this on Xbox 360 hinged on knowing whether a given pixel was ‘alive’ or not, and then only accumulating overdraw for the first live pixel in each shading quad. As far as I’m aware, there’s no official way of detemining this on PC through standard graphics APIs, but some features of DX11 – namely Unordered Access Views (UAVs) and atomic operations – will allow us to arrive at the same result via a different route.
The right way
What I was after was an implementation that was as simple as before, involving three steps:
Render depth pre-pass (optional; do whatever the regular rendering path does for this)
Render scene (material/lighting passes) with special overdraw shader
Display results
A straightforward, safe option is to gather a list of triangles per screen quad, filtering by ID (a combination of SV_PrimitiveID and object ID). This filtering can be performed during the overdraw pass or as a post-process.
What’s unsatisfying with this approach is that it involves budgeting memory for the worst case, or accepting an upper bound on displayable overdraw. Whilst I can imagine that a multi-pass variation is doable, that just adds unwanted complexity to what ought to be a simple debug rendering mode.
The wrong way
So, in order to overcome these limitations, I started toying around with something a lot simpler:
The intent here is to use a UAV to keep track of the current triangle per screen quad. Through InterlockedExchange, we both update the ID and use the previous state to determine if we’re the first pixel to write this ID (prevID != id). If so, we increment an overdraw counter in a second UAV. This is similar in the spirit to the Xbox 360 version, in that we’re selecting one of the live pixels in a shading quad to update the overdraw count. Finally, we can display the results in a fullscreen pass:
On paper, this appears to elegantly avoid the storage and complexity of the previous approach. Alas, it relies on one major, dubious assumption: that quads are shaded sequentially! In reality, GPUs process pixels in larger batches of warps/wavefronts and there’s no guarantee that UAV operations are ordered between quads – hence the name: unordered. So, during the shading of pixels in a quad for one triangle, it’s perfectly possible for another unruly triangle to stomp over the quad ID and break the whole process!
The cheat’s way
Fortunately, we can get around this issue with a few modifications. The basic idea here is to loop and use InterlockedCompareExchange to attempt to lock the screen quad:
RWTexture2D<uint>lockUAV:register(u0);RWTexture2D<uint>overdrawUAV:register(u1);[earlydepthstencil]voidOverdrawPS(float4vpos:SV_Position,uintid:SV_PrimitiveID){uint2quad=vpos.xy*0.5;uintprevId;uintunlockedID=0xffffffff;boolprocessed=false;intlockCount=0;for(inti=0;i<16;i++){if(!processed)InterlockedCompareExchange(lockUAV[quad],unlockedID,id,prevID);[branch]if(prevID==unlockedID){// Wait a bit, then unlock for other quadsif(++lockCount==2)InterlockedExchange(lockUAV[quad],unlockedID,prevID);processed=true;}if(prevID==id)processed=true;}if(lockCount)InterlockedAdd(overdrawUAV[quad],1);}
This leads to three outcomes for unprocessed pixels:
If prevID == unlockedID, then the pixel holds the lock for its shading quad
If prevID == id, another pixel in the shading quad holds the lock
Otherwise, no pixel in the shading quad holds the lock
In the first case we mark the pixel as processed and increment a lock counter. After an additional iteration, we release the lock. This ensures that pixels with the same ID see the state of the lock (second case), so that they can be filtered out. Finally, pixels that held the lock update the quad overdraw.
Ideally we’d loop until the pixel has been tagged as processed, but I haven’t had success with current NVIDIA drivers and UAV-dependent flow control, i.e.:
As a workaround, I’ve simply set the iteration count to a number that works well in practice across NVIDIA and AMD GPUs (those that I’ve had a chance to test, anyway).
Four, Three, Two, One
Now that we have a working system in place, it’s easy to gather other stats. For instance, although we can’t determine directly if a pixel is alive, we can count the number of live pixels in each shading quad, since Interlocked* operations are masked out for dead pixels. With this, we can tally up the number of quads with 1 to 4 live pixels in yet another UAV:
RWTexture2D<uint>lockUAV:register(u0);RWTexture2D<uint>overdrawUAV:register(u1);RWTexture2D<uint>liveCountUAV:register(u2);RWTexture1D<uint>liveStatsUAV:register(u3);[earlydepthstencil]voidOverdrawPS(float4vpos:SV_Position,uintid:SV_PrimitiveID){uint2quad=vpos.xy*0.5;uintprevID;uintunlockedID=0xffffffff;boolprocessed=false;intlockCount=0;intpixelCount=0;for(inti=0;i<64;i++){if(!processed)InterlockedCompareExchange(lockUAV[quad],unlockedID,id,prevID);[branch]if(prevID==unlockedID){if(++lockCount==4){// Retrieve live pixel count (minus 1) in quadInterlockedAnd(liveCountUAV[quad],0,pixelCount);// Unlock for other quadsInterlockedExchange(lockUAV[quad],unlockedID,prevID);}processed=true;}if(prevID==id&&!processed){InterlockedAdd(liveCountUAV[quad],1);processed=true;}}if(lockCount){InterlockedAdd(overdrawUAV[quad],1);InterlockedAdd(liveStatsUAV[pixelCount],1);}}
To my surprise, incrementing a 4-wide UAV didn’t lead to a massive slowdown here. That said, one can certainly use a number of buckets for intermediate results (indexed by the lower bits of the screen position, for instance), if this proves to be a problem.
With these numbers, it’s trivial to add a pie chart to the final pass:
Figure 2: Quad overdraw (dark blue = 1x, to green = 4x), and proportion of live pixels per quad (yellow = 4, to dark red = 1)
Demolition
For your convenience, I’ve packaged things up into a simple demo. Please let me know if you hit any compatibility issues, or come up with any enhancements.
As with last year, I’m gathering links to SIGGRAPH content, to complement Ke-Sen Huang’s invaluable Technical Papers list. Please let me know if you have anything to add to the list.
]]><![CDATA[Blending in Detail]]>2012-07-10T02:40:00-04:00http://blog.selfshadow.com/2012/07/10/blending-in-detail
+
=
I’ve added a new article, Blending in Detail, written together with Colin Barré-Brisebois, on the topic of blending normal maps. We go through various techniques that are out there, as well as a neat alternative (“Reoriented Normal Mapping”) from Colin that I helped to optimise.
This is by no means a complete analysis – particularly as we focus on detail mapping – so we might return to the subject at a later date and tie up some loose ends. In the meantime, I hope you find the article useful. Please let us know in the comments!
]]>
<![CDATA[Travelling Without Moving]]>2012-04-11T23:44:00-04:00http://blog.selfshadow.com/2012/04/11/travelling-without-movingLately I’d been getting increasingly frustrated with the limitations of WordPress(.com), so I longed for a change. With the Easter weekend, I finally had a little extra time and energy to make the switch to Octopress, plus a dedicated web host. Hopefully that’ll encourage me to start posting again, or at least remove one major grumble. I’m also looking forward to such liberties as the ability to embed WebGL, though I can’t entirely promise that I’ll wield such power responsibly.
Existing post URLs remain the same, but if you’re one of the illustrious few who subscribe to the blog via RSS, I’m guessing that you’ll need to change over to the new feed.Update: I’m redirecting the old feed URL now, so everything should be back to normal! Speaking of RSS, as I’m now using MathJax for $\LaTeX$, it appears that I’ll need to implement a fallback there, in addition to tracking down a rendering issue with Chrome. Please let me know if you spot any other oddities.
]]><![CDATA[Righting Wrap (Part 2)]]>2012-01-07T21:07:12-05:00http://blog.selfshadow.com/2012/01/07/righting-wrap-part-2Wrapping Paper
I first tinkered with SH wrap shading (as described in part 1) for Splinter Cell: Conviction, since we were using a couple of models [1][2] for some character-specific materials. Unfortunately, due to the way that indirect character lighting was performed, it would have required additional memory that we couldn’t really justify at that point in development. Consequently, this work was left on the cutting room floor and I only got as far as testing out Green’s model [1].
Recently, however, I spotted that Irradiance Rigs [3] covers similar ground. At the very end of the short paper, they briefly present a generalisation of Valve’s Half Lambert model [2] and the SH convolution terms for the first three bands:
This tidily combines the tunability of [1] with the tighter falloff of [2], albeit at the cost of a few extra instructions in the case of direct lighting. It’s not energy-conserving though, so for kicks I went through the maths – see appendix – and made the necessary adjustments:
I would suggest this as a good workout if your calculus skills are a little on the rusty side; think of it as a much-needed trip to the maths gym: sure it’s going to hurt at first, but you’ll feel better afterwards!
The same authors have since written a more in-depth paper, Wrap Shading [4], which Derek Nowrouzezahrai has kindly made available here. I recommend checking it out, since there’s some nice analysis and plenty of background information. One notable insight is that their model is perfectly represented by 3rd-order SH when $a = 1$ (i.e, Half Lambert). This becomes clear when you consider that the model is effectively unclamped in that case, so appropriate scaling of the constant, linear and quatratic bands ($y_{0}^{0}, y_{1}^{0}, y_{2}^{0}$) will match the function:
A similar observation can be made with Green’s model: it’s perfectly represented by 2nd-order SH when $a = 1$.
Shrink Wrap
But wait, at the end of the part 1, didn’t I promise that there would be a discussion of optimisation in this post? You’re quite right. Well, it just so happens that a snippet of reference shader code from this last paper makes for a neat little case study on improving shader performance.
Reference Version
This is pretty much the reference implementation for generating the normalised convolution terms of their generalised model:
12345678910111213
float3GeneralWrapSH(floatfA){// Normalization factor for our model.floatnorm=0.5*(2+fA)/(1+fA);float4t=float4(2*(fA+1),fA+2,fA+3,fA+4);returnnorm*float3(t.x/t.y,2*t.x/(t.y*t.z),t.x*(fA*fA-t.x+5)/(t.y*t.z*t.w));}float4main(floatfA:TEXCOORD):COLOR{returnGeneralWrapSH(fA).xyzz;}
The only thing that I’ve changed – beyond adding calling code – is to pass in the wrap parameter fA from the vertex shader. It was previously a user-supplied constant, which doesn’t make for a particularly credible example, since in that case all of the maths could simply be moved to the CPU and performed just the once!
Note that there’s been some attempt to pull out common terms, particularly for the final component, where instead of fA*fA - 2*fA + 3 (see $\mathbf{f}$) we now have fA*fA - t.x + 5.
Without further ado, let’s see how this stacks up in terms of ps_3_0 instructions:
123456789101112131415161718192021222324252627
//
// Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111
//
// fxc /Tps_3_0 /O3 shader.hlsl
//
ps_3_0
def c0, 2, 1, 3, 4
def c1, 0.5, 4, 5, 0
dcl_texcoord v0.x
add r0, c0, v0.x
add r1.x, r0.y, r0.y
mad r1.y, v0.x, v0.x, -r1.x
add r1.y, r1.y, c1.z
mul r1.y, r1.y, r1.x
mul r0.z, r0.z, r0.x
mul r0.w, r0.w, r0.z
rcp r0.z, r0.z
rcp r0.w, r0.w
mul r2.zw, r0.w, r1.y
mul r1.yz, r0.xxyw, c1.xxyw
rcp r0.y, r0.y
rcp r0.x, r0.x
mul r2.xy, r0.xzzw, r1.xzzw
mul r0.x, r0.y, r1.y
mul oC0, r2, r0.x
// approximately 16 instruction slots used
Ouch! 16 is fairly substantial, but perhaps not all that surprising going by the HLSL. Since this is device-independent assembly, I decided to check the ALU count on Xbox 360 for comparison. In that case it’s a somewhat more reasonable 10 operations, because 5 scalar ops get dual-issued with vector ops. So, in summary, we have:
DX9: 16, X360: 10(+5) ALU ops
Cancellation
Immediately, a simple but very effective change we can make is to cancel through by the normalisation term, which leaves us with $\mathbf{\hat{f}}$ directly:
Don’t expect the compiler to do intelligent optimisations like this; constant folding yes, factoring sometimes, sophisticated symbolic manipulation? Good luck!
For instance, even seemingly ‘obvious’ opportunities like (a/b)/(b/a) will go unnoticed by FXC. This isn’t down to the compiler trying to maintain special-case behaviour such as divide by zero either, because it will happily replace a/a with 1 in the absence of any knowledge about the value of a.
Apologies if that was already perfectly clear and all I’ve done is insult your intelligence, but I’ve seen some people blithely leave everything up to the compiler and not scrutinise what it’s generating. Of course, high-level algorithmic optimisations are hugely important as well, but so is this lower-level stuff when a shader is being executed for millions of pixels!
Just look at what this small amount of effort has netted us:
DX9: 10, X360: 5(+3) ALU ops
Factorisation
Next we can factor fA*fA - 2*fA + 3 again – this time as (fA + 1)(fA + 3) - 6*fA – to reduce the numerator of the third term to a single multiply-add:
I’ve also taken the opportunity to manually vectorise the addition of fA, plus a subsequent pair of multiplications between resulting terms. In fact, the compiler does this anyway, as it’s relatively good at vectorising code. Still, one shouldn’t assume that it will always get things right!
Whether there’s a gain or not, manual vectorisation – which is often quick to do – makes it easier to sanity check the output assembly. Just scanning through, you might expect add, mul, mov, rcp, mul, mad, rcp, mul and you’d be pretty much spot on.
So, for DX9 we’ve reduced the op count by 2, but what about Xbox 360? Here, we’ve only succeeded in shaving off one paired scalar op. However, this may turn into a real gain once the function is part of a larger shader.
DX9: 8, X360: 5(+2) ALU ops
Rescaling
This next trick involves rescaling so that the second term becomes 1/t.y, or a single rcp:
You might wonder why I’m using an external constant here. Well, it turns out that FXC will misoptimise when it knows the values. Bad compiler! Again, there’s one less paired scalar op on Xbox 360:
DX9: 7, X360: 5(+1) ALU ops
Expansion
Rather than factoring terms, we could have expanded $\mathbf{\hat{f}}$ instead:
This is a win for ps_3_0 but not for Xbox 360, as it removes the opportunity for pairing. It’s possible that some clever variation could fix this, but it doesn’t matter because we haven’t exhausted our optimisation options…
DX9: 6, X360: 6 ALU ops
Fitting
There are potentially significant gains to be had from numerical fitting, so it’s worth taking the time familiarise yourself with the various techniques, maths packages and libraries out there.
In this instance, I’m performing a cubic fit – i.e. $ax^3 + bx^2 + cx + d$ – for the 2nd and 3rd bands. Polynomials are attractive for performance because they can be efficiently evaluated as a series of mad instructions when written in Horner form: $x(x(ax + b) + c) + d$
With careful vectorisation, this collapses to the following:
Xbox 360 does all this in one less operation because placing 1 into r.x can be achieved with a destination register modifier:
DX9: 4, X360: 3 ALU ops
I could present graphs showing how the cubic approximations fare, but take it from me that they are extemely close. In fact, we can arguably drop down to a quadratic fit and save a further mad in the process. This is still acceptable:
Figure 1: Comparison between original and quadratic fit for 2nd and 3rd bands (left, right)
In both cases – cubic and quadratic – I’ve actually constrained the fitting process so that the curves go through the endpoints. This reduces the worst case error a little and maintains the nice property of exactness when $a = 1$. Of course, something has to give and so the average error is a little higher.
In practice, this quadratic approximation has little effect on the end result. When lighting with a single directional source – a worst-case scenario – the difference is slight and far less significant than the error that comes from using 3rd-order SH in the first place.
And yet, we’re still not done! The DX9 figure suggests that we might pay the instruction cost of moving 1 into r.x with some GPUs, and although it could go away when the terms are actually used, it would be cute if we could get rid of it just in case.
Notice that the two curves are monotonically decreasing and within the range [0, 1]. If we negate the intermediate result of the first mad, saturate and then negate again, there will be no overall effect. By doing this, we can take r.x along for the ride and force it to 0 through one of the negative constants, then add 1 via the final mad:
Here’s a WebGL sample that encapsulates this mini-series on wrap shading.
In conclusion, shader optimisation is critical for video game rendering, so you shouldn’t defer to the compiler. To quote Michael Abrash: “The best optimizer is between your ears”. Don’t forget it, train it!
]]><![CDATA[Righting Wrap (Part 1)]]>2011-12-31T18:39:22-05:00http://blog.selfshadow.com/2011/12/31/righting-wrap-part-1A while back, Steve McAuley and I were discussing physically-based rendering miscellanea over a quiet pint – hardly a stretch of the imagination, since we’re English 3D programmers after all! Anyway, it turned out that we both had plans to write up a few thoughts in relation to wrap shading, and, following some gentle arm-twisting, Steve has posted his. I suggest that you go and read that first if you haven’t already, then return here for a continuation of the subject.
Bad Wrap
Wrap shading has its uses when more accurate techniques are too expensive, or simply to achieve a certain aesthetic, but common models [1][2] have some deficiencies out of the box. Neither of these is energy conserving and they don’t really play well with shadows either. On top of that, Valve’s Half Lambert model [2] has a fixed amount of wrap, so it can’t be tuned to suit different materials (or, perhaps, to limit shadow oddities). I’ll come back to the point about flexibility in part 2, but first I’d like to discuss another factor that’s easily overlooked: environmental lighting.
If you’re set on using some form of wrap shading, then it’s not just a matter of applying it to your standard direct sources – directional, point and spot lights, for instance – it ought to be carried through to environmental lighting as well! Naturally, the importance of this depends on how strong and directional your secondary lighting is; obviously if you’re only using constant ambient then there’s no problem, but these days it’s fairly common to encode indirect lighting in Spherical Harmonics (SH) [3] and perhaps some additional lights as well. Fortunately, wrap shading in the context of SH lighting is easy, and much like energy conservation it’s a relatively cheap or free addition, so it’s worth considering even if the results prove to be subtle.
Looking After the Environment
So, how do we accomplish this? Well, that’s best explained with a quick recap. If you recall, for diffuse SH lighting, we first project the lighting environment, $f$, into SH:
Finally, a division by $\pi$ gives us outgoing (or exit) radiance. Personally, I find it convenient to roll these extra terms into $h$ itself. The nice thing about this is that the convolution kernel then boils down (via analytical integration) to easy to remember values for the first three SH bands:
For further details, you can find a complete and approachable account in [4].
Now, back to wrap: adjusting things for our shading model of choice is simply a matter of replacing $\hat{h}$. Let’s try this for the simple wrap model from Green [1] that Steve already discussed:
From Steve’s post, we know that we need an additional normalisation factor of $1 + w$ for energy conservation, so the full formula for our new convolution, which I’ll call $\hat{g}$, is:
You can go through a similar process of analytical integration as Steve did, only now with the additional SH basis terms $y_{l}^{0}$, or if you’re lazy like me, you can throw the formula at a package like Mathematica. Either way, once you’re done, you’ll arrive at the following (or something equivalent):
We can clearly see that this reduces to the cosine convolution kernel, $\hat{h}$, when $w = 0$. In visual terms, the effect of changing $w$ is evident with a single directional light, as you would expect:
Figure 1: Variable wrap shading (0, 0.5, 1) with a single directional light in SH
On the other hand, the difference is a lot subtler with a more uniform lighting environment:
Figure 2: Variable wrap shading (0, 0.5, 1) with a general SH environment (Grace Cathedral)
It’s a Wrap?
Okay, confession time: this post wasn’t just about wrap shading, since it also serves as a foundation for future posts. For instance, part 2 will conveniently segue into shader optimisations, which is a topic I’ve been planning to write – or, perhaps more accurately, rant – about in general, and I’ll also be returning to Spherical Harmonics down the line.
Figure 1: Major axes for original (left), swizzle (mid) and perpendicular (right) vectors
Introduction
Two months ago, there was a question (and subsequent discussion) on Twitter as to how to go about generating a perpendicular unit vector, preferably without branching. It seemed about time that I finally post something more complete on the subject, since there are various ways to go about doing this, as well as a few traps awaiting the unwary programmer.
Solution Quartet
Here are four options with various trade-offs. If you happen to know of any others, by all means let me know and I’ll update this post.
Note: in all of the following approaches, normalisation is left as an optional post-processing step.
Quick ‘n’ Dirty
A quick hack involves taking the cross product of the original unit vector – let’s call it $\mathbf{u}(x, y, z)$ – with a fixed ‘up’ axis, e.g. $(0, 1, 0)$, and then normalising. A problem here is that if the two vectors are very close – or equally, pointing directly away from each other – then the result will be a degenerate vector. However, it’s still a reasonable approach in the context of a camera, if the view direction can be restricted to guard against this. A general solution in this situation is to fall back to an alternative axis:
Listing 1: A quick way to generate a perpendicular vector
Hughes-Möller
In a neat little Journal of Graphics Tools paper [1], Hughes and Möller proposed a more systematic approach to computing a perpendicular vector. Here’s the heart of it:
Or, as the paper also states: “Take the smallest entry (in absolute value) of $\mathbf{u}$ and set it to zero; swap the other two entries and negate the first of them”.
Figure 2: Distribution of v over the sphere
However, there’s a problem with this as written: it doesn’t handle cases where multiple components are the smallest, such as $(0, 0, 1)$! I hit this a few years back when I needed to generate an orthonormal basis for some offline geometry processing, and it’s easily remedied by replacing $<$ with $\leq$. Here’s a corrected version in code form:
More recently, Michael M. Stark suggested some improvements to the Hughes-Möller approach [2]. Firstly, his choice of permuted vectors is almost the same – differing only in signs – but even easier to remember:
$$
\mathbf{\bar{v}} = \begin{pmatrix} x \\ y \\ z \end{pmatrix} \times
\begin{pmatrix}
\left[x \text{ is smallest}\right] \\
\left[y \text{ is smallest}\right] \\
\left[z \text{ is smallest}\right]
\end{pmatrix}
$$
In plain English: the perpendicular vector $\mathbf{\bar{v}}$ is found by taking the cross product of $\mathbf{u}$ with the axis of its smallest component. (Note: the same care is needed when multiple components are the smallest.)
Figure 3 visualises this intermediate ‘swizzle’ vector over the sphere:
Figure 3: Intermediate swizzle vector
Secondly, Michael also provides a branch-free implementation:
I know what you’re thinking, there’s a problem here too: it should be zm = 1^(xm | ym)! Although still robust, the effect of this error is that the nice property of even, symmetrical distribution over the sphere is lost:
Figure 4: Broken symmetry in the perpendicular vector(that’s 7 years bad luck!)
XNAMath
Finally, another branch-free solution is provided in the form of XMVector3Orthogonal, which is part of the XNAMath library. Here’s the actual code taken from the DirectX SDK (June 2010):
I’ve failed, thus far, to pinpoint the origin of or thought process behind this approach. That said, some insight can be gained from visualising the resulting vectors:
Figure 5: Covering one’s axis
Their maximum component is in the x axis, except close to the +/-ve x poles. Essentially, Microsoft’s solution ensures robustness without concern for distribution, much like the initial ‘quick’ approach.
Performance
I haven’t benchmarked these implementations, since in cases where I’ve needed to generate perpendicular vectors, absolute speed wasn’t important or the call frequency was vanishingly small. Even in performance-critical situations, it really depends on what properties/restrictions you can live with and your target architecture(s). Still, I can’t help but think that XMVector3Orthogonal is doing a little bit more than it needs to, so maybe there’s cause to revisit this subject at a later date.
Conclusion
I hope you’ve learnt something about generating perpendicular vectors, or that I’ve at least made you aware of some of the minor issues in previous work on the subject. On that note, if you spot any new errors here, please let me know!
References
[1] Hughes, J. F., Möller, T., “Building an Orthonormal Basis from a Unit Vector”, Journal of Graphics Tools 4:4 (1999), 33-35.
[2] Stark, M. M., “Efficient Construction of Perpendicular Vectors without Branching”, Journal of Graphics Tools 14:1 (2009), 55-61.
]]><![CDATA[Hidden Costs]]>2011-10-01T18:56:14-04:00http://blog.selfshadow.com/2011/10/01/hidden-costsI recently added two of my existing publications, one about high performance dynamic visibility and the other on how to display pixel quad overshading in real-time on Xbox 360.
The first of these was originally published in GPU Pro 2. Unfortunately, I missed some errors that crept into the typeset version, so I was pleased to finally correct those and I took the opportunity to rework a few sentences for greater clarity as well. Now that it’s online, I’ll also be able to refer directly to certain sections in follow-up blog posts on the subject.
The second took the form of a journal entry for the Microsoft Game Developer Network, which went up in the spring. It may have flown under your radar, as I’ve since spoken to a few developers who hadn’t seen it, yet were keen to have such a tool in their engine. For NDA reasons, I can’t go into all of the implementation details here, so think of it as a ‘graphical appetiser’.
In a way, the two topics are related: the primary goal of a visibility system is to efficiently remove parts of the world that can’t be seen from a given viewpoint, whereas the purpose of a debug overshading mode is to directly visualise pixel shader work, some of which can likewise have zero contribution to the final image.
I think it’s also fair to say that keeping both forms of redundancy in check is a critical part of optimising the rendering performance of most AAA titles. For that reason, I hope you find these articles useful, and as always, please let me know what you think!
Both HPG and SIGGRAPH were a blast and I’m intending to write up a full report soon, but here are some links to conference content in the meantime. If you have any additional sources, please let me know in the comments section and I’ll update the post accordingly.
HPG
Practically all of the slides and posters are available here, including Hot3D presentations. For everything else, see Ke-Sen Huang’s list.
SIGGRAPH
If you’re playing catch up and wondering where to start, the Real-Time Rendering blog has your back.
Although Cory Doctorow’s keynote isn’t about rendering as such, the topic of digital rights is still highly relevant to the videogame industry, so here it is (via Naty).
As always, Ke-Sen Huang has tirelessly gathered links to all of the available technical papers and associated media here. There’s also a video preview online, as well as a PDF (warning: 184MB) containing the first page of every paper. (Via the Real-Time Rendering blog.)
]]><![CDATA[Specular Showdown in the Wild West]]>2011-07-22T10:45:34-04:00http://blog.selfshadow.com/2011/07/22/specular-showdown
“You see, in this world there’s two kinds of people, my friend: Those with loaded guns and those who dig. You dig.” - Blondie
Saddle Up!
In this post, I’ll be reviewing some existing methods for attaining well-behaved specular lighting. I’ll also cover a simple twist on these that fits better with current game lighting approaches and console memory constraints.
What do I mean by well-behaved? I’m talking about avoiding specular highlight shimmering on bumpy surfaces, as well as achieving the right appearance in the distance: the combined effect of these bumps as individual wrinkles and irregularities become too small to make out. Can we do all of this on a budget? Let’s hit the trail and find out!
The Good, the Bad and the Ugly
Some days it feels like those of us involved in videogame rendering are a bunch of cowboys: we play fast and loose with the laws of the land as we rush to get things up on screen in time, wrangling pixels along the way to produce the look that we (and our artists) want.
Lately we’ve been righting some wrongs by adopting linear lighting [1] and physically-based shading models [2] - something even our more civilised neighbours in film have only recently been transitioning to (see other presentations from the same course). That’s all well and good, but before we get ahead of ourselves and start believing that the Wild West days are over, there’s another major area that we need to be tackling better: aliasing.
As Dan Baker rightly argues [3], aliasing is one key differentiator between us real-time folk and the offline guys. Whilst they can afford to throw more samples at the problem, heavy supersampling would be too slow for us. (Admittedly, I haven’t tried [4] in production, but I’m not expecting it to perform well on current consoles.) MSAA is also ineffective as it only handles edge aliasing, so under-sampling artefacts within shading will remain - specular shimmering being a prime example. Post-process AA (for which there are now many potential options [5]) isn’t really helpful either since it does nothing to address sharp highlights popping in and out of existence as the camera or objects move. Finally, you might think that temporal AA could be a solution, but that’s really just a poor man’s supersampling across frames, with added reliance on temporal coherency [6].
In summary, the standard AA techniques we use in games are pretty hopeless for combating shimmering - particularly for higher specular powers - and none of them achieve the distance behaviour that we want either. Performance aside, I strongly suspect that even supersampling falls short in that case unless an obscene number of samples are used together with custom texture filtering, otherwise bump information will be averaged away. (Well, not entirely; more on this later.)
So, what other options do we have? At an earlier conference [7], Dan covered two workarounds commonly employed by developers: scaling down bumpiness or glossiness in the distance. The first is really wrong though, as it gives us the opposite of what we want: rather than a bumpy surface looking duller when further from the camera, flattening the normal map leads to a more glossy appearance! Instead, reducing the specular power is - to a first approximation - the right thing to do. Although it’s something that needs to be tweaked on a case-by-case basis and doesn’t work correctly for normal maps with both bumpy and flat areas, it’s still better than simply living with aliasing or avoiding high powers altogether. I should add that texture-space lighting also gets a mention, but it’s another heavyweight alternative with its own set of problems, so I won’t discuss it further here. (The idea of marrying this with a dynamic virtual texture cache boggles my mind though!)
All told, are we resigned to being ransacked by badly behaving specular and ugly shimmering? Maybe not, as there’s a new sheriff in town…
CLEANing up the Streets
LEAN Mapping [8] is a recent approach for robust filtering of specular highlights across all scales (with the possible exception of magnification - more on this in a bit). Not only does it model the macro effect of surface roughness in the distance really well - even generating anisotropic highlights from ridged normal maps - but it also supports combining layers of dynamic bumps at runtime (for a few dollars more, naturally).
There isn’t really the space to go into the gritty details of how it works - for that, you can check the references - but it’s definitely at the more practical end of the solution spectrum compared to many previous techniques. It shipped with Civilization V after all, and from my experience so far the results are impressive.
That said, there are some significant roadblocks preventing immediate, widespread adoption:
Heavy storage demands
Anisotropic, tangent-space Beckmann formulation
Off the bat, memory requirements will be a limiting factor for many developers. Not only does LEAN Mapping need two textures in place of a standard normal map (and maybe more to combine layers), but the overhead is compounded by pesky precision requirements, for similar reasons as Variance Shadow Maps [9]. Firaxis did manage to squeeze things down to 8-bit per-channel storage in some cases, but the paper advises: “In general, 8-bit textures only make sense if absolutely needed for speed or space”.
Unsurprisingly, this rules out DXT1 or DXT5, which are two of the most common cross-platform formats for normal maps on current consoles. By comparison, we could be facing at least 8 times the storage cost (possibly less if the normal is recovered from the core LEAN terms and other data is packed in its place). Yowzers!
Things get even rougher for deferred rendering, where G-buffer space is typically a scarce commodity and to make matters worse, the lighting formulation further complicates things. Even assuming that everything could be moved to a common space (such as world space), we would still need to store an additional tangent vector! There’s also more maths involved with LEAN Mapping, so that’s another important consideration regardless of how we choose to light our environments.
So, overall, it’s certainly not the drop-in replacement that it might first appear. Fortunately the sheriff has a deputy who’s quicker on the draw: at this year’s GDC, Dan showed off a cut down version, CLEAN (Cheap LEAN) Mapping [3], that sacrifices anisotropy for lower storage requirements - roughly half - and slightly higher performance. That’s a significant improvement and losing anisotropy effectively resolves the second issue as well, but the footprint over DXT is still rather steep for my taste.
What I’d really like is something that can be applied liberally without having to seriously ‘re-evaluate’ art budgets (and by that, I of course mean making cuts elsewhere). Don’t get me wrong, I think that (C)LEAN Mapping is a really exciting advance, I just don’t expect to see it being used with wild abandon on current generation consoles as things stand. Still, it’s a great option to keep in mind for specific situations, with Civ 5’s water serving as a case in point.
We Need To Go Cheaper
Is there anything out there that could give us more bang for our buck? Actually, way back in 2004, Michael Toksvig presented a beautifully simple technique [10] that - much like (C)LEAN Mapping - takes advantage of MIP-mapping and hardware texture filtering, but estimates bump variance directly from the lengths of the averaged normal vectors stored in an existing normal map. Since it doesn’t have a catchy name as such, I’ll refer to it as Toksvig AA.
From the paper, the original Blinn-Phong formulation is:
$N_a$ is the averaged normal read from the texture, from which we calculate the so-called Toksvig Factor, $f_t$. This is then used to modulate the specular exponent, $s$, and the overall intensity depending on the variance (roughness).
Here’s the same thing in code, with some minor tweaks:
In the original version, all of this is stored in a 2D LUT that’s indexed by dot(Na, H) and dot(Na, Na), in which case the CLAMP texture address mode would take care of the saturate() for us. Whereas using a LUT made more sense back in the days of Pixel Shader version 2.0 and lower, we can just evaluate things directly in the shader, which removes the need for a LUT per specular exponent and thus sidesteps potential cache and precision issues.
The first part of the equation, $\frac{1+f_ts}{1 + s}$, also looks a bit bogus since it’s only handling energy conservation in one direction and I’m pretty sure that those 1s should really be 2s [2]. Treating Blinn-Phong as a Normal Distribution Function (NDF) [2], what I suspect you really want is:
$$spec = \frac{p + 2}{8}\left(N.H\right)^{p} \text{, where } N = \frac{N_a}{|N_a|} \text{ and } p = f_ts$$
For those of you already using energy conserving Blinn-Phong, this should look familiar and also quite elegant: you’re just scaling the specular exponent and then everything just works.
This may be obvious, but for deferred rendering you’ll want to do this during your initial scene pass, as then you’re scaling just the once before packing the exponent into the G-buffer, rather than for every light. It also means that you’re still free to use clever G-buffer encodings of normals, such as BFN [11] or two components in view space [12].
Unfortunately, this brings us on to a major problem with Toksvig AA: you can’t use it with two-component input normals, such as 3Dc and (typically) DXT5. The whole basis of the technique is that local roughness (divergent normals) is approximately captured by the length of the filtered normal, so it’s no good trying to do this with encodings that reconstruct a unit vector! Furthermore, even if we use DXT1 exclusively for normal maps, compression will still mess with vector length. It’s easy to forget this since we often get acceptable results for lighting after renormalisation, but it can really play havoc with Toksvig AA. Given all that, it’s not surprising that the technique hasn’t been picked up by developers.
Was this yet another dead end? Actually, no, as it’s brought us a bit closer to a solution.
The Wild Hunch
Here’s an embarrassingly simple idea: Toksvig and LEAN Mapping exploit texture filtering, so why don’t we do this offline? Let’s see what happens if we take our normal map (Figure 1) and pre-compute the Toksvig Factors from a gaussian-filtered version of each MIP level.
Figure 1: Normal map
What we end up with is shown in Figure 2. The results are immediately intuitive: areas in the original normal map that were flat are white (glossy), whereas noisy, bumpy sections are darker.
Figure 2: Toksvig map
With the right kernel size, this Toksvig Map matches up really well against the run-time Toskvig method, which you can see for yourself with the demo at the end. It’s even better under magnification (compare Figure 3a and 3b), as Toksvig AA shows blockiness due to the discontinuous nature of bilinear filtering - cubic interpolation would fix this. (C)LEAN mapping suffers from the same kind of artefacts, but we don’t get this with the baked version because we’ve pre-filtered with a gaussian.
Figure 3a: Toksvig AA under magnification
Figure 3b: Toksvig map under magnification
What we effectively have here is an auto-generated anti-aliasing gloss map, and unlike the (C)LEAN terms, it compresses really well! Also, if this map is generated as part of the art import pipeline, then we’re free to use high precision three-component normals, prior to the compression method of our choosing. So, with this straightforward change, we’ve overcome the two big obstacles of memory consumption and precision.
Using this map is trivial as it’s just like any other gloss map:
However, the observant reader will notice that the Toksvig Map has been generated with a particular specular power in mind. Fortunately, you can bake with a fixed power that gives reasonable contrast - around 100 seems to work well - and then convert later if you need that flexibility (e.g. for texture reuse or material property changes):
1
ft/=lerp(s/fixed_s,1,ft);
Another option is to store the adjusted power, $f_ts$, as the exponent (range [0,1]) of a maximum specular power, as suggested by Naty Hoffman [2]. This log-space glossiness term could be more intuitive for artists to work with and may remove the need for a separate material property for the exponent, plus it’s also a convenient format for direct G-buffer storage [13]. A third option is to store the variance or the length of the normal instead and do the remaining maths at runtime.
The data itself could be packed alongside a specular mask or used to modulate an existing gloss map. I could even imagine the map being used directly as a starting point for artists to paint on top of, but in that case care will be needed to avoid reintroducing aliasing!
Gloss maps are arguably more important than specular masks in the context of physically-based shading [2], but artists are typically more comfortable with the latter. Additionally, with the high intensity range that’s possible with energy-conserving specular, anti-aliasing is even more critical. For those reasons, I find the whole idea of an auto-generated texture to be really appealing, even though I haven’t tested all of this out in production yet. I’m a big fan of art tools that do 80-90% of the work automatically, but still provide a way to go in and directly, locally tweak the output. The result is less hair pulling and more time dedicated to polishing. Hopefully this is another example of that.
Gunfight at the O.K. Corral
How does Toksvig Mapping compare to LEAN Mapping? Well, it’s certainly not as good on account of the lack of anisotropy, which can mean over-broadening of the specular highlight in some cases and not enough (leaving dampened aliasing) in others. However, it’s possible to bake CLEAN Mapping instead, which reduces remaining shimmering at the cost of a little more highlight blooming, since it tends to conservatively attenuate.
The reason we can do this is that the main thing separating Toksvig and CLEAN in practice is the measure of variance ($\sigma^2$):
Everything I already mentioned earlier with respect to Toksvig Maps - baking, evaluating, converting between powers, etc. - applies here too. I’ve even had some success baking LEAN, but I’ll leave talking about that for another time.
Even though LEAN Mapping is closer to the ground truth, baked Toksvig/CLEAN Mapping is still a hell of a lot better than doing nothing, which is precisely what most of us are doing at the moment.
Ride ‘em, Cowboy!
Here’s a simple WebGL demo that allows you to toggle between standard Blinn-Phong (default), Toksvig AA and Toksvig Map. You can also edit the shader code on the fly!
Demo update: I’ve had reports of visual issues with the Toksvig Map option and NVIDIA GPUs. If it appears as though you’re missing MIP-maps, then upgrading to the latest drivers (285.62+) should fix the problem.
Besides the shimmering with Blinn-Phong, notice how the teapot remains shiny when you zoom out (mouse wheel) as bumps disappear. In contrast, the material maintains its appearance with Toksvig.
I’ve also created another little example that demonstrates the filtering process.
…Into the Sunset
Phew, that was a rather long post! Perhaps I could have just said: “Store the Toksvig Factor in a texture”, but I enjoyed the journey of getting there and I hope you found it interesting too!
I’ll follow up with more thoughts and an expanded demo at a later date once I’ve had time to investigate further alternatives. In the meantime, I’m interested to hear from anyone who’s explored this area. Besides clearly being a subject that keeps Dan Baker up at night, I spotted that Jason Mitchell had experimented with SpecVar maps [14] for Team Fortress 2 [15] and Naty Hoffman also shared some thoughts on earlier work here [16]. Beyond that though, I haven’t seen much discussion outside of the referenced literature, so I’m all ears!
Acknowledgements
In addition to the great work of all the cited authors, I’d also like to acknowledge sources of inspiration and code for the demo. The live editing environment is (or will be) heavily inspired by Iñigo Quilez’s Shader Toy and Tom Beddard’s Fractal Lab. It makes use of several open source components such as Ace and jQuery, plus various UI widgets. Some WebGL utility code was taken from Mike Acton’s #AltDevBlogADay post, which builds on/refactors this Chromium demo. I’ll save more details on the editor for a future post.
Finally, the normal map is borrowed from RenderMonkey. I trust that this is okay as I couldn’t find a licence anywhere, but please let me know if that isn’t the case. There isn’t a whole lot of freely available data out there and that particular texture, with its rough and smooth areas, makes for a good example.
