Bringing an Accurate Fresnel to Real-Time Rendering: a Preintegrable Decomposition

Bringing an Accurate Fresnel to Real-Time Rendering

a Preintegrable Decomposition

Laurent Belcour

Megane Bati

Pascal Barla

Offline / Real-Time Convergence

Goal: use game engines across productions
- Not just in previz.
- Not just for backgrounds
Challenge: a unified workflow
- Assets
- ...
Asset appearance must match
- That includes material models
- All models rely on Fresnel reflectance/transmittance

Production of Love & 50 Megatons

One of our goals is the convergence between real-time and offline rendering *click* In this context, we aim to allow real-time rendering engines, such as Unity, to be used more in productions that would traditionally use offline rendering software. Our goal is to push the use of game engines in all of the different stages of the production pipeline. Not restricting its use to previsualization or background rendering. *click* However, this poses many challenges. One of them being that the asset workflow must match. Once an artist has authored an asset in a content creation tool, it should be interpreted the same way in both the offline and the real-time worlds. *click* In this talk, I will focus on appearance. To achieve the same appearance, the PBR material models should match or at least resemble each other closely. While the industry has converged to the use of microfacet theory to unify surface shading, there still is an issue with the match of one of its components, which is Fresnel reflectance.

What is Fresnel reflectance?

Amount of light reflected by a surface
- Basic ingredient of Microfacet models
- Based on the Index of Refraction $\eta + i \kappa$

Microfacet model

$ \theta $

Fresnel reflectance

$ \theta $

What is Fresnel reflectance?

Amount of light reflected by a surface
- Basic ingredient of Microfacet models
- Based on the Index of Refraction $\eta + i \kappa$
A better parameterization exists
- Designed for metals [Gulbrandsen 2014]

Parameterization $(r, g)$

$r$ $g$

$r = $ 0.0
$g = $ 0.0

$\eta = $ 1.5
$\kappa = $ 0.0 Fresnel reflectance

What is Fresnel reflectance?

Amount of light reflected by a surface
- Basic ingredient of Microfacet models
- Based on the Index of Refraction $\eta + i \kappa$
A better parameterization exists
- Designed for metals [Gulbrandsen 2014]
- Not linear with respect to edge tint

Gulbrandsen's parameterization, blue to red

What is Fresnel reflectance?

Amount of light reflected by a surface
- Basic ingredient of Microfacet models
- Based on the Index of Refraction $\eta + i \kappa$
A better parameterization exists
- Designed for metals [Gulbrandsen 2014]
- Not linear with respect to edge tint
Approximations exist
- The famous Schlick Fresnel [Schlick 1994]
- Not controllable at grazing angles

Another Issue: Lack of Unification

Multiple "Standards"
- Exact Fresnel with Gulbrandsen's parameterization
- Approximate Fresnel using Schlick
War of the Fresnels (which to choose?)
- Some with strong opinion [Hoffman 2019]
- My take: this is not my battle
No cross-compatibility
- Schlick Fresnel is not a valid Fresnel
- Conversion impacts material perception
Issue: game engines only use Schlick
- Restricts the use of game engines in production
- Let's do something about it!

Autodesk Standard Surface [Georgiev et al.]

Disney's Principled BRDF [Burley 2012]

Same reflectivity! (r = 0.25)

Now the co-existence of both Schlick's and Gulbrandsen's methods creates another problem in my opinion. *click*

Why so? Because, it creates multiple standards. For example, companies such as Autodesk expose Gulbrandsen's parameterization in their uber shader, while Disney's principled BRDF model uses Schlick. You can even have a combination of both methods. *click*

There isn't a consensus to use one or the other within the graphics community. My take on this is a little bit different. I don't think it is up to me to decide for the artist which model is better. If they want to use edge tint, they should have tools and renderers that support it. *click*

My problem though lies with two facts:
First: those two models are not compatible. Schlick Fresnel does not represent a valid Fresnel reflectance. Hence, for a given reflectivity (r), there is no edge tint (g) that will produce the same curve as Schlick Fresnel. As you can see on the right, converting from one to the other might change the appearance. *click*
Second: *click* games engines (such as Unity) are limited to Schlick Fresnel, which can restrict their use in film or TV production.

In this talk, I will show how we can change that.

What is the Constraint of Real Time?

Precomputed integrals (e.g., Split-Sum Integral [Karis 2013])

$$ L_o = \int_\Omega \rho(\omega_i, \omega_o) L_i(\omega_i) \mbox{d}\omega_i $$

Too hard to evaluate analytically

What is the Constraint of Real Time?

Precomputed integrals (e.g., Split-Sum Integral [Karis 2013])

$$ L_o \simeq \color{green}{ \int_\Omega \rho(\omega_i, \omega_o) \mbox{d}\omega_i } \times \color{blue}{ \int_\Omega D(\omega_h) L_i(\omega_i) \mbox{d}\omega_i } $$

Separate the integral into directional albedo (FGD) and light terms (LD).

What is the Constraint of Real Time?

Precomputed integrals (e.g., Split-Sum Integral [Karis 2013])

$$ \color{green}{ \int_\Omega \rho(\omega_i, \omega_o) \mbox{d}\omega_i } $$

FGD Depends on roughness, view direction, reflectivity, and edge tint.

What is the Constraint of Real Time?

Precomputed integrals (e.g., Split-Sum Integral [Karis 2013])

$$ \color{green}{ \int_\Omega \rho(\omega_i, \omega_o) \mbox{d}\omega_i } = r \times \quad\quad\quad\quad + \,\, (1-r) \times \, $$

Schlick Fresnel allows us to reduce the dimensionality.

Requirements for a Cross-Production Fresnel

	Schlick	Gulbrandsen	"New model"
Accuracy	✗	✓	✓
Real Time	✓	✗	✓
Linear parameters	✓	✗	✓
Optional edge tint	✗	?	✓

Let's summarize: "What are the requirements of a Fresnel model that works across production pipelines? *click*

First, we would like this model to be accurate, so as to get a close match between real-time and offline use. *click*

Second, we would like this model to be compatible with the real-time constraints of the split-sum integral. *click*

Third, we would like its parameter to be as linear as possible to avoid the incorrect blending I showed previously. *click*

Last, we might want to have an optional edge tint that is a default configuration, where the material behaves similar to Schlick but following a real Fresnel reflectance. *click*

As mentioned, Schlick Fresnel fulfills real-time constraints and has linearity, but it fails to be accurate and does not enable us to control the grazing angles. *click*

Gulbrandsen's parameterization on the other hand uses the exact Fresnel formula, so it is accurate and provides an edge tint parameter to control the grazing angles. However, it fails to meet the linearity and real-time requirements. *click*

In this talk, I am going to show to build a model that fullfills those criteria. *click*

But what I am going to present to not just a model, but rather a framework to build and rectify any Fresnel model you would like. Sort of the swiss knife for building Fresnel models that work across industries.

Did You Say Precomputed?

What does it mean?
- Fresnel is mostly used through a LUT
- So, why bother having an analytical form?
One constraint though
- Fresnel must be a linear decomposition
- This decomposition is known ahead of rendering
This was the starting point for our research

An Empirical Fresnel Decomposition

Extract a Fresnel model from data
- Linear decomposition of Fresnel reflectance $$ F(\theta) \simeq \sum_i w_i b_i(\theta) $$
- What we need is a basis tailored to Fresnel reflectance
How many basis functions do we need?
- Eigenvalues for dieletrics → 3

Eigenvalues for dieletrics

An Empirical Fresnel Decomposition

Extract a Fresnel model from data
- Linear decomposition of Fresnel reflectance $$ F(\theta) = \sum_i w_i b_i(\theta) $$
- What we need is a basis tailored to Fresnel reflectance
How many basis functions do we need?
- Eigenvalues for dieletrics → 3
Use the SVD to extract a basis
- But we enforce some constraints
- And use a tailored fitting procedure

Fitting A Fresnel Reflectance

$$ = c_0 $$

$$ + \, c_1 $$

$$ +\, c_2 $$

$$ + \, c_3 $$

$$ = r $$

$$ + \, {\scriptstyle (1-r)} $$

$$ +\, c_2 $$

$$ + \, c_3 $$

Results: Curves

$\eta $ $\kappa $

Results: Comparison with Schlick Fresnel

Rhodium (Rh)

Results: Comparison with Schlick Fresnel

Tungsten (W)

Results: Comparison with Schlick Fresnel

Chromium (Cr)

Where Are We So Far?

	"New model"
Accuracy	✓
Real Time	✓
Linear parameters	?
Optional edge tint	?

where Are We So Far?

	"New model"
Accuracy	✓
Real Time	✓
Linear parameters	?
Optional edge tint	?

Building a Parameterization

We could build a color parameterization
- Use the zero crossing of $b_3(\theta)$
- There the reflectance is determined by $c_2$ only
Has its drawbacks
- Can break energy conservation (need clamping)
- Can introduce unwanted colours

Let's focus on the linear parameterization first.*click*

We could build a parametrization based on a colour similar to the extented Lazanyi. And for that we only need to look for the zero crossing of the last basis element which is at roughly 78 degrees.

At this specific point, for a given reflectivity, the Fresnel reflectance is solely determined by the c_2 parameter. Which makes F(78) a valid parametrization.*click*

Now I wont go into the details of this parameterization, because Naty already talked about this idea in the previous part of the course. Since it is equivalent to the extended Lazanyi parametrization it will share its drawbacks: it requires care to avoid breaking energy conservation. It can also introduce unwanted colours such as the bluish tint at the grazing angle here when a gray grazing colour is specified.

Building a Parameterization

Take advantage of $(c_0, c_2)$ space
- Our model is linear with respect to $c_i$
- Any non-linearity will show in this space
Display the Gulbrandsen parameterization
- Regular grid of $(r,g)$ shows the non-linearity
- Any Fresnel reflectance is constrained to this space
Find $c_2(r, \beta)$, then $c_3(r, c_2)$

$c_0 = r$ $c_2$

So instead I will show you how we can rectify the non-linearity in Gulbrandsen's parameterization. *click*

And for that, we are going to use the space of coefficient as a way to ensure this linearity. Remember that coefficient behave linearly. So any non linearity of the parameters, will show in the space of coefficients. Here on the right is the coefficient space with c_0 the x-axis and c_2 the y-axis. *click*

What I can do in this space is to display Gulbrandsen's parameterization. To do so, I transform a regular grid over (r,g) in coefficient space. Each point corresponds to the first and third coefficient when fitting an (r, g) pair. We can see that dots are not space regularly along the y-axis, meaning there is a non linearity. We can also see that all of Fresnel reflectances mapped by the parameterization are enclosed in a wave shape. *click*

And this particular point will allow us to develop our new parameterization. We are going to linearize with respect to the c2 coefficient and retreive the remainging coeffcient afterwards. *click*

Building a Parameterization

$c_0 = r$ $c_2$

Offline Renders

Gulbrandsen's parameterization, blue to red

Our parameterization, blue to red

Performance

OpenGL demo
- Running on an Nvidia RTX 2070 at 720p
- 0.02 ms overhead compared to Karis' split-sum integral

Unity Demo

Editing session
- Implementation by Thomas Deliot
- Custom Forward shader in Unity HDRP
- Running on an Nvidia RTX 2070

where Are We So Far?

	"New model"
Accuracy	✓
Real Time	✓
Linear parameters	✓
Optional edge tint	?

No Edge Tint Default

Many possibilities
- $\beta_o = 0$ is a bad idea
- $\beta_o = \text{argmin}\left(c_2^2 + c_3^2 \right)$ → bad with edge-tint control
Best solution we found: $\beta_o = 0.75$
- No grazing angle componnent
- Allows back-and-forth edits of r and g

Default using Gulbrandsen (g = 0.75)

Schlick with Equivalent r

$r $ $\beta $

We found that there are practically as many potential ways to obtain a default parameter for the edge tint as you can think of. *click*

Of course, setting beta to 0 is a bad idea. *click*

At some point, we experimented with the idea of looking for the beta parameter that minimizes the impact of the belly. While this works well (sorry for not showing it), I think it has a practical drawback if you want to include edge-tint control in the model. The issue is that the default beta parameter is dependent on the reflectivity. *click*

For example, let's say that an artist paints a reflectivity map and then edits the edge tint map with the default value as the base color. If they later want to edit the reflectivity, they might introduce an edge tint in regions that were not intended to have one. *click*

So instead, we settled on beta = 0.75, since it provides a Schlick-like behavior. *click*

There is no grazing angle appearing on the curve for this parameter *click*

and it prevents the issue with the edge tint 'leaking' during editing. *click*

Here are renderings of using this default beta for different reflectivity. *click*

We can compare this default with Schlick and it provides pretty similar results.

Standardize Fresnel?

Let's start a community discussion about it!
Use Fresnel models coherently!
- Replace Schlick by a no edge tint default?
- Add edge tint in a opt-in/out fashion
An editing format for artists
- Linear edge-tint parameter?
- Artistic $F(X^{\circ})$?
- Your parameterization here?
An exchange format
- Even share coefficients?
- Compression / conversion / ... should be defined

Before concluding, I would like to start a discussion on our use of Fresnel and how we could look in the future and avoid the pitfall of mixing Shlick and the analytical Fresnel I described in the beginning. *click*

It is quite unfortunate that due to recent events we cannot have this discussion in one room, but that doesn't mean we can't still have it. What follows is a brain dump and should not be considered a set of guidelines, but more as the catalyst for a discussion. *click*

My first point is that mixing models with non overlaping default is a bad idea. Now we could build an extended Schlick Fresnel as Naty pointed (or constraint our decomposition to include Schlick Fresnel). But a possibility could be to replace Schlick entirely from our pipeline. And use a a no-edge-tint default in place. I mean, we shouldn't be tied to the exact formula of Schlick Fresnel, but rather aim for Fresnels that offer a range of dieletric and conductors over a unique reflectivity range. In that sens, we could even build a specific model using the decomposition method.

More importantly, I think this would allow users/artists to choose if they are fine with a model that has no edge tint. And provide this feature in an opt-in. *click*

Then there is the issue of the parameterization if we want to include grazing angle control in the model. Since the edge-tint parameterization and the artistic colour parameterization can produce the same unwanted colour issues. So maybe a not-so-strict definition of the grazing angles that is the edge-tint is fine.

Maybe there is a parameterization out there that is still to be discovered and will solve all our issues.

But in the end, maybe this is a choice that only affects the production of assets and not its use for rendering. *click*

Because maybe what we could use is another format that is used to exchange assets between engines in a principled way. There as probably many forms that this format could take, as long as it enable to accurately reproduce the desired reflectivity. But any operation on this exchange format should be well defined.

Summary

A framework to build Fresnel model
- Compatible with real-time rendering
- Accurate with respect to reference Fresnel
Improve Fresnel parameterization
- More linear than Gulbrandsen's
- Or with colour control
More details at belcour.github.io/blog
- Source code, notebook, ...

Thanks to: Thomas Deliot for the Unity prototype. Eric Heitz, Jonathan Dupuy, and Kenneth Vanhoey for feedbacks. Naty Hoffman, Stephen Hill, Emmanuel Turquin, and Sebastien Lagarde for discussions on Fresnel.

Slides use reveal.js framework with juxtapose for image comparisons. Studio and shader ball assets available at blendswap.