Advanced Ray Tracer: Part 5

In this part, I implemented some new type of lights and HDR tonemapping. I think this part really stepped up what the ray tracer can do and the outputs look really cool.

First, we will dive into HDR tone mapping to not stop the flow of the post as some lights require this feature. Then we will go through directional, spot and environment lights in this order.

All images seen in the post are rendered by my raytracer. The tests are conducted using the Apple MacBook M3 Pro chip with 5 performance cores with 4.06 GHz and 6 efficiency cores with 2.8 GHz.

Glass Sphere Environment Light

HDR Tonemapping

Up until this point in our color calculations, we clamped the end result to fit the [0, 255] RGB color limits. This is obviously not a good thing to do as it means that we give a lot of color complexity away, especially in the brightest and darkest parts. You can think of this such that up until now, in every scene we rendered, we did not get light visuals as bright as in real life and proper contrast between colors. What we can do is handle this clamping operation better so it follows the real color calculations more realistically.

HDR is the acronym for ‘High Dynamic Range’. For our outputs, it is basically the proper unclamped version of our computations. The [0, 255] range is called LDR: ‘Low Dynamic Range’. We can actually create images that utilize HDR and also read stuff like textures from them. We used .exr files for the rendering of .exr scenes. But we would also want to create LDR images in the format of .png as it is more lightweight, easier to use and extremely wide spread.

The process of conversion from HDR to LDR is like draining the liquid of a stew we have cooked for hours, so we can carry it places and eat it easier. But in the process we lose all of that good herbs, spices and fat that would give our stew that extra flavor. Up until this point we just drained all the liquid and ended up with just dry meat, but now we will try to keep that extra spice and juiciness that was already created, but ignored.

Before going forward I want to mention that I used the tinyexr library to handle the read and write operations of .exr files. It is pretty easy to use and lightweight, I recommend it to anyone who wants do something similar.

Background

There are different approaches to tone mapping. The basic classification is global and local tone mapping operators. With global tone mapping operators, we apply the same transformation to every single pixel based on its luminance and global data like the average luminance. This is simpler to do, faster and supposed to be pretty consistent. But as we approach it globally, we might miss local contrasts and miss some detail in the extreme values. Local tone mapping operators on the other hand apply different transformations to different regions based on local data. This approach might seem better overall but there can be some problems:

Those algorithms are more complicated than the global ones; they can show artifacts (e.g. halo effect and ringing); and the output can look unrealistic, but they can (if used correctly) provide the best performance, since human vision is mainly sensitive to local contrast. — (Wikipedia, n.d.)

In this part, we will use the photographic tone mapping algorithm intruduced by Reinhard et. al. in 2002 with the paper Photographic Tone Reproduction for Digital Images which is a global tone mapping algorithm. This method is inspired by Ansel Adams’s Zone System:

Scene Zones — (Reinhard, Stark, Shirley, & Ferwerda, 2002)

The mapping from scene zones to print zones. Scene zones at either extreme will map to pure black (zone 0) or white (zone X) if the dynamic range of the scene is eleven zones or more. — Reinhard, Stark, Shirley, & Ferwerda, 2002

I believe there are some related terms we should get out of the way before I start explaining the whole process:

Middle gray: subjective middle brightness of the scene. Typically mapped to Zone V.

Dynamic range: difference between the highest and lowest zones to which a scene will be mapped

Key: the overall subjective brightness of the scene

Dodging-and burning: locally inhibiting or administering light to a certain region of the print

—( Course Notes by AO Akyüz)

So having these in mind, let’s look at the actual steps to tone map a scene.

Photographic Tone Mapping

In the 1st step, we will get an initial luminance value by multiplying the color value’s RGB components with some constants:

Luminance Calculation — Course Notes by AO Akyüz

The coefficients are decided based on the human visual system. This topic is closely related to how we actually percieve light. So these coefficients are decided based on the receptor sensitivity to light in our eyes.

Now for the 2nd step, we need to calculate the log-average luminance to scale our initially calculate luminance into a zone. This is the middle-gray mentioned just a little while ago, and it is estimated as such:

Log-average Luminance Estimation — Course Notes by AO Akyüz

According to the book Fundamentals of Computer Graphics 5th Edition, in the Tone Reproduction part actually written by Erik Reinhard, there is a simple note about why we add an epsilon(written as delta in the formula but acts as a very small constant like we usually represent with epsilon) value:

… a small constant δ is introduced to prevent the average to become zero in the presence of black pixels. — (Marschner & Shirley, 2022, p. 580)

So this value matters in the calculation of the average estimation when our scene has a lot of black pixels.

In the 3rd step, we will also be given a key value for each scene to determine how aggresively this compression is done and divide this by the average approximation. The aggressiveness refers to how strongly the tone mapping operator reduces high luminance values. A higher key value would do a less aggressive change, so the higher luminance values would remain and the image would be britghter. If it is lower, it would be more aggressively compressed and it would be darker. So here the key value determines how bright the scene is and we generally want to map the middle grey to %18 reflectance so we set the key to be 0.18. This value comes from the Zone System as Zone V has approximately 18% reflectance also, which is the middle ground. This would give us the most optimal results in terms of realism in general. Lastly, we can multiply with the luminance value to get the new ‘scaled’ luminance, and we are done with this part:

Key Mapping — Course Notes by AO Akyüz

After calculating this, in the 4th step, we can apply the main tone mapping operator(TMO):

Reinhard TMO — (Reinhard, Stark, Shirley, & Ferwerda, 2002)

This part is extremely simple, but it is where we actually compress high luminances. This is called sigmodial compression. This piece of basic math is very useful, and also explained in the paper:

Note that high luminances are scaled by approximately 1/L, while low luminances are scaled by 1. The denominator causes a graceful blend between these two scalings. This formulation is guaranteed to bring all luminances within displayable range. However, as mentioned in the previous section, this is not always desirable. — (Reinhard, Stark, Shirley, & Ferwerda, 2002)

Here the result is guaranteed to be in the [0,1] range and we can easily gamma correct and scale to our usual display range easily.

The ‘not always desirable’ case is when we want to directly burn out high luminance pixels for artistic purposes and visual aesthetic. In that case we will be given a percentage on how much from the top we will luminate the maximum amount when we sort all the luminance values. With this, our TMO formula also changes a bit:

Reinhard TMO Burnout — (Reinhard, Stark, Shirley, & Ferwerda, 2002)

As we are finished with the TMO, we can move onto the 4th step which will be calculating the new RGB values using a saturation parameter. This is the saturation you can see in games or if you have done anything about photography, video or image editing etc. you might be familiar with it. The s is the saturation parameter. If it is lower, we will get a duller result that will look more greyish. If it is higher, we will get an image with more vivid colors, but they might look unrealistic. So the key is to determine s by keeping the artistic choice and or realism in mind.

We can achieve this effect by dividing the inital RGB values by the luminance values we found in the first step and take its power to s. Then we can multiply this with the result of the tonemapping function. I am not extremely clear on why we use this exact way to do this but it makes sense:

Saturation — Course Notes by AO Akyüz

After this, we can move onto the 5th and the final step where we will do gamma correction. After the saturation step, we will have a linear color-space. We will encounter a problem about this with the state of modern monitors. I found this hard to explain myself, but I found out that learnopengl.com did it pretty nicely:

From learnopengl.com:

Because the human eyes prefer to see brightness colors according to the top scale, monitors (still today) use a power relationship for displaying output colors so that the original physical brightness colors are mapped to the non-linear brightness colors in the top scale.

This non-linear mapping of monitors does output more pleasing brightness results for our eyes, but when it comes to rendering graphics there is one issue: all the color and brightness options we configure in our applications are based on what we perceive from the monitor and thus all the options are actually non-linear brightness/color options. Take a look at the graph below:

Gamma Curves — (learnopengl.com, n.d.)

Here the 2.2 gamma curve aligns closely with humen eye sensitivity. But we want to somewhat preserve our generated color values to best of our ability. Applying gamma correction hereensures that the image’s actual color and the display’s response results in an image that we as humans can see the colors of better.

You can just think of it such that we have color values on the line at the middle, but our monitor will display it by transforming it by taking its power by some value g. We will try to even that out by taking the power of the color values by 1/g beforehand so the color values we end up with in the display fall closer to the line, preserving the actual values. Another good explanation for this I found was in a photography webpage:

Gamma Correction — (Cambridge in Colour, n.d.)

The difference of applying gamma correction vs. not doing so is extremely appearent in rendered scenes:

Without vs With Gamma Correction — (Rosen, 2010)

As you can see the gamma corrected scene on the right looks way more realistic than the not applied case on the left.

So after applying gamma correction, we can display the scenes in .png format by rounding to the nearest integer. Here are some basic results:

Cube Point Light — 0.184074 seconds

Cube Point Light HDR Tonemapping — Key = 0.18–0.225341 seconds

As you can see the image with HDR tonemapping looks properly luminated and we can clearly see darker and brighter sides and differentiate between them. I also wanted show how the image changes with different key and saturation values.

Changing the key value:

Cube Point Light HDR Tonemapping — Key = 0.01, Saturation = 1.0

Cube Point Light HDR Tonemapping — Key = 0.18, Saturation = 1.0

Cube Point Light HDR Tonemapping — Key = 0.72, Saturation = 1.0

As you can see, as we increase the key value, the image gets brighter which is expected and is the point of this parameter.

Changing the saturation:

Cube Point Light HDR Tonemapping — Saturation = 0.5, Key = 0.18

Cube Point Light HDR Tonemapping — Saturation = 1.0, Key = 0.18

Cube Point Light HDR Tonemapping — Saturation = 3.0, Key = 0.18

All of these outputs are expected and although 0.18 should be the best for realism in most cases, it is an artistic choice.

We can also read .exr files and use them as textures for our objects and get outputs as we can now apply tone mapping:

Sphere Point HDR Texture — 0.29178 seconds

Beware that each image in the environment lighting part were rendered in .exr format and also tonemapped to fit the .png format. I can not upload .exr files to the blog so each scene you will see from now excluding the directional and spot lights are the tonemapped versions as we read .exr files which are high dynamic range.

Having this done, let’s move onto lighting!

Directional Light

This was probably the easiest thing to implement in this homework. With directional lights, we assume that our point light is moved to infinity at a direction with a radiance value that does not decrease as objects are further away. So it lights every point the same amount, and it always comes from a specific direction universally for the scene.

So what we can just use the inverse of this direction vector to check if a point is in shadow, and otherwise give it the default radiance value of the light. It is dead simple. Here is the output:

Cube Directional Light — 0.178197 seconds

Spot Light

Spot lights are much more interesting. In this case, we are given all components of a point light plus a coverage and a fall-off angle. The coverage angle is the angle the light luminates with most angular difference from its direction. The fall-off angle is the angle difference from the light direction which we will start to attenuate the intensity based on the angle difference. Here is how it looks:

Spot Light — Course Notes by AO Akyüz

So the angle difference lower than the fall-off angle will be lighted with the given intensity. Between the fall-off angle and the coverage angle, we will lower the intensity as we go further away from the light direction so the outer parts of the spotlight will be smooth. We need to multiply the value by a factor we can calculate like this to attenuate:

Spot Light Attenuation — Course Notes by AO Akyüz

Here, alpha is the angle between the light direction and the point we are doing this calculation for shading. Lastly, outside of the coverage angle, the light will not affect any point.

Here are some cool results:

Dragon Spot Light — 100 samples— 24.2889 seconds

We can also give these lights color by using different intensity values and create some cool scenes:

Dragon New with Spot Lights — 4.04563 seconds

Environment Light

Environment lights are also called spherical directional lights. They are basically similar to directional lights but the difference is that for every single direction we want to sample light from, there is a different value. We achieve this by thinking that the whole scene is in a sphere with an infinite radius that will act as our directional light for each small point.

I think environment lights were the trickiest to implement in this part. We use .exr files to store environment maps and should sample from them properly with u, v values like we did in texture mapping. There are 2 different types of environment map formats we used: equirectangular (lat-long) and spherical(probe).

I want to clarify how we see the actual image in the background of our scenes first. As these images surround the whole scene, if our rays do not hit any objects, we can directly give the radiance value we get by sampling the environment using the ray direction. So we see the environment map even when there is no object in the scene.

Equirectangular

Obviously the first thing we should do is find the u and v values. With equirectangular maps, we have the image domain of u = [0,2] and v = [0,1]. We will try to get a sample from the image with these.

At this point, we use the direction vector we have in these formulas:

Equirectangular Map Texture Coordinates — Course Notes by AO Akyüz

x, y and z represent the components of the direction vector. After getting these u and v values, we need to convert them to image pixel values. As I said before, we get the u value to be in the range [0,2]. While getting actual sampling pixel coordinate, we need to also divide the u value by 2, then multiply u by the image width and v by the image height. After doing all this and sampling like we did in texture mapping, we get some cool results. Here is a scene with no object so there is no intersection and we just give the environment map samples as output:

Empty Environment Light Latitude-Longitude — 0.052338 seconds

To get a better idea of equirectangular maps and some examples, you can check out USC’s vision and graphics lab page: https://vgl.ict.usc.edu/Data/HighResProbes/

Spherical

The spherical format is also very simple to implement. In this case, we will actually do the mapping like we are stepping inside a sphere. The operation is extremely easy. We need to beware that we get u and v values both in the range [-1,1]:

Spherical Environment Map — Course Notes by AO Akyüz

So we calculate r and multiply with the x and y components of the direction vector(negate for v as we go down). Then we map it in the range [0,1] by adding 1 and dividing by 2. Here is the result:

Empty Environment Light Probe — 0.04819 seconds

You can check out Paul Debevec’s page about this to get a better idea: https://www.pauldebevec.com/Probes/

Lighting Objects

So we know how to get samples when there is no intersection, but have not talked about shading actual objects yet. The thing to consider here is to decide which direction to sample from to shade a point. We did not calculate this ourselves before, we just knew where the lights were and calculated the direction between the point and the lights. Here we can not do that as there is light in every direction, but we can sample a direction randomly in a way that will give us proper results.

Random Rejection Sampling

The main idea of random rejection sampling is extremely simple. We have a condition in which we accept the sample, and until this condition is satisfied, we generate random samples and do the condition check. In our case for shading, we want our sample to be in the upper hemisphere of the surface where the point we are shading resides in.

We can just create samples by generating 3 values for x, y and z in the range [-1, 1]. To check if the sample is in the upper hemisphere, we can just get the dot product of the randomly generated sample and the surface normal. We can then normalize and use this vector to sample from the environment map. Finally, we must not forget to multiply the radiance we find by 2PI. The thing is that we just sampled from the 1 out of 2PI possible divided parts of the hemisphere, so we need to even things out. We will divide the radiance we get by the probability of sampling this specific vector, which is equal to multiplying it with 2PI. I forgot to do this for a while and it caused the objects to look darker. Need to keep this in mind as it is an important step.

Here, the samples will still be random and it will be noisy with low sample counts. If we increase the image pixel sample count, the whole shading will even out nicely and we will get the whole environment lighting really well. You can see this especially in the Audi TT results. Here are the outputs:

Glass Sphere Environment Light — 0.285928 seconds

Mirror Sphere Environment Light — 0.190534 seconds

Sphere Environment Light — 900 samples — 65.3405 seconds

Audi TT Glacier Environment Lİght — 16 samples — 27.039 seconds

Audi TT Pisa Environment Lİght — 16 samples — 26.6108 seconds

As you can see there is too much noise in the Audi TT images. I decided to increase the sample count to get sharper ones:

Audi TT Glacier Environment Lİght — 400 samples — 678.536 seconds

Audi TT Pisa Environment Lİght — 400 samples — 675.624 seconds

These are still noisier that the ground truth images but still look pretty good.

I also decided to visit the webpages I previously talked about for equilateral maps and found some new maps. Here are cool results with another one:

Glass Sphere Environment Light — Different Map — 0.51765 seconds

Mirror Sphere Environment Light — Different Map — 0.414982 seconds

The thing I do not like about this mirror image is that light comes in from the background and the mirror sphere’s side we see looks seperated from all the light coming inside. Perhaps the steps we will take in the next part would be better for a scene similar to this.

Encountered Problems

I encountered a problem with the head environment light scene. At first I did not see the head at all. So I decided to translate it further back:

Head Environment Light — Problematic Test

This looked kind of scary, but I later realized that all head intersections happened between the camera and the near plane. In the ‘motion blur boxes’ scene from part 3, taking these as proper intersections caused issues. The near distance was 10.01 and there was a wall at z = 10.00. If that was count as an intersection, we only saw a wall. I think this opposes that scene. If I count these as proper intersections in the head environment light scene, it works. I decided to handle this by adding an IgnoreNp option to the camera in the input .xml file. I give it a 0 or 1 value to check if I want to get the intersections between the camera and the near plane. This works properly, but I added 1 line to the head environment light scene. Here is the output:

Head Environment Light — 900 samples — 107.476 seconds

The bad thing here is I still did not implement smooth shading and the absence of it is jarringly appearent in this scene. The other thing is there was a comment on the top of the .xml file and that caused a problem for some reason and it was fixed when I deleted the comment. I would not call this a problem or anything but wanted to mention this subtle change.

Conclusion

This part was extremely enjoyable. The scenes look extremely cool and the new features added a whole new level of realism to our images. Also, this part was easy to understand and implement, especially as there are no problems in the previous parts (this would be harder if I had to debug texture sampling).

The thing here is, I suspect that the next part will be way more difficult but also pretty cool again as we will dive into BRDF’s and path tracing. See you in the next part!

References

Akyüz, A.O. Lecture Slides from CENG795 Advanced Ray Tracing. Middle East Technical University

Cambridge in Colour. (n.d.). Gamma correction. Retrieved December 28, 2024, from https://www.cambridgeincolour.com/tutorials/gamma-correction.htm

LearnOpenGL. (n.d.). Gamma correction. Retrieved December 28, 2024, from https://learnopengl.com/Advanced-Lighting/Gamma-Correction

Marschner, S., Shirley, P., Ashikhmin, M., Gleicher, M., Hoffman, N., Johnson, G., Munster, T., Reinhard, E., Thompson, W.B., Willemsen, P., & Wyvill, B. (2021). Fundamentals of computer graphics (5th ed.). A K Peters/CRC Press.

Reinhard, E., Stark, M., Shirley, P., & Ferwerda, J. (2002). Photographic tone reproduction for digital images. ACM SIGGRAPH. https://doi.org/10.1145/566654.566575

Rosen, D. (2010, February 17). Gamma-correct lighting. Wolfire Blog. Retrieved December 28, 2024, from http://blog.wolfire.com/2010/02/Gamma-correct-lighting

Wikipedia contributors. (n.d.). Tone mapping. In Wikipedia. Retrieved December 28, 2024, from https://en.wikipedia.org/wiki/Tone_mapping