## MIS with IBL

### MIS with IBL

Hi all,

I have a question about something that I can't seem to wrap my head around.

Here's the situation:

In a unidirectional path tracer with next event estimation and MIS lights can be reached by a random bounce and by explicitly sampling the light source. For MIS I thus take both pdfs into consideration, both when I stumble upon the light and when I sample it explicitly. So far so good.

Now, I add in IBL. In my first implementation, I randomly select either the skydome or one of the lights for next event estimation (50/50). When I stumble upon the light, I use the pdf for sampling a light scaled by 0.5 to account for the fact that I could have sampled it with 50% probability. Same for the sky. This produces a correct image, so I believe this method is correct and unbiased. Any other probability for light/sky works, including 0 and 100%.

Obviously, some parts of the scene benefit more from lights than the skydome. So, I want to change the probability of sampling the light or the sky depending on potential contribution. So I evaluate a random point on the skydome (importance sampled) and a random point on a random light. Based on these 'potential contributions' I determine the probability of sampling a light or the sky. This probability is recorded, so that when I stumble upon a light or the sky I can use it like I used the 0.5 in the previous scenario.

Sadly, this introduces bias, although subtle, and I can't figure out what I am doing wrong.

One thing that I realized is that when the random selected light happens to be behind the point that is being evaluated the potential contribution is 0, and therefore the sky will be selected with a probability of 1. However, another light may in fact be visible, so when I stumble upon it the MIS probability for sampling it explicitly is wrongly set to 0. So I clamped the light/sky probability to the range of 0.1..0.9 to ensure that both at least have a chance of being selected, but this feels wrong, bias remains (although the situation is better) and it looks like I am not understanding something here.

Any ideas?

I have a question about something that I can't seem to wrap my head around.

Here's the situation:

In a unidirectional path tracer with next event estimation and MIS lights can be reached by a random bounce and by explicitly sampling the light source. For MIS I thus take both pdfs into consideration, both when I stumble upon the light and when I sample it explicitly. So far so good.

Now, I add in IBL. In my first implementation, I randomly select either the skydome or one of the lights for next event estimation (50/50). When I stumble upon the light, I use the pdf for sampling a light scaled by 0.5 to account for the fact that I could have sampled it with 50% probability. Same for the sky. This produces a correct image, so I believe this method is correct and unbiased. Any other probability for light/sky works, including 0 and 100%.

Obviously, some parts of the scene benefit more from lights than the skydome. So, I want to change the probability of sampling the light or the sky depending on potential contribution. So I evaluate a random point on the skydome (importance sampled) and a random point on a random light. Based on these 'potential contributions' I determine the probability of sampling a light or the sky. This probability is recorded, so that when I stumble upon a light or the sky I can use it like I used the 0.5 in the previous scenario.

Sadly, this introduces bias, although subtle, and I can't figure out what I am doing wrong.

One thing that I realized is that when the random selected light happens to be behind the point that is being evaluated the potential contribution is 0, and therefore the sky will be selected with a probability of 1. However, another light may in fact be visible, so when I stumble upon it the MIS probability for sampling it explicitly is wrongly set to 0. So I clamped the light/sky probability to the range of 0.1..0.9 to ensure that both at least have a chance of being selected, but this feels wrong, bias remains (although the situation is better) and it looks like I am not understanding something here.

Any ideas?

### Re: MIS with IBL

When your ray stumble upon a light source and you want to do MIS, you need to know the probability of it being selected. It's impossible to know that probability without having sampled random points on all other light sources in your sampling method. I'm not sure if doing that make sense and will converge to a correct result.

I don't understand that part, if you have say 3 lights and one is behind the point, you'd be left selecting between 2 lights. The probability being I assume derived from some weighted sum.However, another light may in fact be visible, so when I stumble upon it the MIS probability for sampling it explicitly is wrongly set to 0.

Last edited by patlefort on Tue Jul 11, 2017 1:39 am, edited 1 time in total.

### Re: MIS with IBL

If you have 3 lights, then you need to consider all 3 when weighting them for selection. How exactly are you generating your weights? The simplest way is just using the (luminous) power of each light, then you can just normalize them to get probabilities, but obviously you have to consider all the lights.One thing that I realized is that when the random selected light happens to be behind the point that is being evaluated the potential contribution is 0, and therefore the sky will be selected with a probability of 1. However, another light may in fact be visible, so when I stumble upon it the MIS probability for sampling it explicitly is wrongly set to 0. So I clamped the light/sky probability to the range of 0.1..0.9 to ensure that both at least have a chance of being selected, but this feels wrong, bias remains (although the situation is better) and it looks like I am not understanding something here.

More complex schemes would approximate the irradiance, e.g. by calculating the solid angle subtended by the lights' shape (this would naturally handle your case of a light being behind the current vertex's hemisphere).

Even more complex schemes take into account the BSDF of the surface.

The tradeoff is of course, the more accurate your approximation when selecting a light, the lower your variance, but the more expensive each event is to find. You're trading poor performance with small numbers of lights for high performance with large numbers of lights.

### Re: MIS with IBL

This sounds like you are mixing up the sampling probability with the contribution of the light source. If you have selected the light source without taking the visibility (or being infront/to the back of the point being shaded) into account, then the probability of this sampling strategy should be non-zero (and the sky probability < 1).jbikker wrote: One thing that I realized is that when the random selected light happens to be behind the point that is being evaluated the potential contribution is 0, and therefore the sky will be selected with a probability of 1.

Otherwise, if your sampling strategy only samples lights that are visible (or in the upper hemisphere), this should be reflected in the algorithm, e.g. by discarding the sampled light source until a valid one is found or all are discarded. In this scenario the probability of sampling the sky would be 1, only if none of the remaining light sources are "visible".

### Re: MIS with IBL

Thanks for your replies.

1. The last vertex was specular, so there is no other strategy to sample the light, so the contribution is simply throughput * light color.

2. The last vertex was not specular, so we now have a second strategy, so MIS is used. For that I calculate the light pdf = dist^2 / solid angle, and the brdf pdf, which I stored when I did the diffuse bounce. The combined pdf is then simply the sum of these.

This part is working, and agrees with a reference renderer (not using next event estimation).

What also works is IBL by itself: I can sample it with next event estimation and importance sampling, and MIS works too (similar to what I just described). But when I want to combine IBL and lights things get tricky:

1. If I either sample a light *or* the skydome everything is clear; the probability of sampling the sky is now e.g. 50% of what it was, which causes no problems in MIS. Same for the lights. Any other probability also works just fine.

2. If however I chose a probability based on some heuristic (in my case: 'potential contribution') things go bad, even when I can reproduce that exact probability when I apply MIS, and even when it's never zero for a light or the skydome. That's the part that is puzzling me...

Conformance to reference is checked by the way by averaging the energy for each pixel and reporting this value. This quickly converges (even if there's tons of noise) and gives a very good indication of the accuracy of an approach.

Currently all lights have an equal probability of being selected. So when I stumble upon a light I have two options:patlefort wrote:When your ray stumble upon a light source and you want to do MIS, you need to know the probability of it being selected. It's impossible to know if that probability without having sampled random points on all other light sources in your sampling method. I'm not sure if doing that make sense and will converge to a correct result.

1. The last vertex was specular, so there is no other strategy to sample the light, so the contribution is simply throughput * light color.

2. The last vertex was not specular, so we now have a second strategy, so MIS is used. For that I calculate the light pdf = dist^2 / solid angle, and the brdf pdf, which I stored when I did the diffuse bounce. The combined pdf is then simply the sum of these.

This part is working, and agrees with a reference renderer (not using next event estimation).

What also works is IBL by itself: I can sample it with next event estimation and importance sampling, and MIS works too (similar to what I just described). But when I want to combine IBL and lights things get tricky:

1. If I either sample a light *or* the skydome everything is clear; the probability of sampling the sky is now e.g. 50% of what it was, which causes no problems in MIS. Same for the lights. Any other probability also works just fine.

2. If however I chose a probability based on some heuristic (in my case: 'potential contribution') things go bad, even when I can reproduce that exact probability when I apply MIS, and even when it's never zero for a light or the skydome. That's the part that is puzzling me...

But isn't any (non-zero) probability fine, in principle? I am not skipping any lights, nor any part of the skydome.javor wrote:This sounds like you are mixing up the sampling probability with the contribution of the light source. If you have selected the light source without taking the visibility (or being infront/to the back of the point being shaded) into account, then the probability of this sampling strategy should be non-zero (and the sky probability < 1).

Conformance to reference is checked by the way by averaging the energy for each pixel and reporting this value. This quickly converges (even if there's tons of noise) and gives a very good indication of the accuracy of an approach.

### Re: MIS with IBL

Re-reading your first comment, you say that another light may be visible when sampling the sky. When you select the sky, you have to shoot a shadow ray just like any other light source. If that shadow ray is occluded by another light source, it must not contribute anything, because it's the sky being sampled and tested for visibility.

### Re: MIS with IBL

Just to be sure, the sample you're taking to calculate the probabilities is being discarded right after that and not being reused?jbikker wrote:Obviously, some parts of the scene benefit more from lights than the skydome. So, I want to change the probability of sampling the light or the sky depending on potential contribution. So I evaluate a random point on the skydome (importance sampled) and a random point on a random light. Based on these 'potential contributions' I determine the probability of sampling a light or the sky. This probability is recorded, so that when I stumble upon a light or the sky I can use it like I used the 0.5 in the previous scenario.

Out of curiosity, how do you notice that there's bias?Sadly, this introduces bias, although subtle, and I can't figure out what I am doing wrong.

### Re: MIS with IBL

Correct.stefan wrote:Just to be sure, the sample you're taking to calculate the probabilities is being discarded right after that and not being reused?

Well the setup is a simple box with a single-sided quad light well below the ceiling. One side of the box is open, letting in illumination from the skydome. The walls can thus 'see' the light up to a certain height. With the reference path tracer, the transition from 'below light level' to 'above light level' is somewhat fuzzy. The approach where lights or the sky are sampled with a custom probability causes this line to be unnaturally hard. Apart from that the overall energy level appears to be correct.stefan wrote:Out of curiosity, how do you notice that there's bias?

### Re: MIS with IBL

The problem is that the probability you use for choosing a light source is itself a random variable. If you write out the estimator you're building with your sampling techniques, its expected value won't be what you want. To get an unbiased estimator, you'll need to divide not by that random probability (which you got by sampling one random point on each light) but by its integral for all possible points on the lights, which is some unknown constant. Obtaining this in practice is obviously not feasible.

You can easily test whether this indeed is the problem by using a light selection probability that is only a function of deterministically known things at the shading point (i.e. every time you land at this point you will compute the exact same light selection probability).

You can easily test whether this indeed is the problem by using a light selection probability that is only a function of deterministically known things at the shading point (i.e. every time you land at this point you will compute the exact same light selection probability).

### Re: MIS with IBL

That is indeed correct: if I use a fixed probability for sampling the light or the sky, everything is OK.ingenious wrote:You can easily test whether this indeed is the problem by using a light selection probability that is only a function of deterministically known things at the shading point (i.e. every time you land at this point you will compute the exact same light selection probability).

You are correct that the light sampling / sky sampling probability itself became a random variable... So I am basically falling in the 'Resampled Importance Sampling' trap again.

- Jacco.