understanding sampling, filters, motion blur, DoF,...

[MohamedSakr]

forgive me for asking such a noob questions (and very general), but really I think I can't get the full image of how ray tracing works!! "not about the algorithm itself like (path tracing, VCM, BDPT, ...,) but about the components of a ray tracer like (sampler, filter, acceleration structure, ...)"

**correct me as I'm going, I may be understand things completely wrong
1- about samplers "like LD, random sampler, MLT, etc,...", what I understand is that this is used only to get a good (x,y) position inside a pixel to initialize a camera path, is this correct?
2- about filters "I read about them from PBRT" like "box filter, gaussian, etc...", what I understand (from image processing) is that they are a fast post process to clean the image (may be for anti aliasing) , is this correct? //I sense I can't understand filters so far
3- how we can do anti aliasing? "from commercial render engines, I see anti aliasing is increasing render time almost linearly, so I guess it is just similar to rendering a higher res image and averaging result into smaller image
4- how we can simulate motion blur? "I tried to figure out how to intersect a ray hitting a triangle at time T, how is this possible? we store multiple BVHs for each time T?", and how we treat the results with respect to physical camera shutter speed
5- how we can simulate DoF? "from what I understand we are shooting from the same pixel multiple rays in a conic angle (instead of discrete direction in pinhole camera), is this correct? how we treat the result with respect to physical camera FStop?
6- in BDPT, we sample also lights, how do we do this? (I only encountered MIS and MLT) "so LD may be used also?? like how we treat camera samples"
7- for BRDFs, is there any special treatment with any sampler? "what I understand is that it takes an incident ray (hit,direction,power) and with probabilities/BRDF function gets outgoing ray (direction and power)


feel free to let me understand how things works(hope you won't get angry as the questions looks very bad ), mention any related good papers that clarifies these stuff

[sriravic]

Okay. So I'll try to answer your questions one by one in the order you've listed.

1. The primary objective of a sampler is to provide samples in a generic domain. They are not related to pixels per se but are used to get good sampling locations within a pixel [domain]. Think of the area of the pixel as the domain and the sampler picks good samples from this domain. You can extend this to any other domains also like sampling on a light [the physical light itself is the domain and you try to get good sampling locations on this light], sampling the lens[the lens itself is the domain]. This also would apply to abstract concept of light paths or camera paths [they are domains, and you can sample them i.e. choose one or many paths in the entire domain of paths].

2. Filtering from the perspective of image processing is used not only to do anti-aliasing, but it can be used to do more. You can have a clamp filter to do thresholding operations on the image, etc. Think of filter as the generic filter you use in daily life - filter something. PBRT has an excellent material on filtering, but I suggest you can look at any image processing texts to understand this.

3. Anti aliasing is a very broad topic and can be done in many ways. The most simplest would be as you suggested would be to render a bigger version of the image and downsample it(average). But this is not feasible most of the times and we have many filters that can perform better with very less samples. I suggest you take a look at this http://iryoku.com/aacourse/ for how different aa methods are used(this is a real time scenario but the theory should hold I guess).

4. Well for motion blur, the solution is a little bit tricky. You have a time field associated with each object as well as the ray. So when a ray and triangle is intersected, we estimate how much the triangle has moved from the beginning by linearly interpolating between start and end time. More over we also do the same for the ray. If the ray intersects the object during that interpolated time, then its recorded as a hit. So you can see how much problems that this can create. It might be that nearby pixels would test positive and negative at the same time which normally you'd not expect. Hence we need a lot of samples (time is the domain - 1st question) and we sample in the time domain to get more samples for rays to get smooth motion blur effects.

5. Similarly to motion blur, depth of field is simulated by sampling in the lens. Normally, the thin lens model is such a way that different points on the lens leads to different kinds of rays which would simulate depth. I suggest you read wiki articles on thin lens model(try to see the ray diagrams in wikipedia to understand further)

6. In BDPT, we sample lights similar to camera samples albeit differently. First if we have many lights in the scene, we have to randomly choose some lights(according to some pdf). Then we must choose a light sample (on the light itself) and estimate the radiance from that light sample. Once we are done with this, the random walk of the light path is almost the same as camera path (bsdf and adjoint bsdf's come into picture here -Refer to Veach's thesis). So you can use the sampler to sample the domain in any way (uniform sampler to choose uniformly among the lights, stratified sampler to choose the light points on the light surface in a stratified manner).

7. Well I guess wrt BRDFs, samplers are used to sample directions for evaluating radiance or importance. You have to understand that its the BRDF is an analytical function and evaluating it at every point is infeasible. So you use sampling to sample points the BRDFs domain (which is angular directions in the hemisphere and area(SSS)). Once the sample direction is chosen, we can then feed the values in the brdf function and get the contribution as well as probability of choosing this direction(pdf is used in monte carlo calculations).

Hope this clears your doubts.

[MohamedSakr]

@sriravic thanks a lot!!! you really made my day

4. Well for motion blur, the solution is a little bit tricky. You have a time field associated with each object as well as the ray. So when a ray and triangle is intersected, we estimate how much the triangle has moved from the beginning by linearly interpolating between start and end time. More over we also do the same for the ray. If the ray intersects the object during that interpolated time, then its recorded as a hit.

just 1 more question about motion blur , I'm confused about how we intersect rays with objects, if objects BVH are stored at time T1, and rays are trying to hit at time T2, how will they intersect?
it is like intersecting ghost objects

[sriravic]

You never store objects with time. You store animated transforms (transforms that have a time value associated with them. PBRT to the rescue again.). With this animated transform, you can find the position of the object at any time and use that for intersecting.

Note that animated transform blur (or transformation motion blur) is notoriously complicated and its a production essential feature.

[cessen]

Note that animated transform blur (or transformation motion blur) is notoriously complicated and its a production essential feature.

I found it to be pretty straight-forward, actually. I feel like its reputation for being difficult/complex comes from the days when e.g. KD-trees were the preferred acceleration structure for ray tracing. But with BVH's it's pretty trivial.

To start off, consider a case where we only have one triangle to intersect. We have the location of the triangle vertices stored for the beginning and end of the frame (corresponding to times 0.0 and 1.0 respectively). If you have a ray with time 0.35 and you need to test it against the triangle, you interpolate the vertices of the triangle to time 0.35 and test the ray against that interpolated triangle.

Now let's say we want to test against the triangle's bounding box before testing against the triangle. So we create the bounding box for the triangle at time 0.0, and the bounding box for the triangle at time 1.0. To test against the bounding box, we again can just do a simple interpolation of the bounding box to time 0.35 and test the ray against it. If it hits the bounding box, then we test against the triangle as in the previous paragraph. If it misses, we skip.

Now let's say we have two triangles, each with bounding boxes, and we want to test against a parent bounding box before testing against the children. Again, create the bounding box at time 0.0 and time 1.0, and then interpolate to test. If it hits, test against the children. If not, skip.

Then you can have a parent of the parent, and so on, and you have a bvh for motion blur. This is deformation motion blur, of course. But transformation motion blur isn't too much more difficult. You store two transform matrices, and interpolate between those, and transform the ray before testing. It gets a little trickier if you want to decompose the matrix before interpolating (specifically, making sure you get the bounds right). But the concepts are pretty straight-forward. It also get slightly trickier with multi-segment motion blur. And for a well-tuned implementation you really only want to store multiple bounds for parts of the BVH tree that actually have motion.

But the basic idea is quite straight-forward, and my experience was that a simple implementation just to get your feet wet is fairly easy.

[MohamedSakr]

@cessen thanks a lot for clarification for this part , so far what I understand is:
at any frame N, we need 2 BVHs, for the current frame N, and for the next frame N+1, and we interpolate between both of them depending on the chosen sample

**correct me as I'm trying to justify my understanding:
so considering a very slow shutter speed "like 1 sec", and a frame rate of 24 fps, this means at frame N we will need samples from N to N+24 ? making it needs 25 BVHs considering animation is dense (all frames are filled with different positions)

[friedlinguini]

@cessen thanks a lot for clarification for this part , so far what I understand is:
at any frame N, we need 2 BVHs, for the current frame N, and for the next frame N+1, and we interpolate between both of them depending on the chosen sample

**correct me as I'm trying to justify my understanding:
so considering a very slow shutter speed "like 1 sec", and a frame rate of 24 fps, this means at frame N we will need samples from N to N+24 ? making it needs 25 BVHs considering animation is dense (all frames are filled with different positions)

You could do it that way, but I'm not sure why you would. Motion blur is essentially anti-aliasing, but in the time domain instead of the raster domain. If you were trying to render an image that had a 24-pixel wide filter function, you could sample across that entire region of the image for every pixel and wait for the image to eventually converge, but it's probably faster to just use a 1-pixel wide filter and blur the whole thing as a post-process. Similarly, if you have a shutter speed that is longer than one or two frames for some reason, you could sample across that entire time domain, but it's probably just easier to render a bunch of frames with a 1-frame shutter speed and then blend a bunch of frames together.

Why do you want a shutter speed that is slower than your frame time? Is there some analogue for physical cameras?

[cessen]

so far what I understand is:
at any frame N, we need 2 BVHs, for the current frame N, and for the next frame N+1, and we interpolate between both of them depending on the chosen sample

When you say "2 BVHs" it makes me think that you mean two separate trees, which is wrong. There is only one tree. But each node has more than one bounding box, which represent the bounds of that node at different times.

For a simple "getting your feet wet" implementation I would recommend giving each node two bounding boxes, representing the bounds at the beginning and end of the frame. But in a more advanced implementation, some nodes might only have one bounding box (i.e. none of its children are moving, and so don't need multiple bounds), and others might have eight or more bounding boxes (for fast curved motion). So it's important to keep your mental model as multiple bounds per node, not multiple BVHs.

**correct me as I'm trying to justify my understanding:
so considering a very slow shutter speed "like 1 sec", and a frame rate of 24 fps, this means at frame N we will need samples from N to N+24 ? making it needs 25 BVHs considering animation is dense (all frames are filled with different positions)

When I say "frame" I mean the span of time of the image being rendered. If you have an image that spans one second of time, then the frame is one second long. So for a simple "getting your feet wet" implementation, given your example, just take one sample from N, and one sample from N+24.

And when I say "sample" in this context, I am of course referring to getting geometry and creating bounding boxes. The rays themselves should have random times between 0.0 and 1.0, so that the rendered motion blur converges to a smooth result. I hope I'm not being too confusing.

[MohamedSakr]

@friedlinguini thanks, I totally understand what you mean , this may be an option in the renderer

@cessen you clarified a really critical part here about BVH for me , actually my experience in accelerated ray intersection structures was in Octrees, so it was kinda straight forward
but about BVH, I have read about SBVH, MSBVH "which is what we are discussing here for interpolating bounding boxes"

1- big question about BVH to understand it more, what is the structure of the BVH, I thought it is just some bounding boxes which contains a set of triangles, and I won't lie I've seen the term "node" and I didn't understand it that time , so I may need a clarification of the basic structure of a BVH

2- what I know is building SBVH is really slow, so do I build it at every different frame? "seems logical for animations", what about multiple bounding boxes as you stated? "are they fast to build or will require a huge extra time"

[tarlack]

@friedlinguini : don't you have structured aliasing patterns with your approach ? I remember blender doing this, and the number of images to avoid ghosting effects were large as soon as an object was going really fast relatively to the aperture time. For instance, reproducing a long exposure time photo (typically with these nice and so appealing curved headlights trails in an urban setting) requires a continuous sampling (or a prohibitively large number of images), maybe with some time-domain filtering.

For motion-blur and raytracing, it seems to me that having two bounding boxes and interpolating between them does not allow to correctly take into account non linear transformations, such as rotors, a drifting car, or even the path of an object with very long exposure times. Maybe a more accurate way would be to have a single BVH in 4D ? Or, for an more intuitive point of view, a standard 3D BVH, where the bounding box of each object is simply the union of the bounding box of the object at all times during the aperture time. Then, when you hit a bbox, you compute the exact intersection for your ray's t value, for which you can use an acceleration structure based on time specific to the object to accelerate this computation. This way I think it should be possible to handle any non-linear motion blur.