Update 2 on Exposure Render

There is now a stable release of Exposure Render, the open source direct volume renderer with physically based lighting on the GPU: http://code.google.com/p/exposure-render/

It now even runs on a laptop with a 8600M GT (and all the example datasets fit in the GPU memory). The photoreal lighting makes this renderer stand out when compared to other volume renderers. And it converges surprisingly fast. Some pictures (rendered for only 20 seconds on the ultraslow 8600M GT):

OTOY partners with Autodesk to stream real-time path traced graphics from the cloud

Awesome news!!! In addition to the partnerships with Refractive Software (makers of Octane Render) and the development team behind the Brigade path tracer (which I blogged about here), OTOY now announced a partnership with Autodesk to integrate Octane Render into the viewport of Autodesk's 3D Studio Max 2012 and stream real-time path traced visuals for animations, movies and games from remote GPU clusters in the cloud to the client. Real-time GPU path tracing is big business, and the best part is that it's going to bring game graphics up to ultrarealistic levels in the not too distant future.

VentureBeat article (includes an image of Octane's subsurface scattering implementation): http://venturebeat.com/2011/10/20/otoy-teams-up-with-autodesk-to-create-cloud-rendering-platform/

Yahoo biz (Business Wire): http://finance.yahoo.com/news/OTOY-and-Autodesk-to-Create-bw-395764800.html?x=0&.v=1

Some quotes from the article and official press release:

Real Time Path Tracing in the Viewport - A New Level of Photorealism

OTOY's 3D rendering tools, including Octane Render™ and Brigade™, are the premier rendering solutions for next generation 'path traced' games and films.

Path tracing significantly reduces the cost and complexity of high quality rendering by automatically generating effects traditionally handled through manual post processing – including spectral rainbows, lens flares, unbiased motion blur and depth of field.

“A year ago, path tracing was considered too expensive to be used even in high-end Hollywood blockbusters. Today, thanks to advances in GPU rendering, OTOY is bringing real time path-tracing technology to millions of artists by leveraging GPU hardware that costs only a few hundred dollars. This is a game-changer on any budget,” said Jules Urbach, CEO of OTOY.

“Autodesk is the leader in 3D design software for film and video game production. We are incredibly excited about our partnership and proud to be bringing their industry leading tools to an ever-expanding market through our cloud solutions,” said Alissa Grainger, President of OTOY.

One of the images in the VentureBeat article looks like the Kajiya scene (which can now be rendered at more than 10 fps with the Brigade engine, including caustics) in an IBL environment:

Very cool!

Real-time path tracing of animated meshes on the GPU (2): OGRE

Another test in my never relenting quest for real-time photorealistic graphics. This time I was inspired by one of the first animations rendered with unbiased Monte Carlo path tracing. The animation was made by Daniel Martinez Lara from Pepeland in 1999 and can be seen here:

http://www.youtube.com/watch?v=VNyknZ2zrsM

It’s one of the first animations that uses Arnold, the Monte Carlo path tracing based production renderer developed by Marcos Fajardo, that is currently taking Hollywood VFX by storm: it was used in e.g. Cloudy with a Chance of Meatballs, 2012, Alice in Wonderland and the Assassin’s Creed: Brotherhood and Revelations CG trailers and is giving PRMan and mental ray a run for their money (probably making them obsolete soon, mainly because of ease of use and huge artist time savings). The animation shows a very lifelike clay figure coming to life. Despite the simplicity of the scene, the whole looks very believable thanks to physically accurate global illumination and materials and an extensive use of depth-of-field and camera shake.

In an attempt to reproduce that particular scene, I’ve used the animated Ogre model from a ray tracing demo developed by Javor Kalojanov which can be found at http://javor.tech.officelive.com/tmp.aspx. The Ogre model (which was created by William Vaughan) consists of 50,855 triangles and was also used in the excellent paper “Two-level grids for ray tracing on GPUs” by Javor Kalojanov and Philipp Slusallek (really great people btw, whom I've had the pleasure to meet in person recently. The conversations I've had with them inspired me to finally try triangles as primitive for my real-time path tracing experiments (instead of just spheres), which led to this Ogre demo. To my surprise, triangle meshes are not that much slower to intersect compared to spheres. I think this is due to the fact that the cost of primitive intersection is becoming increasingly smaller compared to the cost of shading).

The following videos show an animated Ogre path traced in real-time with real-time, per frame update of the acceleration structure of the Ogre’s 50k triangle mesh (watch in 480p):

http://www.youtube.com/watch?v=qPFjBuVEGec

http://www.youtube.com/watch?v=z7IjCiazdJ0

http://www.youtube.com/watch?v=u0a0mPR8jdM

Path tracing is performed entirely on the GPU, in this case a GTS 450 (a low-end GPU by today’s standards). The framerate of the walk animation is about 4 fps max on my card but should be around 15-20 fps on a GTX 580. The image converges extremely fast to a very high quality result (in about 1-2 seconds). The movement of the Ogre (translation, rotation, animation) is actually much more fluid in real life without Fraps, the overhead of the video capturing software almost halfs performance.

The images below were each rendered at 20 samples per pixel in under 2 seconds on a GTS 450 (it would take less then 0.5 seconds on a GTX 580):

Note the very subtle color bleeding from the ground plane onto the Ogre:

With a glossy floor:

If you’re interested in trying this scene out yourself, send me an e-mail at sam [dot] lapere [at] live [dot] be. A CUDA enabled GPU is required (minimum compute capability 1.1).

I’m planning to build a (very) simple game with this tech. The possibilities are really endless. We're on the cusp of having truly photorealistic games.

Real-time path tracing of animated meshes on the GPU (1)

Another experiment: no spheres this time, but a real-time animated triangle mesh (the hand is from the Utah 3D Animation Repository, a textured mesh containing 15,855 triangles). The goal was to create a simple animated scene and achieve a look as close to photorealism as possible with completely dynamic, physically accurate global illumination in real-time using path tracing.

The following animation was rendered in real-time on a GTS 450 (stretching the compute capabilities of my GPU to the max):

http://www.youtube.com/watch?v=akZNUe0L8y8

Details will follow.

The image below was rendered with path tracing at 26 samples per pixel in 0.8 seconds on a GeForce GTS 450 (192 cuda cores at factory clocks, it would render 3 to 4 times faster on a GTX 580, which has 512 cuda cores and is clocked higher):

Update on Exposure Render

Two new great looking videos of Exposure Render, the real-time CUDA-based volume path tracer have appeared on Youtube. It's quite cool to see these anatomical volume datasets being rendered with physically based shading in near real-time:

http://www.youtube.com/watch?v=cZaPIEo6PPs
http://www.youtube.com/watch?v=4D2HfJ5Cwqc


Someone has also put the video from the paper "On filtering the noise from the random parameters in Monte Carlo rendering" on Youtube, making it more convenient to retrieve and watch:

http://www.youtube.com/watch?v=ZQ6TFZE5QEU

The results for filtered images using only 8 input samples per pixel are extremely impressive.

Unbiased subsurface scattering on the GPU in next version of Octane Render

Wow, the developers behind Octane render never cease to amaze. After being the first GPU renderer to implement Population Monte Carlo (a more complex rendering method than plain path tracing which borrows concepts from Metropolis light transport and energy redistribution path tracing to handle scenes with difficult lighting more efficiently), Octane render is now adding unbiased subsurface scattering along with other features such as instancing. Here's the announcement post from radiance (Octane's main developer) over at the Octane forum:

"We have fully working and VERY fast SSS ready for release in the next test version. It renders about as fast (a tiny bit slower) as a glossy specular material. And, it's unbiased/bruteforce SSS, eg no bias introducing photon grids or other precomputed approximations.
[...]
This is the prime new feature in the next test release, along with instancing support and FAST voxelisation, and another suprise feature, aswell as the soon to be publically released first of a series of new products, OctaneRender for 3DS Max."

Surprise feature? Refractive Software knows how to keep their audience hyped ;-)

Some screenshots with the new SSS method can be seen in this thread.

I think it will eventually be possible to implement every feature found in traditional CPU renderers on the GPU and make it an order of magnitude faster. For example, Radiance hinted at bidirectional path tracing + PMC:

"Bidirectional pathtracing (and PMC) should make renders like this one converge MUCH faster and bidirectional pathtracing + PMC is something we will be starting work on next, after 2.5 is out.
PMC + bidirectional will be ideal, it will be as efficient as the popular standard in CPU based unbiased engines (MLT+bidir), and this combined with the power of GPUs should really take things to a new level."

GPU rendering is going to redefine every area of rendering from movies, animation, visualization and design to games, simulation and virtual reality. Truly the most exciting time for rendering in decades. I'm very happy that this paradigm shift is in full swing and that things are evolving at nauseating speed :-)

Real-time physically based volume rendering with CUDA

Just saw a very impressive video of "Exposure Render", an open source CUDA based volumetric path tracer which will be presented at EuroGraphics 2012:

http://www.youtube.com/watch?v=qzFv0draRG8

The video shows an eerily realistic representation of a person scanned with CT (computed tomography), where anatomical layers of skin, muscle, cartilage, connective tissue and bone can be peeled away and rendered with photorealistic quality global illumination in real-time. This could be an extremely useful training tool for medical students and aspiring surgeons. The physically based photorealistic 3D renders of skull, bones and soft tissue could also benefit radiologists in diagnostic decision making.

Most medical volume rendering software uses ray casting, which is fast but provides very unrealistic looking lighting. With volumetric path tracing finally being feasible in real-time thanks to the GPU, this will hopefully change soon. I think we will see an implementation of real-time path traced sparse voxel octrees pop up sometime in the very near future.

An executable and source code (Oct 3rd) are available at http://code.google.com/p/exposure-render/

In-depth video on micropolygons in OpenGL

Anyone interested in OpenGL and micropolygons should definitely check out the following 1-hour-long video presentation by Kayvon Fatahalian, "Evolving the OpenGL Graphics Pipeline in Pursuit of Real-Time, Film-Quality Rendering":

http://www.youtube.com/watch?v=KfGX73NOA6I

Lots of interesting insights in the future of GPU hardware.

Real-time dynamic GPU path tracing

A small teaser of something I'm working on, rendered in real-time on a GeForce GTS 450:

http://www.youtube.com/watch?v=1Q0z2E_mfXY

12 samples per pixel, max path length = 4, 584x266 render resolution, Stanford Bunny with 69k triangles (rendered in about 0.4 seconds on a low-end GTS450):

This will eventually become a full-fledged game inspired by this video:

http://www.youtube.com/watch?v=pmu5sRIizdw

A few details on Imagination's PowerVR GPU with Caustic Graphics hardware ray tracing

A few days ago, an article appeared on the Japanese website 4gamer.net about an in-development PowerVR GPU, the PowerVR RTX, which is said to have integrated ray tracing hardware from Caustic Graphics (acquired by Imagination Technologies in December 2010). The article is in Japanese, and the Google translated version is barely understandable:

http://translate.google.com/translate?hl=en&sl=auto&tl=en&u=http%3A%2F%2Fwww.4gamer.net%2Fgames%2F017%2FG001762%2F20110920023%2F

Video of Unbiased FPS + a new video of Brigade!

The following two videos of "Unbiased FPS" were rendered on a GTS 450 and demonstrate mouse look, shooting mechanics and how adjusting the number of averaged frames affects the amount of noise and blurring. Youtube has a terrible video compression algorithm for videos containing fine-grained noise (such as Monte Carlo noise), so these came out much worse than expected:

The frame averaging technique (implemented by Kerrash) significantly reduces the amount of noise without affecting the framerate, but the resulting blurring of fast moving objects is of course less than ideal. At the upcoming Siggraph Asia 2011, a paper entitled "Image-space bidirectional scene reprojection" will be presented which could potentially solve the blurring issue by reconstructing additional frames without distracting blurry artefacts (there are two versions, a raster based version and an image-based one). The video accompanying the paper shows a lot of promise and is definitely worth checking out: http://research.microsoft.com/en-us/um/people/hoppe/proj/bireproj/

On a sidenote, I found a new, short video of the Brigade path tracer, showing a real-time pathtraced animated character: http://www.youtube.com/watch?v=RbxoZK6Apj8 (no details on the hardware used)

The video shows very nice soft shadows and path traced ambient light coming from the skydome (calculated in real-time contrary to the constant ambient term which is used by many games to approximate indirect lighting).

Unbiased FPS

Another small experiment to see how a path traced first person shooter game might look like (all screenshots rendered on a 8600M GT):

4spp, reusing samples from the previous 6 frames, 720p resolution:


720p resolution, 4spp with 12 averaged frames (reusing samples from the previous 12 frames, resulting in heavy blur for moving objects, but crisp image quality for the rest of the scene):

Some features:

- a first person "gun" sticks to the camera and shoots chrome balls with physical weight

- mouse look is semi-functional, you can move the mouse cursor up to the borders of the screen

- the car can be controlled with the arrow keys

- the robot has an initial angular momentum, but essentially does nothing

- the camera is controlled with mouse + WASD keys (the classic FPS controls)

- the number of averaged frames can now be set with a slider (from 0 to 12). It regulates the amount of "blur" caused by reusing samples from previous frames and is very useful for comparison purposes. Setting the slider to 1 or higher greatly reduces the amount of noise at the expense of a more blurry image and doesn't have any impact on the framerate. You can for example quadruple the number of samples per pixel (thereby halving the amount of noise) for free and if the framerate is high enough, the blurring isn't even that distracting. Setting the slider to 12 with just 4 spp results in a very high quality image (effectively more than 100 spp) at high framerates, but at the expense of lots of motion blur when the car or camera are moving

There are no acceleration structures used (not even a bounding box around the car or the robot). Implementing those should give a nice ray tracing performance boost.

Executable is available at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Video of Octane Render rendering in real-time on 8x GTX580s

I just saw a very impressive video on Youtube, showing real-time rendering of a complex interior scene with Octane Render using eight GTX 580s. Octane Render has very recently improved its "direct lighting/AO" kernel which includes a new ambient occlusion preset and a diffuse indirect lighting option (there are now separate sliders for specular, glossy and diffuse path depth and a slider for AO distance) so that it is now capable of rendering very realistic looking interior scenes extremely fast:

http://www.youtube.com/watch?v=CZ1IRQTqMMY

Screengrab from Youtube:

Some observations:

- in contrast to the type of scenes that is usually used to show off unbiased renderers (wide open outdoor scenes with shiny cars and lots of direct lighting from a skydome or sun/sky) this is an interior scene that is mostly lit by indirect lighting with many glossy surfaces

- the scene in the video contains more than 1.3 million triangles

- rendered at very high resolution (1500x1000 pixels)

- perfect scaling with number of GPUs (99-100% extra performance per additional GPU)

- while navigating through the scene, the image remains remarkably clear and recognizable without ever degenerating into a pixelated mess of big, blocky (and often black) pixels

- convergence to a noise-free image is extremely quick

It's not hard to imagine that this renderer will be truly real-time with the upcoming generation of GPUs (according to Nvidia, Kepler will be more than twice as fast at path tracing as Fermi, just like Fermi is 2-4x as fast as Tesla (GT200) thanks to caches and other improvements (see http://www.youtube.com/watch?v=0IC2NIogWR4) and AMD's Graphics Core Next will be much more focused on GPGPU computing than previous architectures). These graphics can be rendered noise-free in real-time at high resolution with techniques like adaptive sampling, image reconstruction, compressed sensing (a hot topic in medical imaging currently), edge-aware filtering of indirect lighting (e.g. the a-trous wavelet noise filter), extraction of spatiotemporal coherence with reprojection, path regeneration, reusing samples with frame averaging and frameless rendering. Rendering complex, 100% photorealistic scenes in real-time is much closer than commonly believed and cloud rendering will play a key role in accelerating this process.

UPDATE: another interior animation rendered on a Nvidia GTX 590 with Octane Render's new indirect diffuse lighting/AO kernel: http://www.youtube.com/watch?v=O2Aufjr44g4Hi
Rendertime per frame: ~1 minute, 1024 samples per pixel (see screenshot in this thread). This is absolutely amazing...

Unbiased Stunt Racer in HD!

I've changed the 'Unbiased Stunt Racer' demo a little bit: I've added the buildings from my Futuristic Buildings demo to the scene and also added a function to adjust the speed of the physics simulation at runtime with the N and M keys. The platform and sphere are also moving at a speed that's independent of the framerate, so the game is still playable on high-end cards like the GTX 480 and higher. The new executable contains 480p and 720p executables. I find it mindblowing that real-time path tracing is feasable today at HD resolutions using only one GPU.

Below is a 720p image of the new demo, rendered on a 8600M GT at 8 samples per pixel:

Some 480p screenshots (all captured on a 8600m GT):

A video will follow soon and the source code will also be uploaded. The code is very messy with lots of obsolete comments and there are heaps of global variables, which isn't good programming practice but it does the job. :)

The executable "Unbiased Stunt Racer HD Final" is available at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

This will be one of my last demos involving only spheres and boxes as primitives. The limitations on the number of objects and shapes are starting to outweigh the benefits of these cheap-to-intersect primitives, so I'm moving to the wonderful world of triangles and BVHs soon.

UPDATE: source code is online at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

UPDATE 2: sneak peek at an upcoming demo (robot and car will be user controlled and "shootable")

UPDATE 3: Finally found some time to make an HD video of Unbiased Stunt Racer HD:

720p "fly-through":

480p gameplay:

Executable and source can be found at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list




There is also a new paper and video on reducing Monte Carlo rendering noise called "Random parameter filtering" with amazing results (thanks to ompf.org forum): http://agl.unm.edu/rpf/ The video is definitely worth the download and shows great potential for real-time path tracing.

Unbiased Stunt Racer!

For the last couple of days, I've been busy creating "Unbiased Stunt Racer", a new real-time path traced demo involving physics, a vehicle and a shooting robot. It's actually a mix of a platformer and a stunt driving game. First you have to reach a platform by driving the vehicle over a ramp, then you have to reach another platform with a robot which is shooting at you, by driving over two rails where a nasty chrome sphere hinders the passage. The final goal is to push the robot off of his cosy place. The sunlight is constantly changing position and color, casting nice soft shadows over the scene. Below is a walkthrough video (rendered on GTS450) and some screenshots:

The number of samples per pixel can now be altered at runtime with the O and P keys. Pressing space resets the car. I've converted some of the stratified sampling in the original code back to purely random sampling, resulting in a slightly noisier but more realistic image.

An executable demo for Unbiased Stunt Racer is available at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

The biggest advantage of using real-time ray tracing/path tracing for these "games" (besides having higher quality graphics) is that almost anyone can immediately start creating simple games without worrying about lighting, shadows, multiple reflections, refractions and all combinations of these effects. Everything just works as expected and without any effort on the artist side. It's like a "push button" game engine: just create the scene and the renderer takes care of everything else.

New video of Unbiased Physics Games

Changed the ground color and brightened up the sky in preparation for an upcoming demo called "Unbiased Stunt Racer" (courtesy of Anonymous blog reader :). It looks a little bit more realistic now (480p video on GTS 450):

"Unbiased Physics Games v1.0" executable demo at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

OTOY partners with GPU path tracing renderers Octane and Brigade!

Wowowow... this is superawesome news! An article on Venturebeat today revealed that OTOY will be partnering with two fantastic renderers, Octane Render from Refractive Software and Brigade from Jacco Bikker, both pioneers in interactive/realtime high quality GPU based rendering.

I've blogged about these three developers before, and the fact that all three will be joining forces is just breathtakingly awesome! It's a dream come true:

- Octane Render is by far the fastest, most feature rich and highest quality unbiased GPU renderer, trouncing traditional biased and unbiased CPU renderers

- Brigade is a realtime CPU/GPU path tracer targeted at games (with many noise reducing optimizations) developed by Jacco Bikker, who also developed the GPU path tracing code used in my latest demos

- OTOY is an emerging cloud rendering/cloud gaming company (probably best known from the AMD Radeon 4800 and 5800 launch events and the Ruby demo which used voxel ray tracing) that also owns LightStage, a state-of-the-art dome-shaped capturing environment for creating 100% photorealistic digital actors, which was used in blockbuster movies like Benjamin Button, Avatar and The Social Network

This new partnership means that there is now a very high concentration of high-quality rendering know-how and I've got a feeling that there will be more to come. This will have a profound impact on the movie/game/rendering industry... I'm extremely happy and excited to see what's coming from all this :D.

The VentureBeat article is here: http://venturebeat.com/2011/08/30/otoy-cloud-video-games/

UPG: Unbiased Physics Games!

I've been working on some new real-time path traced demos during the past week involving physics, driving and shooting, three popular ingredients in action games. The demos will be bundled under a new 3 letter acronym, i.e. UPG (GPU in reverse), which stands for Unbiased Physics Games.

I've created two new small games:

- a simple driving game where you can pull some driving stunts with lighting effects (the camera follows the car):

- a simple shooter game where you can hit the car and a robot-like character which will respond to the impact of a cannonball:

Both games use the GPU path tracer developed by Jacco Bikker for the Simplex Paternitas demo. The videos were recorded on a GTS450 with 16 samples per pixel (which already offers very good image quality with little noise) at a resolution of 640x480.

The robot and car in the second demo are both encapsulated by a "hitbox" (an axis aligned bounding box to maximize the path tracing performance), which position is updated every frame.

I'm going to implement a movable and "shootable" light hanging from a chain, which should create dramatic shadow effects. Some basic AI path finding code so the robot starts chasing and shooting the user-controlled car might also be an option. Still thinking about implementing oriented bounding boxes which would enable some cool collapsing structures. Plenty of ideas still and new ones are coming every day, there's just not enough time to execute all of them :) On the rendering side, I think I'll shift the geometric focus of the demos to triangle meshes and two-level BVH or grid, because there's only so much you can do with spheres and boxes - even though they're relatively cheap to intersect.

I've also tried porting the code to OpenCL with the recently released OpenCL1.1 drivers from Nvidia but the OpenCL kernel refuses to build for some reason and there's no easy way to find out what's causing the error. It will take some time to get this right.

Download the executables at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Ray tracing in CryEngine 3!

Crytek has pioneered a lot of well known rendering techniques like screen space ambient occlusion and light propagation volumes for real-time dynamic global illumination. Proving that they're still at the forefront of real-time rendering, Crytek revealed at Siggraph that they are using a limited form of ray tracing in CryEngine 3 to enable real-time local reflections. The technique involves screen space ray marching along the direction of the reflection vector. More info about this method and lots of other interesting details in their Siggraph presentation at http://advances.realtimerendering.com/s2011/index.html

It's nice to see that ray tracing is finally being used by a leading game developer (without a doubt the most talented PC game engine developer of the moment), albeit in a limited form. If SSAO is any indication, we'll be seeing ray tracing based reflections in all of the other major game engines within the next year. From there, it's only a small step to use full ray tracing for reflections, shadows, refractions and ultimately global illumination. :)

Unbiased Truck with Plausible Bullet Physics!

The physics in the previous demo were funny and also a bit strange, because the chassis dove down when the wheels were going up. I discovered that the Z-angle of the chassis had to have the same sign as the Z-angle of the wheels, which gives a much more plausible result, although the wheels are still quite jerky and are jumping around because of the loose suspension. The car will pull itself to its original state in most cases, so I've decided to leave the wonky physics as is.

Rendered on 8600m gt:



Unbiased truck test drive in the Cornell Box:

New executable at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Some people reported problems with the last executable, so I included an updated version of "cudart.dll" this time (the CUDA 4.0 version).