OTOY partners with GPU path tracing renderers Octane and Brigade!

Wowowow... this is superawesome news! An article on Venturebeat today revealed that OTOY will be partnering with two fantastic renderers, Octane Render from Refractive Software and Brigade from Jacco Bikker, both pioneers in interactive/realtime high quality GPU based rendering.

I've blogged about these three developers before, and the fact that all three will be joining forces is just breathtakingly awesome! It's a dream come true:

- Octane Render is by far the fastest, most feature rich and highest quality unbiased GPU renderer, trouncing traditional biased and unbiased CPU renderers

- Brigade is a realtime CPU/GPU path tracer targeted at games (with many noise reducing optimizations) developed by Jacco Bikker, who also developed the GPU path tracing code used in my latest demos

- OTOY is an emerging cloud rendering/cloud gaming company (probably best known from the AMD Radeon 4800 and 5800 launch events and the Ruby demo which used voxel ray tracing) that also owns LightStage, a state-of-the-art dome-shaped capturing environment for creating 100% photorealistic digital actors, which was used in blockbuster movies like Benjamin Button, Avatar and The Social Network

This new partnership means that there is now a very high concentration of high-quality rendering know-how and I've got a feeling that there will be more to come. This will have a profound impact on the movie/game/rendering industry... I'm extremely happy and excited to see what's coming from all this :D.

The VentureBeat article is here: http://venturebeat.com/2011/08/30/otoy-cloud-video-games/

UPG: Unbiased Physics Games!

I've been working on some new real-time path traced demos during the past week involving physics, driving and shooting, three popular ingredients in action games. The demos will be bundled under a new 3 letter acronym, i.e. UPG (GPU in reverse), which stands for Unbiased Physics Games.

I've created two new small games:

- a simple driving game where you can pull some driving stunts with lighting effects (the camera follows the car):

- a simple shooter game where you can hit the car and a robot-like character which will respond to the impact of a cannonball:

Both games use the GPU path tracer developed by Jacco Bikker for the Simplex Paternitas demo. The videos were recorded on a GTS450 with 16 samples per pixel (which already offers very good image quality with little noise) at a resolution of 640x480.

The robot and car in the second demo are both encapsulated by a "hitbox" (an axis aligned bounding box to maximize the path tracing performance), which position is updated every frame.

I'm going to implement a movable and "shootable" light hanging from a chain, which should create dramatic shadow effects. Some basic AI path finding code so the robot starts chasing and shooting the user-controlled car might also be an option. Still thinking about implementing oriented bounding boxes which would enable some cool collapsing structures. Plenty of ideas still and new ones are coming every day, there's just not enough time to execute all of them :) On the rendering side, I think I'll shift the geometric focus of the demos to triangle meshes and two-level BVH or grid, because there's only so much you can do with spheres and boxes - even though they're relatively cheap to intersect.

I've also tried porting the code to OpenCL with the recently released OpenCL1.1 drivers from Nvidia but the OpenCL kernel refuses to build for some reason and there's no easy way to find out what's causing the error. It will take some time to get this right.

Download the executables at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Ray tracing in CryEngine 3!

Crytek has pioneered a lot of well known rendering techniques like screen space ambient occlusion and light propagation volumes for real-time dynamic global illumination. Proving that they're still at the forefront of real-time rendering, Crytek revealed at Siggraph that they are using a limited form of ray tracing in CryEngine 3 to enable real-time local reflections. The technique involves screen space ray marching along the direction of the reflection vector. More info about this method and lots of other interesting details in their Siggraph presentation at http://advances.realtimerendering.com/s2011/index.html

It's nice to see that ray tracing is finally being used by a leading game developer (without a doubt the most talented PC game engine developer of the moment), albeit in a limited form. If SSAO is any indication, we'll be seeing ray tracing based reflections in all of the other major game engines within the next year. From there, it's only a small step to use full ray tracing for reflections, shadows, refractions and ultimately global illumination. :)

Unbiased Truck with Plausible Bullet Physics!

The physics in the previous demo were funny and also a bit strange, because the chassis dove down when the wheels were going up. I discovered that the Z-angle of the chassis had to have the same sign as the Z-angle of the wheels, which gives a much more plausible result, although the wheels are still quite jerky and are jumping around because of the loose suspension. The car will pull itself to its original state in most cases, so I've decided to leave the wonky physics as is.

Rendered on 8600m gt:



Unbiased truck test drive in the Cornell Box:

New executable at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Some people reported problems with the last executable, so I included an updated version of "cudart.dll" this time (the CUDA 4.0 version).

Unbiased Truck with Bullet Physics!!

After lots of trial-and-error, I have finally implemented car physics in the real-time path traced Unbiased Truck demos. Bullet doesn't like it when physical objects are not placed according to Newtonian rules, which has caused some unexpected behaviour and crashes.

The Bullet SDK provides a vehicle physics example with parameters for wheel friction, suspension, chassis stiffness, etc. The "car" consists of a rigid chassis with four independent wheels. Matching the physics world with the rendered world took some time to get right: for example, since each of the four wheels can move independently from each other and from the car chassis, each "rim" had to match up perfectly with its respective wheel.

Crashing the car into the walls is just plain fun. When hitting a wall, the car seems to disintegrate at first, after which it pulls its wheels and chassis together again.

The video below was rendered on a 8600M GT at low resolution, spp and max path length to achieve a somewhat playable framerate (it should look much better on a high end Fermi GPU):

512x256 resolution, max path length=3, 4 samples per pixel/frame:

256x256 resolution, max path length=4, 4 samples per pixel/frame:

A better video with higher framerate and image quality will follow soon. The new executable demo "Unbiased Truck Bullet Physics" can be downloaded from:

http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list


This is all work in progress. I'm planning to port this to the GPU path tracer developed by Jacco Bikker (which was also used in the Futuristic Buildings demos), which has support for AABB acceleration structures, better shadows, and which generally runs faster. A camera will follow the car, not only by targeting the car's center, but by moving along with it behind the car. Some basic shooting mechanics will be included as well, and maybe also some ragdoll physics with one of these:

WebGL water ray tracing demo

I always like a nice real-time ray tracing demo, especially when it involves physics. Evan Wallace released a very impressive WebGL demo featuring real-time raytraced water with caustics and physics (waves, buoyancy).

The demo is playable (Google Chrome browser required) at http://madebyevan.com/webgl-water/

Youtube: http://www.youtube.com/watch?v=i6aEQ5pY1aQ

Source code is available in the Chrome browser (right-click on the page > Inspect element > choose the "Scripts" button > choose "renderer.js" from the drop-down menu)

John Carmack: "Eventually ray tracing will win"

Aaah, delightful August... Holiday, warm, sunny, relaxing, enjoying mountains and beaches alternated with reports from QuakeCon, Siggraph and High Performance Graphics :-)

John Carmack has said some interesting things about ray tracing during the Q&A following his QuakeCon keynote this year (which was mostly about the practical problems of working with MegaTextures in a production environment). I made a transcript of the relevant Q&A portion which can be seen in its entirety on youtube: http://www.youtube.com/watch?v=00Q9-ftiPVQ#t=18m30s

"I expect, as we look at next-gen console and broad enough adoption of PC stuff, that there will be some novelty games that are all raytraced or all voxeled on there. But when you look at the top end triple A titles, I do not expect that transition happening in that timeframe but I keep looking at it.

There’s a lot of these things that we look at every five years on and on and eventually… you know I do think that some form of raytracing, of forward tracing or reversed tracing rather than forward rendering will eventually win because there’s so many things that just get magically better there. There’s so much crap that we deal with in rasterisation with, okay let’s depth fade or fake our atmospheric stuff using environment maps, use shadows. And when you just say “well just trace a ray” a lot of these problems vanish. But one interesting thing that people say “look real-time raytracing on current hardware”, that’s what I did in OpenCL recently and I did some interesting work with that. But the real truth is, you don’t just trace one ray it goes sixty frames per second.

To do the things that people want to see out of ray tracing, you’re gonna need to trace a dozen rays on there if you want your soft reflections or even sharp reflections if you got bump mapping on there and not to look like a mess of noise on there, you start needing an order of magnitude more rays than that. And most people still avert their gaze from the whole problem of dynamic geometry. There’s some interesting work that goes on with very rapid GPU assisted KD tree construction that’s starting to look at some of that, but it’s still a long ways off. It’s always the problem of fighting against an entrenched incumbent where we’re doing polygon… Vertex fragment polygon based rasterizers are so far and away the most successful parallel computing architecture ever it’s not even funny. I mean all the research projects and everything else just haven’t added up to one fraction of the value that we get out of that. And there’s a lot of work, lots of smart people, lots of effort and lots of great results coming out of it. Eventually ray tracing will win, but it’s not clear exactly when it’s gonna be."

There is also a brand new video and interview with Carmack on PCPer where he's rambeling about voxels, ray tracing and Larrabee, Fusion, GPUs with ARM cores (Project Denver/Maxwell):

http://www.pcper.com/reviews/Editorial/John-Carmack-Interview-GPU-Race-Intel-Graphics-Ray-Tracing-Voxels-and-more

From the article:

"On the topic of ray tracing, a frequently debated issue on PC Perspective, Carmack does admit to finding its uses quite surprising and has spent some time building ray tracing engines for development testing. He seemed surprised by the results in his initial attempt to replace rasterization with a ray tracing engine, getting as much as 60 FPS at 720p resolutions on a Fermi-based NVIDIA graphics card. When performance dropped below the targeted frame rate he was able to either produce an image of a lower resolution and display that or use previous frames to sort of “blend” them for a smoother image. The issue though is the delta between a tech demo and fully implemented gaming engine – demos deal with static models, there is no character animation, etc. And while the fact that it runs at 60 FPS at 720p sounds impressive, if you were drawing that with a traditional rasterization engine it would be running at 1000 FPS!"

A transcript of the PCPer interview with Carmack (with some in-depth thoughts about ray tracing) can be found here: http://www.pcper.com/reviews/Editorial/John-Carmack-Interview-GPU-Race-Intel-Graphics-Ray-Tracing-Voxels-and-more/Intervi

There's also new interesting research material from HPG, mainly about ray tracing: active thread compaction for GPU path tracing (Wald) (performance boost is much less than initially thought), improving SIMD efficiency for GPU Monte Carlo rendering with stream compaction (van Antwerpen), 10x faster HLBVH (Garanzha), raytraced moblur (Grünschloss), VoxelPipe (Pantaleoni), CUDA rasterization (Laine):

- Slides: http://www.highperformancegraphics.org/media/Papers

- Posters: http://www.highperformancegraphics.org/media/Posters The idea in this poster is quite intruiging: www.highperformancegraphics.org/media/Posters/HPG2011_Posters_Keller2_abstract.pdf

GTS 450 video of Unbiased Sphere Stack Shooter game

Much better quality video (480p) of "Unbiased Sphere Stack Shooter" rendered on a GTS 450:

The photoreal lighting of the scene is much more obvious in this video (less noise, higher framerate and higher resolution than the previous vid which was rendered on a 8600M GT) Unfortunately Youtube puts a nasty filter over the image, the original image quality is much crisper.

The executable demo of TOKAP Unbiased Sphere Stack Shooter is available at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list (CUDA 4.0 required)

After much tinkering with the Bullet code (converting quaternions to degrees) the car physics for Unbiased Truck Soccer are finally working (just one car atm, but it looks hilarious), so I'll be posting a new demo and video soon.

Small update on CentiLeo

Just found this: http://gpgpu.org/2011/08/04/centileo-out-of-core-ray-tracer I wrote about this amazing out-of-core GPU path tracer (which will be presented at Siggraph this week) before (see http://raytracey.blogspot.com/2011/04/centileo-brand-new-interactive-out-of.html and http://raytracey.blogspot.com/2011/06/update-on-centileo-out-of-core-gpu-ray.html) .

There is also a dedicated website about CentiLeo at http://www.centileo.com. Some interesting quotes from the website:

"CPU-based rendering solutions (even many-core, e.g. 16-core) are much slower than our GPU-based software (around 25x slower). Such 16-core CPUs are 3-4x more expensive than our GPU."

"The target of CentiLeo development is to move all the flexible rendering related tools from slow and expensive CPU-based rendering solutions to new innovative GPU-based solutions.

Release version should provide with the libraries and ray tracing acceleration tools which will allow:

- to setup own standalone graphics rendering applications;

- to integrate our accelerated individual rendering stages into existing rendering solutions and pipelines in various industries such as engineering design, architecture, movie industry, visualization and etc."

Unbiased Sphere Stack Shooter, the first real-time path traced shooter

A physics based demo without user interaction with the physical objects in the scene is only half as fun, so I've been working on a little physics based game where you can shoot a ball into the scene and hit physics objects. It's not a real game, it's just mindless fun and the graphics are as good as photoreal.

Features:

- real-time path traced graphics on the GPU: produces ultra-high quality physically accurate global illumination, soft shadows and reflections

- integration of the Bullet physics engine: start/pause the physics simulation by pressing 'm'

- first person camera: move forward, backwards and sideways with the arrow keys

- mouse look: click and hold the left mouse button to look around the scene (already working, but needs a bit more work to make it more robust)

- shooting mechanics: fire a ball into the stack or try hitting a bouncing sphere by pressing 'Enter'. The ball is shot from the current camera position at the current view target in the center of the screen. Due to performance reasons, only one ball is shot at a time (the same ball is 'recycled' with every shot).

Make the physics go berserk!

Video rendered on 8600M GT (a better quality video on GTS 450 will follow soon):

Download the executable for this game at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Tokap Chrome spheres stack: video on GTS 450

Download exe at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Celebrating the third anniversary of this blog (first post on Aug 1, 2008: http://raytracey.blogspot.com/2008/08/ruby-voxels-and-ray-tracing.html) Cheers!

TOKAP with Bullet: Mirror physics

Update to my previous post, just to show that the scene remains fully customizable at runtime:

480p video rendered on 8600M GT:

Executable for "Tokap Sphere Stack with Bullet Physics Release" at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list (this version is the release version and should run without needing the Visual C++ 2008 Redistributable package for x86)

I'm really hooked on this path tracer, the photorealistic quality is so damn awesome.

Coming up next:

Bullet Physics demo 3: Collapsing stack of balls + EXE

Yesterday I got a new idea for a real-time path traced Bullet physics animation: a collapsing stack of spheres. The plan:



I've implemented the scene and physics in both the tokaspt and the Simplex Paternitas path tracers to see the difference in framerate and realism.

Picture from Tokap (some circular artefacts which look kinda cool :) :


Picture from the Futuristic Buildings (Simplex Paternitas) path tracer:

All videos below were rendered in real-time on my poor little 8600M GT, probably one of the weakest CUDA-enabled cards in existence. Every animation below should run smoothly and at much higher quality on a GTX 260 or higher:

Tokap with Bullet 4 spp:


Futuristic Buildings with Bullet 12 spp:


Futuristic Buildings with Bullet 24 spp:


Futuristic Buildings with Bullet 4 spp, Eagle's view:

Download the executable for "TOKAP Bullet Sphere Stack" at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

UPDATE: Some people have reported that the Bullet demos don't work on their systems, receiving the following message:

"Activation context generation failed for "programme name".

Dependent Assembly Microsoft.VC90.DebugCRT, processorArchitecture="x86", publicKeyToken = "1fc8b3b9a1e18e3b", type="win32", version="9.0.21022.8" could not be found. Please use sxstrace.exe for detailed diagnosis."

This error is due to the fact that the demos were compiled with the Bullet library in debug mode and are dependent on the Visual C++ 2008 runtime library. Downloading the Microsoft Visual C++ 2008 Redistributable Package (x86) from http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=29 should solve the issue.

Bullet Physics in Simplex Paternitas path tracer

Someone on youtube suggested to make a version of the physics animation from my previous post where the scene is only lit by the small white ball on top. For this purpose I've incorporated the Bullet library to the Simplex Paternitas path tracer of Jacco Bikker. The results are pretty wicked:

This video was rendered on a 8600M GT, 12 spp, 480x360 resolution:



Without caustic noise (only reflective and diffuse balls, 12 spp, 640x480 resolution on 8600M GT):


The new executable and source code are available at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Damn this stuff is really addictive!

TOKAP with Bullet demo 2

While working on the car physics for the Futuristic Buildings demo, I've decided to first finish something that I wanted to do for a long time: a real-time version of a Bullet Physics animation created by Chiaroscuro (Phymec on Youtube) which was rendered off-line with SmallptGPU (OpenCL path tracing) and can be seen at

http://www.youtube.com/watch?v=33rU1axSKhQ

I've recreated the scene in TOKAP, added physical properties to the balls with Bullet Physics et voilà:


The above video was rendered on a ultra-low end GPU (8600M GT). The animation should run at 60 fps on a GTX 580 at 8spp at default resolution. Physically accurate lighting + physics in real-time!!!

Caustics:


Download the executable at:

http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Latest version of Futuristic Buildings with AABB

I've implemented tight fitting AABB hitboxes around the car and the two buildings, which gives a nice performance boost of about 50-200%, depending on the viewpoint. Since the car can be moved (with the I, J, K, L keys), the AABB around the car is moving with it and its position is updated every frame.

Download the executables and the source code for "Futuristic Buildings v2.9 with AABB" from http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

Real car physics are coming up next. After those are in, OBBs (oriented bounding boxes) or oriented cylinders (depending on which one's cheaper to intersect) are on the todo list for some crazy real-time path traced physics simulations.

VoxelPipe paper and a preview of real-time path traced global illumination

Just read Nvidia's HPG 2011 paper "VoxelPipe: A Programmable Pipeline for 3D Voxelization" by Jacopo Pantaleoni. One thing that immediately caught attention was the preview of a real-time global illumination system, in which primary rays are traced against the regular triangle version of the scene, but secondary incoherent rays are traced against a voxelized version of the scene which is much more efficient intersection-wise. Scene voxelization is performed every frame (it takes only a few milliseconds even for multi-million triangle scenes), so this allows for completely dynamic scenes with (path traced) global illumination in real-time. From the paper:

"Finally, in Figure 5 we show a proof of concept of a real-time global illumination system that we plan to disclose in the near future. The system relies on our voxelization pipeline to create a proxy of the scene geometry that is used to trace incoherent rays. The proxy is rebuilt every frame, allowing support of fully dynamic geometry, materials and lighting."

Combined with previous work from Pantaleoni on PantaRay (an out-of-core GPU ray tracing renderer for precomputing GI with LOD techniques, which was used in "Avatar") and on HLBVH and a yet to be released paper from Pantaleoni and Garanzha on "Simpler and Faster HLBVH with Work Queues" (which describes a system for real-time raytracing of dynamic scenes), this new voxelization technique could do wonders for real-time high quality global illumination and also for unbiased real-time GPU renderers.

Writing real-time path tracing demos

Several readers of this blog (woohoo I've got readers ;-) have asked me what skills you need to learn to make the demos that I've posted. To those, I want to give some C++ pointers (pun intended) in this post.

Six months ago, I didn't know anything about programming (let alone C++). After seeing some amazing real-time path tracing demos on the net (like Jacco Bikker's Brigade), I wanted to write my own stuff because I had some ideas that I wanted to see materialized (such as the Pong game for which I got the help from Kerrash, a programmer with CUDA experience). I didn't want to be too dependent on other programmers so I decided to learn C++ on my own in my spare time. I've read some C++ tutorials on the net and some excellent books that have been a great help in understanding the C++ language and writing your own programs. This is the list of books and websites that I've read:

- "Beginning programming with C++ for Dummies", an excellent beginner's book, easy to read, explains the concept of object-oriented programming very well

- "C++ without Fear", another great book, which delves deeper into the basics than the first book and also discusses code optimization

- tutorials on www.learncpp.com, this is a great website with lots of small Q&A's, and also discusses smaller subjects like enums

- "Practical C++ Programming", this is a little bit harder than the first two books (especially the exercises)

Other books that were recommended by others (that I haven't read yet):

- "Thinking in C++"

- "Accelerated C++"

- "Effective C++"

- www.cprogramming.com

- www.cplusplus.com

Reading these books and doing the exercises with perseverance, I slowly got a hold on the C++ language. Frankly I'm amazed that the language becomes almost second nature so quickly. I'm still getting compiler errors that I don't understand, but luckily there is Google, which quickly solves the problem in most cases.

In order to write a real-time path tracing demo, learning C++ is not enough. Some basic knowledge about ray tracing and path tracing is necessary as well. "Realistic Ray Tracing" by Shirley is an excellent resource for this. There are also lots of powerpoint presentations about ray tracing and Monte Carlo rendering on the net.

Not having the patience to reinvent the wheel of real-time path tracing, I wanted to base my path tracing demos on existing open source code that I knew was ultrafast and ultra-efficient. So I did some research on the net. This is what I came up with:

- tokaspt, an excellent real-time CUDA path tracer

- smallpt, a very short but complete CPU path tracer by Kevin Beason (not real-time)

- smallptGPU, a real-time OpenCL path tracer by Dade from LuxRender

- mxadd.org, another CUDA path tracer with source code

- Javor's CUDA path tracer (click on the thumbnail to download the demo)

- Understanding the Efficiency of Ray Traversal on GPUs, ultra-fast CUDA path tracing code by Timo Aila and Samuli Laine

- Brigade path tracer, real-time CPU/GPU(cuda) path tracer by Jacco Bikker

- Kajiya demo, a real-time CUDA path tracer by Jacco Bikker and Jeroen van Schijndel, very well optimized

- Simplex Paternitas, another real-time CUDA path tracer by Jacco Bikker, this is an excellent starting point for learning to write new code and add new objects and animations (and see the results immediately). You don't have to touch the CUDA part of the code (although it's quite easy to understand once you're familiar with C++). Every frame, all the framedata is passed from C++ to CUDA, so you only need to know C++ to write new animations and to add a physics library like Bullet Physics.

Adding the Bullet Physics library to the code (as shown in my latest demo) and learning the Bullet API was actually not so difficult in retrospect (the Hello World example included in the Bullet SDK was very helpful in this respect). Right now, I'm incorporating the vehicle physics from the Bullet SDK into the Futuristic Buildings demo (which is based on the code framework of the Simplex Paternitas demo). It's kinda working right now, but I won't make a vid until I'm satisfied. The AABB hitboxes for the car and buildings are almost working as well (after identifying which variable triggered the ray shooting functions using the hitboxes).

To summarize, learning the basics of C++ is a must. You have to invest some time in it (it took me about 3 months of learning in my spare time to get a good grasp on the basics and an additional month to read and understand someone else's code and adding own stuff). Understanding the structure of a program (which functions call other functions, which loop is the control flow entering, what stuff has to be initialized before rendering the first frame, how are variables efficiently shared between files, ...) is extremely important. From that point, you'll need creativity and perseverance to make an interesting demo and the possibilities will only be limited by imagination. A cool idea would be this one (this should be possible with fewer planks (20-30) in real-time once I've added support for oriented bounding boxes to represent the planks).

Paper on accelerating path tracing using eye path reprojection available

The paper "Accelerating path tracing by eye path reprojection" by Niklas Henrich et al., which I blogged about earlier is available from http://www.rendering.ovgu.de/publikationen.html. The algorithm achieves similar quality as regular path tracing in half the time and can easily be ported to the GPU. The authors also plan to incorporate their technique into a real-time raytracer.

Videos on GTS 450

I've made two video's of somewhat better quality, resolution and framerate on the GTS 450, which is about 5x faster than a 8600M GT in Tokap (and about 8x faster in Futuristic Buildings).

The animation is now being rendered at 60 fps, 8 spp per frame and 512x256 resolution. The physics simulation itself runs internally at 60 fps, so the physics are rendered in "true" real-time.

Video of Future Buildings v2 ( 8 spp per frame, 640x480 resolution, still no acceleration structures):

The source code for "TOKAP with Bullet Physics" is available at http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list