CUDA 3.1 out

Just read that cuda 3.1 toolkit is available (has been for some time in fact)

http://www.theregister.co.uk/2010/07/22/nvidia_cuda_parallel_nsight/

As you can see from the release notes, CUDA 3.1 also gives 16-way kernel concurrency, allowing for up to 16 different kernels to run at the same time on Fermi GPUs. Banks said a bunch of needed C++ features were added, such as support for function pointers and recursion to allow for more C++ apps to run on GPUs as well as a unified Visual Profiler that supports CUDA C/C++ as well as OpenCL. The math libraries in the CUDA 3.1 SDK were also goosed, with some having up to 25 per cent performance improvements, according to Banks.

The support for recursion and concurrent kernels should be great for CUDA path tracers running on Fermi and I'm curious to see the performance gains. Maybe the initial claims that Fermi will have 4x the path tracing performance of GT200 class GPUs could become true after all.

OnLive + Mova vs OTOY + LightStage

I've just read Joystiq's review of OnLive, which is very positive regarding the lag issue: there is none...

As it stands right now, the service is -- perhaps shockingly -- running as intended. OnLive still requires a faster than normal connection (regardless of what the folks from OnLive might tell you), and it requires a wired one at that, but it absolutely, unbelievably works. Notice I haven't mentioned issues with button lag? That's because I never encountered them. Not during a single game (even UE3).

A related recent Joystiq article about OnLive mentions Mova, a sister company of OnLive developing Contour, a motion capture technology using a curved wall of camera's, very reminiscent of OTOY's LightStage (although the LightStage dome is bigger and can capture the actor from 360 degrees at once). The photorealistic CG characters and objects that it produces are the real advantage of cloud gaming (as was being hinted at when the Lightstaged Ruby from the latest Ruby demo was presented at the Radeon HD 5800 launch):

What he stressed most, though, was Perlman's other company, Mova, working in tandem with OnLive to create impressive new visual experiences in games. "This face here," Bentley began, as he motioned toward a life-like image that had been projected on a screen before us, "is computer generated -- 100,000 polygons. It's the same thing we used in Benjamin Button to capture Brad Pitt's face. Right here, this is an actress. You can't render this in real time on a standard console. So this is the reason OnLive really exists." Bentley claims that Mova is a big part of the reason that a lot of folks originally got involved with OnLive. "We were mind-boggled," he exclaimed. And mind-boggling can be a tremendous motivator, it would seem -- spurring Bentley to leave a successful startup for a still nascent, unknown company working on the fringes of the game industry.

In fairness, what we saw of Mova was terrifyingly impressive, seemingly crossing the uncanny valley into "Holy crap! Are those human beings or computer games?" territory. Luckily for us, someone, somewhere is working with Mova for games. Though Bentley couldn't say much, when we pushed him on the subject, he laughed and responded, "Uhhhh ... ummm ... there's some people working on it." And though we may not see those games for quite some time, when we do, we'll be seeing the future.

Just like OTOY, I bet that OnLive is developing some voxel ray tracing tech as well, which is a perfect fit for server side rendering due to it's massive memory requirements. Now let's see what OTOY and OnLive with their respective cloud servers and capturing technologies will come up with :-)

Real-time Energy Redistribution Path Tracing in Brigade!

A lot of posts about Brigade lately, but that's because the pace of development is going at break neck speeds and the intermediate updates are very exciting. Jacco Bikker and Dietger van Antwerpen, the coding brains behind the Brigade path tracer, seem unstoppable. The latest contribution to the Brigade path tracer is the implementation of ERPT or Energy Redistribution Path Tracing. ERPT was presented at Siggraph 2005 and is an unbiased extension of regular path tracing which combines Monte Carlo path tracing and Metropolis Light Transport path mutation to obtain lower frequency noise and converge faster in general. Caustics benefit greatly as well as scenes which are predominantly lit by indirect lighting. The original ERPT paper can be found at http://rivit.cs.byu.edu/a3dg/publications/erPathTracing.pdf and offers a very in-depth and understandable insight into the technique. A practical implementation of ERPT can be found in the paper "Implementing Energy Redistribution Path Tracing" (http://www.cs.ubc.ca/~batty/projects/ERPT-report.pdf).

The algorithm seems to be superior than (bidirectional) path tracing and MLT in most cases, while retaining it's unbiased character. And they made it work on the GPU! You could say that algorithm-wise, the addition of ERPT makes Brigade currently more advanced than the other GPU renderers (Octane, Arion, LuxRays, OptiX, V-Ray RT, iray, SHOT, Indigo GPU, ...) which rely on "plain" path tracing.

The following video compares path tracing to ERPT in Brigade at a resolution of 1280x720(!) on a GTX 470: http://www.youtube.com/watch?v=d9X_PhFIL1o&feature=channel

This image directly compares path tracing on the left with ERPT on the right (the smeary pixel artefacts in the ERPT image are mostly due to the youtube video + JPEG screengrab compression, but I presume that there are also some noise filters applied as described in the ERPT paper):
ERPT seems to be a little bit darker than regular path tracing in this image, which seems to be a by product of the noise filters according to http://pages.cs.wisc.edu/~yu-chi/research/pmc-er/PMCER_files/pmc-er-egsr.pdf.

On a side note, the Sponza scene in the video renders very fast for the given resolution and hardware. When comparing this with the video of Sponza rendering in the first version of SmallLuxGPU on a HD 4870 (which I thought looked amazing at the time), it's clear that GPU rendering has made enormous advancements in just a few months thanks to more powerful GPU's and optimizations of the path tracing code. I can hardly contain my excitement to see what Brigade is going to bring next! Maybe Population Monte Carlo energy redistribution for even faster convergence? ;)

Sparse voxel octree and path tracing: a perfect combination?

I have been wondering for some time if SVO and path tracing would be a perfect solution for realtime GI in games. Cyril Crassin has shown in his paper "Beyond triangles: Gigavoxels effects in videogames" that secondary ray effects such as shadows can be computed very inexpensively by tracing a coarse voxel resolution mipmap without tracing all the way down to the finest voxel resolution, something that is a magic intrinsic property of SVO's and which is to my knowledge not possible when tracing triangles (unless there are multiple LOD levels), where all rays have to be traced to the final leaf containing the triangle, which is of course very expensive. A great example of different SVO resolution levels can be seen in the video "Efficient sparse voxel octrees" by Samuli Laine and Tero Karras (on http://code.google.com/p/efficient-sparse-voxel-octrees/ , video link at the right, a demo and source code for CUDA cards is also available).

I think that this LOD "trick" could work with all kinds of secondary ray effects, not just shadows. Particularly ambient occlusion and indirect lighting could be efficiently calculated in this manner, even on glossy materials. One limitation could be high frequency content, because every voxel is an average of the eight smaller voxels that it's constituted of, but in such a case the voxels could be adaptively subdivided to a higher res (same for perfectly specular reflections). For diffuse materials, the cost of computing indirect lighting could be drastically reduced.

Another idea is to compute primary rays and first bounce secondary rays at full resolution, and all subsequent bounces at lower voxel LODs with some edge-adaptive sampling, since the 2nd, 3rd, 4th, ... indirect bounces contribute relatively little to the final pixel color compared to direct + 1st indirect bounce. Not sure if this idea is possible or how to make it work.

Voxelstein3d has already implemented the idea of pathtracing SVOs for global illumination (http://raytracey.blogspot.com/2010/03/svo-and-path-tracing-update.html) with some nice results. Once a demo is released, it's going to be interesting to see if the above is true and doesn't break down with non-diffuse materials in the scene.

UPDATE: VoxLOD actually has done this for diffuse global illumination and it seems to work nicely:
http://voxelium.wordpress.com/2010/07/25/shadows-and-global-illumination/

Tokaspt, an excellent real-time path tracing app

Just stumbled upon this very impressive CUDA based path tracer: http://code.google.com/p/tokaspt/ for exe and source (The app itself has been available since January 2009)

Although the scenes are quite simple (spheres and planes only), it's extremely fast and it converges literally in a matter of milliseconds to a high quality image. Navigation is as close to real-time as it gets. There are 4 different scenes to choose from (load scene with F9) and they can be modified at will: parameters are sphere size, color, emitting properties, 3 material BRDFs (diffuse (matte), specular (mirror) and refractive (glass)) and sphere position. Path trace depth and spppp (samples per pixel per pass) can also be altered on the fly thanks to the very convenient GUI with sliders. When moving around and ghosting artefacts appear, press the "reset acc" button to clear the accumulation buffer and get a clean image. Definitely worth messing around with!

New version of Brigade path tracer

Follow the link in this post to download. There's some new features + performance increase. Rename cudart.dll to cudart32_31_9.dll to make it work.

The next image demonstrates some of the exceptional strenghts of using path tracing:

- indirect lighting with color bleeding: notice that every surface facing down (yellow arrows) picks up a slightly greenish glow from the floor plane, due to indirect light bouncing off (this picture uses path trace depth 6)
- soft shadows
- indirect shadows
- contact shadows (ambient occlusion)
- superb anti-aliasing
- depth of field
- natural looking light with gradual changes

all of these contribute to the photorealistic look and it's all interactive (on high end cpu+gpu)!

Crayon Physics Deluxe

This has to be one of the coolest games ever. Media Molecule should hire these guys:

http://vimeo.com/1849263

http://vimeo.com/2739375

Demo here

Brigade path tracer comparison

The following screenshots are taken from the Brigade real-time path tracer demo, available at http://igad.nhtv.nl/~bikker/
Rendered with CPU only at resolution 832x512
Images with 100 and 800 spp were taken without frame averaging (only 1 iteration)
Images with 2, 8, 16, 32 spp taken with frame averaging (averaging samples of several frames)

2 spp


8 spp


16 spp


32 spp


100 spp


800 spp



To top it off, one big image comparing 800, 8, 16 and 32 spp. It amazes me that the quality of just 8 samples is already great and with some filtering it could rival the quality of the 800 spp image:

Gaikai's cloud bussiness model: play games for free in your browser

Interview with Gaikai's Dave Perry on Joystiq:
http://www.joystiq.com/2010/06/30/dave-perry-on-the-innovation-of-gaikai/

Gaikai focuses on delivering game demo's, not complete games: you see an advertisement of a game on a website (could be Gamespot, EA.com, Eurogamer) or read a game review, and with just one click you can play a demo of that game in your browser without paying a cent. The game publisher pays for your playing time, 1 cent per minute per user.
This approach is economically safer, more practical and more retail/publisher/gamer friendly than what OnLive is doing. With the current network infrastructure of the internet and its bandwidth limitations, this is probably the most successful route for cloud gaming. Gaikai has already signed EA (http://games.venturebeat.com/2010/06/17/gaikai-signs-ea-as-digital-distribution-partner/), so I think cloud gaming is gonna get big pretty soon.

UPDATE: another video of OnLive, showing mouse latency in F.E.A.R. 2 behind a router: http://www.youtube.com/watch?v=Edf5xsqST90

Crytek aims for the cloud, sparse voxel octrees and sparse surfel octrees

Crytek's HPG 2010 presentation is available at http://www.crytek.com/fileadmin/user_upload/inside/presentations/2010_HPG_Keynotes/Notes.ppt According to the slides, Crytek is already using sparse voxel octree in the game asset pipeline. Because they are Crytek, they are also researching a variation on SVO, called the sparse surfel octree (I vaguely remember surfels from Michael Bunnell's point based ambient occlusion and indirect lighting technique in GPU Gems 2). I wonder what that is going to look like.

There are a few slides on server side rendering, which could "drastically change the direction" of the real-time rendering pipeline.

Other interesting tidbits:
- a configurable hardware scheduler which could make raytracing much faster
- Larrabee 2 still in the race for next-gen consoles?
- DX11 Compute Shaders suck
- Perception-driven graphics are the key to more efficient use of rendering resources (this is a very interesting future direction)

DysKinect

This one made me LOL:

Tom's Hardware review of OnLive

Tom's Hardware review: OnLive Cloud-Based Gaming: Is This the End of High-End PCs?

Interesting review, but the assumption in the article's title is already fundamentally flawed: high end PC gaming is already nearing it's end and it has nothing to do with cloud gaming, but everything with consoles. The article's conclusion is that OnLive doesn't come close to displace a high-end PC in gaming, but what the author seems to forget is that game publishers - not PC gamers - will ultimately decide if cloud gaming will succeed and they also have the power to kill off high-end PC gaming in no time (as they have been doing for the last 5 years).

Gone are the days when I was eagerly anticipating cutting-edge PC games like Doom III (before the name change to Doom 3), Half-Life 2 and Unreal 2. That was about 8 years ago. The last PC game worthy of being called cutting-edge was Crysis and that came out 3 years ago. Consoles have become the main focus of id Software, Epic games and now also Crytek. For a graphics enthusiast like myself, this is a very unfortunate trend, all dictated by economical laws.

So in essence, high-end PC gaming is already dead. Crysis still beats every other PC game (ported from consoles) in advanced effects. Cranking up resolution, AA, and AF settings in Modern Warfare 2 will not change the poor lighting and shadowing. I can't wait until the consoles, aka the high-end PC gaming killers, will be made obsolete themselves by cloud gaming. Considering the enthusiasm of game publishers and developers for the cloud (e.g. Crytek is a big proponent of server side rendering cfr. Yerli's presentation at HPG 2010), this might happen much sooner than thought, just like stereoscopic 3D is taking console gaming by storm.

UPDATE: CNET also has an excellent review of OnLive up, with an analysis of the benefits of cloud gaming for all actors in the playing field http://news.cnet.com/8301-17938_105-20009033-1.html

iray officially in the open with Bunkspeed Shot

Bunkspeed Shot (final version) was officially released last week. More importantly, it also means that the long awaited iray, which is powering SHOT, is finally available for everyone (it should have been here much earlier with 3ds max 2011, but was not integrated for some obscure reason).

Bunkspeed SHOT press release

You can download a 30-day trial demo at http://bunkspeed.com/shot/demo/index.html (Don't bother if you don't have a CUDA enabled card with at least 1GB of VRAM, else the software defaults to CPU only, totally retarded limitation if you ask me).

Funny enough, the press release makes it look like you'll need a Quadro or Tesla to run the software, but a Geforce should do fine and is probably faster. The amount of GPU memory will be the real decisive factor and Octane has shown that you can do a lot within a 1GB VRAM budget (it even has procedural textures now, which take up almost no memory).

Waiting for V-Ray GPU... hopefully another "Siggraph surprise" :-D

Demo of Brigade real-time path tracer out!

Available at http://igad.nhtv.nl/~bikker/

Anyone (remotely) interested in real-time raytracing, owning a CUDA enabled GPU or a powerful CPU, must definitely try this excellent demo! It works with Geforce 8000 cards and upwards, but can also use the CPU only if you don't have a CUDA card. (UPDATE: some people on XP machines cannot run the program because of a msvcrt.dll error. Removing opengl32.dll from the Brigade folder seems to solve the problem). There are 4 different scenes to choose from, some are animated with "planes" flying around. You can simply edit "scene.txt" to change scene, resolution, samples per pixel and so on. This is a video of one of the scenes: http://www.youtube.com/watch?v=qC2zKIqttzk

It's an amazing piece of software with huge potential. Afaik, the CPU uses bidirectional path tracing while divergent rays are path traced on the GPU. Scenes and materials are very simple, but nevertheless I am still stunned that real-time path tracing with animated objects is possible today. (Bidirectional) path tracing solves most limitations encountered by other real-time GI methods which rely on image space techniques or can only be used in diffuse and semi-glossy scenes. Speed is the only limit. I can't wait to see Brigade run on a cluster of PC's each containing multiple Fermi's (as is suggested in the readme file). Looking very forward to the games that the Dutchies will produce with this technology ;-)

Cloud gaming just works

The last couple of days, I have been reading a lot of impressions from gamers who have tried the OnLive service and I must say that I'm surprised at the amount of positive feedback. Many people don't seem to perceive any lag and the ones that do don't mind it too much and it never makes the games unplayable. I am stunned reading that OnLive works so well, I've been hoping that cloud gaming would work great, but this is even better than I had expected. I thought it would initially be plagued by major lag fluctuations, stuttering, connections shutting down when the service launched, but everything seems fine till now.

When cloud gaming really catches on and other services like OTOY and Gaikai will join the battlefield, these game clouds will have the capability to go beyond what consoles and even high-end PCs can offer in terms of graphics processing:

- insane geometrical detail with e.g. sparse voxel octree raycasting/-tracing for environments and characters
- advanced lighting and global illumination through GPU accelerated raytracing
- physics on every dynamic object
- procedural sound
- more human-like A.I. (just hook up the server to Blue Gene ;-))

This will be imo the ultimate argument to drop restricted console architecture in favor of cloud gaming.

Cloud gaming is also a great way for offering time-limited game demo's, which is Gaikai's main focus: play a demo of a soon to be released game right in your browser. How easy and customer friendly can it get? There are many other options beyond gaming, such as a Facebook-ish virtual reality world (LivePlace powered by OTOY), CAD programs, Photoshop, Matlab, anything compute intensive...

UPDATE: Two interesting short articles on Dave Perry's Gaikai:

http://www.nowgamer.com/news/3548/perry-big-3-will-embrace-cloud-gaming
http://www.nowgamer.com/news/3547/perry-gaikai-demo-surprised-publishers

Video review of OnLive

Cloud gaming, it's finally here: http://www.youtube.com/watch?v=Ir4B0rgta0Y

Two written reviews:

http://gizmodo.com/5567770/onlive-streaming-game-service-tested-at-home-finally

http://blog.wolfire.com/2010/06/Thoughts-on-OnLive

The reviews are surprisingly positive about the lag: some say it's noticeable, others say it isn't, but in either case it's not really an issue during gameplay, not even for "fast twitched" games like Unreal Tournament 3 and Batman Arkham Asylum. I(UPDATE: I have been watching 5 video reviews from casual gamers on YouTube, and every single one says that lag is unnoticeable, which is fuck awesome, I guess this settles anyones doubts about latency!). mage quality is a mixed bag: one reviewer says it looks almost identical to 720p local play, another says it's significantly worse. due to compression. I guess it depends on the bandwidth connection. Image quality is imo definitely not the biggest hurdle for cloud gaming, latency is much more important and apparently it's all very playable. Games are the most challenging software to make work through cloud computing and OnLive has apparently succeeded at this daunting task. OTOY and Gaikai will surely follow. When these services mature and gain popularity, consoles will face a difficult time. Ultimately, when every thinkable piece of software can be run on the cloud, who will still need Windows?

There's no doubt that cloud gaming is the future of games, the killer app for iPad and iPhone, probably sooner than most people think, and a serious problem for next-gen consoles. I think Larrabee/MIC/Knight's Corner might resurface in a cloud game server environment instead of being sold as a stand-alone PC card. I also think that at some point in the future, Nvidia and AMD will make hardware that is specifically aimed at game cloud servers and will be more efficient at memory use and power management in order to serve multiple users with the same hardware resources (like OTOY). A super beefed-up version of AMD Fusion for example.

Using the GPU for precomputing GI in game development

I just read on the Real-Time Rendering blog, that Ubi Montreal used GPUs to precompute ambient occlusion for Splinter Cell Conviction. The technique used was invented by Toshiya Hachisuka and described in GPU Gems 2 in the chapter "High-Quality Global Illumination Rendering Using Rasterization".

I also read that Bungie uses a GPU-accelerated photon mapping technique from the Siggraph 2009 paper "An Efficient GPU-based Approach for Interactive Global Illumination" by Rui Wang et al. to precompute GI in some of the Halo games (ODST?, Reach?).

It's nice to see that GPUs are actually used for precomputating lighting in games and movies (e.g. PantaRay in Avatar) and I believe this is a very interesting trend. On a PC stuffed with multiple Fermi's, some of these techniques might be close to real-time and achieve very high quality. With the latest breakthroughs in GPU-accelerated GI algorithms (path tracing, bidirectional path tracing (Brigade), soon realtime MLT?, (image space) GPU photon mapping, sppm) it should be possible to have movie-quality real-time GI on the next generation of consoles coming in 2012 (at the earliest). Or maybe not on consoles, but definitely on GPU clouds. :-).

Stochastic progressive photon mapping in Luxrender GPU

The guys behind Luxrender have released smallppmGPU and smallsppmgpu, two demos that incorporate ppm and sppm, which are very interesting unbiased algorithms that are much more efficient in rendering caustics than other unbiased methods (plain path tracing in particular) and which also offers nice DOF and motion blur. Link: http://www.luxrender.net/forum/viewtopic.php?f=34&t=4024 (registration is needed)

The next step for the GPU renderers which rely on brute force path tracing is to investigate more efficient and faster algorithms such as bidirectional path tracing and Metropolis light transport running entirely on the GPU. There is already research going on in this area e.g. "Path Regeneration for Interactive Path Tracing" by Novak, Havran and Dachsbacher describes an efficient bidirectional path tracer running on the GPU (http://www.vis.uni-stuttgart.de/~novakjn/paper/eg2010_pt.pdf).

Another logical evolution is getting biased algorithms (photon mapping, irradiance cache) to work efficiently on the GPU. This seems to be a much more difficult (but not impossible) task than having unbiased rendering on the GPU because these biased algo's are much more difficult to parallellize. Some recent papers in this area:

Morgan McGuire and David Luebke: Hardware-Accelerated Global Illumination by Image Space Photon Mapping

Bartosz Fabianowski and John Dingliana: Compact BVH Storage for Ray Tracing and Photon Mapping

Rui Wang et al.: An Efficient GPU-based Approach for Interactive Global Illumination

Maybe Chaos Group will stun us again at Siggraph 2010 with a biased GPU renderer, which renders 10 times faster than V-Ray GPU :-). Lots of interesting approaches to be explored and more exciting times ahead!

1 more screen of Design Garage

There are lots of amazing screenshots from Nvidia's Design Garage demo for the new Fermi cards. I like this interior shot in particular, because it reminds me of the Cinema2.0/OTOY/Ruby demo:


From http://www.evga.com/forums/tm.aspx?m=289470&mpage=6

Intel pronounces Larrabee dead, how will this affect Unreal Engine 4?

You might remember this interview with Epic Games' ever so humble president Mike Capps from a few weeks ago, in which he said "if you look at what’s happening in the PC market – Larrabee and all that – it’s really taking off, and I think the jump to next generation’s going to be another really big one". Sadly, in an unexpected turn of events, Intel decided otherwise and decided to kill off the GPU that was partly Tim Sweeney's baby. In numerous occasions (e.g. Siggraph '09) Sweeney has stated that Epic's next-gen game technology Unreal Engine 4 was built specifically with Larrabee's multi-core architecture in mind.

Hopefully, this devastating revelation from Intel will not hinder Unreal Engine 4's supremacy in the next console generation, because I dare not imagine what console graphics would have looked like if it wasn't for UE3's anti-alias free dominance... oh the Humanity!