Generic European Driving World

Our most recent military project is a tank training simulation. As with most of our vehicle training simulations, we have built hardware that closely resembles the internal cabin of the real vehicle (to the point where it feels as if you're in the real tank). For this particular sim the driver compartment is mounted upon a three degree of freedom motion platform. Since we're simulating multiple episcopes, external cameras, thermal imaging, night vision etc, we use multiple 'nodes' (i.e. multiple PCs connected via a network), with different graphics nodes controlling the visuals of different screens, and a core dynamics node running the physics engine, controlling vehicle internal logics and general driving dynamics (this is common practice for us, our sim framework is geared toward this kind of distributed architecture).

I can't really provide too much information about the vehicle aspect of the simulation (to avoid non-disclosure clause issues, I prefer to be cautious when it comes to military projects). However, what I can talk about is the generic European world we have built for this project.

My role was lead software developer for the world team, as well as technical lead for implementing new sim features to meet the requirements of the world. This is the largest and most complex world we've built, and so getting started on the project was a daunting task initially. There were many new, high-risk features and a limited time frame, and so the beginning R&D phase required a fair bit of planning and brain-storming.

Some of the features and tasks required for this project were:

• Taking fuller advantage of our dynamic time and weather system. I ported this code across from my own engine, Transcendence, a few years back, but we haven't been using it to its full capacity until now.
• Support for season transitions. We now support summer and winter.
• Support for snow precipitation. Previously the only precipitation model supported was rain. Besides the visual aspect (implemented via particle systems), this had repurcussions for our dynamics team (driving surfaces are affected by precipitation, for example, by becoming more slippery).

• A procedural terrain texturing system. The clients wanted vast tracts of drivable terrain, and so we needed to provide high fidelity, detailed terrain throughout the world. The world was far too massive for the artists to texture it by hand in the given timeframe, and so I coded a terrain shader that did this procedurally (with some degree of user control via the use of what I call the 'control map'). We were very satisfied with the outcome, the terrain looks good.
• Support for snow-covered terrain during winter. I modified the aforementioned terrain shader to dynamically alter the amount of snow on the terrain based on the surface normal of the landscape at any given point. This gives a more realistic look, with upward facing surfaces receiving more snow than steeply-sloped surfaces. So, for example, cliff faces are rocky rather than snowy, but plains and mountain peaks are covered in snow.
• Support for dry snow, wet snow and icy roads.

• A complete re-working of the way we construct pathing data. The world was just too massive for it to be feasible to manually construct pathing (this was the case for Durban driving world as well). For this particular project the art department used a 3rd-party city construction toolkit to build the roads. The framework allows the exporting of the spline information used to construct the roads, but unfortunately provides no documentation about the formatting of this information, and so I to figure it out (this entailed a fair bit of trial-and-error and guess-work). This turned out to be a success, and it allowed me to procedurally generate all pathing data from this spline information (I basically wrote a configurable command-line application to convert their splines to our pathing format).
• The previous change (as well as the scope of the world) required various modifications to our traffic system (specifically, I needed to re-work the pathing algorithms to no longer rely on pre-computed paths).

• Implementation of various miscellaneous world features (for example, see the train in the image below).
• Implementation of a set of exercises and associated scoring checks (too many to discuss in any reasonable amount of space).

Implementing thermal imaging.

Over the last two days I've been working on simulating the thermal imaging sensor (TIS) for a military training simulator project. This supports two modes, white hot and black hot. In white hot mode hotter objects are brighter. The sky tends to be dark grey to black, the terrain varies from dark grey to white (for hot sand and roads during the days), and people and vehicles tend to be light grey to white. Black hot is just the opposite (hotter objects are darker), which means I only really needed to implement white hot, and added a final step in my post-processing shader which inverts the image when black hot is enabled.

Previously I had implemented night vision and incorporated it into our post-fx pipeline. This entailed adding a custom high contrast bright pass filter, a green filter (tweaked in various ways to match the reference videos) and a grainy effect (similar to film grain). The excessive contrast works well with the bloom part of the pipeline to give a realistic looking dramatic blooming effect when parts of the rendered image becomes brighter (for example, when weapons are fired).

I used the same part of the post-fx pipeline to implement thermal imaging, but applied a greyscale filter instead, and used significantly less contrast. At this stage the result was a grainy greyscale image, but there was no differentiation between hot and cold objects.

Looking at the reference videos I observed that there is significantly more contrast on thermal imaging camera during the day than at night. I implemented this in the application code itself rather than in shaders. We have a dynamic time of day and weather system which allows for any time of day and weather combination, each with its own set of diffuse, ambient, fog and sky parameters. When thermal imaging is enabled I intercept the calls to set these parameters and supply an alternative set of parameters that match the reference videos. Fortunately, I had reference videos showing TIS enabled during transitions from day to night and vice versa, so I could run the video alongside our simulator, dramatically increase the time elapse rate in our simulator, and observe the results as I tweaked the values. As for the sky, I modified the dynamic skydome shader so that it bypasses the part where it derives colour for various regions of the sky based on time of day and weather, and outputted a single colour based on time of day instead.

In the reference videos the foliage tends to range from dark grey to black (it would seem that plants aren't particularly warm). This was easy enough, I simply intercepted the calls to the SpeedTree API and modified lighting and fog parameters to give the desired look.

At this stage the scene looked very close to the reference videos for the most part, except for hot objects -- we needed some way to make infantry and vehicles stand out white against the dark backdrop. Initially I was going to do this with shaders -- either in the specific shaders for the infantry and vehicles or via post-processing. However, in the real world various vehicles have very specific heat signatures. For example, some tanks are specifically designed so that they have low heat signatures, so that they don't stand out when TIS is enabled. Similarly, many tanks, trucks and even infantry show very specific heat patterns. For example, a tank may appear dark everywhere except around its engine, which glows white, allowing it to be uniquely identified by a knowledgeable observer. As a result, a generic solution such as this would not suffice.

To obtain realistic results we needed the artists to provide an additional set of customized textures, one for each vehicle and infantry unit type. When thermal imaging is enabled, the graphics engine is instructed to swap out the default set of diffuse maps for an alternative set of heat maps. The final result is a final rendered image that is very close to the real thing.