nDisplay is Unreal Engine’s tool for CAVE and other cluster rendering, allowing simulation to be displayed on any amount of screens from any amount of computers. However, getting nDisplay to work for driving simulation can be challenging.

First, I recommend reading the Synchronization in nDisplay article, which didn’t exist when I started using the tool, and would have been of great help if it did.

Non-deterministic physics

One of the first hurdle I encountered with nDisplay was when I noticed that as time went on during the simulation, cars’ position’s would start to diverge from one screen to the next. In the first seconds, it would be near invisible, but after a few minutes, some cars could be at totally different places on each monitor.

This is caused by PhysX’s non-deterministic model. Behind the scenes, nDisplay synchronizes all instances’ clocks, and each instance then “solves” the simulation for that given time. But if the physics model is non-deterministic, you can then have two instances having very slight differences in their outputs, placing cars at very slightly different positions. Over the course of a whole scenario, this can lead to drastic differences.

As I’m writing this (4.26), Chaos is still experimental and not part of the official release. But this new physics engine might solve this issue by itself, as it seems Epic Games added a deterministic mode for nDisplay use cases.

Our workaround, as shown above, is to disable all car’s physics on all cluster slaves, having only the master solving their position. Cars’ transform are then sync’d with slaves using Actor Replication.

Swap sync policy

Before the Synchronization in nDisplay page existed, I spent way too much time discovering that the swap_sync_policy existed. My issue was that for some reason, when using nDisplay, the simulation wouldn’t run past 30Hz, even though it could run at 120Hz on a single instance without issue. But as soon as I connected any slave, it would cap at 30Hz.

It turned out that our simulator’s TVs are 50Hz, and swap_sync_policy defaults to 1, which, now that we have this new document, we can have a better understanding of:

This means VSync=1. All nodes show their frames after RenderThreadBarrier WITH synchronization to VBlank.

So if you have 50Hz TVs, you probably want a swap_sync_policy of 0. Which now means that you have tearing all over your screen. Which is the current situation I’m at.

Fake ego-chassis

If you disabled swap sync and you’re in a setup where you display ego’s chassis on screen for better immersion (e.g. not full scale simulator), you’ll end up having a lot of stutter on said chassis.

I’m still not sure why it happens, but I’m guessing it’s because without swap sync, each slave renders when it wants, and at the time each one does that, some have received the latest ego position and some have not (from the actor replication mentioned above). This is a very common problem when rendering on a cluster without proper synchronization.

To solve this issue, and as illustrated above, we simply hide ego’s chassis, and on each slave we create a new “fake” one that we attach to nDisplay’s root actor. With such attachment, this fake chassis will always have the same position relative to nDisplay’s root, and therefore relative to each screen. No more chassis stuttering!

Mirrors

One of the specificity of driving simulators is mirrors. Not only do you want to render a full scene from the driver’s point of view, you also want to render mirrors that show the same scene from a different point of view.

Depending on your simulator hardware, you might render mirrors from within the scene itself, as shown above. This is ideal for small-scale simulator, where the driver might not be in an actual car.

On larger scale simulator, you can have a dedicated screen for each mirror, as illustrated above. If you look closely, you’ll notice that there’s an issue with it: the image isn’t mirrored, i.e. isn’t flipped horizontally, as it would normally be when looking in a mirror. In other words the left of the displayed image should be on the right, and vice versa.

Currently, nDisplay offers no way of flipping an image. I’ve tried multiple hacks in the configuration file (e.g. negative viewport width), but nothing worked.

Our planned workaround is to use dedicated hardware, such as an HDMI MirrorBox, that would simply flip the output HDMI stream and therefore display the proper view on the screen.

Cluster events

Cluster Events are nDisplay’s way of making all cluster nodes act simultaneously on an event. They can be emitted from your simulation, but also from external tools. There are multiple cases where those are needed, which aren’t necessarily documented.

For example, in our platform, we have a small UI where the user can choose the scenario he’d like to run. This implies calling OpenLevel with the correct parameters. However, if this is done on the master only, the simulation crashes. Any call to OpenLevel must be propagated using cluster events, so that all nodes actually open the level. And that’s not all, I’ve noticed that by just doing that, the simulation would still sometimes crash on level loading. I’ve had to add a 1s delay between the message being received by all listeners and the actual OpenLevel call.

As of 4.26, a new action required using cluster events: pause. It’s the same idea as OpenLevel, and will also cause a crash if not done through cluster events.

Stutter

An ongoing issue we have with nDisplay is heavy stuttering, with some ticks over 300ms. Our simulation runs fine outside nDisplay, or with master only; but if we add any slave to the cluster, we get the following (from profiling session on master).

As of right now, we don’t have much clue as to what causes it. But as soon as we have an answer, we’ll let you know!