tinySG - Computational Fluid Dynamics

Friends of tinySG, Air flow around Space Shuttle

computational fluid dynamics has been in tinyScenegraphs project queue for a long time. In August 2013, a first milestone was reached with the GPU-based voxeliser. Two months later it is time to harvest some fruits grown on top of this base technology: Computational Fluid Dynamics (CFD) on polygonal models, calculated directly in tinySG.

The crowd of plugins for tinySG's scenegraph editor has grown by yet another member, the tinyFluids plugin: It provides an interface to the tinyFluids solver library as well as pre- and postprocessing capabilities on scene- and simulation data.

Pre-Processing scene data

The preprocessing step involves creating a simulation domain on a loaded scene. This basically involves the definition of (sub-)volume boundaries of the scene where fluid dynamics computations shall be performed. It also includes simulation parameter definitions like the direction and magnitude of flow or resolution (accuracy) of the simulation, as well as transformation of polygonal scene data into finite elements suitable for physics calculations.

Simulation domain: A special mouse event handler intercepts all mouse input in the scene viewer and allows to adjust a simulation sub-volume by dragging/resizing a cuboid. Once the desired area has been defined, the user may start the voxeliser, converting all polygons inside the sub-volume into voxels of a 3D texture.
Simulation parameters: Each simulation requires a set of parameters, like external forces, speed of flow, fluid viscosity, etc.
The tinyFluids preprocessor also involves the selection of a solver (tinyFluids and OpenLB are available), assigning available compute resources (like GPUs or CPU cores).
Finite elements creation: This task is performed by tinySG's Voxeliser: It transforms polygons inside the sub-volume into voxels of a 3D texture.
This step is extremely fast as it is processed entirely on the GPU, using scene data already available in GPU memory. A FirePro W8000 converts a scene with about 1 million triangles into a volume of 512x256x256 voxels (=33 MegaVoxels) in less than a second, running tinySG's voxeliser.

Running a simulation

Running a simulation is all about selecting a solver and feeding it with parameters determined in the preprocessor step. The tinyFluids solver can run on either the CPU cores using OpenMP or the GPU using OpenGL compute shaders.

To aid easy simulation domain setup, the solver may run with "empty" border cells, so that the fluid may flow around the specified simulation domain.

A running simulation may update the scene with intermediate results regularly, so the user can verify early on that the parameters converge against a reasonable result or choose to abort before the maximum simulation step count is reached.

Once the simulation has finished, it's results may be saved to a vti file for later re-import or analysis with the visualisation toolkit VTK.

Post-Processing simulation data

Finally, the post processor visualises the simulation results as the last step in simulation. A simulation may produce a scalar pressure field, a field of velocity vectors, temperature gradients, and many more results. tinyFluids aids all these applications of CFD by providing visualisation methods for both scalar and vector fields in both 2D and 3D, involving

Streamlines: tinyFluids renders streamlines as illuminated hermite splines, representing the flow through the simulation domain.
Velocity vectors: Small lines representing the fluid's velocity at the simulation domains grid points. May be visualised as color-coded lines showing the flow/speed at grid coordinates.
Iso surfaces:Iso surfaces are made of points of equal pressure, temperature, etc. They are created using the marching cube algorithm.
The post processor allows to adjust the iso surface parameter on the fly, so the user may use a slider to browse through shells of i.e. equal pressure.
Particle system:
Annotated model: The CAD model can be annotated by assigning additional attributes to each of it's vertices. For example, a vertex may receive a color attribute based on it's nearest simulation grid point(s), coding the local pressure or speed of flow at it's position. Vertex attributes can be scalars or vectors and GLSL shaders can evaluate them in arbitrary ways.

Visualisation of simulation data is handled as Meta Geometry: Although Meta Geometry is made up by regular scenegraph nodes, it is not added to the CAD dataset. Instead, the tinySG renderer handles it separately, and the plugin maintains a handle to the topmost group node so it can influence all nodes properties.


Field of velocity vectors. Image shows 50% of the vectors in a 64x16x32 field.	Visual representation of a voxelised dataset (128x32x64 voxels).

Performance

Since the tinyFluids solver may run on either the CPU or the GPU, performance comparisons between the two architectures are possible. The results are particularly interesting, because the CFD algorithm scales very well with available compute resources. If you ever wondered what use to make of all your CPU cores, here is an application that could set hundreds of cores under fire.

The charts below show the CPU scalability for a number of different machines with very different price tags. The x-axis enumerates the processor type, along with the number of available cores. The z-axis specifies the number of cores being used for the simulation. The y-axis shows the total simulation run time in seconds on 800 iterations over 90,000 cells (lower values are better).

The sweet spot is often the native core count. Exceptions are the Core i5 and Core i7 machines which actually benefit from hyperthreading (if load is balanced evenly across virtual cores). Going beyond the native core count tends to slow down the simulation again.
Please note that the comparison is not entirely fair: Processors spread over various generations from Nehalem to Ivy Bridge, the T1090 ran on Linux, while all other systems utilised Windows 7. Nevertheless, the chart gives a good overview on scalability and a rough impression on relative system performance.

Outlook

As always, there is much more left to do: The next step is definitely going to be a GPU implementation of the solver, using either OpenGL compute shaders or OpenCL. The algorithm should be a perfect match for a parallel implementation, using the full computational power of a GPU. It will be interesting to see how GPUs perform compared to an OpenMP implementation running on the CPU's cores.

The fluid model used in tinyFluids is limited to special cases and cannot simulate turbulent flows. There are many extensions described in scientific papers that wait for their implementation.

Furthermore, having this great solver code available, it should be interesting to utilise it for some more physics use cases, like heat dissipation or simulation of distribution of sound.

Finally, I cannot neglect to concede that this project made some appetite for more physics. A likely candidate is the integration of the bullet physics engine into tinySG, taking care of some rigid body collisions/simulations.

Keep rendering,
Christian

Acknowledgements and links:

OpenLB is an open source implementation of the Lattice Bolzmann method on solving the Navier/Stokes equations. It served as a validation tool for the tinyFluid implementation and is also available in the tinyFluids plugin as an alternative solver.
The paper of Dragos Chirila has been a most valuable resource on Lattice-Bolzmann simulation. I'd suggest anybody interested in the mathematics, physics and algorithm behind tinyFluids to read this excellent work.
The Space Shuttle dataset is used by courtesy of NASA and is available for educational or informational purposes on NASA's 3D resource web page.