Turbulence is ubiquitous in environmental and engineering fluid flows, and is encountered routinely in on a regular basis life. A greater understanding of those turbulent processes may present invaluable insights throughout quite a lot of analysis areas — bettering the prediction of cloud formation by atmospheric transport and the spreading of wildfires by turbulent vitality alternate, understanding sedimentation of deposits in rivers, and bettering the effectivity of combustion in plane engines to scale back emissions, to call a couple of. Nevertheless, regardless of its significance, our present understanding and our capacity to reliably predict such flows stays restricted. That is primarily attributed to the extremely chaotic nature and the large spatial and temporal scales these fluid flows occupy, starting from energetic, large-scale actions on the order of a number of meters on the high-end, the place vitality is injected into the fluid move, all the best way right down to micrometers (μm) on the low-end, the place the turbulence is dissipated into warmth by viscous friction.
A robust software to grasp these turbulent flows is the direct numerical simulation (DNS), which offers an in depth illustration of the unsteady three-dimensional flow-field with out making any approximations or simplifications. Extra particularly, this strategy makes use of a discrete grid with sufficiently small grid spacing to seize the underlying steady equations that govern the dynamics of the system (on this case, variable-density Navier-Stokes equations, which govern all fluid move dynamics). When the grid spacing is sufficiently small, the discrete grid factors are sufficient to symbolize the true (steady) equations with out the lack of accuracy. Whereas that is engaging, such simulations require super computational assets to be able to seize the proper fluid-flow behaviors throughout such a variety of spatial scales.
The precise span in spatial decision to which direct numerical calculations have to be utilized is dependent upon the duty and is decided by the Reynolds number, which compares inertial to viscous forces. Sometimes, the Reynolds quantity can vary between 102 as much as 107 (even bigger for atmospheric or interstellar issues). In 3D, the grid measurement for the decision required scales roughly with the Reynolds quantity to the ability of 4.5! Due to this robust scaling dependency, simulating such flows is mostly restricted to move regimes with reasonable Reynolds numbers, and usually requires entry to high-performance computing systems with thousands and thousands of CPU/GPU cores.
In “A TensorFlow simulation framework for scientific computing of fluid flows on tensor processing units”, we introduce a brand new simulation framework that allows the computation of fluid flows with TPUs. By leveraging newest advances on TensorFlow software program and TPU-hardware structure, this software program software permits detailed large-scale simulations of turbulent flows at unprecedented scale, pushing the boundaries of scientific discovery and turbulence evaluation. We show that this framework scales effectively to accommodate the size of the issue or, alternatively, improved run instances, which is outstanding since most large-scale distributed computation frameworks exhibit lowered effectivity with scaling. The software program is out there as an open-source undertaking on GitHub.
Giant-scale scientific computation with accelerators
The software program solves variable-density Navier-Stokes equations on TPU architectures utilizing the TensorFlow framework. The single-instruction, multiple-data (SIMD) strategy is adopted for parallelization of the TPU solver implementation. The finite difference operators on a colocated structured mesh are forged as filters of the convolution operate of TensorFlow, leveraging TPU’s matrix multiply unit (MXU). The framework takes benefit of the low-latency high-bandwidth inter-chips interconnect (ICI) between the TPU accelerators. As well as, by leveraging the single-precision floating-point computations and extremely optimized executable by way of the accelerated linear algebra (XLA) compiler, it’s attainable to carry out large-scale simulations with glorious scaling on TPU {hardware} architectures.
This analysis effort demonstrates that the graph-based TensorFlow together with new varieties of ML particular goal {hardware}, can be utilized as a programming paradigm to resolve partial differential equations representing multiphysics flows. The latter is achieved by augmenting the Navier-Stokes equations with bodily fashions to account for chemical reactions, heat-transfer, and density adjustments to allow, for instance, simulations of cloud formation and wildfires.
It’s price noting that this framework is the primary open-source computational fluid dynamics (CFD) framework for high-performance, large-scale simulations to completely leverage the cloud accelerators which have turn into frequent (and turn into a commodity) with the development of machine studying (ML) lately. Whereas our work focuses on utilizing TPU accelerators, the code will be simply adjusted for different accelerators, reminiscent of GPU clusters.
This framework demonstrates a technique to tremendously scale back the associated fee and turn-around time related to working large-scale scientific CFD simulations and allows even better iteration velocity in fields, reminiscent of local weather and climate analysis. Because the framework is carried out utilizing TensorFlow, an ML language, it additionally allows the prepared integration with ML strategies and permits the exploration of ML approaches on CFD issues. With the overall accessibility of TPU and GPU {hardware}, this strategy lowers the barrier for researchers to contribute to our understanding of large-scale turbulent programs.
Framework validation and homogeneous isotropic turbulence
Past demonstrating the efficiency and the scaling capabilities, it’s also crucial to validate the correctness of this framework to make sure that when it’s used for CFD issues, we get affordable outcomes. For this goal, researchers usually use idealized benchmark issues throughout CFD solver improvement, a lot of which we adopted in our work (extra particulars within the paper).
One such benchmark for turbulence evaluation is homogeneous isotropic turbulence (HIT), which is a canonical and properly studied move during which the statistical properties, reminiscent of kinetic vitality, are invariant below translations and rotations of the coordinate axes. By pushing the decision to the boundaries of the present state-of-the-art, we have been capable of carry out direct numerical simulations with greater than eight billion levels of freedom — equal to a three-dimensional mesh with 2,048 grid factors alongside every of the three instructions. We used 512 TPU-v4 cores, distributing the computation of the grid factors alongside the x, y, and z axes to a distribution of [2,2,128] cores, respectively, optimized for the efficiency on TPU. The wall clock time per timestep was round 425 milliseconds and the move was simulated for a complete of 400,000 timesteps. 50 TB knowledge, which incorporates the rate and density fields, is saved for 400 timesteps (each 1,000th step). To our data, this is among the largest turbulent move simulations of its variety performed so far.
As a result of complicated, chaotic nature of the turbulent move discipline, which extends throughout a number of magnitudes of decision, simulating the system in excessive decision is important. As a result of we make use of a fine-resolution grid with eight billion factors, we’re capable of precisely resolve the sphere.
 |
| Contours of x-component of velocity alongside the z midplane. The excessive decision of the simulation is crucial to precisely symbolize the turbulent discipline. |
The turbulent kinetic vitality and dissipation charges are two statistical portions generally used to research a turbulent move. The temporal decay of those properties in a turbulent discipline with out extra vitality injection is because of viscous dissipation and the decay asymptotes observe the anticipated analytical power law. That is in settlement with the theoretical asymptotes and observations reported within the literature and thus, validates our framework.
 |
| Strong line: Temporal evolution of turbulent kinetic vitality (okay). Dashed line: Analytical energy legal guidelines for decaying homogeneous isotropic turbulence (n=1.3) (Ⲧl: eddy turnover time). |
 |
| Strong line: Temporal evolution of dissipation fee (ε). Dashed line: Analytical energy legal guidelines for decaying homogeneous isotropic turbulence (n=1.3). |
The vitality spectrum of a turbulent move represents the vitality content material throughout wavenumber, the place the wavenumber okay is proportional to the inverse wavelength λ (i.e., okay ∝ 1/λ). Usually, the spectrum will be qualitatively divided into three ranges: supply vary, inertial vary and viscous dissipative vary (from left to proper on the wavenumber axis, beneath). The bottom wavenumbers within the supply vary correspond to the most important turbulent eddies, which have essentially the most vitality content material. These giant eddies switch vitality to turbulence within the intermediate wavenumbers (inertial vary), which is statistically isotropic (i.e., basically uniform in all instructions). The smallest eddies, equivalent to the most important wavenumbers, are dissipated into thermal vitality by the viscosity of the fluid. By advantage of the advantageous grid having 2,048 factors in every of the three spatial instructions, we’re capable of resolve the move discipline as much as the size scale at which viscous dissipation takes place. This direct numerical simulation strategy is essentially the most correct because it doesn’t require any closure mannequin to approximate the vitality cascade beneath the grid measurement.
 |
| Spectrum of turbulent kinetic vitality at totally different time situations. The spectrum is normalized by the instantaneous integral size (l) and the turbulent kinetic vitality (okay). |
A brand new period for turbulent flows analysis
Extra just lately, we prolonged this framework to foretell wildfires and atmospheric flows, which is related for climate-risk evaluation. Other than enabling high-fidelity simulations of complicated turbulent flows, this simulation framework additionally offers capabilities for scientific machine learning (SciML) — for instance, downsampling from a advantageous to a rough grid (model reduction) or constructing fashions that run at decrease decision whereas nonetheless capturing the proper dynamic behaviors. It may additionally present avenues for additional scientific discovery, reminiscent of constructing ML-based fashions to higher parameterize microphysics of turbulent flows, together with bodily relationships between temperature, strain, vapor fraction, and so on., and will enhance upon numerous management duties, e.g., to scale back the vitality consumption of buildings or discover extra environment friendly propeller shapes. Whereas engaging, a essential bottleneck in SciML has been the supply of information for coaching. To discover this, we’ve been working with teams at Stanford and Kaggle to make the information from our high-resolution HIT simulation obtainable by way of a community-hosted web-platform, BLASTNet, to supply broad entry to high-fidelity knowledge to the analysis group through a network-of-datasets strategy. We hope that the supply of those rising high-fidelity simulation instruments along side community-driven datasets will result in important advances in numerous areas of fluid mechanics.
Acknowledgements
We want to thank Qing Wang, Yi-Fan Chen, and John Anderson for consulting and recommendation, Tyler Russell and Carla Bromberg for program administration.
