Loci

Loci: Logic Programming for Parallel Computational Field Simulations

The Loci Framework

The Loci framework was originally developed in the late 1990s with support from the NSF ERC for Computational Field Simulations to simplify the development of complex numerical models that can take advantage of massively parallel high end computing systems. The framework provides a rule-based programming model whereby an application is described in terms of a collection of simple computational kernels. A key feature of the Loci framework are that these computational kernels are documented such that the data-flow between kernels can be extracted using relational annotations that are provided by the rules. In this system the given data is provided as a set of facts (relations). The application is then given in the form of a collection of transformation rules. Finally, the user makes a query (e.g. what is the heat flux at this surface?) From this the scheduler provided by the Loci framework provide a composition of the provided transformations that will answer the query. The effect of the programming model is to place numerical programming in a sphere similar to one of information management using relation databases where problems are solved using powerful relational queries.

The advantage of the Loci framework is that it views the data-structures of a scientific computation abstractly as a relation. Computational kernels then read and write data using relational queries. Relations provide a mechanism for abstracting the data-structure from the computations. In consideration of performance most data-structures in Loci are implemented using distributed arrays, but the model does not preclude alternative implementations such as hashed containers. This separation of concerns gives the Loci programming model an ability to morph an application onto a wide variety of architectural configurations.

To facilitate flexible scheduling of computations, the majority of computational transformations described by Loci rules have single-assignment semantics and are referentially transparent. Loci takes advantage of these features in order to optimize the scheduling of computations in order to factor in non-uniform cost structures presented by modern high end computing systems. For example, Loci provides a work replication optimization that replicates computations on remote processors in order to reduce inter-processor bandwidth requirements. These optimization have been shown to improve application scalability on clusters that use low-cost and subsequently low-bandwidth interconnect. Optimizations that change the order of computations in order to enhance locality have also been implemented and shown to improve application performance.

A New Era for Loci: Introducing the Loci Software Consortium

Established through a National Science Foundation Pathways to Enable Open-Source Ecosystems (POSE) award, the Loci Software Consortium is the formal structure guiding Loci’s transition to a community-driven open-source ecosystem. It brings together industry, government, and academic users and stakeholders to set Loci’s direction, maintain its quality, and keep it at the forefront of computational fluid dynamics for years to come. With a shared governance model and technical leadership ladder, the consortium offers a way for the broader Loci community to shape what comes next for. For more information, contact Alex Way at asw241@msstate.edu or John Walters at john.walters@msstate.edu.

Why Is The System Called Loci?

The name of this system, Loci, is the plural of locus. The meaning of locus that is implied here is the set of points that satisfy some given set of rules. For example, a circle is the the locus of points that are equidistant from a given center point. The Loci system derives the control structure by finding the set of "entities" that satisfy the rules given to the system. Thus, in a discrete sense, the program control structure in the Loci system is derived by computing a set of locus. From this the process the Loci system borrows its name.

Simulation Tools

A variety of simulation tools have been developed using the Loci framework. While many of these applications were developed at Mississippi State University along with the Loci framework, several tools have also been developed by other companies and organizations. Below is a list of Loci-based solvers with a brief description of their capabilities and associated organizations.

Loci/CHEM
CHEM is a full featured Navier-Stokes solver for non-equilibrium flows involving chemical reactions. CHEM was developed at Mississippi State University and was the first application developed using the Loci framework. The CHEM solver uses advanced generalized grid algorithms based on finite-volume methods and high resolution Riemann solvers.

flowPsi
The flowPsi solver is an open source implementation of a fluid flow solver using computational fluid dynamics model to solve the Navier Stokes equations for compressible ideal gas flows. It is suitable for academic use or for problems that do not involve chemical reactions.

Loci/GGFS
Loci/GGFS is a Eulerian-Eulerian gas-granular flow solver for modeling granular material efficiently and accurately. The solver was developed by CFD Research Corporation (CFDRC) and has been applied to large scale simulations of plume-soil interactions, upwards of 100M cells, and 8-10K processors. The results include cratering characteristics, debris and regolith characteristics post impact.

Loci/Boltzmann
CFD Research has developed a Eulerian rarefied gas-dynamics flow solver which can provide more than an order-of-magnitude superior performance compared to the Monte Carlo based solvers. The solver can also be coupled to Loci/CHEM to allow combined continuum-rarefied flow simulations accurately.

Loci/THRUST
Loci/THRUST is a finite-element solver of the compressible Navier-Stokes equation based on the discontinuous Galerkin method. Developed by Mississippi State University, THRUST is typically deployed to solve problems for which numerical dissipation must be kept to a minimum (e.g. aeroacoustics, combustion/stability analysis).

Loci-Stream
Loci-Stream is a comprehensive tool developed by Streamline Numerics that is capable of efficiently solving fluid-flow and heat transfer in large scale industrial applications. Loci-Stream employs pressure-based methods from the SIMPLE family of algorithms to provide a unified framework for the solution of both incompressible and compressible flows.

Loci/VOF
Loci/VOF is a fully unstructured volume-of-fluid (VOF) solver module that links with Loci/CHEM and Loci/STREAM. The module was developed by CFD Research and has been applied for slosh-dynamics modeling, surface tension driven flows in micro-gravity, ignition-over pressure mitigation and other applications.

Loci/RTE
Loci/RTE is a generalized radiative transfer equation solver developed by CFD Research for analyzing combustion systems. The solvers and radiative property models allow close to line-by-line accuracy for radiation transport modeling in combustion systems. Multiple RTE solvers including spherical harmonics, finite-volume discrete ordinates, and photon Monte Carlo methods are available.

Representative Papers

  • "A Rule-Based Specification System for Computational Fluid Dynamics", PhD Dissertation, Mississippi State University, December 1999
  • "Loci: A Deductive Framework for Graph-Based Algorithms", ISCOPE'99 Copyright Springer-Verlag pp 142-153
  • E. Luke and T. George, "Loci: A Rule-Based Framework for Parallel Multidisciplinary Simulation Synthesis," Journal of Functional Programming, Special Issue on Functional Approaches to High-Performance Parallel Programming, Volume 15, Issue 03, pp. 477-502, Cambridge University Press
  • Y. Zhang and E. Luke, "Dynamic Memory Management in the Loci Framework," Parallel and Distributed Computing Practices , Volume 7, Number 3, September, 2006
  • P. Adhikari, E. Luke, and E. Allen, "Verification of a Loop Scheduling Protocol using Finite State Verification," ISCA 22nd International Conference on parallel and Distributed Computing and Communication Systems (PDCCS-2009), September 24-26, Louisville, Kentucky, 2009
  • Y. Zhang and E. Luke, "The Design and Implementation of Dynamic Irregular Parallel Computations using the Key-Value-Reference Model," ISCA 22nd International Conference on parallel and Distributed Computing and Communication Systems (PDCCS-2009), September 24-26, Louisville, Kentucky, 2009
  • K. Soni, N. Cain, and E. Luke, "Work Replication: A Communication Optimization in Loci," ISCA 21st International Conference on Parallel and Distributed Computing and Communication Systems (PDCCS-2008), September 24-26, 2008, New Orleans, LA
  • Y. Zhang and E. Luke, "Concurrent Composition Using Loci," Computing in Science and Engineering, Volume 11, Issue 3 (May 2009), pp. 27-35
  • Y. Zhang and E. Luke, "Using the Loci Framework for Automated Program and Component Generation," Workshop on Automated Program Generation for Computational Science, ICCS May 31-Jun 2, 2010, Amsterdam, The Netherlands, Procedia Computer Science, Volume 1, Issue 1, May 2010, pp. 1855-1861
  • E. Luke, "Loci: Automated Synthesis for Terascale Computing Systems," 27th Army Science Conference, Orlando, Fl, November 29th - December 2nd, 2010, CP-08
  • S. Medders, E. Allen and E. Luke, "Using Rule Structure to Evaluate the Completeness of Rule -Based System Testing," Journal of Software Engineering and Knowledge Engineering, Volume 20, Issue 7, pp. 975-986, 2010

Example Applications

The Open Source FlowPsi CFD Solver

The flowPsi solver is an open source implementation of a fluid flow solver using computational fluid dynamics model to solve the Navier Stokes equations for compressible ideal gas flows. It can be found on source forge at flowPsi.

Chem a Finite-Rate Chemistry Solver

The Loci/CHEM code, originally developed as a technology demonstrator for the Loci framework, has become a well developed and mature simulation code for complex multiphysics simulations. The solver that forms the central core of the code is based on high-resolution Godonov methods implemented for multicomponent flows using an implicit time integration scheme that makes the code effective for high Reynolds number flows at high speeds. The core algorithims have been extended to accurately model flows at low speeds through the use of PDE preconditioning techniques.

Since the original development in 1999, the Loci/CHEM code has been significantly extended under support from NASA Marshall Space Flight Center in order to simulate cryogenics, high pressure combustion, multiphase flows, coupled non-gray radiation effects. More recently support from the Air Force, Department of Homeland Security and US Army are extending the capabilities of this mature production code into areas of hypersonic flows and blast simulations. Although the first version of the code was released in 1999, by 2006 the Loci/CHEM code was among the top 30 in CPU hours requested on the DoD MSRC centers. Today it is used widely by NASA and industry predominately in the simulation of aerospace related problems. Notably it has been used extensively as the reference model in external aerodynamics simulations for the ARES I. More information about the CHEM code can be found here.