We are thrilled to announce that LFortran can now successfully compile and run lanl/SNAP, marking a significant milestone in our journey to beta. SNAP is the seventh production-grade, third-party code that LFortran can compile, bringing us closer to our goal of compiling 10 such codes—a critical step toward a beta-quality compiler.

About SNAP

SNAP (SN Discrete Ordinates Application Proxy) is a proxy application designed to emulate the performance of modern discrete ordinates neutral particle transport codes. It replicates the computational workload, memory requirements, and communication patterns of the Los Alamos National Laboratory code PARTISN, without including actual physics. By using domain decomposition and parallelization across spatial, angular, and energy domains, SNAP is ideal for performance testing on cutting-edge high-performance computing systems.

It is also a great application to test LFortran on, as it exercises the typical Fortran features used for computational physics, has a simple build system, and it is about 10K lines. By being able to compile it and run it successfully (reproducing results from other compilers), LFortran is well on its way of becoming a solid compiler that you can depend on.

How to Compile SNAP with LFortran

Follow the commands shown below to build and compile SNAP with LFortran.

Set up the Environment

conda create -n lfortran lfortran=0.40.0 make
conda activate lfortran
git clone https://github.com/certik/SNAP
cd SNAP
git checkout -t origin/lf11
cd src

Note: Currently, compilation is only supported with MPI and OpenMP turned off. Ensure that these features are disabled in the build flags to avoid any issues.

Build in Debug mode

make FORTRAN=lfortran FFLAGS= MPI=no OPENMP=no
./gsnap ../qasnap/sample/inp out

Build in Release mode

make clean
make FORTRAN=lfortran FFLAGS="--fast --skip-pass=promote_allocatable_to_nonallocatable" MPI=no OPENMP=no
./gsnap ../qasnap/sample/inp out

Compilation Benchmarks

We conducted several benchmarks to compare LFortran with GFortran and Flang in compiling the SNAP codebase. Using lfortran=0.40.0 on a Surface 5 laptop:

While LFortran shows promising results, especially in the module-based approach, there is still room for improvement in single-file compilation where GFortran currently holds an advantage. Ongoing work to optimize LFortran’s integration with LLVM is expected to enhance these benchmarks further. After we reach beta, we also plan to invest into our alternative backends like WASM that allow much faster compilation speed.

Also, we shall note that there are two small workarounds that are currently needed:

• Namelist: issue#1999
• Wrap a global subroutine into a module: issue#4175

We will implement namelists later, and the global subroutine issue we plan to fix after beta. The workaround however could be argued is an improvement to the original code, since a compiler cannot in general check argument types when calling global subroutines, while it checks everything if they are inside a module.

There is also a bug in the build system on macOS, which we fixed and submitted a PR upstream: SNAP#21.

Development Overview

Achieving this milestone was not without its challenges. We encountered some technical hurdles, but our team’s collaborative effort and determination ensured that these were swiftly addressed. Below are some of the key areas where we focused our development:

Provide nullify support

We implemented nullify mechanism in the LLVM backend. We used the LLVM API and the utilities we built before to make the array variable point to a null pointer. The issue was fixed via PR#4517.

Improving pointer management with nullify

We also worked on addressing an issue related to the nullify statement, where LFortran had difficulties handling pointer-arrays allocation after nullifying it. This led to further enhancements in LFortran’s allocation technique and also opened our eyes to a subtle bug that we may encounter later. The related issue is tracked at issue#4653.

Assignment to array of structs

We had to provide an implmentation to handle such an expression a%num = 44 where a is an array. We had to detect the existence of an array and then to build a loop on its indices to set their member num with value 44. Due to the flexibility in the ASR builder and the strong utilities: such a task was easy to implement. One may track it at issue#4579.

Where statement for looping on every dimension

We missed the case of doing nested loops on all of the array dimensions so we can test the predicate of the where statment. Fixing that was easy with the help of utilites and the separate-function responsibilites, We just got a recursive call to build nested DO loops from ground up. Feel free to lookup issue#4510.

Support printing and writing of tuple in ImpliedDoLoop

Fortran syntax doesn’t support compound data type such as tuples, except in one special case: implied do loops in print or write statements. There are multiple ways to implement this feature in a compiler, and we chose to use tuples since LCompilers internally already support tuples for LPython. We got the ASR builder identifying tuple in the implied do loops, but we had to extend our implementation to support printing it. So, we now support printing and writing of tuple in ImpliedDoLoop, e.g.

print *, (i, A(i), i = 1, 2)

now works. The changes are detailed in this PR#4683.

String format enhancement

We had to provide runtime implementation of computing the formatting of the resulting print statement. The implementation required array handling in the runtime, which made us able to print arrays with the desired format. It is tracked via issue#4507.

Overcoming the global subroutine linking issue

We encountered an issue related to the linking of global subroutines. It was challenging to determine whether this was due to an error in SNAP’s code or a bug in LFortran. After extensive investigation, we identified it as a bug in LFortran, which was subsequently reported and tracked at issue #4435. This issue underscores the need for improvements in LFortran’s handling of global subroutines and their linkage, especially when they are not encapsulated within modules.

Overshadow variables

We encountered the case where we had to overshadow the variables in the order of their declaration, so we added this feature in the process of building the ASR, and we also raised a warning for the user to be aware of the behavior the compiler is following. The related issue is tracked here.

Improve interface matching

We encountered a case where matching function with interface didn’t work properly due a missing edge case in the implementation of the matcher. With the help of how the nodes are structred, we were able to implement that with no problem. Detailed discussion over it is done at issue#4131.

What’s Next?

As of this writing, LFortran compiles seven third-party codes:

• Legacy Minpack (part of SciPy) and several more SciPy packages
• Modern Minpack
• fastGPT
• dftatom
• SciPy (60%)
• stdlib (85%)
• SNAP

Here is our issue to track priorities to reach beta quality.

Our primary goal is to bring LFortran from alpha to beta and we think compiling 10 third-party codes is a key milestone toward achieving this. Our approach involves prioritizing features based on their importance, ensuring that LFortran supports all language features present in the selected codes. Progress towards beta will be measured by successfully compiling and running these codes without alterations. We will keep on announcing each one as LFortran is able to fully compile and run. The codes we are targeting on next are Fortran Package Manager (fpm), LAPACK, PRIMA and remaining parts of Fortran stdlib and SciPy. Once we reach the milestone of compiling 10 codes, we’ll collaborate with the community to identify additional steps needed for beta. A beta-quality compiler, in our context, is defined as one that executes user code without errors, though it may still contain bugs.

Additionally, we are prioritizing support for newer LLVM versions to ensure LFortran remains compatible with the latest technologies, including LLVM 17, 18, and beyond.

Join Us

We welcome new contributors to join our journey. If you’re interested, please reach out on Zulip. Working on LFortran is both challenging and rewarding, offering ample opportunities for learning and growth.

Acknowledgements

We want to thank:

• Sovereign Tech Fund (STF)
• NumFOCUS
• QuantStack
• Google Summer of Code
• John D. Cook
• LANL
• GSI Technology
• Our GitHub, OpenCollective and NumFOCUS sponsors
• All our contributors (84 so far!)

Discussions

• Fortran Discourse: https://fortran-lang.discourse.group/t/lfortran-compiles-snap/8532
• Twitter/X: https://x.com/lfortranorg/status/1828459991046435083