LFortran: the Fastest Open-Source Compiler in Compile-Time Evaluation of an Array Benchmark

After successful compilation of stdlib and Fortran On Web Using LFortran, we focused on improving the support for compile time computation and we are excited to announce that LFortran can now compile the example from the Computing at compile time Fortran Discourse topic.

This example can be used to benchmark the speed of compilation and compile-time array evaluation by increasing the size of the arrays. On this particular example LFortran outperforms other open-source Fortran compilers by an order of magnitude for large array sizes. For small array sizes each compiler has a different constant overhead, which becomes negligible for large arrays and this benchmark measures how quickly the compiler can internally apply the symbolic function on a large array numerically.

You can try LFortran yourself on other examples and if you find that other compilers significantly outperform it in the speed of compilation, please let us know.

Why care about the speed of compilation? When developing large Fortran codes, it is essential to compile as quickly as possible to make the edit-compile-run cycle short. It must be a pleasure to develop in Fortran. Fast compilation, beautiful error messages both at compile time and runtime in Debug mode (no segfaults).

When running in production, we then want the fastest performance at runtime. LFortran’s goal is to be fast to compile in Debug mode and fast to run in Release mode.

LFortran is currently alpha software, meaning that users must continue expecting that LFortran will fail compiling or running their codes. Please report all bugs that you find.

Compilation Benchmark

The following program computes the integral of sin(x) from 0 to pi at compile-time using the simplest linear approximation:

compile_time.f90

program compile_time
implicit none
integer, parameter :: N = 65535
integer, parameter :: sp = kind(1.0)
integer, parameter :: dp = kind(1.d0)
real(dp), parameter :: pi = 2*asin(1._dp)
real(dp), parameter :: a = 0, b = pi
real(dp), parameter :: dx = (b-a)/N
integer :: i
real(dp), parameter :: X(*) = [(sin(a+(b-a)*i/N), i = 1, N)]
real(dp), parameter :: S = sum(X)*dx
logical, parameter :: l = S < 2
real(kind=merge(sp, dp, l)) :: y
if (kind(y) == sp) then
    print *, "true"
else
    print *, "false"
end if
print *, S
end program

Now we benchmark the compilation speed with various compilers depending on the array size N (the number of integration points).

On MacBook Air M2 8GB RAM

On Surface Laptop 5; WSL, Ubuntu 22.04, 12th Gen Intel(R) Core(TM) i7-1265U

On Ryzen 7 5800, 16G RAM, Ubuntu

The above tables are visualized in the graphs below:

How to Run the Benchmark Yourself

We installed LFortran, GFortran and Flang using Conda, that way you can install exactly the same binary as well, and that way if there is any issue how each compiler was compiled into the conda package, one can simply send a PR against the conda-forge repository to improve it, and the compiler developers can ensure that their compiler is compiled correctly with maximum performance.

Create a conda environment with exactly the versions that we used:

mamba create -n ct lfortran=0.36.0 gfortran=13.2.0 flang=18.1.6
mamba activate ct

Benchmark the speed of compilation:

time lfortran compile_time.f90 -o a.out
time gfortran -fmax-array-constructor=10000000 compile_time.f90 -o a.out
time flang-new compile_time.f90 -o a.out

Note: Flang currently does not have a conda package for Apple Silicon, so we only benchmarked it on Linux.

Development Overview

The roots of development lead us to PR#3564 which aimed to handle compile time evaluation of elemental functions. At that time we only had ArrayConstant represented in ASR grammar as:

ArrayConstant(expr* args, ttype type, arraystorage storage_format)

That means ArrayConstant was being used to handle both compiletime and runtime values. Thereafter we decided to split it into ArrayConstructor for handling runtime value and ArrayConstant which contains compiletime values.

Thus in PR#3579, the grammar was changed to:

ArrayConstructor(expr* args, ttype type, expr? value, arraystorage storage_format) ArrayConstant(expr* args, ttype type, arraystorage storage_format)

From then, ArrayConstant is only allowed other *Constant in it (but not another ArrayConstant) whereas the ArrayConstructor is allowed other nodes. For example [sin(x), y], or two ArrayConstants.

Later, we realized that we should improve upon the internal representation of ArrayConstant further, so we documented approach at issue#3581.

Now,ArrayConstant should contain a type (that includes kind) of the element, shape, and then a pointer to contiguous data. Thus the ASR grammar was changed to:

ArrayConstant(int n_data, void data, ttype type, arraystorage storage_format)

Implementation details can be found at PR#3859.

There were also a bunch of minor patches that contributed to performance improvement for evaluating compile time values.

Contributing

Help us get to beta faster by reporting bugs and submitting PRs to fix things. We will help you get started, no prior compiler experience needed.

Acknowledgements

We want to thank:

• Sovereign Tech Fund (STF)
• NumFOCUS
• QuantStack
• Google Summer of Code
• GSI Technology
• LANL
• Our GitHub, OpenCollective and NumFOCUS sponsors
• All our contributors (78 so far!)

Discussions

• Fortran Discourse: https://fortran-lang.discourse.group/t/compile-time-computation-example-now-works-with-lfortran/8139
• Twitter: https://twitter.com/lfortranorg/status/1797999009849868783
• LinkedIn: https://www.linkedin.com/posts/pranavgoswami1_fortran-arrays-performance-activity-7203772703198052352-Q1RR/