We’re thrilled to share that LFortran can now successfully compile and execute libprima/PRIMA. PRIMA marks the eighth production-grade, third-party code that LFortran has compiled with bit-for-bit alignment to GFortran’s output. This milestone brings us closer to our goal of compiling 10 such codes—an essential step toward achieving a beta-quality compiler.

About PRIMA

PRIMA is a package developed by Zaikun Zhang for solving general nonlinear optimization problems without using derivatives. It provides the reference implementation for Powell’s derivative-free optimization methods, i.e., COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. PRIMA means Reference Implementation for Powell’s methods with Modernization and Amelioration, P for Powell. The package is written in Fortran and is widely used in the scientific community for solving optimization problems.

It utilizes a range of Modern Fortran features, including extensive use of optional variables, function pointer passing, and a randomized test driver, among others. Successfully compiling PRIMA requires a compiler with a robust and mature backend, as well as well-developed intermediate passes and a capable parser.

How to Compile PRIMA with LFortran

Follow the steps shown below to build, compile and run PRIMA examples and tests with LFortran.

Set up the Environment

conda create -n lfortran lfortran=0.46.0 make cmake
conda activate lfortran
git clone https://github.com/Pranavchiku/prima.git
cd prima
git checkout -t origin/lf-prima-12

For macos

export LFORTRAN_RUNNER_OS='macos';

For linux

export LFORTRAN_RUNNER_OS='linux';

git clean -dfx
FC="lfortran --cpp" cmake -S . -B build -DCMAKE_INSTALL_PREFIX=$(pwd)/install  && cmake --build build --target install

To execute examples

./build/fortran/example_bobyqa_fortran_1_exe
./build/fortran/example_bobyqa_fortran_2_exe
./build/fortran/example_cobyla_fortran_1_exe
./build/fortran/example_cobyla_fortran_2_exe
./build/fortran/example_lincoa_fortran_1_exe
./build/fortran/example_lincoa_fortran_2_exe
./build/fortran/example_newuoa_fortran_1_exe
./build/fortran/example_newuoa_fortran_2_exe
./build/fortran/example_uobyqa_fortran_1_exe
./build/fortran/example_uobyqa_fortran_2_exe

To execute tests

cd fortran/
test_name=test_xx.f90 FC=lfortran ./script.sh
cd ../

Build with optimisations

git clean -dfx
FC="lfortran --cpp --fast" cmake -S . -B build -DCMAKE_INSTALL_PREFIX=$(pwd)/install  && cmake --build build --target install

Compilation Benchmarks

To ensure no performance loss, we conducted several benchmarks comparing LFortran against GFortran, both without any optimisations when compiling PRIMA. For these tests, we used lfortran=0.46.0 on a MacBook Air M2 with 8GB of RAM.

For compiling single binary ( for example: cobyla ), one has to apply the following diff and then compile the code.

diff --git a/fortran/CMakeLists.txt b/fortran/CMakeLists.txt
index cf16524a..b1673397 100644
--- a/fortran/CMakeLists.txt
+++ b/fortran/CMakeLists.txt
@@ -147,13 +147,13 @@ macro (prima_add_f_test name number)
   add_dependencies(examples example_${name}_fortran_${number}_exe)
 endmacro ()
 
-prima_add_f_test (cobyla 1)
+# prima_add_f_test (cobyla 1)
 prima_add_f_test (cobyla 2)
-prima_add_f_test (bobyqa 1)
-prima_add_f_test (bobyqa 2)
-prima_add_f_test (newuoa 1)
-prima_add_f_test (newuoa 2)
-prima_add_f_test (uobyqa 1)
-prima_add_f_test (uobyqa 2)
-prima_add_f_test (lincoa 1)
-prima_add_f_test (lincoa 2)
+# prima_add_f_test (bobyqa 1)
+# prima_add_f_test (bobyqa 2)
+# prima_add_f_test (newuoa 1)
+# prima_add_f_test (newuoa 2)
+# prima_add_f_test (uobyqa 1)
+# prima_add_f_test (uobyqa 2)
+# prima_add_f_test (lincoa 1)
+# prima_add_f_test (lincoa 2)

Example Benchmarks

We’re pleased to report that LFortran compiles the PRIMA code as is—without any workarounds or modifications—except for the necessary adjustments to integrate LFortran into the existing PRIMA build system.

Development Overview

PRIMA has been on our list since we set out to achieve our goal of compiling 10 production-level codes. However, compiling it was no cakewalk—it came with its own set of challenges. Despite this, our team’s collaborative effort and determination allowed us to tackle them effectively. Here are some key areas where we focused our development efforts:

Strengthening support for procedure variables

PRIMA extensively utilizes function pointers as arguments, relying on procedure variables for linking. This posed a challenge due to LFortran’s initial improper handling of procedure variables in its intermediate passes. Addressing this issue required multiple pull requests, which can be found under label-prima.

Passing arrays with unequal strides not equal to one to functions

PRIMA presented cases where array sections with non-unit strides were passed to functions, requiring us to generate temporary arrays at runtime. To handle this efficiently, we developed a dedicated pass to detect such cases and create the necessary temporaries. The implementation details can be found at PR#6363

Precision loss while using power ** operator

We encountered precision loss issues in PRIMA when using the power operator **, particularly with double-precision variables with large exponents. This led to incorrect results for expressions like v**3, as LFortran internally cast the exponent to real(3, kind=8). This issue was resolved by using llvm.powi through a dedicated PR#.

Along with these there were several other issues that were resolved to successfully compile PRIMA with LFortran. The complete list of issues and PRs can be found here.

What’s Next?

As of this writing, LFortran compiles eight third-party codes:

• Legacy Minpack (part of SciPy) and several more SciPy packages
• Modern Minpack
• fastGPT
• dftatom
• SciPy (60%)
• stdlib (85%)
• SNAP
• PRIMA

Here is our issue to track priorities to reach beta quality.

Our primary objective is to transition LFortran from alpha to beta, and we believe that successfully compiling 10 third-party codes is a key milestone in this journey. Our strategy prioritizes features based on their significance, ensuring that LFortran fully supports all language constructs used in these selected codes.

Progress toward beta will be measured by our ability to compile and execute these codes without modifications. Currently, another team is working on POT3D, which is nearly compiling but requires further refinement. We will continue announcing each successful compilation as LFortran achieves full compatibility.

Next, we plan to target Fortran Package Manager (fpm), LAPACK, other codes listed under label:code-to-be-compiled, and the remaining parts of Fortran stdlib and SciPy.

Once we reach our goal of compiling 10 third-party codes, we will collaborate with the community to determine additional steps needed for beta. In our definition, a beta-quality compiler is one that successfully executes user code without errors, though it may still contain some bugs.

Join Us

We welcome new contributors to join our journey. If you’re interested, please reach out on Zulip. Working on LFortran is both challenging and rewarding, offering ample opportunities for learning and growth.

Acknowledgements

We want to thank:

• Sovereign Tech Fund (STF)
• NumFOCUS
• QuantStack
• Google Summer of Code
• John D. Cook
• LANL
• GSI Technology
• Our GitHub, OpenCollective and NumFOCUS sponsors
• All our contributors (104 so far!)

Discussions

• Fortran Discourse: https://fortran-lang.discourse.group/t/lfortran-compiles-prima/9296
• Twitter/X: https://x.com/lfortranorg/status/1896958812365713906