Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Flang] LLVM_ENABLE_RUNTIMES=flang-rt #110217

Open
wants to merge 1 commit into
base: users/meinersbur/flang_runtime_move-files
Choose a base branch
from

Conversation

Meinersbur
Copy link
Member

@Meinersbur Meinersbur commented Sep 27, 2024

Extract Flang's runtime library to use the LLVM_ENABLE_RUNTIME mechanism.

Motivation:

  • Consistency with LLVM's other runtime libraries (compiler-rt, libc, libcxx, openmp offload, ...)
  • Allows compiling the runtime for multiple targets at once using the LLVM_RUNTIME_TARGETS configuration options
  • Installs the runtime into the compiler's per-target resource directory so it can be automatically found even when cross-compiling

Potential future directions:

  • Uses CMake's support for compiling Fortran files, including dependency resolution of Fortran modules
  • Improve robustness of compiling libomp.mod when openmp is available
  • Remove Flang's dependency from flang-rt's RTNAME function declarations (tblgen?)
  • Reduce Flang's build-time dependency from flang-rt's REAL(16) support

See RFC discussion at https://discourse.llvm.org/t/rfc-use-llvm-enable-runtimes-for-flangs-runtime/80826

Patch series:

Patch for lab.llvm.org buildbots:

@Meinersbur Meinersbur changed the base branch from main to users/meinersbur/flang_runtime_move-files September 27, 2024 18:58
Copy link

github-actions bot commented Oct 1, 2024

⚠️ Python code formatter, darker found issues in your code. ⚠️

You can test this locally with the following command:
darker --check --diff -r 7e369b5626b6de43de394212efe5024a4fdebf48...f67081627ea71065635f5dd84172bcd68904f4f6 flang-rt/test/NonGtestUnit/lit.cfg.py flang-rt/test/Unit/lit.cfg.py flang-rt/test/lit.cfg.py flang/test/lit.cfg.py
View the diff from darker here.
--- flang-rt/test/lit.cfg.py	2025-01-24 15:50:07.000000 +0000
+++ flang-rt/test/lit.cfg.py	2025-01-24 18:46:12.948136 +0000
@@ -78,23 +78,21 @@
         "%clang",
         command=FindTool("clang"),
         extra_args=isysroot_flag,
         unresolved="fatal",
     ),
-    ToolSubst("%cc",
-        command=config.cc,
-        extra_args=isysroot_flag,
-        unresolved="fatal"
-    ),
+    ToolSubst("%cc", command=config.cc, extra_args=isysroot_flag, unresolved="fatal"),
 ]
 llvm_config.add_tool_substitutions(tools)
 
 # Let tests find LLVM's standard tools (FileCheck, split-file, not, ...)
 llvm_config.with_environment("PATH", config.llvm_tools_dir, append_path=True)
 
 # Include path for C headers that define Flang's Fortran ABI.
-config.substitutions.append(("%include", os.path.join(config.flang_source_dir, "include")))
+config.substitutions.append(
+    ("%include", os.path.join(config.flang_source_dir, "include"))
+)
 
 # Library path of libflang_rt.a (for lib search path when using non-Flang driver for linking)
 config.substitutions.append(("%libdir", config.flang_rt_output_resource_lib_dir))
 
 # For CUDA offloading, additional steps (device linking) and libraries (cudart) are needed.

Copy link

github-actions bot commented Oct 1, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

@Meinersbur
Copy link
Member Author

Meinersbur commented Oct 1, 2024

I reverted the change that moved flang/module/*.f90 to FortranRuntime/module/*.f90. A lot of Flang tests depend on the *.mod files to be available. Moving the modules to FortranRuntime would either require moving those tests to FortranRuntime as well, or mark them as requiring the module_files (REQUIRES: module_files) and build those before check-flang:

flowchart TB
 subgraph Flang
   direction TB
   flang-new
   check-flang
 end
 subgraph FortranRuntime
   direction TB
   iso_fortran_env_impl("iso_fortran_env_impl.f90")
   lib("libFortranRuntime.a")
   module_files
   check-FortranRuntime
 end

 flang-new --> iso_fortran_env_impl --> lib --> check-FortranRuntime
 flang-new --> module_files --> check-flang
 flang-new --> check-flang
 flang-new --> check-FortranRuntime
Loading

Except the PPC modules, the mod files are not target-specific and hence would not profit from compiling multiple targets from a bootstrap build anyway. However, they are products of the Flang compiler and therefore IMHO logically fits better to the runtime. Also, it would have been nice to make use of CMake's dependency resolution rather than hardcoding Into the CMakeLists.txt. I think it is best to keep that decision out of the current patch.

Meinersbur added a commit that referenced this pull request Oct 7, 2024
Due to missing lit.cfg, will only be executing again after #110217.
@Meinersbur Meinersbur marked this pull request as ready for review October 7, 2024 20:57
@Meinersbur Meinersbur requested a review from a team as a code owner October 7, 2024 20:57
@Meinersbur Meinersbur changed the title [Flang][DRAFT] LLVM_ENABLE_RUNTIMES=FortranRuntime [Flang] LLVM_ENABLE_RUNTIMES=FortranRuntime Oct 7, 2024
FortranRuntime/CMakeLists.txt Outdated Show resolved Hide resolved
llvm/runtimes/CMakeLists.txt Outdated Show resolved Hide resolved
runtimes/CMakeLists.txt Show resolved Hide resolved
llvm/runtimes/CMakeLists.txt Outdated Show resolved Hide resolved
Copy link
Member

@ldionne ldionne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the global setting of flags is the last comment I have from the perspective of "integration with the runtimes build". I haven't reviewed anything else inside flang-rt/ though.

flang-rt/CMakeLists.txt Outdated Show resolved Hide resolved
@jplehr
Copy link
Contributor

jplehr commented Oct 31, 2024

I tested this locally and it appears that it requires are more modern CMake version than what was installed (3.22).
According to the LLVM docs (https://releases.llvm.org/12.0.0/docs/GettingStarted.html#id8) currently CMake 3.20 is the minimum required version.

@Meinersbur Meinersbur requested a review from jhuber6 November 6, 2024 14:36
flang-rt/CMakeLists.txt Outdated Show resolved Hide resolved
flang-rt/CMakeLists.txt Outdated Show resolved Hide resolved
llvm/runtimes/CMakeLists.txt Show resolved Hide resolved
llvm/cmake/modules/LLVMExternalProjectUtils.cmake Outdated Show resolved Hide resolved
@jhuber6 jhuber6 changed the title [Flang] LLVM_ENABLE_RUNTIMES=FortranRuntime [Flang] LLVM_ENABLE_RUNTIMES=flang-rt Nov 6, 2024
@Meinersbur Meinersbur requested a review from sscalpone November 7, 2024 15:48
@jhuber6
Copy link
Contributor

jhuber6 commented Nov 7, 2024

Overall I think the patch is fine pending some naming nits, my one concern is the -U__GLIBCXX stuff, because undefining internal vars seems really sketchy. Do we use -nostdlib++ to make sure we don't link the C++ library?

@jeanPerier
Copy link
Contributor

jeanPerier commented Nov 8, 2024

When building the runtime out of tree and running check-flang-rt, I am seeing an error about LLVM header not found

edit: My -DLLVM_DIR was incorrect, never mind.

@Meinersbur
Copy link
Member Author

I think with >1600 commits and >300kLoC changes, something went wrong here with the merging. As mentioned by myself and others, it would be good to rebase this and condense the commits that belong into #110298 resp. this one.

This happens when I push the branch, that has origin/HEAD merged into, before doing the same with the PR that it is based on. In this case I pushed the wrong branch, one of the PRs that has already been merged. Sorry about that. I now wrote a script that pushes in the right order so I hope this doesn't happen anymore.

It seems that I will not be able to apply llvm/llvm-zorg#333 in a timely manner minimize the time the buildbots are red. I am currently working on keeping the old CMake code that builds the runtime such that the buildbot builders can be updated iteratively. It has the side-effect that I need to keep the flang/runtime/CMakeLists.txt working which I originally wanted to avoid, but also allows splitting this patch into smaller PRs.

@Meinersbur
Copy link
Member Author

Meinersbur commented Jan 8, 2025

It is an old problem, see #87866 (comment)

Can we raise an issue for this?

Created #122152

I don't expect anything come out of it, I think moving to LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON by default is deliberate.

@Meinersbur
Copy link
Member Author

Meinersbur commented Jan 9, 2025

The library is present under

$PREFIX/flang-rt/lib/x86_64-unknown-linux-gnu/libflang_rt.a

(where it got installed through the changes in this PR, without any specific overrides).

If $PREFIX is CMAKE_INSTALL_PREFIX, it is the wrong location. it should be in $PREFIX/lib/clang/20/lib/x86_64-unknown-linux-gnu/libflang_rt.a. I don't see how the flang_rt dir could get in there. Can you try with the latest update?

If $PREFIX is CMake's build directory, are you using a runtimes-standalone (non-bootstrap) build and running flang from there? Then $PREFIX for flang and flang-rt are different. flang looks into its own buildir only. You will have to -L to that directory or install them into the same CMAKE_INSTALL_PREFIX.

@Meinersbur Meinersbur marked this pull request as draft January 9, 2025 02:07
@Meinersbur Meinersbur changed the base branch from users/meinersbur/flang_runtime_move-files to users/meinersbur/flang_runtime_FortranSupport January 9, 2025 02:07
@Meinersbur Meinersbur changed the base branch from users/meinersbur/flang_runtime_FortranSupport to users/meinersbur/flang_runtime_move-files January 9, 2025 02:08
@Meinersbur Meinersbur marked this pull request as ready for review January 9, 2025 02:08
@Meinersbur Meinersbur force-pushed the users/meinersbur/flang_runtime branch from e92a529 to 301af7f Compare January 9, 2025 03:22
Copy link
Contributor

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

saw some minor merge damage

flang/lib/Optimizer/Transforms/CMakeLists.txt Outdated Show resolved Hide resolved
flang/lib/Parser/CMakeLists.txt Outdated Show resolved Hide resolved
flang/lib/Semantics/CMakeLists.txt Outdated Show resolved Hide resolved
@h-vetinari
Copy link
Contributor

If $PREFIX is CMAKE_INSTALL_DIR, it is the wrong location. it should be in $PREFIX/lib/clang/20/lib/x86_64-unknown-linux-gnu/libflang_rt.a. I don't see how the flang_rt dir could get in there. Can you try with the latest update?

Both flang and flang-rt get installed into the same -DCMAKE_INSTALL_PREFIX=$PREFIX. I did try with the latest state of this PR, and it runs into the same error.

I then tried shifting -DLLVM_ENABLE_RUNTIMES="flang-rt;compiler-rt" to the flang-rt build, but this seems really pointless and wasteful, because at that point we already have a compiled & working compiler-rt (and failed because it ends up building libcxx, which needs a newer GCC than our current default). I don't get all the involved configuration here, but as I said before:

Not sure at what point the ability of the driver to look into ${prefix}/lib/clang/20/lib/windows gets set, but it sounds like we'd want to force that to always be true.

I don't mind patching clang/flang/whatever for that to be the case, but I'd really like to avoid rebuilding compiler-rt just to nudge the build in that direction.

@Meinersbur
Copy link
Member Author

We are mixing up the Windows issue of finding the right clang_rt.builtins (which is an issue unrelated to this PR) and the UNIX install path of libflang_rt.a. Let's separate these issues.

Windows name of clang_rt.builtins

I added LLVM_ENABLE_RUNTIMES=compiler-rt to building flang, but that didn't change anything. Not sure at what point the ability of the driver to look into ${prefix}/lib/clang/20/lib/windows gets set, but it sounds like we'd want to force that to always be true.

It also needs to be built, i.e.

$ ninja builtins
$ find -name "clang_rt.builtins*.lib"
./lib/clang/20/lib/windows/clang_rt.builtins-x86_64.lib   

Can you confirm that it is there?

I then tried shifting -DLLVM_ENABLE_RUNTIMES="flang-rt;compiler-rt" to the flang-rt build, but this seems really pointless and wasteful, because at that point we already have a compiled & working compiler-rt (and failed because it ends up building libcxx, which needs a newer GCC than our current default). I don't get all the involved configuration here, but as I said before:

I don't get your setup. Why is it compiling libcxx if it is not in LLVM_ENABLE_RUNTIMES? How are you using GCC on Windows? According to https://llvm.org/docs/GettingStarted.html, LLVM does not support being built on Windows using GCC.

Not sure at what point the ability of the driver to look into ${prefix}/lib/clang/20/lib/windows gets set, but it sounds like we'd want to force that to always be true.

I don't mind patching clang/flang/whatever for that to be the case, but I'd really like to avoid rebuilding compiler-rt just to nudge the build in that direction.

This should do it (untested):

diff --git a/clang/lib/Driver/ToolChain.cpp b/clang/lib/Driver/ToolChain.cpp
index 9f174fbda398..b77cbb72e7ec 100644
--- a/clang/lib/Driver/ToolChain.cpp
+++ b/clang/lib/Driver/ToolChain.cpp
@@ -773,7 +773,7 @@ std::string ToolChain::getCompilerRT(const ArgList &Args, StringRef Component,
       buildCompilerRTBasename(Args, Component, Type, /*AddArch=*/true);
   SmallString<128> OldPath(getCompilerRTPath());
   llvm::sys::path::append(OldPath, CRTBasename);
-  if (Path.empty() || getVFS().exists(OldPath))
+  if (Path.empty() || getVFS().exists(OldPath) || getTriple().isWindowsMSVCEnvironment())
     return std::string(OldPath);

   // If none is found, use a file name from the new layout, which may get        

Non-Windows install location of libflang_rt.a

If $PREFIX is CMAKE_INSTALL_DIR, it is the wrong location. it should be in $PREFIX/lib/clang/20/lib/x86_64-unknown-linux-gnu/libflang_rt.a. I don't see how the flang_rt dir could get in there. Can you try with the latest update?

Both flang and flang-rt get installed into the same -DCMAKE_INSTALL_PREFIX=$PREFIX. I did try with the latest state of this PR, and it runs into the same error.

I have no idea what could be going on. Would you mind adding some printf so I can see the result?

diff --git a/flang-rt/CMakeLists.txt b/flang-rt/CMakeLists.txt
index 51dbe6b6cc09..50dac3037ba1 100644
--- a/flang-rt/CMakeLists.txt
+++ b/flang-rt/CMakeLists.txt
@@ -114,12 +114,20 @@ else ()
   extend_path(FLANG_RT_OUTPUT_RESOURCE_LIB_DIR "${FLANG_RT_OUTPUT_RESOURCE_DIR}" "lib${LLVM_LIBDIR_SUFFIX}")
 endif ()
 extend_path(FLANG_RT_INSTALL_RESOURCE_LIB_PATH "${FLANG_RT_INSTALL_RESOURCE_PATH}" "${toolchain_lib_subdir}")
+message("LLVM_TREE_AVAILABLE: ${LLVM_TREE_AVAILABLE}")
+message("toolchain_lib_subdir: ${toolchain_lib_subdir}")
+message("FLANG_RT_OUTPUT_RESOURCE_DIR: ${FLANG_RT_OUTPUT_RESOURCE_DIR}")
+message("FLANG_RT_INSTALL_RESOURCE_PATH: ${FLANG_RT_INSTALL_RESOURCE_PATH}")
+message("FLANG_RT_OUTPUT_RESOURCE_LIB_DIR: ${FLANG_RT_OUTPUT_RESOURCE_LIB_DIR}")
+message("FLANG_RT_INSTALL_RESOURCE_LIB_PATH: ${FLANG_RT_INSTALL_RESOURCE_LIB_PATH}")
 cmake_path(NORMAL_PATH FLANG_RT_OUTPUT_RESOURCE_DIR)
 cmake_path(NORMAL_PATH FLANG_RT_INSTALL_RESOURCE_PATH)
 cmake_path(NORMAL_PATH FLANG_RT_OUTPUT_RESOURCE_LIB_DIR)
 cmake_path(NORMAL_PATH FLANG_RT_INSTALL_RESOURCE_LIB_PATH)

# for all files of Flang-RT.
if (EXISTS "${FLANG_RT_LIBCUDACXX_PATH}/include")
target_include_directories(obj.${name}PTX AFTER PRIVATE "${FLANG_RT_LIBCUDACXX_PATH}/include")
target_compile_definitions(obj.${name}PTX PRIVATE RT_USE_LIBCUDACXX=1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also need:

      target_include_directories(${name} AFTER PRIVATE "${FLANG_RT_LIBCUDACXX_PATH}/include")
      target_compile_definitions(${name} PRIVATE RT_USE_LIBCUDACXX=1)

Because the flang_rt.a library itself is also built with CUDA.

Sorry for not catching it earlier, I missed the build warnings from this and only caught this while doing some end to end.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed that changed. However, instead of tons of warnings I now get errors:

/home/meinersbur/src/llvm-project-flangrt/flang-rt/../flang/include/flang/Common/variant.h:18:10: fatal error: cuda/std/variant: No such file or directory
   18 | #include <cuda/std/variant>
      |          ^~~~~~~~~~~~~~~~~~

CUDA's variant is located in /usr/include/cuda/std/detail/libcxx/include/variant (From libcu++-dev Ubuntu 24.04.1). There is no package where it is located in cuda/std/variant.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And you were not getting those when building the PTX lib?
What path did you provide as FLANG_RT_LIBCUDACXX_PATH?
Can you check if you have /usr/include/cuda/std/variant?

In which case I think you should maybe just provide /usr as FLANG_RT_LIBCUDACXX_PATH.

@vzakhari, FYI.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, cuda/std/variant was added after libcu++ project was converged/replaced with cccl (https://github.com/nvidia/cccl). Here is how the buildbot is configured https://lab.llvm.org/buildbot/#/builders/152/builds/889/steps/4/logs/stdio: b'-DFLANG_LIBCUDACXX_PATH=/home/buildbot/worker/third-party/nv/cccl/libcudacxx'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not set FLANG_RT_LIBCUDACXX_PATH at all. I assumed it had found it in a default search path since it compiled without error.

If this requires custom environments I suggest for Nvidia to check and fix it in their own environments. Some documentation would be appreciated as well. I put in the work to keep the old CMake code functional so nothing breaks until LLVM_ENABLE_RUNTIMES=flang-rt is added.

#===------------------------------------------------------------------------===#


add_flangrt_library(CufRuntime STATIC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the change to STATIC only here?

We also currently build and test the share version in our current CUDA Fortran feature development (@clementval).

Can you add SHARED in #121782?

Copy link
Member Author

@Meinersbur Meinersbur Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this:
https://github.com/llvm/llvm-project/blob/main/flang/cmake/modules/AddFlangOffloadRuntime.cmake#L19-L23

Do you build FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT=CUDA and FLANG_CUF_RUNTIME=ON independently? Not yet one of the configurations I have tested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you build FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT=CUDA and FLANG_CUF_RUNTIME=ON independently?

Yes, the Cuda Fortra runtime does not need (and I think should not?) be built with FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT=CUDA option. @clementval is the expert that can confirm here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CUDA Fortran runtime is (at least for now) host only entry points. So it is not called from the device so @jeanPerier is right. This is independent of FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT and should be buildable for static or shared.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added support for shared CufRuntime to #121782 in 6c03632

@h-vetinari
Copy link
Contributor

Thanks for the answer @Meinersbur! I'm planning to respond as soon as our builds produce more results, but in the meantime, the most recent changes (as of 4b65d4b; plus the clang patch you suggested) produce the following on windows:

[347/354] Generating ../../include/flang/iso_fortran_env.mod
FAILED: include/flang/iso_fortran_env.mod C:/bld/flang-split_1736519732226/work/build/include/flang/iso_fortran_env.mod 
C:\Windows\system32\cmd.exe /C "cd /D %SRC_DIR%\build\tools\f18 && %BUILD_PREFIX%\Library\bin\cmake.exe -E make_directory C:/bld/flang-split_1736519732226/work/build/include/flang && %SRC_DIR%\build\bin\flang.exe -cpp -fsyntax-only -module-dir C:/bld/flang-split_1736519732226/work/build/include/flang C:/bld/flang-split_1736519732226/work/flang/module/iso_fortran_env.f90"
error: Semantic errors in C:/bld/flang-split_1736519732226/work/flang/module/iso_fortran_env.f90
C:/bld/flang-split_1736519732226/work/flang/module/iso_fortran_env.f90:25:7: error: Cannot read module file for module 'iso_fortran_env_impl': Source file 'iso_fortran_env_impl.mod' was not found
    use iso_fortran_env_impl, only: &
        ^^^^^^^^^^^^^^^^^^^^

@h-vetinari
Copy link
Contributor

Still erroring the same way (both on linux & windows, which are the only platforms I'm testing right now) with

[374/376] Generating ../../include/flang/iso_fortran_env.mod
FAILED: include/flang/iso_fortran_env.mod $SRC_DIR/build/include/flang/iso_fortran_env.mod 
cd $SRC_DIR/build/tools/f18 && $BUILD_PREFIX/bin/cmake -E make_directory $SRC_DIR/build/include/flang && $SRC_DIR/build/bin/flang -cpp -fsyntax-only -module-dir $SRC_DIR/build/include/flang $SRC_DIR/flang/module/iso_fortran_env.f90
error: Semantic errors in $SRC_DIR/flang/module/iso_fortran_env.f90
$SRC_DIR/flang/module/iso_fortran_env.f90:25:7: error: Cannot read module file for module 'iso_fortran_env_impl': Source file 'iso_fortran_env_impl.mod' was not found
    use iso_fortran_env_impl, only: &
        ^^^^^^^^^^^^^^^^^^^^

as of 21229e0. This is during the flang-portion of the build, before we even get to flang-rt.

@Meinersbur
Copy link
Member Author

Thanks for the answer @Meinersbur! I'm planning to respond as soon as our builds produce more results, but in the meantime, the most recent changes (as of 4b65d4b; plus the clang patch you suggested) produce the following on windows:

[347/354] Generating ../../include/flang/iso_fortran_env.mod
FAILED: include/flang/iso_fortran_env.mod C:/bld/flang-split_1736519732226/work/build/include/flang/iso_fortran_env.mod 
C:\Windows\system32\cmd.exe /C "cd /D %SRC_DIR%\build\tools\f18 && %BUILD_PREFIX%\Library\bin\cmake.exe -E make_directory C:/bld/flang-split_1736519732226/work/build/include/flang && %SRC_DIR%\build\bin\flang.exe -cpp -fsyntax-only -module-dir C:/bld/flang-split_1736519732226/work/build/include/flang C:/bld/flang-split_1736519732226/work/flang/module/iso_fortran_env.f90"
error: Semantic errors in C:/bld/flang-split_1736519732226/work/flang/module/iso_fortran_env.f90
C:/bld/flang-split_1736519732226/work/flang/module/iso_fortran_env.f90:25:7: error: Cannot read module file for module 'iso_fortran_env_impl': Source file 'iso_fortran_env_impl.mod' was not found
    use iso_fortran_env_impl, only: &
        ^^^^^^^^^^^^^^^^^^^^

Thanks for testing. I had fixed it in #122336 with 57e9dd1. I wrongfully assumed that iso_fortran_env_impl wasn't needed if not compiling the runtime.

@Meinersbur Meinersbur force-pushed the users/meinersbur/flang_runtime_move-files branch from 30b6800 to 7e369b5 Compare January 24, 2025 18:43
@Meinersbur Meinersbur force-pushed the users/meinersbur/flang_runtime branch from 93d7651 to f670816 Compare January 24, 2025 18:43
@llvmbot llvmbot added cmake Build system in general and CMake in particular clang Clang issues not falling into any other category clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' flang Flang issues not falling into any other category labels Jan 24, 2025
@llvmbot
Copy link
Member

llvmbot commented Jan 24, 2025

@llvm/pr-subscribers-clang

Author: Michael Kruse (Meinersbur)

Changes

Extract Flang's runtime library to use the LLVM_ENABLE_RUNTIME mechanism.

Motivation:

  • Consistency with LLVM's other runtime libraries (compiler-rt, libc, libcxx, openmp offload, ...)
  • Allows compiling the runtime for multiple targets at once using the LLVM_RUNTIME_TARGETS configuration options
  • Installs the runtime into the compiler's per-target resource directory so it can be automatically found even when cross-compiling

Potential future directions:

  • Uses CMake's support for compiling Fortran files, including dependency resolution of Fortran modules
  • Improve robustness of compiling libomp.mod when openmp is available
  • Remove Flang's dependency from flang-rt's RTNAME function declarations (tblgen?)
  • Reduce Flang's build-time dependency from flang-rt's REAL(16) support

See RFC discussion at https://discourse.llvm.org/t/rfc-use-llvm-enable-runtimes-for-flangs-runtime/80826

Patch series:

  • #110244
  • #112188
  • #121997
  • #122069
  • #122334
  • #122336
  • #122341
  • #110298
  • #110217 (this PR)
  • #121782
  • #124126

Patch for lab.llvm.org buildbots:


Patch is 108.72 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/110217.diff

41 Files Affected:

  • (modified) clang/lib/Driver/ToolChains/Flang.cpp (+9-5)
  • (added) flang-rt/.clang-tidy (+2)
  • (added) flang-rt/CMakeLists.txt (+248)
  • (added) flang-rt/CODE_OWNERS.TXT (+14)
  • (added) flang-rt/LICENSE.TXT (+234)
  • (added) flang-rt/README.md (+188)
  • (added) flang-rt/cmake/modules/AddFlangRT.cmake (+186)
  • (added) flang-rt/cmake/modules/AddFlangRTOffload.cmake (+101)
  • (added) flang-rt/cmake/modules/GetToolchainDirs.cmake (+125)
  • (added) flang-rt/lib/CMakeLists.txt (+18)
  • (added) flang-rt/lib/FortranFloat128Math/CMakeLists.txt (+136)
  • (added) flang-rt/lib/Testing/CMakeLists.txt (+20)
  • (added) flang-rt/lib/flang_rt/CMakeLists.txt (+213)
  • (added) flang-rt/lib/flang_rt/CUDA/CMakeLists.txt (+33)
  • (modified) flang-rt/lib/flang_rt/io-api-minimal.cpp (+1-1)
  • (added) flang-rt/test/CMakeLists.txt (+59)
  • (modified) flang-rt/test/Driver/ctofortran.f90 (+5-24)
  • (modified) flang-rt/test/Driver/exec.f90 (+4-4)
  • (added) flang-rt/test/NonGtestUnit/lit.cfg.py (+22)
  • (added) flang-rt/test/NonGtestUnit/lit.site.cfg.py.in (+14)
  • (modified) flang-rt/test/Runtime/no-cpp-dep.c (+3-2)
  • (added) flang-rt/test/Unit/lit.cfg.py (+21)
  • (added) flang-rt/test/Unit/lit.site.cfg.py.in (+15)
  • (added) flang-rt/test/lit.cfg.py (+102)
  • (added) flang-rt/test/lit.site.cfg.py.in (+19)
  • (added) flang-rt/unittests/CMakeLists.txt (+111)
  • (added) flang-rt/unittests/Evaluate/CMakeLists.txt (+21)
  • (added) flang-rt/unittests/Runtime/CMakeLists.txt (+48)
  • (added) flang-rt/unittests/Runtime/CUDA/CMakeLists.txt (+18)
  • (modified) flang/CMakeLists.txt (+26-27)
  • (added) flang/cmake/modules/FlangCommon.cmake (+43)
  • (modified) flang/docs/GettingStarted.md (+58-50)
  • (modified) flang/docs/ReleaseNotes.md (+7-1)
  • (modified) flang/module/iso_fortran_env_impl.f90 (+1-1)
  • (modified) flang/test/lit.cfg.py (-20)
  • (modified) flang/test/lit.site.cfg.py.in (-3)
  • (modified) llvm/CMakeLists.txt (+7-1)
  • (modified) llvm/cmake/modules/LLVMExternalProjectUtils.cmake (+15-1)
  • (modified) llvm/projects/CMakeLists.txt (+3-1)
  • (modified) llvm/runtimes/CMakeLists.txt (+18-7)
  • (modified) runtimes/CMakeLists.txt (+1-1)
diff --git a/clang/lib/Driver/ToolChains/Flang.cpp b/clang/lib/Driver/ToolChains/Flang.cpp
index 68a17edf8ca341..17a8a4dd8d0a87 100644
--- a/clang/lib/Driver/ToolChains/Flang.cpp
+++ b/clang/lib/Driver/ToolChains/Flang.cpp
@@ -342,11 +342,15 @@ static void processVSRuntimeLibrary(const ToolChain &TC, const ArgList &Args,
                                     ArgStringList &CmdArgs) {
   assert(TC.getTriple().isKnownWindowsMSVCEnvironment() &&
          "can only add VS runtime library on Windows!");
-  // if -fno-fortran-main has been passed, skip linking Fortran_main.a
-  if (TC.getTriple().isKnownWindowsMSVCEnvironment()) {
-    CmdArgs.push_back(Args.MakeArgString(
-        "--dependent-lib=" + TC.getCompilerRTBasename(Args, "builtins")));
-  }
+
+  // Flang/Clang (including clang-cl) -compiled programs targeting the MSVC ABI
+  // should only depend on msv(u)crt. LLVM still emits libgcc/compiler-rt
+  // functions in some cases like 128-bit integer math (__udivti3, __modti3,
+  // __fixsfti, __floattidf, ...) that msvc does not support. We are injecting a
+  // dependency to Compiler-RT's builtin library where these are implemented.
+  CmdArgs.push_back(Args.MakeArgString(
+      "--dependent-lib=" + TC.getCompilerRTBasename(Args, "builtins")));
+
   unsigned RTOptionID = options::OPT__SLASH_MT;
   if (auto *rtl = Args.getLastArg(options::OPT_fms_runtime_lib_EQ)) {
     RTOptionID = llvm::StringSwitch<unsigned>(rtl->getValue())
diff --git a/flang-rt/.clang-tidy b/flang-rt/.clang-tidy
new file mode 100644
index 00000000000000..ee3a0ab2201bf3
--- /dev/null
+++ b/flang-rt/.clang-tidy
@@ -0,0 +1,2 @@
+Checks: '-llvm-include-order,readability-braces-around-statements,-readability-identifier-naming,-clang-diagnostic-*'
+InheritParentConfig: true
diff --git a/flang-rt/CMakeLists.txt b/flang-rt/CMakeLists.txt
new file mode 100644
index 00000000000000..655d0a55b40044
--- /dev/null
+++ b/flang-rt/CMakeLists.txt
@@ -0,0 +1,248 @@
+#===-- CMakeLists.txt ------------------------------------------------------===#
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+#===------------------------------------------------------------------------===#
+#
+# Build instructions for the flang-rt library. This is file is intended to be
+# included using the LLVM_ENABLE_RUNTIMES mechanism.
+#
+#===------------------------------------------------------------------------===#
+
+if (NOT LLVM_RUNTIMES_BUILD)
+  message(FATAL_ERROR "Use this CMakeLists.txt from LLVM's runtimes build system.
+      Example:
+        cmake <llvm-project>/runtimes -DLLVM_ENABLE_RUNTIMES=flang-rt
+    ")
+endif ()
+
+set(LLVM_SUBPROJECT_TITLE "Flang-RT")
+set(FLANG_RT_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}")
+set(FLANG_RT_BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}")
+set(FLANG_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../flang")
+
+# CMake 3.24 is the first version of CMake that directly recognizes Flang.
+# LLVM's requirement is only CMake 3.20, teach CMake 3.20-3.23 how to use Flang.
+if (CMAKE_VERSION VERSION_LESS "3.24")
+  cmake_path(GET CMAKE_Fortran_COMPILER STEM _Fortran_COMPILER_STEM)
+  if (_Fortran_COMPILER_STEM STREQUAL "flang-new" OR _Fortran_COMPILER_STEM STREQUAL "flang")
+    include(CMakeForceCompiler)
+    CMAKE_FORCE_Fortran_COMPILER("${CMAKE_Fortran_COMPILER}" "LLVMFlang")
+
+    set(CMAKE_Fortran_COMPILER_ID "LLVMFlang")
+    set(CMAKE_Fortran_COMPILER_VERSION "${LLVM_VERSION_MAJOR}.${LLVM_VERSION_MINOR}")
+
+    set(CMAKE_Fortran_SUBMODULE_SEP "-")
+    set(CMAKE_Fortran_SUBMODULE_EXT ".mod")
+
+    set(CMAKE_Fortran_PREPROCESS_SOURCE
+      "<CMAKE_Fortran_COMPILER> -cpp <DEFINES> <INCLUDES> <FLAGS> -E <SOURCE> > <PREPROCESSED_SOURCE>")
+
+    set(CMAKE_Fortran_FORMAT_FIXED_FLAG "-ffixed-form")
+    set(CMAKE_Fortran_FORMAT_FREE_FLAG "-ffree-form")
+
+    set(CMAKE_Fortran_MODDIR_FLAG "-module-dir")
+
+    set(CMAKE_Fortran_COMPILE_OPTIONS_PREPROCESS_ON "-cpp")
+    set(CMAKE_Fortran_COMPILE_OPTIONS_PREPROCESS_OFF "-nocpp")
+    set(CMAKE_Fortran_POSTPROCESS_FLAG "-ffixed-line-length-72")
+
+    set(CMAKE_Fortran_COMPILE_OPTIONS_TARGET "--target=")
+
+    set(CMAKE_Fortran_LINKER_WRAPPER_FLAG "-Wl,")
+    set(CMAKE_Fortran_LINKER_WRAPPER_FLAG_SEP ",")
+  endif ()
+endif ()
+enable_language(Fortran)
+
+
+list(APPEND CMAKE_MODULE_PATH
+    "${FLANG_RT_SOURCE_DIR}/cmake/modules"
+    "${FLANG_SOURCE_DIR}/cmake/modules"
+  )
+include(AddFlangRT)
+include(GetToolchainDirs)
+include(FlangCommon)
+include(HandleCompilerRT)
+include(ExtendPath)
+
+
+############################
+# Build Mode Introspection #
+############################
+
+# Determine whether we are in the runtimes/runtimes-bins directory of a
+# bootstrap build.
+set(LLVM_TREE_AVAILABLE OFF)
+if (LLVM_LIBRARY_OUTPUT_INTDIR AND LLVM_RUNTIME_OUTPUT_INTDIR AND PACKAGE_VERSION)
+  set(LLVM_TREE_AVAILABLE ON)
+endif()
+
+# Path to LLVM development tools (FileCheck, llvm-lit, not, ...)
+set(LLVM_TOOLS_DIR "${LLVM_BINARY_DIR}/bin")
+
+# Determine build and install paths.
+# The build path is absolute, but the install dir is relative, CMake's install
+# command has to apply CMAKE_INSTALL_PREFIX itself.
+get_toolchain_library_subdir(toolchain_lib_subdir)
+if (LLVM_TREE_AVAILABLE)
+  # In a bootstrap build emit the libraries into a default search path in the
+  # build directory of the just-built compiler. This allows using the
+  # just-built compiler without specifying paths to runtime libraries.
+  #
+  # Despite Clang in the name, get_clang_resource_dir does not depend on Clang
+  # being added to the build. Flang uses the same resource dir as clang.
+  include(GetClangResourceDir)
+  get_clang_resource_dir(FLANG_RT_OUTPUT_RESOURCE_DIR PREFIX "${LLVM_LIBRARY_OUTPUT_INTDIR}/..")
+  get_clang_resource_dir(FLANG_RT_INSTALL_RESOURCE_PATH)
+
+  extend_path(FLANG_RT_OUTPUT_RESOURCE_LIB_DIR "${FLANG_RT_OUTPUT_RESOURCE_DIR}" "${toolchain_lib_subdir}")
+else ()
+  # In a standalone runtimes build, do not write into LLVM_BINARY_DIR. It may be
+  # read-only and/or shared by multiple runtimes with different build
+  # configurations (e.g. Debug/Release). Use the runtime's own lib dir like any
+  # non-toolchain library.
+  # For the install prefix, still use the resource dir assuming that Flang will
+  # be installed there using the same prefix. This is to not have a difference
+  # between bootstrap and standalone runtimes builds.
+  set(FLANG_RT_OUTPUT_RESOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}")
+  set(FLANG_RT_INSTALL_RESOURCE_PATH "lib${LLVM_LIBDIR_SUFFIX}/clang/${LLVM_VERSION_MAJOR}")
+
+  extend_path(FLANG_RT_OUTPUT_RESOURCE_LIB_DIR "${FLANG_RT_OUTPUT_RESOURCE_DIR}" "lib${LLVM_LIBDIR_SUFFIX}")
+endif ()
+extend_path(FLANG_RT_INSTALL_RESOURCE_LIB_PATH "${FLANG_RT_INSTALL_RESOURCE_PATH}" "${toolchain_lib_subdir}")
+cmake_path(NORMAL_PATH FLANG_RT_OUTPUT_RESOURCE_DIR)
+cmake_path(NORMAL_PATH FLANG_RT_INSTALL_RESOURCE_PATH)
+cmake_path(NORMAL_PATH FLANG_RT_OUTPUT_RESOURCE_LIB_DIR)
+cmake_path(NORMAL_PATH FLANG_RT_INSTALL_RESOURCE_LIB_PATH)
+
+
+#################
+# Build Options #
+#################
+
+# Important: flang-rt user options must be prefixed with "FLANG_RT_". Variables
+# with this prefix will be forwarded in bootstrap builds.
+
+option(FLANG_RT_INCLUDE_TESTS "Generate build targets for the flang-rt unit and regression-tests." "${LLVM_INCLUDE_TESTS}")
+
+
+set(FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT "" CACHE STRING "Compile Flang-RT with GPU support (CUDA or OpenMP)")
+set_property(CACHE FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT PROPERTY STRINGS
+    ""
+    CUDA
+    OpenMP
+  )
+if (NOT FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT)
+  # Support for GPUs disabled
+elseif (FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT STREQUAL "CUDA")
+  # Support for CUDA
+  set(FLANG_RT_LIBCUDACXX_PATH "" CACHE PATH "Path to libcu++ package installation")
+  option(FLANG_RT_CUDA_RUNTIME_PTX_WITHOUT_GLOBAL_VARS "Do not compile global variables' definitions when producing PTX library" OFF)
+elseif (FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT STREQUAL "OpenMP")
+  # Support for OpenMP offloading
+  set(FLANG_RT_DEVICE_ARCHITECTURES "all" CACHE STRING
+      "List of OpenMP device architectures to be used to compile the Fortran runtime (e.g. 'gfx1103;sm_90')"
+    )
+
+  if (FLANG_RT_DEVICE_ARCHITECTURES STREQUAL "all")
+    # TODO: support auto detection on the build system.
+    set(all_amdgpu_architectures
+      "gfx700;gfx701;gfx801;gfx803;gfx900;gfx902;gfx906"
+      "gfx908;gfx90a;gfx90c;gfx940;gfx1010;gfx1030"
+      "gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036"
+      "gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151"
+      "gfx1152;gfx1153")
+    set(all_nvptx_architectures
+      "sm_35;sm_37;sm_50;sm_52;sm_53;sm_60;sm_61;sm_62"
+      "sm_70;sm_72;sm_75;sm_80;sm_86;sm_89;sm_90")
+    set(all_gpu_architectures
+      "${all_amdgpu_architectures};${all_nvptx_architectures}")
+      set(FLANG_RT_DEVICE_ARCHITECTURES ${all_gpu_architectures})
+  endif()
+  list(REMOVE_DUPLICATES FLANG_RT_DEVICE_ARCHITECTURES)
+else ()
+  message(FATAL_ERROR "Invalid value '${FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT}' for FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT; must be empty, 'CUDA', or 'OpenMP'")
+endif ()
+
+
+option(FLANG_RT_ENABLE_CUF "Compile CUDA Fortran runtime sources" OFF)
+if (FLANG_RT_ENABLE_CUF)
+  find_package(CUDAToolkit REQUIRED)
+endif()
+
+
+########################
+# System Introspection #
+########################
+
+include(CheckCXXSymbolExists)
+include(CheckCXXSourceCompiles)
+check_cxx_symbol_exists(strerror_r string.h HAVE_STRERROR_R)
+# Can't use symbol exists here as the function is overloaded in C++
+check_cxx_source_compiles(
+  "#include <string.h>
+   int main() {
+     char buf[4096];
+     return strerror_s(buf, 4096, 0);
+   }
+  "
+  HAVE_DECL_STRERROR_S)
+
+
+# Search for clang_rt.builtins library. Need in addition to msvcrt.
+if (WIN32)
+  find_compiler_rt_library(builtins FLANG_RT_BUILTINS_LIBRARY)
+endif ()
+
+
+# Check whether the compiler can undefine a macro using the "-U" flag.
+# Aternatively, we could use
+#   CMAKE_CXX_COMPILER_FRONTEND_VARIANT STREQUAL "GNU"
+# but some older versions of CMake don't define it for GCC itself.
+check_cxx_compiler_flag("-UTESTFLAG" FLANG_RT_SUPPORTS_UNDEFINE_FLAG)
+
+
+#####################
+# Build Preparation #
+#####################
+
+if (FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT AND FLANG_RT_INCLUDE_TESTS)
+  # If Fortran runtime is built as CUDA library, the linking
+  # of targets that link flang-rt must be done
+  # with CUDA_RESOLVE_DEVICE_SYMBOLS.
+  # CUDA language must be enabled for CUDA_RESOLVE_DEVICE_SYMBOLS
+  # to take effect.
+  enable_language(CUDA)
+endif()
+
+
+# C++17 is required for flang-rt; user or other runtimes may override this.
+# GTest included later also requires C++17.
+set(CMAKE_CXX_STANDARD 17 CACHE STRING "C++ standard to conform to")
+set(CMAKE_CXX_STANDARD_REQUIRED YES)
+
+
+configure_file(cmake/config.h.cmake.in config.h)
+
+
+# The bootstrap build will create a phony target with the same as the top-level
+# directory ("flang-rt") and delegate it to the runtimes build dir.
+# AddFlangRT will add all non-EXCLUDE_FROM_ALL targets to it.
+add_custom_target(flang-rt)
+
+
+###################
+# Build Artifacts #
+###################
+
+add_subdirectory(lib)
+
+if (FLANG_RT_INCLUDE_TESTS)
+  add_subdirectory(unittests)
+  add_subdirectory(test)
+else ()
+  add_custom_target(check-flang-rt)
+endif()
diff --git a/flang-rt/CODE_OWNERS.TXT b/flang-rt/CODE_OWNERS.TXT
new file mode 100644
index 00000000000000..649243aa1e8fec
--- /dev/null
+++ b/flang-rt/CODE_OWNERS.TXT
@@ -0,0 +1,14 @@
+This file is a list of the people responsible for ensuring that patches for a
+particular part of Flang are reviewed, either by themself or by someone else.
+They are also the gatekeepers for their part of Flang, with the final word on
+what goes in or not.
+
+The list is sorted by surname and formatted to allow easy grepping and
+beautification by scripts. The fields are: name (N), email (E), web-address
+(W), PGP key ID and fingerprint (P), description (D), snail-mail address
+(S) and (I) IRC handle. Each entry should contain at least the (N), (E) and
+(D) fields.
+
+N: Steve Scalpone
+E: [email protected]
+D: Anything not covered by others
diff --git a/flang-rt/LICENSE.TXT b/flang-rt/LICENSE.TXT
new file mode 100644
index 00000000000000..53bb2e7fbc7643
--- /dev/null
+++ b/flang-rt/LICENSE.TXT
@@ -0,0 +1,234 @@
+==============================================================================
+The LLVM Project is under the Apache License v2.0 with LLVM Exceptions:
+==============================================================================
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+    1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+    2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+    3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+    4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+    5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+     ...
[truncated]

@llvmbot
Copy link
Member

llvmbot commented Jan 24, 2025

@llvm/pr-subscribers-clang-driver

Author: Michael Kruse (Meinersbur)

Changes

Extract Flang's runtime library to use the LLVM_ENABLE_RUNTIME mechanism.

Motivation:

  • Consistency with LLVM's other runtime libraries (compiler-rt, libc, libcxx, openmp offload, ...)
  • Allows compiling the runtime for multiple targets at once using the LLVM_RUNTIME_TARGETS configuration options
  • Installs the runtime into the compiler's per-target resource directory so it can be automatically found even when cross-compiling

Potential future directions:

  • Uses CMake's support for compiling Fortran files, including dependency resolution of Fortran modules
  • Improve robustness of compiling libomp.mod when openmp is available
  • Remove Flang's dependency from flang-rt's RTNAME function declarations (tblgen?)
  • Reduce Flang's build-time dependency from flang-rt's REAL(16) support

See RFC discussion at https://discourse.llvm.org/t/rfc-use-llvm-enable-runtimes-for-flangs-runtime/80826

Patch series:

  • #110244
  • #112188
  • #121997
  • #122069
  • #122334
  • #122336
  • #122341
  • #110298
  • #110217 (this PR)
  • #121782
  • #124126

Patch for lab.llvm.org buildbots:


Patch is 108.72 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/110217.diff

41 Files Affected:

  • (modified) clang/lib/Driver/ToolChains/Flang.cpp (+9-5)
  • (added) flang-rt/.clang-tidy (+2)
  • (added) flang-rt/CMakeLists.txt (+248)
  • (added) flang-rt/CODE_OWNERS.TXT (+14)
  • (added) flang-rt/LICENSE.TXT (+234)
  • (added) flang-rt/README.md (+188)
  • (added) flang-rt/cmake/modules/AddFlangRT.cmake (+186)
  • (added) flang-rt/cmake/modules/AddFlangRTOffload.cmake (+101)
  • (added) flang-rt/cmake/modules/GetToolchainDirs.cmake (+125)
  • (added) flang-rt/lib/CMakeLists.txt (+18)
  • (added) flang-rt/lib/FortranFloat128Math/CMakeLists.txt (+136)
  • (added) flang-rt/lib/Testing/CMakeLists.txt (+20)
  • (added) flang-rt/lib/flang_rt/CMakeLists.txt (+213)
  • (added) flang-rt/lib/flang_rt/CUDA/CMakeLists.txt (+33)
  • (modified) flang-rt/lib/flang_rt/io-api-minimal.cpp (+1-1)
  • (added) flang-rt/test/CMakeLists.txt (+59)
  • (modified) flang-rt/test/Driver/ctofortran.f90 (+5-24)
  • (modified) flang-rt/test/Driver/exec.f90 (+4-4)
  • (added) flang-rt/test/NonGtestUnit/lit.cfg.py (+22)
  • (added) flang-rt/test/NonGtestUnit/lit.site.cfg.py.in (+14)
  • (modified) flang-rt/test/Runtime/no-cpp-dep.c (+3-2)
  • (added) flang-rt/test/Unit/lit.cfg.py (+21)
  • (added) flang-rt/test/Unit/lit.site.cfg.py.in (+15)
  • (added) flang-rt/test/lit.cfg.py (+102)
  • (added) flang-rt/test/lit.site.cfg.py.in (+19)
  • (added) flang-rt/unittests/CMakeLists.txt (+111)
  • (added) flang-rt/unittests/Evaluate/CMakeLists.txt (+21)
  • (added) flang-rt/unittests/Runtime/CMakeLists.txt (+48)
  • (added) flang-rt/unittests/Runtime/CUDA/CMakeLists.txt (+18)
  • (modified) flang/CMakeLists.txt (+26-27)
  • (added) flang/cmake/modules/FlangCommon.cmake (+43)
  • (modified) flang/docs/GettingStarted.md (+58-50)
  • (modified) flang/docs/ReleaseNotes.md (+7-1)
  • (modified) flang/module/iso_fortran_env_impl.f90 (+1-1)
  • (modified) flang/test/lit.cfg.py (-20)
  • (modified) flang/test/lit.site.cfg.py.in (-3)
  • (modified) llvm/CMakeLists.txt (+7-1)
  • (modified) llvm/cmake/modules/LLVMExternalProjectUtils.cmake (+15-1)
  • (modified) llvm/projects/CMakeLists.txt (+3-1)
  • (modified) llvm/runtimes/CMakeLists.txt (+18-7)
  • (modified) runtimes/CMakeLists.txt (+1-1)
diff --git a/clang/lib/Driver/ToolChains/Flang.cpp b/clang/lib/Driver/ToolChains/Flang.cpp
index 68a17edf8ca341..17a8a4dd8d0a87 100644
--- a/clang/lib/Driver/ToolChains/Flang.cpp
+++ b/clang/lib/Driver/ToolChains/Flang.cpp
@@ -342,11 +342,15 @@ static void processVSRuntimeLibrary(const ToolChain &TC, const ArgList &Args,
                                     ArgStringList &CmdArgs) {
   assert(TC.getTriple().isKnownWindowsMSVCEnvironment() &&
          "can only add VS runtime library on Windows!");
-  // if -fno-fortran-main has been passed, skip linking Fortran_main.a
-  if (TC.getTriple().isKnownWindowsMSVCEnvironment()) {
-    CmdArgs.push_back(Args.MakeArgString(
-        "--dependent-lib=" + TC.getCompilerRTBasename(Args, "builtins")));
-  }
+
+  // Flang/Clang (including clang-cl) -compiled programs targeting the MSVC ABI
+  // should only depend on msv(u)crt. LLVM still emits libgcc/compiler-rt
+  // functions in some cases like 128-bit integer math (__udivti3, __modti3,
+  // __fixsfti, __floattidf, ...) that msvc does not support. We are injecting a
+  // dependency to Compiler-RT's builtin library where these are implemented.
+  CmdArgs.push_back(Args.MakeArgString(
+      "--dependent-lib=" + TC.getCompilerRTBasename(Args, "builtins")));
+
   unsigned RTOptionID = options::OPT__SLASH_MT;
   if (auto *rtl = Args.getLastArg(options::OPT_fms_runtime_lib_EQ)) {
     RTOptionID = llvm::StringSwitch<unsigned>(rtl->getValue())
diff --git a/flang-rt/.clang-tidy b/flang-rt/.clang-tidy
new file mode 100644
index 00000000000000..ee3a0ab2201bf3
--- /dev/null
+++ b/flang-rt/.clang-tidy
@@ -0,0 +1,2 @@
+Checks: '-llvm-include-order,readability-braces-around-statements,-readability-identifier-naming,-clang-diagnostic-*'
+InheritParentConfig: true
diff --git a/flang-rt/CMakeLists.txt b/flang-rt/CMakeLists.txt
new file mode 100644
index 00000000000000..655d0a55b40044
--- /dev/null
+++ b/flang-rt/CMakeLists.txt
@@ -0,0 +1,248 @@
+#===-- CMakeLists.txt ------------------------------------------------------===#
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+#===------------------------------------------------------------------------===#
+#
+# Build instructions for the flang-rt library. This is file is intended to be
+# included using the LLVM_ENABLE_RUNTIMES mechanism.
+#
+#===------------------------------------------------------------------------===#
+
+if (NOT LLVM_RUNTIMES_BUILD)
+  message(FATAL_ERROR "Use this CMakeLists.txt from LLVM's runtimes build system.
+      Example:
+        cmake <llvm-project>/runtimes -DLLVM_ENABLE_RUNTIMES=flang-rt
+    ")
+endif ()
+
+set(LLVM_SUBPROJECT_TITLE "Flang-RT")
+set(FLANG_RT_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}")
+set(FLANG_RT_BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}")
+set(FLANG_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../flang")
+
+# CMake 3.24 is the first version of CMake that directly recognizes Flang.
+# LLVM's requirement is only CMake 3.20, teach CMake 3.20-3.23 how to use Flang.
+if (CMAKE_VERSION VERSION_LESS "3.24")
+  cmake_path(GET CMAKE_Fortran_COMPILER STEM _Fortran_COMPILER_STEM)
+  if (_Fortran_COMPILER_STEM STREQUAL "flang-new" OR _Fortran_COMPILER_STEM STREQUAL "flang")
+    include(CMakeForceCompiler)
+    CMAKE_FORCE_Fortran_COMPILER("${CMAKE_Fortran_COMPILER}" "LLVMFlang")
+
+    set(CMAKE_Fortran_COMPILER_ID "LLVMFlang")
+    set(CMAKE_Fortran_COMPILER_VERSION "${LLVM_VERSION_MAJOR}.${LLVM_VERSION_MINOR}")
+
+    set(CMAKE_Fortran_SUBMODULE_SEP "-")
+    set(CMAKE_Fortran_SUBMODULE_EXT ".mod")
+
+    set(CMAKE_Fortran_PREPROCESS_SOURCE
+      "<CMAKE_Fortran_COMPILER> -cpp <DEFINES> <INCLUDES> <FLAGS> -E <SOURCE> > <PREPROCESSED_SOURCE>")
+
+    set(CMAKE_Fortran_FORMAT_FIXED_FLAG "-ffixed-form")
+    set(CMAKE_Fortran_FORMAT_FREE_FLAG "-ffree-form")
+
+    set(CMAKE_Fortran_MODDIR_FLAG "-module-dir")
+
+    set(CMAKE_Fortran_COMPILE_OPTIONS_PREPROCESS_ON "-cpp")
+    set(CMAKE_Fortran_COMPILE_OPTIONS_PREPROCESS_OFF "-nocpp")
+    set(CMAKE_Fortran_POSTPROCESS_FLAG "-ffixed-line-length-72")
+
+    set(CMAKE_Fortran_COMPILE_OPTIONS_TARGET "--target=")
+
+    set(CMAKE_Fortran_LINKER_WRAPPER_FLAG "-Wl,")
+    set(CMAKE_Fortran_LINKER_WRAPPER_FLAG_SEP ",")
+  endif ()
+endif ()
+enable_language(Fortran)
+
+
+list(APPEND CMAKE_MODULE_PATH
+    "${FLANG_RT_SOURCE_DIR}/cmake/modules"
+    "${FLANG_SOURCE_DIR}/cmake/modules"
+  )
+include(AddFlangRT)
+include(GetToolchainDirs)
+include(FlangCommon)
+include(HandleCompilerRT)
+include(ExtendPath)
+
+
+############################
+# Build Mode Introspection #
+############################
+
+# Determine whether we are in the runtimes/runtimes-bins directory of a
+# bootstrap build.
+set(LLVM_TREE_AVAILABLE OFF)
+if (LLVM_LIBRARY_OUTPUT_INTDIR AND LLVM_RUNTIME_OUTPUT_INTDIR AND PACKAGE_VERSION)
+  set(LLVM_TREE_AVAILABLE ON)
+endif()
+
+# Path to LLVM development tools (FileCheck, llvm-lit, not, ...)
+set(LLVM_TOOLS_DIR "${LLVM_BINARY_DIR}/bin")
+
+# Determine build and install paths.
+# The build path is absolute, but the install dir is relative, CMake's install
+# command has to apply CMAKE_INSTALL_PREFIX itself.
+get_toolchain_library_subdir(toolchain_lib_subdir)
+if (LLVM_TREE_AVAILABLE)
+  # In a bootstrap build emit the libraries into a default search path in the
+  # build directory of the just-built compiler. This allows using the
+  # just-built compiler without specifying paths to runtime libraries.
+  #
+  # Despite Clang in the name, get_clang_resource_dir does not depend on Clang
+  # being added to the build. Flang uses the same resource dir as clang.
+  include(GetClangResourceDir)
+  get_clang_resource_dir(FLANG_RT_OUTPUT_RESOURCE_DIR PREFIX "${LLVM_LIBRARY_OUTPUT_INTDIR}/..")
+  get_clang_resource_dir(FLANG_RT_INSTALL_RESOURCE_PATH)
+
+  extend_path(FLANG_RT_OUTPUT_RESOURCE_LIB_DIR "${FLANG_RT_OUTPUT_RESOURCE_DIR}" "${toolchain_lib_subdir}")
+else ()
+  # In a standalone runtimes build, do not write into LLVM_BINARY_DIR. It may be
+  # read-only and/or shared by multiple runtimes with different build
+  # configurations (e.g. Debug/Release). Use the runtime's own lib dir like any
+  # non-toolchain library.
+  # For the install prefix, still use the resource dir assuming that Flang will
+  # be installed there using the same prefix. This is to not have a difference
+  # between bootstrap and standalone runtimes builds.
+  set(FLANG_RT_OUTPUT_RESOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}")
+  set(FLANG_RT_INSTALL_RESOURCE_PATH "lib${LLVM_LIBDIR_SUFFIX}/clang/${LLVM_VERSION_MAJOR}")
+
+  extend_path(FLANG_RT_OUTPUT_RESOURCE_LIB_DIR "${FLANG_RT_OUTPUT_RESOURCE_DIR}" "lib${LLVM_LIBDIR_SUFFIX}")
+endif ()
+extend_path(FLANG_RT_INSTALL_RESOURCE_LIB_PATH "${FLANG_RT_INSTALL_RESOURCE_PATH}" "${toolchain_lib_subdir}")
+cmake_path(NORMAL_PATH FLANG_RT_OUTPUT_RESOURCE_DIR)
+cmake_path(NORMAL_PATH FLANG_RT_INSTALL_RESOURCE_PATH)
+cmake_path(NORMAL_PATH FLANG_RT_OUTPUT_RESOURCE_LIB_DIR)
+cmake_path(NORMAL_PATH FLANG_RT_INSTALL_RESOURCE_LIB_PATH)
+
+
+#################
+# Build Options #
+#################
+
+# Important: flang-rt user options must be prefixed with "FLANG_RT_". Variables
+# with this prefix will be forwarded in bootstrap builds.
+
+option(FLANG_RT_INCLUDE_TESTS "Generate build targets for the flang-rt unit and regression-tests." "${LLVM_INCLUDE_TESTS}")
+
+
+set(FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT "" CACHE STRING "Compile Flang-RT with GPU support (CUDA or OpenMP)")
+set_property(CACHE FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT PROPERTY STRINGS
+    ""
+    CUDA
+    OpenMP
+  )
+if (NOT FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT)
+  # Support for GPUs disabled
+elseif (FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT STREQUAL "CUDA")
+  # Support for CUDA
+  set(FLANG_RT_LIBCUDACXX_PATH "" CACHE PATH "Path to libcu++ package installation")
+  option(FLANG_RT_CUDA_RUNTIME_PTX_WITHOUT_GLOBAL_VARS "Do not compile global variables' definitions when producing PTX library" OFF)
+elseif (FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT STREQUAL "OpenMP")
+  # Support for OpenMP offloading
+  set(FLANG_RT_DEVICE_ARCHITECTURES "all" CACHE STRING
+      "List of OpenMP device architectures to be used to compile the Fortran runtime (e.g. 'gfx1103;sm_90')"
+    )
+
+  if (FLANG_RT_DEVICE_ARCHITECTURES STREQUAL "all")
+    # TODO: support auto detection on the build system.
+    set(all_amdgpu_architectures
+      "gfx700;gfx701;gfx801;gfx803;gfx900;gfx902;gfx906"
+      "gfx908;gfx90a;gfx90c;gfx940;gfx1010;gfx1030"
+      "gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036"
+      "gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151"
+      "gfx1152;gfx1153")
+    set(all_nvptx_architectures
+      "sm_35;sm_37;sm_50;sm_52;sm_53;sm_60;sm_61;sm_62"
+      "sm_70;sm_72;sm_75;sm_80;sm_86;sm_89;sm_90")
+    set(all_gpu_architectures
+      "${all_amdgpu_architectures};${all_nvptx_architectures}")
+      set(FLANG_RT_DEVICE_ARCHITECTURES ${all_gpu_architectures})
+  endif()
+  list(REMOVE_DUPLICATES FLANG_RT_DEVICE_ARCHITECTURES)
+else ()
+  message(FATAL_ERROR "Invalid value '${FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT}' for FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT; must be empty, 'CUDA', or 'OpenMP'")
+endif ()
+
+
+option(FLANG_RT_ENABLE_CUF "Compile CUDA Fortran runtime sources" OFF)
+if (FLANG_RT_ENABLE_CUF)
+  find_package(CUDAToolkit REQUIRED)
+endif()
+
+
+########################
+# System Introspection #
+########################
+
+include(CheckCXXSymbolExists)
+include(CheckCXXSourceCompiles)
+check_cxx_symbol_exists(strerror_r string.h HAVE_STRERROR_R)
+# Can't use symbol exists here as the function is overloaded in C++
+check_cxx_source_compiles(
+  "#include <string.h>
+   int main() {
+     char buf[4096];
+     return strerror_s(buf, 4096, 0);
+   }
+  "
+  HAVE_DECL_STRERROR_S)
+
+
+# Search for clang_rt.builtins library. Need in addition to msvcrt.
+if (WIN32)
+  find_compiler_rt_library(builtins FLANG_RT_BUILTINS_LIBRARY)
+endif ()
+
+
+# Check whether the compiler can undefine a macro using the "-U" flag.
+# Aternatively, we could use
+#   CMAKE_CXX_COMPILER_FRONTEND_VARIANT STREQUAL "GNU"
+# but some older versions of CMake don't define it for GCC itself.
+check_cxx_compiler_flag("-UTESTFLAG" FLANG_RT_SUPPORTS_UNDEFINE_FLAG)
+
+
+#####################
+# Build Preparation #
+#####################
+
+if (FLANG_RT_EXPERIMENTAL_OFFLOAD_SUPPORT AND FLANG_RT_INCLUDE_TESTS)
+  # If Fortran runtime is built as CUDA library, the linking
+  # of targets that link flang-rt must be done
+  # with CUDA_RESOLVE_DEVICE_SYMBOLS.
+  # CUDA language must be enabled for CUDA_RESOLVE_DEVICE_SYMBOLS
+  # to take effect.
+  enable_language(CUDA)
+endif()
+
+
+# C++17 is required for flang-rt; user or other runtimes may override this.
+# GTest included later also requires C++17.
+set(CMAKE_CXX_STANDARD 17 CACHE STRING "C++ standard to conform to")
+set(CMAKE_CXX_STANDARD_REQUIRED YES)
+
+
+configure_file(cmake/config.h.cmake.in config.h)
+
+
+# The bootstrap build will create a phony target with the same as the top-level
+# directory ("flang-rt") and delegate it to the runtimes build dir.
+# AddFlangRT will add all non-EXCLUDE_FROM_ALL targets to it.
+add_custom_target(flang-rt)
+
+
+###################
+# Build Artifacts #
+###################
+
+add_subdirectory(lib)
+
+if (FLANG_RT_INCLUDE_TESTS)
+  add_subdirectory(unittests)
+  add_subdirectory(test)
+else ()
+  add_custom_target(check-flang-rt)
+endif()
diff --git a/flang-rt/CODE_OWNERS.TXT b/flang-rt/CODE_OWNERS.TXT
new file mode 100644
index 00000000000000..649243aa1e8fec
--- /dev/null
+++ b/flang-rt/CODE_OWNERS.TXT
@@ -0,0 +1,14 @@
+This file is a list of the people responsible for ensuring that patches for a
+particular part of Flang are reviewed, either by themself or by someone else.
+They are also the gatekeepers for their part of Flang, with the final word on
+what goes in or not.
+
+The list is sorted by surname and formatted to allow easy grepping and
+beautification by scripts. The fields are: name (N), email (E), web-address
+(W), PGP key ID and fingerprint (P), description (D), snail-mail address
+(S) and (I) IRC handle. Each entry should contain at least the (N), (E) and
+(D) fields.
+
+N: Steve Scalpone
+E: [email protected]
+D: Anything not covered by others
diff --git a/flang-rt/LICENSE.TXT b/flang-rt/LICENSE.TXT
new file mode 100644
index 00000000000000..53bb2e7fbc7643
--- /dev/null
+++ b/flang-rt/LICENSE.TXT
@@ -0,0 +1,234 @@
+==============================================================================
+The LLVM Project is under the Apache License v2.0 with LLVM Exceptions:
+==============================================================================
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+    1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+    2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+    3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+    4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+    5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+     ...
[truncated]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang Clang issues not falling into any other category cmake Build system in general and CMake in particular flang Flang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.