Build wamrc AOT compiler Error

[jquanC]

I follow the official tutorial to build the wamrc, but i meet a problem.

Env: Ubuntu 18.04.6 LTS

Before I tried to build the wamrc , I built the iwasm successfully. (have to upgrade the cmake version：3.16.0)
Then, I go to build the wamrc as follows:
cd wamr-compiler
./build_llvm.sh

In the first time, the output is：

root@iZf8zb4j1s5w47yxj5z8xmZ:/opt/wasm-micro-runtime/wamr-compiler# ./build_llvm.sh
Collecting requests==2.28.2 (from -r ../build-scripts/requirements.txt (line 1))
Could not find a version that satisfies the requirement requests==2.28.2 (from -r ../build-scripts/requirements.txt (line 1)) (from versions: 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.6, 0.10.7, 0.10.8, 0.11.1, 0.11.2, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.13.4, 0.13.5, 0.13.6, 0.13.7, 0.13.8, 0.13.9, 0.14.0, 0.14.1, 0.14.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.2.1, 2.3.0, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0, 2.6.1, 2.6.2, 2.7.0, 2.8.0, 2.8.1, 2.9.0, 2.9.1, 2.9.2, 2.10.0, 2.11.0, 2.11.1, 2.12.0, 2.12.1, 2.12.2, 2.12.3, 2.12.4, 2.12.5, 2.13.0, 2.14.0, 2.14.1, 2.14.2, 2.15.1, 2.16.0, 2.16.1, 2.16.2, 2.16.3, 2.16.4, 2.16.5, 2.17.0, 2.17.1, 2.17.2, 2.17.3, 2.18.0, 2.18.1, 2.18.2, 2.18.3, 2.18.4, 2.19.0, 2.19.1, 2.20.0, 2.20.1, 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0, 2.25.1, 2.26.0, 2.27.0, 2.27.1)
No matching distribution found for requests==2.28.2 (from -r ../build-scripts/requirements.txt (line 1))
git clone --depth 1 --branch release/15.x [email protected]:llvm/llvm-project.git llvm
Cloning into 'llvm'...
remote: Enumerating objects: 124785, done.
remote: Counting objects: 100% (124785/124785), done.
remote: Compressing objects: 100% (111040/111040), done.
remote: Total 124785 (delta 21963), reused 48349 (delta 9793), pack-reused 0
Receiving objects: 100% (124785/124785), 175.31 MiB | 6.16 MiB/s, done.
Resolving deltas: 100% (21963/21963), done.
Checking out files: 100% (118554/118554), done.
Use the gcc toolchain
CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_ASM_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
See also "/opt/wasm-micro-runtime/core/deps/llvm/build/CMakeFiles/CMakeOutput.log".

It seems I need 'Ninja', and I then installed it, and execute the command again, while have followed output:

root@iZf8zb4j1s5w47yxj5z8xmZ:/opt/wasm-micro-runtime/wamr-compiler# ./build_llvm.sh
Collecting requests==2.28.2 (from -r ../build-scripts/requirements.txt (line 1))
Could not find a version that satisfies the requirement requests==2.28.2 (from -r ../build-scripts/requirements.txt (line 1)) (from versions: 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.6, 0.10.7, 0.10.8, 0.11.1, 0.11.2, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.13.4, 0.13.5, 0.13.6, 0.13.7, 0.13.8, 0.13.9, 0.14.0, 0.14.1, 0.14.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.2.1, 2.3.0, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0, 2.6.1, 2.6.2, 2.7.0, 2.8.0, 2.8.1, 2.9.0, 2.9.1, 2.9.2, 2.10.0, 2.11.0, 2.11.1, 2.12.0, 2.12.1, 2.12.2, 2.12.3, 2.12.4, 2.12.5, 2.13.0, 2.14.0, 2.14.1, 2.14.2, 2.15.1, 2.16.0, 2.16.1, 2.16.2, 2.16.3, 2.16.4, 2.16.5, 2.17.0, 2.17.1, 2.17.2, 2.17.3, 2.18.0, 2.18.1, 2.18.2, 2.18.3, 2.18.4, 2.19.0, 2.19.1, 2.20.0, 2.20.1, 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0, 2.25.1, 2.26.0, 2.27.0, 2.27.1)
No matching distribution found for requests==2.28.2 (from -r ../build-scripts/requirements.txt (line 1))
Use the gcc toolchain
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:206 (message):
Unable to find the program ccache. Set LLVM_CCACHE_BUILD to OFF
-- Configuring incomplete, errors occurred!
See also "/opt/wasm-micro-runtime/core/deps/llvm/build/CMakeFiles/CMakeOutput.log".

I have check out the ../build-scripts/requirements.txt , it only have simple content:

requests==2.28.2

and in the 'CMakeOutput.log', I can't find helpful info (without any 'error' message).

I am relatively new in this field, could someone know how to solve it? Please help me!

[TianlongLiang]

Hi, @jquanC, I am here to help. You should be able to fix this issue by installing ccache, you can use this command in the terminal to install it:

sudo apt-get install ccache

Let me know if there is any other problem.

PS: one funny thing is, this actually happened to me a few days ago when I tried to set up a new environment

[TianlongLiang]

1. Does the ./build_llvm.sh process occupies the CPU too much and thus make the ssh thread crash? // I saw the warning of the CPU load being too high in the cloud console.

Based on your description before:

the first time I executed ./build_llvm.sh, it took a long time and my vscode-remote-ssh lost its connection.
For a while, I couldn't ssh to the server(an ali-could ECS). However, after 30mins, I can ssh to it again.

I would say yes, it drains too much CPU quota for your VM, building LLVM requires significant CPU resources, and using all CPU cores will make other processes on your VM like ssh unstable or even crush.

2. Does the ./build_llvm.sh command work successfully?

I guess no. For this error message:

make[2]: *** No rule to make target '/opt/wasm-micro-runtime/core/deps/llvm/build/lib/libLLVMCore.a', needed by 'wamrc'. Stop.

This error should mean that you didn't build LLVM successfully. You can list the directory /opt/wasm-micro-runtime/core/deps/llvm/build/lib/libLLVMCore.a to check whether this file exists.

I think you may want to try a cloud VM instance with more CPU power. I remember that in AWS, you can change EC2 instance type without losing data by snapshot or something, ali-cloud should have similar feature.

[jquanC]

Thanks @TianlongLiang
Thankyou for your advice!
Still now I couldn't ssh to the server, and can't check out whether '/opt/wasm-micro-runtime/core/deps/llvm/build/lib/libLLVMCore.a' exists.
In the cloud console monitor, the first time CPU load to 100%, while 36% the second time. It's a little confusing.
I asked the cloud technique support and wait for their response.
And I will try as you suggested later, such as upgrading the VM, and give feedback here.

[jquanC]

The problem has been done.
It is the memory bottleneck (as a main reason, also CPU meets its upperbound) that resulted in this issue. I upgrade the memory from 2G to 4G, and it takes about 1.5h to build the LLVM.
@TianlongLiang Thanks!🍖

[TianlongLiang]

I am glad that it was resolved in the end. The cloud platform is exciting but sometimes can be tricky, good luck with the future coding!

[jquanC]

Yes, it is exciting and challenging! There are many things need to learn and I will go on for it!