Failed lowering PolyBench's C code to MLIR SCF dialect

[johnpzh]

Hi there! I was trying Polygeist to lower PolyBench C code to MLIR SCF dialect. I tried gemm.c as the input but it crashed. I am using macOS Ventura 13.7 and Apple Silicon. Here is the command.

$ ./cgeist PolyBenchC/linear-algebra/blas/gemm/gemm.c -O0 -function="*" -S -I /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I PolyBenchC/utilities
In file included from ./PolyBenchC/./linear-algebra/blas/gemm/gemm.c:12:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:93:16: warning: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) [-Wnullability-completeness]
   93 |         unsigned char   *_base;
      |                         ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:93:16: note: insert '_Nullable' if the pointer may be null
   93 |         unsigned char   *_base;
      |                         ^
      |                           _Nullable 
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:93:16: note: insert '_Nonnull' if the pointer should never be null
   93 |         unsigned char   *_base;
      |                         ^
      |                           _Nonnull 
# ... 
# similar warnings 
# ...
RecordType 0x134a10dc0 'struct __sFILEX'
`-Record 0x134a10d38 '__sFILEX'
ST: %struct.__sFILEX = type opaque
fields
types
Assertion failed: (types.size()), function getMLIRType, file clang-mlir.cc, line 5637.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist ./PolyBenchC/./linear-algebra/blas/gemm/gemm.c -O0 -function=* -S -I /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I PolyBenchC/utilities
1.	<eof> parser at end of file
 #0 0x00000001074e6e20 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x102d3ee20)
 #1 0x00000001074e73fc PrintStackTraceSignalHandler(void*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x102d3f3fc)
 #2 0x00000001074e50b0 llvm::sys::RunSignalHandlers() (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x102d3d0b0)
 #3 0x00000001074e8518 SignalHandler(int) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x102d40518)
 #4 0x0000000196262a24 (/usr/lib/system/libsystem_platform.dylib+0x18042ea24)
 #5 0x0000000196233c28 (/usr/lib/system/libsystem_pthread.dylib+0x1803ffc28)
 #6 0x0000000196141ae8 (/usr/lib/system/libsystem_c.dylib+0x18030dae8)
 #7 0x0000000196140e44 (/usr/lib/system/libsystem_c.dylib+0x18030ce44)
 #8 0x00000001047b3618 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000b618)
 #9 0x00000001047b26d4 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000a6d4)
#10 0x00000001047b4198 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000c198)
#11 0x00000001047b31d4 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000b1d4)
#12 0x00000001047b26d4 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000a6d4)
#13 0x00000001047b28b4 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000a8b4)
#14 0x00000001047b26d4 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000a6d4)
#15 0x00000001047b4198 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000c198)
#16 0x00000001047b4198 MLIRASTConsumer::getMLIRType(clang::QualType, bool*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10000c198)
#17 0x00000001047e4a28 MLIRScanner::VisitDeclRefExpr(clang::DeclRefExpr*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10003ca28)
#18 0x00000001047badb4 clang::StmtVisitorBase<std::__1::add_pointer, MLIRScanner, ValueCategory>::Visit(clang::Stmt*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100012db4)
#19 0x00000001047c8e98 MLIRScanner::VisitCastExpr(clang::CastExpr*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100020e98)
#20 0x00000001049de960 clang::StmtVisitorBase<std::__1::add_pointer, MLIRScanner, ValueCategory>::VisitImplicitCastExpr(clang::ImplicitCastExpr*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100236960)
#21 0x00000001047bacf4 clang::StmtVisitorBase<std::__1::add_pointer, MLIRScanner, ValueCategory>::Visit(clang::Stmt*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100012cf4)
#22 0x0000000104a7a93c MLIRScanner::getLLVM(clang::Expr*, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x1002d293c)
#23 0x0000000104a83df0 MLIRScanner::VisitCallExpr(clang::CallExpr*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x1002dbdf0)
#24 0x00000001047baba4 clang::StmtVisitorBase<std::__1::add_pointer, MLIRScanner, ValueCategory>::Visit(clang::Stmt*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100012ba4)
#25 0x0000000104a33b84 MLIRScanner::VisitCompoundStmt(clang::CompoundStmt*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10028bb84)
#26 0x00000001047b9ee4 clang::StmtVisitorBase<std::__1::add_pointer, MLIRScanner, ValueCategory>::Visit(clang::Stmt*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100011ee4)
#27 0x00000001047b1d50 MLIRScanner::init(mlir::func::FuncOp, clang::FunctionDecl const*) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100009d50)
#28 0x00000001047ee604 MLIRASTConsumer::run() (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100046604)
#29 0x00000001047efab0 MLIRASTConsumer::HandleTranslationUnit(clang::ASTContext&) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100047ab0)
#30 0x0000000111515684 clang::ParseAST(clang::Sema&, bool, bool) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10cd6d684)
#31 0x000000010ab664c0 clang::ASTFrontendAction::ExecuteAction() (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x1063be4c0)
#32 0x000000010ab65c8c clang::FrontendAction::Execute() (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x1063bdc8c)
#33 0x00000001047f8ca4 parseMLIR(char const*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>, mlir::OwningOpRef<mlir::ModuleOp>&, llvm::Triple&, llvm::DataLayout&, llvm::Triple&, llvm::DataLayout&) (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x100050ca4)
#34 0x00000001047f29c4 main (/Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist+0x10004a9c4)
#35 0x0000000195edbfd8 
run01.cgeist_c_to_scf.sh: line 12: 35575 Abort trap: 6           /Users/peng599/pppp/comet-amais-memory/Polygeist-main/cmake-build-debug/bin/cgeist ./PolyBenchC/./linear-algebra/blas/gemm/gemm.c -O0 -function=* -S -I /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I PolyBenchC/utilities

Before this test, there were also some failed tests after building Polygeist as the follows.

$ ninja check-polygeist-opt
********************
********************
Failed Tests (6):
  polygeist :: polygeist-opt/cpuifybackprop.mlir
  polygeist :: polygeist-opt/cpuifyhotspot.mlir
  polygeist :: polygeist-opt/cpuifyloopdistribute.mlir
  polygeist :: polygeist-opt/looprecur.mlir
  polygeist :: polygeist-opt/paralleldistribute.mlir
  polygeist :: polygeist-opt/pathfinder.mlir


Testing Time: 9.29s
  Passed           : 80
  Expectedly Failed:  3
  Failed           :  6

For ninja check-cgeist, most of the failure were caused by the error of 'stdio.h' file not found. That's why in the first gemm.c test I used the -I option to include macOS's standard C headers.

$ ninja check-cgeist
********************
********************
Failed Tests (46):
  cgeist :: Verification/addressof.cpp
  cgeist :: Verification/clangbuiltin.cpp
  cgeist :: Verification/cond.c
  cgeist :: Verification/cond2.c
  cgeist :: Verification/freecst.c
  cgeist :: Verification/fscanf.c
  cgeist :: Verification/fw.c
  cgeist :: Verification/fwfree.c
  cgeist :: Verification/gettimeofday.c
  cgeist :: Verification/malloc.c
  cgeist :: Verification/memref-fullrank.c
  cgeist :: Verification/nestloop.c
  cgeist :: Verification/new.cpp
  cgeist :: Verification/palloc.c
  cgeist :: Verification/tobits.c
  cgeist :: Verification/whileset.c
  cgeist :: polybench/datamining/correlation/correlation.c
  cgeist :: polybench/datamining/covariance/covariance.c
  cgeist :: polybench/linear-algebra/blas/gemm/gemm.c
  cgeist :: polybench/linear-algebra/blas/gemver/gemver.c
  cgeist :: polybench/linear-algebra/blas/gesummv/gesummv.c
  cgeist :: polybench/linear-algebra/blas/symm/symm.c
  cgeist :: polybench/linear-algebra/blas/syr2k/syr2k.c
  cgeist :: polybench/linear-algebra/blas/syrk/syrk.c
  cgeist :: polybench/linear-algebra/blas/trmm/trmm.c
  cgeist :: polybench/linear-algebra/kernels/2mm/2mm.c
  cgeist :: polybench/linear-algebra/kernels/3mm/3mm.c
  cgeist :: polybench/linear-algebra/kernels/atax/atax.c
  cgeist :: polybench/linear-algebra/kernels/bicg/bicg.c
  cgeist :: polybench/linear-algebra/kernels/doitgen/doitgen.c
  cgeist :: polybench/linear-algebra/kernels/mvt/mvt.c
  cgeist :: polybench/linear-algebra/solvers/cholesky/cholesky.c
  cgeist :: polybench/linear-algebra/solvers/durbin/durbin.c
  cgeist :: polybench/linear-algebra/solvers/gramschmidt/gramschmidt.c
  cgeist :: polybench/linear-algebra/solvers/lu/lu.c
  cgeist :: polybench/linear-algebra/solvers/ludcmp/ludcmp.c
  cgeist :: polybench/linear-algebra/solvers/trisolv/trisolv.c
  cgeist :: polybench/medley/deriche/deriche.c
  cgeist :: polybench/medley/floyd-warshall/floyd-warshall.c
  cgeist :: polybench/medley/nussinov/nussinov.c
  cgeist :: polybench/stencils/adi/adi.c
  cgeist :: polybench/stencils/fdtd-2d/fdtd-2d.c
  cgeist :: polybench/stencils/heat-3d/heat-3d.c
  cgeist :: polybench/stencils/jacobi-1d/jacobi-1d.c
  cgeist :: polybench/stencils/jacobi-2d/jacobi-2d.c
  cgeist :: polybench/stencils/seidel-2d/seidel-2d.c


Testing Time: 15.64s
  Passed           : 119
  Expectedly Failed:  10
  Failed           :  46

I have tried built Polygeist by both option 1 and option 2, and have tried branches main, cgo24 and ivanradanov-patch-1, but the problem remained.

I was wondering if I had done something wrong. Any suggestions would be appreciated. Thank you for your time!

[johnpzh]

After some attempts and discussion, I feel that this problem would only happen on macOS. A Linux server would work fine. Please check this thread from LLVM forum.