Failed to init stack guard pages in Go

[benozol]

Using the Go bindings in a goroutine can trigger a failure in InitThreadEnv(), which is preceded by the message Failed to init stack guard pages. In the example below, this happens roughly at every second run (MacOS).

The failure is not prevented by using a mutex on the goroutine. I'm unable to reproduce the failure in C using pthread.h.

Is my use of InitThreadEnv() invalid? Is some other locking mechanism required in the context of goroutines?

package test

import (
	"fmt"
	"runtime"
	"sync"

	"github.com/bytecodealliance/wasm-micro-runtime/language-bindings/go/wamr"
)

func main() {
	wamr.Runtime().Init()
	var wg sync.WaitGroup
	for i := 0; i < 3; i++ {
		i := i
		wg.Add(1)
		go func() {
			if wamr.Runtime().ThreadEnvInited() {
				fmt.Printf("Already inited (%d)\n", i)
			} else {
				if ok := wamr.Runtime().InitThreadEnv(); ok {
					fmt.Printf("Inited (%d)\n", i)
					wamr.Runtime().DestroyThreadEnv()
				} else {
					fmt.Printf("Init failed (%d)\n", i)
				}
			}
			wg.Done()
		}()
	}
	wg.Wait()
	wamr.Runtime().Destroy()
}

[benozol]

Apparently the initialization fails when the result of os_thread_get_stack_boundary() is not page-aligned (16384 on MacOS), because the result is passed to os_mprotect(), which requires the address to be page-aligned. Are the go bindings supported in goroutines?

package main

import (
	"fmt"
	"sync"
	"unsafe"
)

/*
   #include <stdio.h>
   #include <stdbool.h>
   #include <stdint.h>
   #include <unistd.h>
   #include <pthread.h>
   #include <errno.h>
   #include <string.h>
   #include <sys/mman.h>
   #define APP_THREAD_STACK_SIZE_MAX (8 * 1024 * 1024)
   #define STACK_OVERFLOW_CHECK_GUARD_PAGE_COUNT 3
   char *check_aligned(uint8_t *addr) {
       if ((intptr_t)addr % 16384 == 0) return "aligned to 16KiB";
       if ((intptr_t)addr % 4096 == 0) return "aligned to 4KiB";
       return "unaligned to 4KiB or 16KiB";
   }
   void check_stack_boundary(int32_t i) {
       pthread_t self = pthread_self();
       uint8_t *addr = (uint8_t *)pthread_get_stackaddr_np(self);
       size_t page_size = getpagesize();
       size_t stack_size = pthread_get_stacksize_np(self);
       size_t max_stack_size = (size_t)(APP_THREAD_STACK_SIZE_MAX + page_size - 1) & ~(page_size - 1);
       fprintf(stdout, "%d.1|stack addr np %s (%p)\n", i, check_aligned(addr), addr);
       fprintf(stdout, "%d.2|page size: %zu\n", i, page_size);
       fprintf(stdout, "%d.3|stack size: %zu\n", i, stack_size);
       fprintf(stdout, "%d.4|max stack size: %zu\n", i, max_stack_size);
       if (addr <= (uint8_t *)&stack_size) addr = addr + stack_size;
       if (stack_size > max_stack_size) stack_size = max_stack_size;
        addr -= stack_size;
        addr += page_size;
        fprintf(stdout, "%d.6|stack boundary %s (%p)\n", i, check_aligned(addr), addr);
        bool ok = mprotect(addr, page_size * STACK_OVERFLOW_CHECK_GUARD_PAGE_COUNT, PROT_NONE) == 0;
        if (ok) fprintf(stdout, "%d.8|mprotect succeeded!\n", i);
        else fprintf(stdout, "%d.8|mprotect failed: %s\n", i, strerror(errno));
   }
*/
import "C"

func main() {
	var wg sync.WaitGroup
	for i := 0; i < 3; i++ {
		i := i
		wg.Add(1)
		go func() {
			C.my_os_thread_get_stack_boundary((C.int32_t)(i))
			wg.Done()
		}()
	}
	wg.Wait()
}

$ go run .|sort
0.1|stack addr np aligned to 4KiB (0x16d7b3000)
0.2|page size: 16384
0.3|stack size: 8384512
0.4|max stack size: 8388608
0.6|stack boundary aligned to 16KiB (0x16cfb8000)
0.8|mprotect succeeded!

1.1|stack addr np aligned to 4KiB (0x16cfab000)
1.2|page size: 16384
1.3|stack size: 8400896
1.4|max stack size: 8388608
1.6|stack boundary aligned to 4KiB (0x16c7af000)
1.8|mprotect failed: Invalid argument

2.1|stack addr np aligned to 16KiB (0x16af8c000)
2.2|page size: 16384
2.3|stack size: 8372224
2.4|max stack size: 8388608
2.6|stack boundary aligned to 16KiB (0x16a794000)
2.8|mprotect succeeded!

[wenyongh]

@benozol wasm_runtime_init or wasm_runtime_full_init should have initialize the thread env of the current thread: wasm_runtime_init -> wasm_runtime_env_init -> runtime_signal_init -> os_thread_signal_init, not sure why it fails. And do you mean the page size in MacOS is 16384? It is a little strange, normally it is 4096?

[benozol]

According to sysctl hw.pagesize (and getpagesize()) the pagesize is 16384 on my Apple M2 (surprisingly sparse information on the internet…). I have updated the repro check_stack_boundary in my comment above and added the output, which shows that the result of pthread_get_stackaddr_np() is aligned to either 16KiB or 4KiB and that subtracting pthread_get_stacksize_np() from the 4KiB-aligned stack addr result in either a 16KiB or 4KiB aligned stack boundary, and the failure occurs in the latter case.