Explain details of compile

[guest271314]

I'm running a live Linux USB. If/when that temporary file system freezes during one of my experiments I lose that live session. I store my experiments and applications on a different USB.

Every time I use wasmtime compile file.wasm that produces f.cwasm, and I lose the current filesystem, when I launch another live Linux session wasmtime file.cwasm fails to run.

That behaviour reminds me of how

bun build --compile

and

npx postject hello NODE_SEA_BLOB sea-prep.blob \
    --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2

work to compile executables. When the source file system is no more the executable doesn't work anymore. The executable expectes the source files to be referenced on the file system.

COuld be something else in wasmtime. Don't know. I know what I describe above has happened multiple times when I try to run that same .,cwasm file produced by wasmtime that executed before I didn't have the exact same file system to execute that file on.

What's going on with the output of wasmtime compile here?

[alexcrichton]

Would you be able to paste any details of snippets or commands you're running? You refer to f.cwasm as the output of wasmtime compile file.wasm but then you later refer to file.cwasm (note f vs file in the filename). I mostly want to confirm there's not a typo in the commands that you're running.

Otherwise to answer your question here: no, a *.cwasm artifact does not depend on the filesystem in any way and can be transferred to another machine as-is and executed there. I'm not familiar with either npx or bun but your file.cwasm produced should be a standalone file disconnected from anything else on the filesystem. The only requirement is that it's loaded into a compatible version of wasmtime-the-CLI. If you're using the same binary though that shouldn't be a concern.

[guest271314]

Soource code https://github.com/guest271314/native-messaging-webassembly/blob/main/nm_javy.js

javy compile nm_javy.js -o nm_javy.wasm

https://github.com/guest271314/native-messaging-webassembly/blob/main/nm_javy.js

chmod +x doesn't work. I have to recompile every I reboot a new live Linux sesssion. I don't have to do that with the C source code compiled to WASM in the same repository.

[guest271314]

This is how the binary is executed https://github.com/guest271314/native-messaging-webassembly/blob/main/nm_c_wasm.sh

#!/usr/bin/env -S /home/user/1234/native-messaging-webassembly/wasmtime -C cache=n --allow-precompiled /home/user/1234/native-messaging-webassembly/nm_javy.cwasm

[guest271314]

@alexcrichton I realize it's perhaps a bit much to reproduce exactly the steps I describe above. Just something I noticed and shared here.

This is what I'm talking about re bun build and Node.js's implementation of their SEA; cf. deno compile Compiling a standalone executable using modern JavaScript/TypeScript runtimes, Compiling npm to a standalone executable: Which runtime can do this out of the box; node, deno, or bun?, bun build --compile does not produce a standalone executable - still depends on external files #14676.

Undoubtedly this current live Linux session I'm running now will freeze up at some point then I'll have the nm_javy.cwasm file and the same wasmtime executable I have now. So perhaps at that point the file can be disaaembled and analyzed.

The reason I know that this is happening with wasmtime and that .cwasm file is when I fetch Chromium when I start a new live Linux session I run a file that lists the time each programming language or JavaScript engine or runtime takes to read stdin and write to stdout. I always have to recompile that .cwasm file produced by wasmtime. Same wasmtime executable stored on an external USB copied to the new temporary file system operating in RAM. Same .cwasm file.

0	'nm_wasm'	0
1	'nm_qjs'	0.11310000002384185
2	'nm_c'	0.11989999997615815
3	'nm_rust'	0.12069999998807907
4	'nm_cpp'	0.129
5	'nm_tjs'	0.2415999999642372
6	'nm_deno'	0.27840000003576276
7	'nm_bun'	0.29010000002384184
8	'nm_nodejs'	0.366
9	'nm_python'	0.41009999996423724
10	'nm_d8'	0.4128999999761581
11	'nm_typescript'	0.4516000000238419
12	'nm_spidermonkey'	0.4697000000476837
13	'nm_llrt'	0.844

[alexcrichton]

Ideally to help investigate this further on our end we'd need information such as steps to reproduce (e.g. exact commands, but we probably don't need to reproduce your entire environment exactly) along with logs/errors/etc that you're seeing. Without that the best we can to is piece together this from what you've listed here.

I can again reiterate though that *.cwasm files don't depend on external files. It sounds like you're running into a situation where you think that they do, but without error messages or a means of reproduction it's hard to say much more.

[guest271314]

The linked repository includes complete source code and instructions. I'm using wasmtime and the .cwasm file as a Native Messaging host https://github.com/guest271314/NativeMessagingHosts#native-messaging-documentation. Here's the test that generates the table data https://gist.github.com/guest271314/7d2118bd395bf1e3475b70b0187910f2.

The running a live Linux USB/CD is perhaps somewhat involved, though possible.

I don't know why what I describe happens.

[cfallin]

@guest271314 Alex had asked for

along with logs/errors/etc that you're seeing.

You say only "fails to run"; could you show us what the actual error is?

Given that we know how our system is meant to operate -- .cwasm files are independent artifacts, and the same wasmtime binary running the same cwasm file should either work or not work deterministically -- my first suspicion would be something about your setup that is corrupting the file. Can you take a hash (e.g. sha256) of the file when it works and when it doesn't, as well as a hash of the wasmtime binary that you're using in both cases?

[guest271314]

@cfallin

Here's the current .cwasm file that is now working after re-compilation, and the original .wasm file. The regular .wasm file works as expected when I boot a new live Linux session. The .cwasm file doesn't.

I understand how you expect the compilation to work. That's what I expect, too.

Nonetheless I've encountered similar cases in different runtimes that also expected the same, and people have filed bugs to fix what is broken. Not saying that's what's going on here.

I don't look forward to my system freezing and starting a new live Linux session,. Though I've done it a few thousand times now. I have a startup script that basically gets me right back to where I am now.

The only way for me to reproduce this is to reboot the machine and run the code.

Would that help you analyze what's going on here?

[guest271314]

wasm-cwasm.tar.gz

[guest271314]

@cfallin Alright, here's what connecting to the working .cwasm file WebAssembly Native Messaging host using this shell script

javy compile nm_javy.js -o nm_javy.wasm

wasmtime compile --optimize opt-level=s nm_javy.wasm

#!/usr/bin/env -S /home/user/native-messaging-webassembly/wasmtime -C cache=n --allow-precompiled /home/user/native-messaging-webassembly/nm_javy.cwasm

looks like from the browser.

.

wasmtime 28.0.0 (2e584e841 2024-12-20)

I want to make I'm checking everything when I turn off and turn back on this machine.

• Make sure the .sh file is executable
• Make sure wasmtime is executable
• Make sure it's not the test script I'm running, so I'll just repeat what I posted above directly, without connecting to all the hosts I have in that script

var runtimes = new Map([
  ["nm_nodejs", 0],
  ["nm_deno", 0],
  ["nm_bun", 0],
  ["nm_tjs", 0],
  ["nm_qjs", 0],
  ["nm_spidermonkey", 0],
  ["nm_d8", 0],
  ["nm_typescript", 0],
  ["nm_llrt", 0],
  ["nm_rust", 0],
  ["nm_cpp", 0],
  ["nm_c", 0],
  ["nm_python", 0],
  ["nm_wasm", 0],
  ["nm_c", 0],
  ["nm_cpp", 0]


]);
for (const [runtime] of runtimes) {
  try {
    const {
      resolve,
      reject,
      promise
    } = Promise.withResolvers();
    const now = performance.now();
    const port = chrome.runtime.connectNative(runtime);
    port.onMessage.addListener((message) => {
      console.assert(message.length === 209715, {message, runtime});
      runtimes.set(runtime, (performance.now() - now) / 1000);
      port.disconnect();
      resolve();
    });
    port.onDisconnect.addListener(() => reject(chrome.runtime.lastError));
    port.postMessage(new Array(209715));
    if (runtime === "nm_spidermonkey") {
      port.postMessage("\r\n\r\n");
    }
    await promise;
  } catch (e) {
    console.log(e, runtime);
    continue;
  }
}
var sorted = [...runtimes].sort(([, a], [, b]) => a < b ? -1 : a === b ? 0 : 1);
console.table(sorted);

I don't think I'm missing any particulars about what I need to do after I reboot and test from Chromium Dev Channel.

Anything particular I should do that I didn't list above?

[cfallin]

Hi @guest271314 -- unfortunately you didn't provide the information that I asked for. To reiterate from above:

You say only "fails to run"; could you show us what the actual error is?

In other words, please copy and paste from a shell session where you run wasmtime and see that it cannot run a .cwasm file that you expect to be runnable and that was previously runnable. Please copy and paste exactly the output, including the command. To reduce unknowns, I would ask you to please try to reproduce this from the command line: you appear to have a quite custom runner setup involving a JavaScript top-level and a browser with subprocesses. To determine whether the problem is Wasmtime and narrow it down if so, let's stick to the CLI.

Can you take a hash (e.g. sha256) of the file when it works and when it doesn't, as well as a hash of the wasmtime binary that you're using in both cases?

Please copy and paste the hash of the wasmtime binary and your cwasm file both in the case that execution works and the case that execution doesn't, to verify that they are the same.

I cannot offer any more advice until you provide the information above. Thanks!

[guest271314]

In other words, please copy and paste from a shell session where you run wasmtime and see that it cannot run a .cwasm file

I did. I posted code and screenshots of Chromium browser DevTools => Sources => Snippets where I run the code from.

Native Messaging protocol (see https://github.com/guest271314/NativeMessagingHosts?tab=readme-ov-file#native-messaging-documentation) provides a communication channel over IPC between the browser and native applications. So the screenshot is literally be executing

#!/usr/bin/env -S /home/user/native-messaging-webassembly/wasmtime -C cache=n --allow-precompiled /homeuser/native-messaging-webassembly/nm_javy.cwasm

from the browser. Sending JSON, getting JSON back in the form of a plain JavaScript object in the browser.

In the above screenshot I'm benchmarking C, C++, Rust, JavaScript, Python, WASM in the form of JavaScript compiled to WASM with javy then .wasm compiled to .cwasm with wasmtime , et al. Native Messaging hosts. See https://github.com/guest271314/NativeMessagingHosts.

The benchmark is testing the speed to send 1 MB from the browser console to the Native Messaging hosts, and the hosts sending back 1 MB back to the browser.

In other Native Messaging hosts I do other stuff, like stream real-time audio to the browser from the native application.

Above I'm just testing the relative speed of different JavaScript runtimes and programming languages, and WASM, and C and C++ and JavaScript compiled to WASM, and that WASM compiled to .cwasm - because I've observed the file size is greater, though the .cwasm is faster then executing the original .wasm file.

Compare the first using .cwasm, second using .wasm.

Got it working.

Recompiled during previous session.

Made sure I am using newest wasmtime.

Turned off machine.

Turned machine back on. Ran startup script. Tested on Chromium Version 134.0.6949.0 (Developer Build) (64-bit).

Thanks.