mini_racer: `rb_context_init_unsafe': External function cannot be found.

[deepj]

truffleruby 19.3.0, like ruby 2.6.2, GraalVM CE Native [x86_64-darwin]

To reproduce:

gem install mini_racer
ruby -rmini_racer -e 'context = MiniRacer::Context.new; context.eval "var adder = (a,b)=>a+b"; puts context.eval "adder(20,22)"'

Error:

~/.gem/truffleruby/2.6.2/gems/mini_racer-0.2.8/ext/mini_racer_extension/mini_racer_extension.cc:752:in `rb_context_init_unsafe': External function _ZN2v82V813InitializeICUEPKc cannot be found. (LLVMLinkerException) (RuntimeError)
Translated to internal error
	from ~/.rubies/truffleruby-19.3.0/lib/truffle/truffle/cext_ruby.rb:37:in `init_unsafe'
	from ~/.gem/truffleruby/2.6.2/gems/mini_racer-0.2.8/lib/mini_racer.rb:156:in `initialize'
	from -e:1:in `new'
	from -e:1:in `<main>'

[eregon]

$ echo _ZN2v82V813InitializeICUEPKc | c++filt 
v8::V8::InitializeICU(char const*)

So the v8 library or its functions seem to not be found for some reason.

[SamSaffron]

I think it is something to do with libv8 being c++, the mix of c++ with c appears a bit challenging for truffle.

[eregon]

When I tried to install mini_racer locally, libv8 failed to compile.
Since libv8 seems independent of the Ruby C API/ABI, we should try to allow installing it as a binary gem too on TruffleRuby, but that seems non-trivial since RubyGems doesn't really have good support for that.

The issue might be that libv8 doesn't have bitcode, and that's currently required if the C/C++ extension directly links against it (not though FFI). We should probably relax that in Sulong and allow to use the native version of libv8.

[eregon]

When trying to install libv8 locally with --platform ruby it fails the same on both MRI and TruffleRuby for me locally:

$ gem install --platform ruby libv8
...
Package glib-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `glib-2.0.pc'
to the PKG_CONFIG_PATH environment variable
Package 'glib-2.0', required by 'virtual:world', not found
Package 'gmodule-2.0', required by 'virtual:world', not found
Package 'gobject-2.0', required by 'virtual:world', not found
Package 'gthread-2.0', required by 'virtual:world', not found
Could not run pkg-config.

With means I need to instal GLib 2, and the only reason it works for me locally on MRI is that it's precompiled on MRI.

Maybe we could use the precompiled version on TruffleRuby, but that would need support in Sulong to mix bitcode functions and native functions.

[eregon]

libv8 does compile successfully on TruffleRuby when GLib 2 is installed (on Linux).
It takes a long while to compile though, both on MRI and TruffleRuby:

$ jt gem i libv8
Fetching libv8-7.3.492.27.1.gem
Building native extensions. This could take a while...
Successfully installed libv8-7.3.492.27.1
1 gem installed
3852.65s user 147.18s system 421% cpu 15:48.07 total

$ chruby 2.6.5
$ gem install --platform ruby libv8
Fetching libv8-7.3.492.27.1.gem
Building native extensions. This could take a while...
Successfully installed libv8-7.3.492.27.1
1 gem installed
3795.04s user 144.30s system 414% cpu 15:49.82 total

[deepj]

@eregon not on macOS though :( (yes, GLib 2 is installed via homebrew)

[eregon]

@deepj And gem install --platform ruby libv8 works for you on macOS on MRI 2.6?

[eregon]

BTW, it looks like the libv8 gem uses its own copy of LLVM to compile:

While installing:
$ ps aux | grep clang++
../../third_party/llvm-build/Release+Asserts/bin/clang++

So we can't just set CC/CXX/PATH to compile it with our LLVM toolchain.

mini_racer also installs on Linux but then:

$ ruby -rmini_racer -e 'p MiniRacer::Context.new'
/home/eregon/.rubies/truffleruby-dev/lib/gems/gems/mini_racer-0.2.9/ext/mini_racer_extension/mini_racer_extension.cc:191:in `init_v8': External LLVMFunction _ZN2v82V813InitializeICUEPKc cannot be found. (LLVMLinkerException) (RuntimeError)
Translated to internal error
	from /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/mini_racer-0.2.9/ext/mini_racer_extension/mini_racer_extension.cc:771:in `rb_context_init_unsafe'
	from /home/eregon/.rubies/truffleruby-dev/lib/truffle/truffle/cext_ruby.rb:39:in `init_unsafe'
	from /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/mini_racer-0.2.9/lib/mini_racer.rb:156:in `initialize'
	from -e:1:in `new'
	from -e:1:in `<main>'

Which is the same error as above, v8::V8::InitializeICU(char const*) only exists as native code, not as bitcode which Sulong currently needs.

[deepj]

@eregon hm, no. I guess rubyjs/libv8#280 can be related to this. Though, I have macOS 10.14 (Mojave), not 10.15 (Catalina). And what I remember, older libv8 gm was able to compile with TruffleRuby. Because I could run mini_racer above.

[eregon]

Previously TruffleRuby would install the precompiled libv8, but in general that's unsafe to do, because C extensions could be precompiled against MRI internals (e.g., if they include <ruby.h>), and we'd want to compile against TruffleRuby's headers instead.

However, libv8 doesn't actually include ruby.h so it might be fine to use the precompiled version on TruffleRuby too. Although mini_racer won't work yet (and never did), because Sulong currently requires all functions part of the same dynamic library to be present in bitcode, but libv8 functions are only present as native code.

To make the situation a little better I'll try to get TruffleRuby to install the precompiled version of libv8, since compiling from source doesn't help here, and it seems hard to change the toolchain to build libv8.

So to get mini_racer working we have two options:

• Add support in Sulong for libraries with some bitcode functions and some native functions.
• Create a new "backend" for mini_racer or for execjs that uses GraalVM JavaScript.

It's unclear what's best currently.

[SamSaffron]

My 2 cents, I really have no idea how complex either options are but my strong pref would be to go with adding support in Sulong for libs with some bitcode and some native.

GraalVM JS is going to have gaps compared to v8, v8 also has a big pile of features which people consume that are more around framework of operation like snapshots, memory limits and so on. Discourse leverages some mini racer specific things and does not use execjs at runtime.

[eregon]

To give a bit more context, if libv8 was compiled and used as a dynamic library, that would probably work fine on TruffleRuby.
But it's compiled as a static library (libv8_monolith.a), which means mixing bitcode+native and native-only code in the same library (containing the mini_racer extension + libv8), which is non-trivial to support as symbols from native-only could refer to symbols from the other part and vice-versa.

It looks like libv8 might have support to use a system, I would assume dynamically-linked, libv8 but that doesn't really seem connected to anything: https://github.com/rubyjs/libv8/blob/5fc4ac7aba79e56acd37c46584a1d63a6bc362d0/ext/libv8/location.rb#L54

GraalVM actually contains a full node.js using Graal JavaScript and there is a v8.h header. However if we want to reuse the C extensions as-is and that header, we'd still need mini_racer/therubyracer to link dynamically to it, so it wouldn't really change the situation.

[eregon]

With daea9b7 TruffleRuby will use the precompiled version of libv8, like MRI. So at least it will gem/bundle install quickly and without extra dependencies. It won't work at runtime yet though.

[eregon]

Notes:

• Context on how mini_racer is used in Discourse
• https://github.com/discourse/discourse/blob/bd3c0dd59f9034688d3ab70ff2317d4b910e9513/lib/pretty_text.rb#L67 seems where Discourse loads the Markdown JS library.
• https://github.com/discourse/discourse/blob/bd3c0dd59f9034688d3ab70ff2317d4b910e9513/script/benchmarks/markdown/bench.rb is a benchmark of rendering Markdown.

There are no npm packages or node-specific stuff.
So it might just work on GraalJS and seems worth an experiment.
Compiling libv8 as a shared rather than static library also seems a good way (needs changes in the libv8 and mini_racer gems) and then mini_racer would run on Sulong, and libv8 as native code.

[bjfish]

There has been some progress on running mini_racer with libv8 which we can now compile as a shared library. The next blocker on running this is a Sulong limitation where it does not currently support returning struct-by-value yet at the boundary between Sulong and native code.

[deepj]

@eregon look at rubyjs/libv8#304 if that doesn't have impact on TruffleRuby

[eregon]

A bit more details, the current blocker is struct-by-value support in Sulong (which is being worked on).

Looks like it's StartupData, defined as:

 class V8_EXPORT StartupData {
  public:
   const char* data;
   int raw_size;
 };

in V8.

And CreateBlob returns StartupData: https://v8docs.nodesource.com/node-7.10/d6/d77/classv8_1_1_snapshot_creator.html#a45faf27c079e8d42b75d2799d9d497a5

We could try to execute V8 as bitcode on top of Sulong to workaround that not-yet-implemented feature in Sulong, but that's probably not the best idea.

Looking at https://github.com/discourse/discourse/blob/bd3c0dd59f9034688d3ab70ff2317d4b910e9513/lib/pretty_text.rb#L67
it seems to not rely on too many MiniRacer-specific features, but mostly loading a bunch of .js files.
So when we have an ExecJS backend using Graal.js (we don't currently, but it should be fairly easy to make one, #2169), I think that might be worth a try.

[eregon]

mini_racer 0.6.3 works on TruffleRuby now 🎉