Segmentation fault CFUNC :vips_foreign_find_load #417
Comments
|
I'm new to diagnosing segfaults but I can trigger this one readily so please let me know what things to try to narrow down the issue here. |
|
Removing the |
|
Given that Rails uses Puma (and the stack traces reference it), I wonder if this relates to #155. Does the issue still occur on macOS if you set the Alternatively, you might try the suggestion in #155 (comment). |
|
Thanks @kleisauke, I guess I've just found a way to reliably produce this problem on my machine. Solid Queue is running in standalone mode |
|
Kicking off Such a strange one :) Could this be the year of the Linux Laptop? |
|
I also tried completely removing Puma from the So that rules out Puma. I think it's something deep in the guts of the Vips stack :( Might have to look at assembling a minimum example of the problem. |
|
Could you also reproduce this when setting $ echo "export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" >> ~/.zshenv(you might need to restart any active Zsh sessions after this) |
|
Yes unfortunately that still results in a segfault. I think the fork safety that it bypasses actually allows the segfault to occur instead of the warning. It's just that for most people, they're just lucky and the code runs fine most of the time even though it's unsafe. My situation seems to always break. My guess is it's one of the many dependencies of vips. |
|
Ah, that's a bummer - let's re-open this. I had a look at the ruby-ffi issues and ffi/ffi#864 looked very similar, though that might be a bit outdated now. Does the example at https://github.com/ffi/ffi#synopsis work on your end? $ ruby -e 'require "ffi"; module MyLib; extend FFI::Library; ffi_lib "c"; attach_function :puts, [:string], :int; end; MyLib.puts "Hello, World using libc!"'
Hello, World using libc!Also, just to rule out any fork-safety issues, does this minimal CLI test trigger those $ ruby -e 'if pid = fork; p Process.wait2(pid) else require "vips" end' |
|
Thanks @kleisauke, I'm usually quite self-sufficient in figuring out issues but this one has got a bit above my pay grade so I appreciate the help :) Here's the output from those commands: I found a thread that mentioned a project that used VIPS on startup and that was causing it to load VIPS before the fork. I wonder if I'm somehow doing that unwittingly. Definitely not processing images on boot though! :) |
|
Thanks! The output looks as expected. It's a bit odd that the segfault is happening in This might be a long shot, but could you try temporarily renaming the |
|
Looking into this further, it may end up calling To debug the dynamic loadable modules, the |
|
Ah interesting :) Renaming the vips-modules directory did the trick. It works with:
Preloading vips with I'll dig into the modules and try to figure out the culprit. |
|
Ah! Found it. It's |
|
So on homebrew, Not sure where I should open an issue from here. Possibly the |
|
Thanks for tracking that down! I suspect the issue lies somewhere in Poppler (or one of its dependencies). I also noticed Poppler depends on cURL, which also had issues with Could you provide a standalone reproducer for this? I'll attempt to reproduce this on a Apple Mac mini M1 (2020) provided by Scaleway, as I currently don't have access to any other macOS device.
I think we can track it here, given that this only occurs when using Rails (possibly due to its use of |
|
Thanks @kleisauke, that's no worries at all :) I've created a private repo here: https://github.com/brendon/poppler-test and have added you as a contributor. It's a bare bones Rails app with an sqlite database. Interestingly when I was setting up Active Storage I couldn't produce the segfault when using the local storage engine so I had to set up a temporary R2 bucket for this. That seems to point more clearly to curl possibly being the problem as it's figured by a fetch of the file from a remote location. Here's the steps to reproduce the segfault: Check out the project then: In one terminal tab: In another tab: Then run the following in the console
I added |
|
I couldn't reproduce the issue with Ruby 3.3.7 or 3.4.3 installed using rbenv on a Apple Mac mini M1 provided by Scaleway, running macOS 15.2. The image was successfully uploaded to the Cloudflare R2 bucket. The good news is that those fork-safety errors can also be resolved by using the linker flags I suggested in puma/puma#1421 (comment). $ rbenv install 3.4.3 -- LDFLAGS="-Wl,-sectcreate,__DATA_CONST,__objc_imageinfo,/dev/null -Wl,-sectcreate,__DATA,__objc_fork_ok,/dev/null"
$ rbenv local 3.4.3
$ curl -LO https://wsrv.nl/zebra.jpg
$ ruby -e 'if pid = fork; p Process.wait2(pid) else require "vips"; p Vips.vips_foreign_find_load "zebra.jpg" end'
"VipsForeignLoadJpegFile"
[17702, #<Process::Status: pid 17702 exit 0>] |
|
Let me try reinstalling that Mac mini with macOS Sequoia 15.3.1 (which takes around two hours to complete 😅). Here's all version info of that successful run: Details$ ruby --version
ruby 3.4.3 (2025-04-14 revision d0b7e5b6a0) +PRISM [arm64-darwin24]
$ rails --version
Rails 8.0.2
$ brew --version
Homebrew 4.4.30
$ brew list --installed-on-request
rbenv
vips
$ curl -V
curl 8.7.1 (x86_64-apple-darwin24.0) libcurl/8.7.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.63.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe UnixSockets
$ system_profiler SPSoftwareDataType
Software:
System Software Overview:
System Version: macOS 15.2 (24C101)
Kernel Version: Darwin 24.2.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Computer Name: e075e6e6-8703-4ea3-b0b8-429b5e3db09a
User Name: m1 (m1)
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled
Time since boot: 1 day, 27 minutes
$ gem list
*** LOCAL GEMS ***
actioncable (8.0.2)
actionmailbox (8.0.2)
actionmailer (8.0.2)
actionpack (8.0.2)
actiontext (8.0.2)
actionview (8.0.2)
activejob (8.0.2)
activemodel (8.0.2)
activerecord (8.0.2)
activestorage (8.0.2)
activesupport (8.0.2)
addressable (2.8.7)
ast (2.4.3)
aws-eventstream (1.3.2)
aws-partitions (1.1087.0)
aws-sdk-core (3.222.1)
aws-sdk-kms (1.99.0)
aws-sdk-s3 (1.183.0)
aws-sigv4 (1.11.0)
base64 (0.2.0)
bcrypt_pbkdf (1.1.1)
benchmark (0.4.0)
bigdecimal (3.1.9)
bindex (0.8.1)
bootsnap (1.18.4)
brakeman (7.0.2)
builder (3.3.0)
bundler (2.6.8, default: 2.6.7, 2.5.22)
capybara (3.40.0)
cgi (default: 0.4.2)
concurrent-ruby (1.3.5)
connection_pool (2.5.0)
crass (1.0.6)
date (3.4.1)
debug (1.10.0)
delegate (default: 0.4.0)
did_you_mean (default: 2.0.0)
digest (default: 3.2.0)
dotenv (3.1.8)
drb (2.2.1)
ed25519 (1.3.0)
english (default: 0.8.0)
erb (default: 4.0.4)
error_highlight (default: 0.7.0)
erubi (1.13.1)
et-orbi (1.2.11)
etc (default: 1.4.5)
fcntl (default: 1.2.0)
ffi (1.17.1 arm64-darwin)
fiddle (default: 1.1.6)
fileutils (default: 1.7.3)
find (default: 0.2.0)
forwardable (default: 1.3.3)
fugit (1.11.1)
globalid (1.2.1)
i18n (1.14.7)
image_processing (1.14.0)
importmap-rails (2.1.0)
io-console (0.8.0)
io-nonblock (default: 0.3.1)
io-wait (default: 0.3.1)
ipaddr (default: 1.2.7)
irb (1.15.2, default: 1.14.3)
jbuilder (2.13.0)
jmespath (1.6.2)
json (2.10.2, default: 2.9.1)
kamal (2.5.3)
language_server-protocol (3.17.0.4)
lint_roller (1.1.0)
logger (1.7.0, default: 1.6.4)
loofah (2.24.0)
mail (2.8.1)
marcel (1.0.4)
matrix (0.4.2)
mini_magick (5.2.0)
mini_mime (1.1.5)
minitest (5.25.5)
msgpack (1.8.0)
net-http (default: 0.6.0)
net-imap (0.5.6)
net-pop (0.1.2)
net-protocol (0.2.2)
net-scp (4.1.0)
net-sftp (4.0.0)
net-smtp (0.5.1)
net-ssh (7.3.0)
nio4r (2.7.4)
nokogiri (1.18.7 arm64-darwin)
open-uri (default: 0.5.0)
open3 (default: 0.2.1)
openssl (default: 3.3.0)
optparse (default: 0.6.0)
ostruct (0.6.1)
parallel (1.27.0)
parser (3.3.8.0)
pathname (default: 0.4.0)
pp (0.6.2)
prettyprint (0.2.0)
prism (1.4.0, default: 1.2.0)
propshaft (1.1.0)
pstore (default: 0.1.4)
psych (5.2.3, default: 5.2.2)
public_suffix (6.0.1)
puma (6.6.0)
raabro (1.4.0)
racc (1.8.1)
rack (3.1.13)
rack-session (2.1.0)
rack-test (2.2.0)
rackup (2.2.1)
rails (8.0.2)
rails-dom-testing (2.2.0)
rails-html-sanitizer (1.6.2)
railties (8.0.2)
rainbow (3.1.1)
rake (13.2.1)
rdoc (6.13.1, default: 6.10.0)
readline (default: 0.0.4)
regexp_parser (2.10.0)
reline (0.6.1, default: 0.6.0)
resolv (default: 0.6.0)
rexml (3.4.1)
rubocop (1.75.2)
rubocop-ast (1.44.1)
rubocop-performance (1.25.0)
rubocop-rails (2.31.0)
rubocop-rails-omakase (1.1.0)
ruby-progressbar (1.13.0)
ruby-vips (2.2.3)
ruby2_keywords (default: 0.0.5)
rubyzip (2.4.1)
securerandom (0.4.1)
selenium-webdriver (4.31.0)
set (default: 1.1.1)
shellwords (default: 0.2.2)
singleton (default: 0.3.0)
solid_cable (3.0.7)
solid_cache (1.0.7)
solid_queue (1.1.4)
sqlite3 (2.6.0 arm64-darwin)
sshkit (1.24.0)
stimulus-rails (1.3.4)
stringio (3.1.6, default: 3.1.2)
strscan (default: 3.1.2)
syntax_suggest (default: 2.0.2)
tempfile (default: 0.3.1)
thor (1.3.2)
thruster (0.1.12 arm64-darwin)
time (default: 0.4.1)
timeout (0.4.3)
tmpdir (default: 0.3.1)
tsort (default: 0.2.0)
turbo-rails (2.0.13)
tzinfo (2.0.6)
un (default: 0.3.0)
unicode-display_width (3.1.4)
unicode-emoji (4.0.4)
uri (1.0.3)
useragent (0.16.11)
weakref (default: 0.1.3)
web-console (4.2.1)
websocket (1.2.11)
websocket-driver (0.7.7)
websocket-extensions (0.1.5)
xpath (3.2.0)
yaml (default: 0.4.0)
zeitwerk (2.7.2)
zlib (default: 3.2.1)I'll also try to install cURL via Homebrew and override the system version in that run - perhaps that'll reproduce it. |
|
I could also not reproduce this on macOS Sequoia 15.3.1, see: Details$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ echo >> ~/.zprofile
$ echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile
$ echo 'eval "$(/opt/homebrew/bin/brew shellenv)"'
$ brew install vips rbenv
$ cd Downloads/poppler-test-main
$ rbenv init
$ rbenv install $(cat .ruby-version)
$ rbenv global $(cat .ruby-version)
$ <Launch new terminal instance>
$ cd Downloads/poppler-test-main
$ bundle
$ gem install rails
$ rbenv rehash
$ rails db:prepare
$ <Launch new terminal instance>
$ cd Downloads/poppler-test-main
$ bin/jobs
$ <Launch new terminal instance>
$ cd Downloads/poppler-test-main
$ rails c
Loading development environment (Rails 8.0.2)
poppler-test(dev)> file = Rails.root.join("public", "image.jpg")
poppler-test(dev)>
poppler-test(dev)* blob = ActiveStorage::Blob.create_and_upload!(
poppler-test(dev)* io: file.open,
poppler-test(dev)* filename: file.basename.to_s
poppler-test(dev)* )
poppler-test(dev)>
poppler-test(dev)> blob.analyze_later
<snip />
byte_size: 794589,
<snip />
@successfully_enqueued=true,
<snip />(and the command running
This also doesn't reproduce. I tried doing: $ brew install curl
$ echo 'export PATH="/opt/homebrew/opt/curl/bin:$PATH"' >> ~/.zshrc
$ echo 'export PKG_CONFIG_PATH="/opt/homebrew/opt/curl/lib/pkgconfig"' >> ~/.zshrc
$ export PATH="/opt/homebrew/opt/curl/bin:$PATH"
$ export PKG_CONFIG_PATH="/opt/homebrew/opt/curl/lib/pkgconfig"
$ curl -V
curl 8.13.0 (aarch64-apple-darwin24.2.0) libcurl/8.13.0 OpenSSL/3.4.1 (SecureTransport) zlib/1.2.12 brotli/1.1.0 zstd/1.5.7 AppleIDN libssh2/1.11.1 nghttp2/1.65.0 librtmp/2.3
Release-Date: 2025-04-02
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd(and repeat the steps above) |
|
Thank you @kleisauke, that's some great detail. Looks like I have potentially got a buggy package installed here via homebrew? I'll compare things and see what I can come up with. |
|
I took a deep dive into what I had installed via homebrew: Interestingly a lot of the dependencies that vips has were listed as 'installed on request' meaning I installed them manually. This might just mean that I was using homebrew before this feature was introduced. Anyway, I began uninstalling things that I knew I didn't use anymore and following the dependency graphs uninstalling the packages that depended on those packages. Vips brings a lot along with it. Every now and again I'd reinstall vips and test my environment out, on about the 3rd or 4th try it worked! I suspect the problem was an out of date dependency (even though I ran Now when I uninstall vips it takes all of its dependencies with it. It's mildly unsatisfying as I can't pinpoint which package caused the problem, but clearly the solution is to ensure all decencies of vips are up to date. Thank you again for taking the time to verify that this wasn't a general problem in the Mac environment. I hope the thread helps anyone else that ends up in my position. If there's a way I can "buy you a beer" or a coffee let me know :D |
|
I'm glad it's working! And good job Kleis. Yes, I wipe and reinstall homebrew occasionally. They have a policy of always enabling all package dependencies, so things like libvips do pull a lot of things in too. (libvips has a "donate" button on the main website, fwiw) |
|
Thanks @jcupitt, yea it was a bit of a mess. Nice to have it tidied up now. I don't see a donate button there? |
|
Top right, it has a ❤️ on the button. |
|
Oh! I see, the GitHub project not the actual libvips website :) |
|
We're having a dev meet in London next week, on the off chance you're in the city. |
|
Oh I wish! :D I couldn't be further away in New Zealand :D Enjoy though! |
Describe the bug
Segfault:
I've attached the full segfault log and the diagnostics report.
segfault.txt
ruby-2025-04-11-180731.ips.txt
To Reproduce
Simply creating a new Blob with ActiveStorage:
This is the file:
But I don't think it's the file. There are many different images that cause this. I was initially getting
[__NSCFConstantString initialize] may have been in progress in another thread when fork() was callederrors but managed to work around these by requiringvipsearly: rails/rails#38560 (comment)Expected behavior
It should not segfault.
Desktop (please complete the following information):
Additional context
I've tried updating homebrew vips to the latest version.
The text was updated successfully, but these errors were encountered: