Get common C-extensions to declare rb_ext_ractor_safe(true)

[mohamedhafez]

In planning ahead to the near future when TruffleRuby can run C-extensions marked with rb_ext_ractor_safe(true) in parallel, I've been raising issues in common gems to try to nudge them to make their C-extensions Ractor-safe (most already are), and then submitting PR's to mark them as rb_ext_ractor_safe(true). This issue will track the progress of all those issues.

Still need to be reviewed for Ractor-safety

• ripper (TruffleRuby might move to an implementation based on prism so this may not be necessary. But leaving this here until it's addressed one way or the other)

---

Gems that will need some work before we can set rb_ext_ractor_safe(true)

• [feature request] Make C-extension threadsafe/Ractor-safe sparklemotion/nokogiri#3283 Possibly some changes needed to conditionally use threadsafe features of libxml if it's using a new enough version. Testing with --cexts-lock=false doesn't have any issues on my app personally though, so a promising sign at least

---

Reviewed for Ractor-safety, waiting for PR
none currently

---

Reviewed for Ractor-safety, PR submitted
none currently

---

PR merged! 🎉

• Add test for RbConfig Ractor safety ruby/ruby#11762 (rbconfig-sizeof)
• Mark as ractor-safe bcrypt-ruby/bcrypt-ruby#280
• Mark as ractor-safe trilogy-libraries/trilogy#200
• Mark as ractor-safe puma/puma#3486
• Mark as ractor-safe socketry/nio4r#320
• Mark as ractor-safe ruby/rbs#2041

---

No interest shown from maintainers, and not very important anyway

• Make C-extension Ractor-safe ruby/syslog#10
• Make C-extension Ractor-safe brianmario/mysql2#1374 (we already have trilogy, TruffleRuby users can always migrate to that)

[flavorjones]

👋 Nokogiri maintainer here, I'm a Truffleruby contributor and supporter. Nokogiri is a large library, but I would like to make it better support ractors, so please consider my request in that issue to be: what specific use cases should we start with? If you're a user of ractors I'd love to chat with you about your concrete needs.

[mohamedhafez]

@flavorjones So I'm not using Ractors, but in JRuby I have multiple threads repeatedly scraping the account pages of substitute teachers looking for newly posted available jobs at their school district (with the explicit, contractual permission of all parties involved). Each thread has it's own Nokogiri object, and is processing different document objects. I want each thread to be able to run with true concurrency, in parallel on different cores of a CPU, without locking globally each time a call is made into the Nokogiri C-extension (as is currently the case when I run the code with TruffleRuby).

In C-Ruby, I would be trying to accomplish the exact same thing but with Ractors instead of threads. However that is not possible because Nokogiri objects can't be used inside of a Ractor currently and throws an error if you try (see sparklemotion/nokogiri#3283 (comment))

[mohamedhafez]

@eregon did TruffleRuby end up moving to a ripper implementation based on prism, or is there more clarity about whether such a move is planned? Just wondering if I can remove it from the checklist.

Also just wanted to verify that running rb_ext_ractor_safe(true) c-extensions in parallel is still on the roadmap!

[eregon]

did TruffleRuby end up moving to a ripper implementation based on prism

No, it didn't, we tried but we found it rather incomplete. It needs changes in Prism.

Also just wanted to verify that running rb_ext_ractor_safe(true) c-extensions in parallel is still on the roadmap!

Yes, I have a WIP PR for it: #3814

[mohamedhafez]

It looks like ActionView uses ripper in Rails < 8.0 (e.g. https://github.com/rails/rails/blob/v7.2.2.1/railties/lib/rails/source_annotation_extractor.rb), i believe as part of the process of rendering ERB files, so perhaps it's worth the effort to review that for Ractor safety and get it marked as Ractor-safe if possible?

[eregon]

I don't know if you have seen but #2136 is done now, so truffleruby-dev can run C extensions in parallel.

Regarding nkf and debug I think it doesn't really matter, those can execute with a global lock.

[mohamedhafez]

@eregon I did see it! 🙌🙌 Amazing news, excited to give things another try when the next release comes around!!

Ok so regarding this issue: I've removed debug, but regarding nkf, my understanding is that while getting it to be Ractor-safe would require upstream fixes that are unlikely to happen, we might already be at the point we can declare rb_ext_thread_safe(true), right? If so, maybe I should change the description of this issue to be "Get common C-extensions to declare rb_ext_ractor_safe OR rb_ext_thread_safe", since either would accomplish what we want here?

[eregon]

excited to give things another try when the next release comes around!!

It would be useful if you try before the next release, e.g. with truffleruby-dev, so if there would be any issue there is time to fix it before the release.

regarding nkf, my understanding is that while getting it to be Ractor-safe would require upstream fixes that are unlikely to happen, we might already be at the point we can declare rb_ext_thread_safe(true), right?

I'm not sure, we would need to look in more details at nkf and that's not really a priority, part of the idea is that C extensions that are not really maintained and are not ractor/thread-safe like nkf just run with a lock and that's fine.
An example is some methods in Etc are not Ractor-safe, and running with a lock is fine and likely as fast as manually locking in that extension.
It doesn't impact other extensions which are marked as rb_ext_ractor_safe() or rb_ext_thread_safe(), those still run in parallel.
So it would only matter if your application spends a lot of time in a non-ractor-safe non-thread-safe extension, which seems unlikely for nkf.

[mohamedhafez]

Sure, that's understandable if nkf isn't used much and isn't well maintained (my app uses it on the critical path but I'm pretty sure I can work around that for my specific case). I've removed it from this issue. Should I remove syslog as well from here in the same vein?

There also doesn't seem to be any interest from the maintainers for brianmario/mysql2#1374, so maybe I should remove mysql2 as well since users can just use trilogy instead, which did accept a patch?

[eregon]

I think there is no harm to leave those issues opened and linked here, so fine as-is.
Maybe just under a different section in the description to make it clearer they are not so important?

[mohamedhafez]

@eregon done! So i guess the only thing really left to do here for TruffleRuby maintainers is threadsafety for ripper, since that's used in .erb rendering in Rails it might actually be pretty important to do i'm guessing?