[GR-62554] Backport to 24.2: Update to Ruby 3.3.7

PullRequest: truffleruby/4489

Author: andrykonchin

.ruby-version

@@ -1 +1 @@
-3.3.5
+3.3.7

CHANGELOG.md

@@ -1,3 +1,24 @@
+# 24.2.1
+
+New features:
+
+
+Bug fixes:
+
+
+Compatibility:
+
+* Updated to Ruby 3.3.7 (@andrykonchin).
+
+Performance:
+
+
+Changes:
+
+
+Memory Footprint:
+
+
 # 24.2.0
 
 New features:

README.md

@@ -119,7 +119,7 @@ See the [contributor workflow](doc/contributor/workflow.md) document if you wish
 We recommend that people trying TruffleRuby on their gems and applications [get in touch with us](#contact) for any help they might need.
 
 TruffleRuby runs Rails and is compatible with many gems, including C extensions.
-TruffleRuby is not 100% compatible with MRI 3.2 yet. Please [report](https://github.com/oracle/truffleruby/issues) any compatibility issues you might find.
+TruffleRuby is not 100% compatible with MRI 3.3 yet. Please [report](https://github.com/oracle/truffleruby/issues) any compatibility issues you might find.
 TruffleRuby [passes around 97% of ruby/spec](https://eregon.me/rubyspec-stats/),
 more than any other alternative Ruby implementation.
 
@@ -193,4 +193,4 @@ For further licensing information, see [LICENCE](LICENCE.md), [3rd_party_license
 
 ## Attribution
 
-TruffleRuby includes infrastructure code from [JRuby](https://github.com/jruby/jruby) (e.g. parser, JCodings, Joni), core library code from the [Rubinius](https://github.com/rubinius/rubinius) project, as well as code from the standard implementation of Ruby, [MRI](https://github.com/ruby/ruby).
+TruffleRuby includes infrastructure code from [JRuby](https://github.com/jruby/jruby) (e.g. JCodings, Joni), core library code from the [Rubinius](https://github.com/rubinius/rubinius) project, as well as code from the standard implementation of Ruby, [MRI](https://github.com/ruby/ruby).

doc/contributor/updating-ruby.md

@@ -27,13 +27,43 @@ To update a specific default gem to a newer version than in the MRI release, run
 ```
 cd ruby
 git checkout -b truffleruby-updates-$VERSION vn_n_n
+cd ..
+git clone git@github.com:ruby/<gem>.git
+cd <gem>
+git checkout v<gem-version>
+cd ../ruby
 ruby tool/sync_default_gems.rb $GEM
 
 git push -u eregon HEAD
 ```
 to update the default gem in MRI.
 Then follow the instructions below to reimport MRI files and to update default gems.
 
+Another way to update a gem to a specific version is to pass a commits range:
+
+```
+ruby tool/sync_default_gems.rb <gem-name> <sha1-of-currently-imported-version>..<sha1-of-a-new-version>
+```
+
+Run `ruby tool/sync_default_gems.rb --help` to get other ways to run the script.
+
+## Updating a specific bundled gem
+
+Set new version of the bundled gem in `gems/bundled_gems` in a Ruby source code directory.
+
+Build Ruby (currently supported version) with the following commands:
+
+```
+./autogen.sh
+mkdir ~/.rubies
+./configure --prefix="${HOME}/.rubies/ruby-$VERSION-updated-gems" --disable-install-doc
+make -j8 install-nodoc
+```
+
+Use `~/.rubies/ruby-$VERSION-updated-gems` directory as a source for copying a new bundled gem version.
+
+Follow instructions how to update default and bundled gems.
+
 ## Setup
 
 Set the environment variable `$VERSION` to the target version:
@@ -165,8 +195,9 @@ Update all of these:
 
 * Update `.ruby-version`, `TruffleRuby.LANGUAGE_VERSION`
 * Reset `truffleruby-abi-version.h` to `$RUBY_VERSION.1` and `lib/cext/ABI_check.txt` to `1` if `RUBY_VERSION` was updated.
+  * use `$RUBY_VERSION.$GRAALVM_VERSION.1` for releases in `truffleruby-abi-version.h`
 * Update `versions.json`
-  * with bundled gem versions provided by `cat ../ruby/gems/bundled_gems | sort`,
+  * with bundled gem versions provided by `cat ../ruby/gems/bundled_gems | sort` (ensure the target Ruby version is checked out in the `../ruby` directory),
   * default gem versions provided by `ls -l lib/gems/specifications/default`
   * and `gem` gem version provided by `grep 'VERSION =' lib/mri/rubygems.rb`
 * Also update version numbers for `debug`, `racc`, and `rbs` in `src/main/c/Makefile`, `mx.truffleruby/suite.py` and `lib/gems/gems/debug-*/ext/debug/extconf.rb`.
@@ -175,9 +206,11 @@ Update all of these:
 * Copy and paste the TruffleRuby `--help` output to `doc/user/options.md` (e.g., with `jt ruby --help | xsel -b`)
 * Update `doc/user/compatibility.md` and `README.md`
 * Update `doc/legal/legal.md`, notably the `Bundled gems` section
-* Update method lists - see `spec/truffle/methods_spec.rb`
+* Update method lists (see `spec/truffle/methods_spec.rb`)
+  * run `jt -u ruby test spec/truffle/methods_spec.rb` to add new methods
+  * run `jt purge spec/truffle/methods_spec.rb` to remove tags for implemented methods
 * Build TruffleRuby (`jt build`).
-* Run `jt test gems default-bundled-gems`
+* Run `jt test gems default-bundled-gems` and commit generated `Gemfile` and `Gemfile.lock` files
 * Get `jt test spec/truffle/rubygems/default_gems_list_spec.rb` to pass
 * Grep for the old Ruby version with `git grep -F x.y.z`
 * Grep for the old Bundler version with `git grep -F x.y.z`

doc/legal/bundled_gems

@@ -9,15 +9,15 @@ minitest        5.20.0  https://github.com/minitest/minitest
 power_assert    2.0.3   https://github.com/ruby/power_assert
 rake            13.1.0  https://github.com/ruby/rake
 test-unit       3.6.1   https://github.com/test-unit/test-unit
-rexml           3.3.6   https://github.com/ruby/rexml
+rexml           3.3.9   https://github.com/ruby/rexml
 rss             0.3.1   https://github.com/ruby/rss
 net-ftp         0.3.4   https://github.com/ruby/net-ftp
-net-imap        0.4.9.1   https://github.com/ruby/net-imap
+net-imap        0.4.19   https://github.com/ruby/net-imap
 net-pop         0.1.2   https://github.com/ruby/net-pop
 net-smtp        0.4.0.1   https://github.com/ruby/net-smtp
 matrix          0.4.2   https://github.com/ruby/matrix
 prime           0.1.2   https://github.com/ruby/prime
 rbs             3.4.0   https://github.com/ruby/rbs
 typeprof        0.21.9  https://github.com/ruby/typeprof
-debug           1.9.1   https://github.com/ruby/debug
+debug           1.9.2   https://github.com/ruby/debug
 racc            1.7.3   https://github.com/ruby/racc

doc/legal/legal.md

@@ -18,7 +18,7 @@ See `epl-2.0.txt`, `gpl-2.txt`, `lgpl-2.1.txt`.
 ## MRI
 
 The standard implementation of Ruby is MRI. TruffleRuby contains code from MRI
-version 3.3.5, including:
+version 3.3.7, including:
 
 * the standard library in `lib/mri`, 
 * Ruby C extension API in `lib/cext/include` and `src/main/c/cext`, 

doc/user/compatibility.md

@@ -7,7 +7,7 @@ permalink: /reference-manual/ruby/Compatibility/
 # Compatibility
 
 TruffleRuby aims to be fully compatible with the standard implementation of
-Ruby, MRI, version 3.3.5, [including C extensions](#c-extension-compatibility).
+Ruby, MRI, version 3.3.7, [including C extensions](#c-extension-compatibility).
 TruffleRuby is still in development, so it is not 100% compatible yet.
 
 TruffleRuby can run Rails and is compatible with many gems, including C extensions.

lib/cext/ABI_check.txt

@@ -1 +1 @@
-9
+1

lib/cext/include/truffleruby/truffleruby-abi-version.h

@@ -20,6 +20,6 @@
 // $RUBY_VERSION must be the same as TruffleRuby.LANGUAGE_VERSION.
 // $ABI_NUMBER starts at 1 and is incremented for every ABI-incompatible change.
 
-#define TRUFFLERUBY_ABI_VERSION "3.3.5.24.2.0.1"
+#define TRUFFLERUBY_ABI_VERSION "3.3.7.24.2.1.1"
 
 #endif

lib/gems/gems/bundler-2.5.16/exe/bundle renamed to lib/gems/gems/bundler-2.5.22/exe/bundle

@@ -7,3 +7,4 @@ gem "rake-compiler"
 gem "test-unit", "~> 3.0"
 gem "test-unit-rr"
 gem "json-schema"
+gem "test-unit-launchable"

lib/gems/gems/bundler-2.5.16/exe/bundler renamed to lib/gems/gems/bundler-2.5.22/exe/bundler

@@ -13,10 +13,10 @@ New debug.rb has several advantages:
   * TCP/IP
   * Integration with rich debugger frontends
 
-     Frontend |  [Console](https://github.com/ruby/debug#invoke-as-a-remote-debuggee) | [VSCode](https://github.com/ruby/debug#vscode-integration) | [Chrome DevTool](#chrome-devtool-integration) |
-     ---|---|---|---|
-     Connection | UDS, TCP/IP | UDS, TCP/IP | TCP/IP |
-     Requirement | No | [vscode-rdbg](https://marketplace.visualstudio.com/items?itemName=KoichiSasada.vscode-rdbg) | Chrome |
+     | Frontend    | [Console](https://github.com/ruby/debug#invoke-as-a-remote-debuggee) | [VSCode](https://github.com/ruby/debug#vscode-integration)                                  | [Chrome DevTool](#chrome-devtool-integration) |
+     | ----------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | --------------------------------------------- |
+     | Connection  | UDS, TCP/IP                                                          | UDS, TCP/IP                                                                                 | TCP/IP                                        |
+     | Requirement | No                                                                   | [vscode-rdbg](https://marketplace.visualstudio.com/items?itemName=KoichiSasada.vscode-rdbg) | Chrome                                        |
 
 * Extensible: application can introduce debugging support in several ways:
   * By `rdbg` command
@@ -289,10 +289,10 @@ You can run your application as a remote debuggee, and the remote debugger conso
 
 There are multiple ways to run your program as a debuggee:
 
-Stop at program start | [`rdbg` option](https://github.com/ruby/debug#rdbg---open-or-rdbg--o-for-short) | [require](https://github.com/ruby/debug#require-debugopen-in-a-program) | [debugger API](https://github.com/ruby/debug#start-by-method)
----|---|---|---|
-Yes | `rdbg --open` | `require "debug/open"` | `DEBUGGER__.open`
-No | `rdbg --open --nonstop` | `require "debug/open_nonstop"` | `DEBUGGER__.open(nonstop: true)`
+| Stop at program start | [`rdbg` option](https://github.com/ruby/debug#rdbg---open-or-rdbg--o-for-short) | [require](https://github.com/ruby/debug#require-debugopen-in-a-program) | [debugger API](https://github.com/ruby/debug#start-by-method) |
+| --------------------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------- |
+| Yes                   | `rdbg --open`                                                                   | `require "debug/open"`                                                  | `DEBUGGER__.open`                                             |
+| No                    | `rdbg --open --nonstop`                                                         | `require "debug/open_nonstop"`                                          | `DEBUGGER__.open(nonstop: true)`                              |
 
 #### `rdbg --open` (or `rdbg -O` for short)
 
@@ -713,7 +713,7 @@ The `<...>` notation means the argument.
 * `eval <expr>`
   * Evaluate `<expr>` on the current frame.
 * `irb`
-  * Invoke `irb` on the current frame.
+  * Activate and switch to `irb:rdbg` console
 
 ### Trace
 
@@ -784,6 +784,30 @@ The `<...>` notation means the argument.
   * Show help for the given command.
 
 
+### Using IRB as the Debug Console
+
+Starting from version `v1.9`, you can now use IRB as the debug console. This integration brings additional features such as:
+
+* Autocompletion
+* Support for multi-line input
+* Access to commands not available in `debug`, like `show_source` or `show_doc`
+* [Configurable command aliases](https://docs.ruby-lang.org/en/master/IRB.html#module-IRB-label-Command+Aliases)
+
+To switch to the IRB console, simply use the `irb` command in the debug console.
+
+Once activated, you'll notice the prompt changes to:
+
+```txt
+irb:rdbg(main):001>
+```
+
+If you want to make IRB the default console for all sessions, configure the `irb_console` setting by either:
+
+* Setting the `RUBY_DEBUG_IRB_CONSOLE=true` environment variable
+* Or adding `config set irb_console 1` to your `~/.rdbgrc`
+
+To disable the IRB console in the current session, execute `config set irb_console 0` in the console.
+
 ## Debugger API
 
 ### Start debugging

lib/gems/gems/debug-1.9.1/ext/debug/debug_version.h

@@ -8,13 +8,13 @@ static VALUE rb_mDebugger;
 
 // iseq
 typedef struct rb_iseq_struct rb_iseq_t;
+const rb_iseq_t *rb_iseqw_to_iseq(VALUE iseqw);
 VALUE rb_iseq_realpath(const rb_iseq_t *iseq);
 
 static VALUE
 iseq_realpath(VALUE iseqw)
 {
-    rb_iseq_t *iseq = DATA_PTR(iseqw);
-    return rb_iseq_realpath(iseq);
+    return rb_iseq_realpath(rb_iseqw_to_iseq(iseqw));
 }
 
 static VALUE rb_cFrameInfo;
@@ -121,26 +121,26 @@ frame_depth(VALUE self)
 
 // iseq
 
-const struct rb_iseq *rb_iseqw_to_iseq(VALUE iseqw);
+const rb_iseq_t *rb_iseqw_to_iseq(VALUE iseqw);
 
 #ifdef HAVE_RB_ISEQ_TYPE
-VALUE rb_iseq_type(const struct rb_iseq *);
+VALUE rb_iseq_type(const rb_iseq_t *);
 
 static VALUE
 iseq_type(VALUE iseqw)
 {
-    const struct rb_iseq *iseq = rb_iseqw_to_iseq(iseqw);
+    const rb_iseq_t *iseq = rb_iseqw_to_iseq(iseqw);
     return rb_iseq_type(iseq);
 }
 #endif
 
 #ifdef HAVE_RB_ISEQ_PARAMETERS
-VALUE rb_iseq_parameters(const struct rb_iseq *, int is_proc);
+VALUE rb_iseq_parameters(const rb_iseq_t *, int is_proc);
 
 static VALUE
 iseq_parameters_symbols(VALUE iseqw)
 {
-    const struct rb_iseq *iseq = rb_iseqw_to_iseq(iseqw);
+    const rb_iseq_t *iseq = rb_iseqw_to_iseq(iseqw);
     VALUE params = rb_iseq_parameters(iseq, 0);
     VALUE ary = rb_ary_new();
 
@@ -167,12 +167,12 @@ iseq_parameters_symbols(VALUE iseqw)
 #endif
 
 #ifdef HAVE_RB_ISEQ_CODE_LOCATION
-void rb_iseq_code_location(const struct rb_iseq *, int *first_lineno, int *first_column, int *last_lineno, int *last_column);
+void rb_iseq_code_location(const rb_iseq_t *, int *first_lineno, int *first_column, int *last_lineno, int *last_column);
 
 static VALUE
 iseq_first_line(VALUE iseqw)
 {
-    const struct rb_iseq *iseq = rb_iseqw_to_iseq(iseqw);
+    const rb_iseq_t *iseq = rb_iseqw_to_iseq(iseqw);
     int line;
     rb_iseq_code_location(iseq, &line, NULL, NULL, NULL);
     return INT2NUM(line);
@@ -181,7 +181,7 @@ iseq_first_line(VALUE iseqw)
 static VALUE
 iseq_last_line(VALUE iseqw)
 {
-    const struct rb_iseq *iseq = rb_iseqw_to_iseq(iseqw);
+    const rb_iseq_t *iseq = rb_iseqw_to_iseq(iseqw);
     int line;
     rb_iseq_code_location(iseq, NULL, NULL, &line, NULL);
     return INT2NUM(line);

lib/gems/gems/debug-1.9.1/CONTRIBUTING.md renamed to lib/gems/gems/debug-1.9.2/CONTRIBUTING.md

@@ -0,0 +1 @@
+#define RUBY_DEBUG_VERSION "1.9.2"

lib/gems/gems/debug-1.9.1/Gemfile renamed to lib/gems/gems/debug-1.9.2/Gemfile

@@ -158,6 +158,22 @@ def update conf
           SESSION.set_no_sigint_hook old, new
         end
       end
+
+      if_updated old_conf, conf, :irb_console do |old, new|
+        if defined?(SESSION) && SESSION.active?
+          # irb_console is switched from true to false
+          if old
+            SESSION.deactivate_irb_integration
+          # irb_console is switched from false to true
+          else
+            if CONFIG[:open]
+              SESSION.instance_variable_get(:@ui).puts "\nIRB is not supported on the remote console."
+            else
+              SESSION.activate_irb_integration
+            end
+          end
+        end
+      end
     end
 
     private def if_updated old_conf, new_conf, key

lib/gems/gems/debug-1.9.1/LICENSE.txt renamed to lib/gems/gems/debug-1.9.2/LICENSE.txt

@@ -143,7 +143,7 @@ def history
       rescue LoadError
         def readline prompt
           print prompt
-          gets
+          $stdin.gets
         end
 
         def history

lib/gems/gems/debug-1.9.1/README.md renamed to lib/gems/gems/debug-1.9.2/README.md

@@ -24,4 +24,14 @@ def activate_irb_integration
       IRB::Context.prepend(IrbPatch)
     end
   end
+
+  class Session
+    def deactivate_irb_integration
+      Reline.completion_proc = nil
+      Reline.output_modifier_proc = nil
+      Reline.autocompletion = false
+      Reline.dig_perfect_match_proc = nil
+      reset_ui UI_LocalConsole.new
+    end
+  end
 end

lib/gems/gems/debug-1.9.1/Rakefile renamed to lib/gems/gems/debug-1.9.2/Rakefile

@@ -5,7 +5,7 @@
 
 # Put the following line in your login script (e.g. ~/.bash_profile) with modified path:
 #
-#   export RUBYOPT="-r /path/to/debug/prelude $(RUBYOPT)"
+#   export RUBYOPT="-r /path/to/debug/prelude ${RUBYOPT}"
 #
 module Kernel
   def debugger(*a, up_level: 0, **kw)