8000 Rewrite Array#each in Ruby using Primitive by k0kubun · Pull Request #9533 · ruby/ruby · GitHub
[go: up one dir, main page]

Skip to content

Rewrite Array#each in Ruby using Primitive #9533

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our 8000 terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 23, 2024

Conversation

k0kubun
Copy link
Member
@k0kubun k0kubun commented Jan 14, 2024

Same as #6687, but race condition-free.

microbenchmark

Interpreter

Thanks to Primitive, the interpreter doesn't slow down.

$ benchmark-driver benchmark/loop_each.yml -v --chruby 'before;after'
before: ruby 3.4.0dev (2024-01-20T01:12:07Z master 99d6e2f1ee) [x86_64-linux]
after: ruby 3.4.0dev (2024-01-20T01:36:08Z primitive-array-each 3aaa6de3ec) [x86_64-linux]
Warming up --------------------------------------
           loop_each      1.539 i/s -       2.000 times in 1.299363s (649.68ms/i)
Calculating -------------------------------------
                         before       after
           loop_each      1.551       1.796 i/s -       4.000 times in 2.578413s 2.227167s

Comparison:
                        loop_each
               after:         1.8 i/s
              before:         1.6 i/s - 1.16x  slower

YJIT

Even faster than #6687.

$ benchmark-driver benchmark/loop_each.yml -v --chruby 'before::before --yjit-call-threshold=1;after::after --yjit-call-threshold=1'
before: ruby 3.4.0dev (2024-01-20T01:12:07Z master 99d6e2f1ee) +YJIT [x86_64-linux]
after: ruby 3.4.0dev (2024-01-20T01:36:08Z primitive-array-each 3aaa6de3ec) +YJIT [x86_64-linux]
Warming up --------------------------------------
           loop_each      1.789 i/s -       2.000 times in 1.118068s (559.03ms/i)
Calculating -------------------------------------
                         before       after
           loop_each      1.790      12.986 i/s -       5.000 times in 2.793497s 0.385044s

Comparison:
                        loop_each
               after:        13.0 i/s
              before:         1.8 i/s - 7.26x  slower

yjit-bench

See #9622.

@k0kubun k0kubun force-pushed the primitive-array-each branch 6 times, most recently from dca75f2 to 35f3b4e Compare January 15, 2024 05:20
@maximecb
Copy link
Contributor

Nice. Happy to see this working. Surprised it's so far in YJIT as well!

Ideally, for YJIT, we'd like to be able to avoid doing a function call at all, so we can generate tight inlined code though, so we might need a specialized instruction instead of a cexpr? What do you think?

@k0kubun
Copy link
Member Author
k0kubun commented Jan 16, 2024

Ideally, for YJIT, we'd like to be able to avoid doing a function call at all, so we can generate tight inlined code though, so we might need a specialized instruction instead of a cexpr? What do you think?

I think it's possible without a new instruction. cexpr! already gets a somewhat special instruction, invokebuiltin (opt_invokebuiltin_delegate for this ISEQ to be precise). Since we use this instruction for converting a C method, each ISEQ typically has up to one invokebuiltin (at least we could assume it for select ISEQs YJIT specializes). If we specialize invokebuiltin for Array#each ISEQ as if it was the new instruction you proposed, we'll get what we want.

@k0kubun k0kubun force-pushed the primitive-array-each branch from 4f30c21 to 6b5e44b Compare January 21, 2024 06:04
@k0kubun k0kubun force-pushed the primitive-array-each branch from 2e2cd3f to d98ac2c Compare January 23, 2024 19:01
@k0kubun k0kubun marked this pull request as ready for review January 23, 2024 19:40
@k0kubun k0kubun enabled auto-merge (squash) January 23, 2024 20:00
@k0kubun k0kubun merged commit c84237f into ruby:master Jan 23, 2024
@k0kubun k0kubun deleted the primitive-array-each branch January 23, 2024 21:05
paracycle added a commit to Shopify/sorbet that referenced this pull request Feb 12, 2024
In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each` visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
paracycle added a commit to Shopify/sorbet that referenced this pull request Feb 12, 2024
In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each` visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
paracycle added a commit to Shopify/sorbet that referenced this pull request Feb 12, 2024
In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each` visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
paracycle added a commit to Shopify/sorbet that referenced this pull request Feb 12, 2024
In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each` visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
neilparikh pushed a commit to sorbet/sorbet that referenced this pull request Feb 13, 2024
In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each` visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
neilparikh pushed a commit to sorbet/sorbet that referenced this pull request Feb 14, 2024
In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each`
8000
 visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
paracycle added a commit to Shopify/sorbet that referenced this pull request Feb 16, 2024
In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each` visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
jez pushed a commit to sorbet/sorbet that referenced this pull request Feb 21, 2024
In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each` visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
jez pushed a commit to sorbet/sorbet that referenced this pull request Feb 21, 2024
)

In order to provide compatibility with TruffleRuby that implements a lot
of its core library methods in Ruby, and for future versions of CRuby
that is increasingly doing the same, we need to be able to filter
all backtrace locations where the `path` starts with `<internal:`.

[This gist](https://gist.github.com/eregon/912e6359e83781c5fa1c638d3768c526)
shows the current state of the methods implemented in Ruby in CRuby,
JRuby and TruffleRuby.

Most recently [CRuby started implementing `Array#each` in Ruby](ruby/ruby#9533), making it usages of `each` visible in backtraces with an `<internal:array>` path. This means that in order to be compatible with CRuby 3.4, Sorbet runtime needs to start filtering out backtrace locations that start with `<internal:`.

By encapsulating the caller location search logic into a singleton
method, we can apply that filtering in a single location and avoid
having to repeat it in multiple places.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0