Change test-all suite to enable running each test in separate ractors. #14672
Conversation
330d4fe to
c32acb5
Compare
❌ 2/67574 Tests Failedtest/json/ractor_test.rb#test_generatetest/rubygems/test_gem_remote_fetcher_local_ssl_server.rb#test_do_not_allow_insecure_ssl_connection_by_default |
25804c4 to
d641b1f
Compare
|
This branch is ready to be merged @ko1 . When you run the make command listed in the first commit, don't add the |
| def after_teardown | ||
| super | ||
| assert_empty(Process.waitall) | ||
| assert_empty(Process.waitall) unless multiple_ractors? |
There was a problem hiding this comment.
Since after_teardown runs after each test, other ractors could still have processes running.
| @__io__ = nil | ||
| @__passed__ = nil | ||
| @@__current__ = self # FIX: make thread local | ||
| Ractor.current[:__test_current__] = self |
There was a problem hiding this comment.
The removed comment said it should be thread local. is it okay to use Ractor local?
| RUN_TEST_TRACE = "#{__FILE__}:#{__LINE__+3}:in `run_test'".freeze | ||
| def run_test(name) | ||
| progname, $0 = $0, "#{$0}: #{self.class}##{name}" | ||
| progname, $0 = $0, "#{$0}: #{self.class}##{name}" if main_ractor? |
There was a problem hiding this comment.
TODO:
if we make main Ractor as a manager Ractor (and other Ractors run tests, a manager Ractor knows which tests are running), we can set the progname with running test names by the main Ractor.
There was a problem hiding this comment.
Yeah, that's a good idea. For future work 😆
tool/lib/test/unit/testcase.rb
Outdated
| end | ||
|
|
||
| def multiple_ractors? | ||
| ENV["RUBY_TESTS_WITH_RACTORS"].to_i > 1 |
There was a problem hiding this comment.
cache the value or define different methods with the envval?
some tests can remove ENVVAL.
| Ractor.receive | ||
| end | ||
| end | ||
| dir = r.take |
There was a problem hiding this comment.
I'll investigate 🤷 Edit: There was a shim in assert_ractor method. I removed it.
This tests for ractor safety issues as well as concurrency issues. You can enable these tests with RUBY_TESTS_WITH_RACTORS=X when running `make test-all TESTS=test/ruby`. Running tests with ractors is currently only working for tests under the "test/ruby" directory. You should also use the `.excludes-ractor` excludes directory. For example: ``` EXCLUDES=test/.excludes-ractor RUBY_TESTS_WITH_RACTORS=3 TESTS=test/ruby make test-all ``` This will create 3 ractors for each test method, run the test method inside each ractor and then call `join` on the ractors. Then, it's ready to process the next test. The reason we do this instead of taking all the tests and dividing them between the number of ractors is: * We want to run each test under concurrency load to test whether or not it is safe under ractors. If it isn't safe, we know it's something to do with this specific test. * If we get a segfault or error, we know the error is coming from this test alone, not interactions with other tests. The disadvantage of this approach is: * It's slower than dividing the tests between the number of ractors running. * This might not uncover ractor scheduling issues that would show up when you have long-running ractors.
As per ko1's comment, tests can mess with ENV. Add a couple more excludes too after running into issues.
d86bda5 to
680c924
Compare
|
MMTK CI failed but we found out why. We're waiting on #14811 to be merged. |
This tests for ractor safety issues as well as concurrency issues.
You can enable these tests with RUBY_TESTS_WITH_RACTORS=X when running
make test-all TESTS=test/ruby. Running tests with ractors is currently only working for tests under the "test/ruby" directory. You should also use the.excludes-ractorexcludes directory.For example:
This will create 3 ractors for each test method, run the test method inside each ractor and then call
joinon the ractors. Then, it's ready to process the next test. The reason we do this instead of taking all the tests and dividing them between the number of ractors is:We want to run each test under concurrency load to test whether or not it is safe under ractors. If it isn't safe, we know it's something to do with this specific test.
If we get a segfault or error, we know the error is coming from this test alone, not interactions with other tests.
The disadvantage of this approach is:
It's slower than dividing the tests between the number of ractors running.
This might not uncover ractor scheduling issues that would show up when you have long-running ractors.