8000 Add `includes` option to `ruby_bundle` rule for per-gem load path cus… · p0deje/rules_ruby-1@da3214d · GitHub
[go: up one dir, main page]
Skip to content

Commit da3214d

Browse files
mmizutanikigster
mmizutani
and
kigster
authored
Add includes option to ruby_bundle rule for per-gem load path customization (bazelruby#102)
* Add register_gem support for non-standard gemspecs * cleanup and workaround * debug * Add includes option to bundle_build * Add description about includes/excludes optional parameters for ruby_bundle * Trim extra whitespace characters * Add example usage of ruby_bundle.includes to README * Add a test for includes option of ruby_bundle rule * tidy up * Fix the default value of ruby_bundle rule * Bump up Ruby version of ruby_bundle includes sample code * Fix new test workspace * Fix bundle build rule path injection * Fix usage of ruby_bundle rule's includes option * More clarification in comments * Clarify that folders in spec.require_paths do not need to be listed in includes hash Co-authored-by: Konstantin Gredeskoul <kigster@gmail.com>
1 parent 5e8aeab commit da3214d

File tree

10 files changed

+194
-27
lines changed

10 files changed

+194
-27
lines changed

‎README.md

Lines changed: 52 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020

2121
Note: we have a short guide on [Building your first Ruby Project](https://github.com/bazelruby/rules_ruby/wiki/Build-your-ruby-project) on the Wiki. We encourage you to check it out.
2222

23-
## Table of Contents
23+
## Table of Contents
2424

2525
- [Ruby Rules® for Bazel Build System](#ruby-rules-for-bazelhttpsbazelbuild-build-system)
2626
- [Build Status & Activity](#build-status-activity)
@@ -99,6 +99,12 @@ load(
9999

100100
ruby_bundle(
101101
name = "bundle",
102+
# Specify additional paths to be loaded from the gems at runtime, if any.
103+
# Since spec.require_paths in Gem specifications are auto-included, directory paths
104+
# in spec.require_paths do not need to be listed in includes hash.
105+
includes = {
106+
"grpc": ["etc"],
107+
},
102108
excludes = {
103109
"mini_portile": ["test/**/*"],
104110
},
@@ -200,7 +206,7 @@ ruby_gem(
200206
"rubocop": "",
201207
},
202208
srcs = [
203-
glob("{bin,exe,lib,spec}/**/*.rb")
209+
glob("{bin,exe,lib,spec}/**/*.rb")
204210
],
205211
deps = [
206212
"//lib:example_gem",
@@ -224,7 +230,7 @@ You will have to be sure to export the `ASDF_DATA_DIR` in your profile since it'
224230

225231
### Rule Dependency Diagram
226232

227-
> NOTE: this diagram is slightly outdated.
233+
> NOTE: this diagram is somewhat outdated.
228234

229235
The following diagram attempts to capture the implementation behind `ruby_library` that depends on the result of `bundle install`, and a `ruby_binary` that depends on both:
230236

@@ -447,7 +453,7 @@ ruby_test(
447453
size,
448454
timeout,
449455
flaky,
450-
local,
456+
local,
451457
shard_count
452458
)
453459
```
@@ -540,7 +546,8 @@ ruby_bundle(
540546
gemfile,
541547
gemfile_lock,
542548
bundler_version = "2.1.4",
543-
excludes = [],
549+
includes = {},
550+
excludes = {},
544551
vendor_cache = False,
545552
ruby_sdk = "@org_ruby_lang_ruby_toolchain",
546553
ruby_interpreter = "@org_ruby_lang_ruby_toolchain//:ruby",
@@ -598,6 +605,29 @@ ruby_bundle(
598605
<p>NOTE: This rule never updates the <code>Gemfile.lock</code>. It is your responsibility to generate/update <code>Gemfile.lock</code></p>
599606
</td>
600607
</tr>
608+
<tr>
609+
<td><code>includes</code></td>
610+
<td>
611+
<code>Dictionary of key-value-pairs (key: string, value: list of strings), optional</code>
612+
<p>
613+
List of glob patterns per gem to be additionally loaded from the library.
614+
Keys are the names of the gems which require some file/directory paths not listed in the <code>require_paths</code> attribute of the gemspecs to be also added to <code>$LOAD_PATH</code> at runtime.
615+
Values are lists of blob path patterns, which are relative to the root directories of the gems.
616+
</p>
617+
</td>
618+
</tr>
619+
<tr>
620+
<td><code>excludes</code></td>
621+
<td>
622+
<code>Dictionary of key-value-pairs (key: string, value: list of strings), optional</code>
623+
<p>
624+
List of glob patterns per gem to be excluded from the library.
625+
Keys are the names of the gems.
626+
Values are lists of blob path patterns, which are relative to the root directories of the gems.
627+
The default value is <code>["**/* *.*", "**/* */*"]</code>
628+
</p>
629+
</td>
630+
</tr>
601631
</tbody>
602632
</table>
603633

@@ -857,17 +887,17 @@ ruby_gem(
857887
<tr>
858888
<td><code>gem_description</code></td>
859889
<td>
860-
<code>String, required</code>
890+
<code>String, required</code>
861891
<p>Single-line, paragraph-sized description text for the gem.</p>
862892
</td>
863893
</tr>
864894
<tr>
865895
<td><code>gem_homepage</code></td>
866896
<td>
867-
<code>String, optional</code>
897+
<code>String, optional</code>
868898
<p>Homepage URL of the gem.</p>
869899
</td>
870-
</tr>
900+
</tr>
871901
<tr>
872902
<td><code>gem_authors</code></td>
873903
<td>
@@ -886,7 +916,7 @@ ruby_gem(
886916
List of email addresses of the authors.
887917
</p>
888918
</td>
889-
</tr>
919+
</tr>
890920
<tr>
891921
<td><code>srcs</code></td>
892922
<td>
@@ -917,7 +947,7 @@ ruby_gem(
917947
Typically this value is just `lib` (which is also the default).
918948
</p>
919949
</td>
920-
</tr>
950+
</tr>
921951
<tr>
922952
<td><code>gem_runtime_dependencies</code></td>
923953
<td>
@@ -938,8 +968,8 @@ ruby_gem(
938968
testing gems, linters, code coverage and more.
939969
</p>
940970
</td>
941-
</tr>
942-
971+
</tr>
972+
943973
</tbody>
944974
</table>
945975

@@ -972,13 +1002,14 @@ After that, cd into the top level folder and run the setup script in your Termin
9721002
This runs a complete setup, shouldn't take too long. You can explore various script options with the `help` command:
9731003

9741004
```bash
975-
❯ bin/setup help
1005+
❯ bin/setup -h
1006+
9761007
USAGE
9771008
# without any arguments runs a complete setup.
9781009
bin/setup
9791010

9801011
# alternatively, a sub-setup function name can be passed:
981-
bin/setup [ gems | git-hook | help | os-specific | main | remove-git-hook ]
1012+
bin/setup [ gems | git-hook | help | main | os-specific | rbenv | remove-git-hook ]
9821013

9831014
DESCRIPTION:
9841015
Runs full setup without any arguments.
@@ -988,7 +1019,13 @@ DESCRIPTION:
9881019
This action removes the git commit hook installed by the setup.
9891020

9901021
EXAMPLES:
991-
bin/setup — runs the entire setup.
1022+
bin/setup
1023+
1024+
Or, to run only one of the sub-functions (actions), pass
1025+
it as an argument:
1026+
1027+
bin/setup help
1028+
bin/setup remove-git-hook
9921029
```
9931030

9941031
#### OS-Specific Setup

‎WORKSPACE

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,11 @@ local_repository(
1919
path = "ruby/tests/testdata/another_workspace",
2020
)
2121

22+
local_repository(
23+
name = "bazelruby_rules_ruby_ruby_tests_testdata_bundle_includes_workspace",
24+
path = "ruby/tests/testdata/bundle_includes_workspace",
25+
)
26+
2227
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
2328
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
2429

‎ruby/private/bundle/create_bundle_build_file.rb

Lines changed: 35 additions & 12 deletions
Original file line num 2851 berDiff line numberDiff line change
@@ -33,14 +33,14 @@
3333
srcs = glob(
3434
include = [
3535
".bundle/config",
36-
"{gem_lib_files}",
37-
"lib/ruby/{ruby_version}/specifications/{name}-{version}.gemspec",
36+
{gem_lib_files},
37+
"{gem_spec}",
3838
{gem_binaries}
3939
],
4040
exclude = {exclude},
4141
),
4242
deps = {deps},
43-
includes = ["lib/ruby/{ruby_version}/gems/{name}-{version}/lib"],
43+
includes = [{gem_lib_paths}],
4444
)
4545
GEM_TEMPLATE
4646

@@ -59,7 +59,11 @@
5959
ALL_GEMS
6060

6161
GEM_PATH = ->(ruby_version, gem_name, gem_version) do
62-
"lib/ruby/#{ruby_version}/gems/#{gem_name}-#{gem_version}"
62+
Dir.glob("lib/ruby/#{ruby_version}/gems/#{gem_name}-#{gem_version}*").first
63+
end
64+
65+
SPEC_PATH = ->(ruby_version, gem_name, gem_version) do
66+
Dir.glob("lib/ruby/#{ruby_version}/specifications/#{gem_name}-#{gem_version}*.gemspec").first
6367
end
6468

6569
require 'bundler'
@@ -147,6 +151,7 @@ class BundleBuildFileGenerator
147151
:repo_name,
148152
:build_file,
149153
:gemfile_lock,
154+
:includes,
150155
:excludes,
151156
:ruby_version
152157

@@ -158,11 +163,14 @@ def initialize(workspace_name:,
158163
repo_name:,
159164
build_file: 'BUILD.bazel',
160165
gemfile_lock: 'Gemfile.lock',
161-
excludes: nil)
166+
includes: nil,
167+
excludes: nil,
168+
additional_require_paths: nil)
162169
@workspace_name = workspace_name
163170
@repo_name = repo_name
164171
@build_file = build_file
165172
@gemfile_lock = gemfile_lock
173+
@includes = includes
166174
@excludes = excludes
167175
# This attribute returns 0 as the third minor version number, which happens to be
168176
# what Ruby uses in the PATH to gems, eg. ruby 2.6.5 would have a folder called
@@ -226,7 +234,16 @@ def remove_bundler_version!
226234

227235
def register_gem(spec, template_out, bundle_lib_paths, bundle_binaries)
228236
gem_path = GEM_PATH[ruby_version, spec.name, spec.version]
229-
bundle_lib_paths << gem_lib_path = gem_path + '/lib'
237+
spec_path = SPEC_PATH[ruby_version, spec.name, spec.version]
238+
base_dir = "lib/ruby/#{ruby_version}"
239+
240+
# paths to register to $LOAD_PATH
241+
require_paths = Gem::StubSpecification.gemspec_stub(spec_path, base_dir, "#{base_dir}/gems").require_paths
242+
# Usually, registering the directory paths listed in the `require_paths` of gemspecs is sufficient, but
243+
# some gems also require additional paths to be included in the load paths.
244+
require_paths += include_array(spec.name)
245+
gem_lib_paths = require_paths.map { |require_path| File.join(gem_path, require_path) }
246+
bundle_lib_paths.push(*gem_lib_paths)
230247

231248
# paths to search for executables
232249
gem_binaries = find_bundle_binaries(gem_path)
@@ -237,8 +254,9 @@ def register_gem(spec, template_out, bundle_lib_paths, bundle_binaries)
237254
warn("registering gem #{spec.name} with binaries: #{gem_binaries}") if bundle_binaries.key?(spec.name)
238255

239256
template_out.puts GEM_TEMPLATE
240-
.gsub('{gem_lib_path}', gem_lib_path)
241-
.gsub('{gem_lib_files}', gem_lib_path + '/**/*')
257+
.gsub('{gem_lib_paths}', to_flat_string(gem_lib_paths))
258+
.gsub('{gem_lib_files}', to_flat_string(gem_lib_paths.map { |p| "#{p}/**/*" }))
259+
.gsub('{gem_spec}', spec_path)
242260
.gsub('{gem_binaries}', to_flat_string(gem_binaries))
243261
.gsub('{exclude}', exclude_array(spec.name).to_s)
244262
.gsub('{name}', spec.name)
@@ -265,6 +283,10 @@ def find_bundle_binaries(gem_path)
265283
.map { |binary| 'bin/' + binary }
266284
end
267285

286+
def include_array(gem_name)
287+
(includes[gem_name] || [])
288+
end
289+
268290
def exclude_array(gem_name)
269291
(excludes[gem_name] || []) + DEFAULT_EXCLUDES
270292
end
@@ -274,18 +296,19 @@ def to_flat_string(array)
274296
end
275297
end
276298

277-
# ruby ./create_bundle_build_file.rb "BUILD.bazel" "Gemfile.lock" "repo_name" "[]" "wsp_name"
299+
# ruby ./create_bundle_build_file.rb "BUILD.bazel" "Gemfile.lock" "repo_name" "{}" "{}" "wsp_name"
278300
if $0 == __FILE__
279-
if ARGV.length != 5
280-
warn("USAGE: #{$0} BUILD.bazel Gemfile.lock repo-name [excludes-json] workspace-name".orange)
301+
if ARGV.length != 6
302+
warn("USAGE: #{$0} BUILD.bazel Gemfile.lock repo-name {includes-json} {excludes-json} workspace-name".orange)
281303
exit(1)
282304
end
283305

284-
build_file, gemfile_lock, repo_name, excludes, workspace_name, * = *ARGV
306+
build_file, gemfile_lock, repo_name, includes, excludes, workspace_name, * = *ARGV
285307

286308
BundleBuildFileGenerator.new(build_file: build_file,
287309
gemfile_lock: gemfile_lock,
288310
repo_name: repo_name,
311+
includes: JSON.parse(includes),
289312
excludes: JSON.parse(excludes),
290313
workspace_name: workspace_name).generate!
291314

‎ruby/private/bundle/def.bzl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,7 @@ def generate_bundle_build_file(runtime_ctx, previous_result):
144144
"BUILD.bazel", # Bazel build file (can be empty)
145145
"Gemfile.lock", # Gemfile.lock where we list all direct and transitive dependencies
146146
runtime_ctx.ctx.name, # Name of the target
147+
repr(runtime_ctx.ctx.attr.includes),
147148
repr(runtime_ctx.ctx.attr.excludes),
148149
RULES_RUBY_WORKSPACE_NAME,
149150
]

‎ruby/private/constants.bzl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,9 @@ BUNDLE_ATTRS = {
8585
"bundler_version": attr.string(
8686
default = DEFAULT_BUNDLER_VERSION,
8787
),
88+
"includes": attr.string_list_dict(
89+
doc = "List of glob patterns per gem to be additionally loaded from the library",
90+
),
8891
"excludes": attr.string_list_dict(
8992
doc = "List of glob patterns per gem to be excluded from the library",
9093
),

‎ruby/tests/testdata/bundle_includes_workspace/BUILD.bazel

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
load(
2+
"@bazelruby_rules_ruby//ruby:defs.bzl",
3+
"ruby_binary",
4+
)
5+
6+
package(default_visibility = ["//:__subpackages__"])
7+
8+
ruby_binary(
9+
name = "script",
10+
srcs = ["script.rb"],
11+
main = "script.rb",
12+
deps = [
13+
"@gems//:grpc",
14+
],
15+
)

‎ruby/tests/testdata/bundle_includes_workspace/Gemfile

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# frozen_string_literal: true
2+
3+
source 'https://rubygems.org'
4+
5+
gem 'grpc'

‎ruby/tests/testdata/bundle_includes_workspace/Gemfile.lock

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
GEM
2+
remote: https://rubygems.org/
3+
specs:
4+
google-protobuf (3.17.3)
5+
googleapis-common-protos-types (1.1.0)
6+
google-protobuf (~> 3.14)
7+
grpc (1.38.0)
8+
google-protobuf (~> 3.15)
9+
googleapis-common-protos-types (~> 1.0)
10+
11+
PLATFORMS
12+
ruby
13+
14+
DEPENDENCIES
15+
grpc
16+
17+
BUNDLED WITH
18+
2.2.22

‎ruby/tests/testdata/bundle_includes_workspace/WORKSPACE

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
workspace(name = "bazelruby_rules_ruby_ruby_tests_testdata_bundle_includes_workspace")
2+
3+
local_repository(
4+
name = "bazelruby_rules_ruby",
5+
path = "../../../..",
6+
)
7+
8+
load(
9+
"@bazelruby_rules_ruby//ruby:deps.bzl",
10+
"rules_ruby_dependencies",
11+
"rules_ruby_select_sdk",
12+
)
13+
14+
rules_ruby_dependencies()
15+
16+
rules_ruby_select_sdk(version = "3.0.1")
17+
18+
load("@bazelruby_rules_ruby//ruby:defs.bzl", "ruby_bundle")
19+
20+
ruby_bundle(
21+
name = "gems",
22+
bundler_version = "2.2.21",
23+
gemfile = "//:Gemfile",
24+
gemfile_lock = "//:Gemfile.lock",
25+
includes = {
26+
# The gemspec of grpc gem lists ['src/ruby/bin', 'src/ruby/lib', 'src/ruby/pb'] as the `require_paths`. When installing
27+
# pre-built versions of the gem using a package downloaded from rubygems.org, these paths are sufficient since the file
28+
# `src/ruby/lib/grpc.rb` in the downloaded gem package does not `require` any file outside these directories.
29+
# However, when installing grpc gem from source using Bundler, `src/ruby/lib/grpc.rb` in the source package does
30+
# `require` 'etc/roots.pem', so the directory containing this `require`-d file also needs to be present in the `$LOAD_PATH`.
31+
# Thus users have to manually add the 'etc' directory to the `$LOAD_PATH` using the `includes` option of `ruby_bundle` rule.
32+
# The `includes` option of `ruby_bundle` rule is a means of workaround for such a peculiar situation.
33+
"grpc": ["etc"],
34+
},
35+
)

0 commit comments

Comments
 (0)
0