8000 Merge branch 'master' into mps-binops-dtype-precedence · pytorch/pytorch@674b933 · GitHub
[go: up one dir, main page]
Skip to content
Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 674b933

Browse files
lhoeniglhoenig
committed
Merge branch 'master' into mps-binops-dtype-precedence
2 parents 4db40e8 + 18d8c54 commit 674b933

File tree

4,102 files changed

+424043
-197590
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

4,102 files changed

+424043
-197590
lines changed

‎.bazelrc

Lines changed: 97 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,103 @@ build:no-tty --curses no
1313
build:no-tty --progress_report_interval 10
1414
build:no-tty --show_progress_rate_limit 10
1515

16-
# Configuration to build with GPU support
17-
build:gpu --define=cuda=true
16+
# Build with GPU support by default.
17+
build --define=cuda=true
18+
# rules_cuda configuration
19+
build --@rules_cuda//cuda:enable_cuda
20+
build --@rules_cuda//cuda:cuda_targets=sm_52
21+
build --@rules_cuda//cuda:compiler=nvcc
22+
build --repo_env=CUDA_PATH=/usr/local/cuda
23+
24+
# Configuration to build without GPU support
25+
build:cpu-only --define=cuda=false
1826
# define a separate build folder for faster switching between configs
19-
build:gpu --platform_suffix=-gpu
27+
build:cpu-only --platform_suffix=-cpu-only
2028
# See the note on the config-less build for details about why we are
21-
# doing this. We must also do it for the "-gpu" platform suffix.
22-
build --copt=-isystem --copt=bazel-out/k8-fastbuild-gpu/bin
29+
# doing this. We must also do it for the "-cpu-only" platform suffix.
30+
build --copt=-isystem --copt=bazel-out/k8-fastbuild-cpu-only/bin
2331
# rules_cuda configuration
24-
build:gpu --@rules_cuda//cuda:enable_cuda
25-
build:gpu --@rules_cuda//cuda:cuda_targets=sm_52
26-
build:gpu --@rules_cuda//cuda:compiler=nvcc
27-
build:gpu --repo_env=CUDA_PATH=/usr/local/cuda
32+
build:cpu-only --@rules_cuda//cuda:enable_cuda=False
33+
34+
# Definition of --config=shell
35+
# interactive shell immediately before execution
36+
build:shell --run_under="//tools/bazel_tools:shellwrap"
37+
38+
# Disable all warnings for external repositories. We don't care about
39+
# their warnings.
40+
build --per_file_copt=^external/@-w
41+
42+
# Set additional warnings to error level.
43+
#
44+
# Implementation notes:
45+
# * we use file extensions to determine if we are using the C++
46+
# compiler or the cuda compiler
47+
# * we use ^// at the start of the regex to only permit matching
48+
# PyTorch files. This excludes external repos.
49+
#
50+
# Note that because this is logically a command-line flag, it is
51+
# considered the word on what warnings are enabled. This has the
52+
# unfortunate consequence of preventing us from disabling an error at
53+
# the target level because those flags will come before these flags in
54+
# the action invocation. Instead we provide per-file exceptions after
55+
# this.
56+
#
57+
# On the bright side, this means we don't have to more broadly apply
58+
# the exceptions to an entire target.
59+
#
60+
# Looking for CUDA flags? We have a cu_library macro that we can edit
61+
# directly. Look in //tools/rules:cu.bzl for details. Editing the
62+
# macro over this has the following advantages:
63+
# * making changes does not require discarding the Bazel analysis
64+
# cache
65+
# * it allows for selective overrides on individual targets since the
66+
# macro-level opts will come earlier than target level overrides
67+
68+
build --per_file_copt='^//.*\.(cpp|cc)$'@-Werror=all
69+
# The following warnings come from -Wall. We downgrade them from error
70+
# to warnings here.
71+
#
72+
# sign-compare has a tremendous amount of violations in the
73+
# codebase. It will be a lot of work to fix them, just disable it for
74+
# now.
75+
build --per_file_copt='^//.*\.(cpp|cc)$'@-Wno-sign-compare
76+
# We intentionally use #pragma unroll, which is compiler specific.
77+
build --per_file_copt='^//.*\.(cpp|cc)$'@-Wno-error=unknown-pragmas
78+
79+
build --per_file_copt='^//.*\.(cpp|cc)$'@-Werror=extra
80+
# The following warnings come from -Wextra. We downgrade them from error
81+
# to warnings here.
82+
#
83+
# unused-parameter-compare has a tremendous amount of violations in the
84+
# codebase. It will be a lot of work to fix them, just disable it for
85+
# now.
86+
build --per_file_copt='^//.*\.(cpp|cc)$'@-Wno-unused-parameter
87+
# missing-field-parameters has both a large number of violations in
88+
# the codebase, but it also is used pervasively in the Python C
89+
# API. There are a couple of catches though:
90+
# * we use multiple versions of the Python API and hence have
91+
# potentially multiple different versions of each relevant
92+
# struct. They may have different numbers of fields. It will be
93+
# unwieldy to support multiple versions in the same source file.
94+
# * Python itself for many of these structs recommends only
95+
# initializing a subset of the fields. We should respect the API
96+
# usage conventions of our dependencies.
97+
#
98+
# Hence, we just disable this warning altogether. We may want to clean
99+
# up some of the clear-cut cases that could be risky, but we still
100+
# likely want to have this disabled for the most part.
101+
build --per_file_copt='^//.*\.(cpp|cc)$'@-Wno-missing-field-initializers
102+
103+
build --per_file_copt='//:aten/src/ATen/RegisterCompositeExplicitAutograd\.cpp$'@-Wno-error=unused-function
104+
build --per_file_copt='//:aten/src/ATen/RegisterCompositeImplicitAutograd\.cpp$'@-Wno-error=unused-function
105+
build --per_file_copt='//:aten/src/ATen/RegisterMkldnnCPU\.cpp$'@-Wno-error=unused-function
106+
build --per_file_copt='//:aten/src/ATen/RegisterNestedTensorCPU\.cpp$'@-Wno-error=unused-function
107+
build --per_file_copt='//:aten/src/ATen/RegisterQuantizedCPU\.cpp$'@-Wno-error=unused-function
108+
build --per_file_copt='//:aten/src/ATen/RegisterSparseCPU\.cpp$'@-Wno-error=unused-function
109+
build --per_file_copt='//:aten/src/ATen/RegisterSparseCsrCPU\.cpp$'@-Wno-error=unused-function
110+
build --per_file_copt='//:aten/src/ATen/RegisterNestedTensorMeta\.cpp$'@-Wno-error=unused-function
111+
build --per_file_copt='//:aten/src/ATen/RegisterSparseMeta\.cpp$'@-Wno-error=unused-function
112+
build --per_file_copt='//:aten/src/ATen/RegisterQuantizedMeta\.cpp$'@-Wno-error=unused-function
113+
build --per_file_copt='//:aten/src/ATen/RegisterZeroTensor\.cpp$'@-Wno-error=unused-function
114+
build --per_file_copt='//:torch/csrc/lazy/generated/RegisterAutogradLazy\.cpp$'@-Wno-error=unused-function
115+
build --per_file_copt='//:torch/csrc/lazy/generated/RegisterLazy\.cpp$'@-Wno-error=unused-function

‎.buckconfig.oss

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,25 @@
1+
[pt]
2+
is_oss=1
3+
14
[buildfile]
2-
name = BUCK.oss
5+
name = BUCK.oss
6+
includes = //tools/build_defs/select.bzl
37

48
[repositories]
59
bazel_skylib = third_party/bazel-skylib/
10+
ovr_config = .
611

712
[download]
813
in_build = true
914

1015
[cxx]
1116
cxxflags = -std=c++17
1217
should_remap_host_platform = true
18+
cpp = /usr/bin/clang
19+
cc = /usr/bin/clang
20+
cxx = /usr/bin/clang++
21+
cxxpp = /usr/bin/clang++
22+
ld = /usr/bin/clang++
1323

1424
[project]
1525
default_flavors_mode=all

‎.circleci/cimodel/data/dimensions.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
"102",
55
"113",
66
"116",
7+
"117",
78
]
89

910
ROCM_VERSIONS = [

‎.circleci/cimodel/data/pytorch_build_data.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ def child_constructor(self):
7575
"vulkan": VulkanConfigNode,
7676
"parallel_tbb": ParallelTBBConfigNode,
7777
"crossref": CrossRefConfigNode,
78+
"dynamo": DynamoConfigNode,
7879
"parallel_native": ParallelNativeConfigNode,
7980
"onnx": ONNXConfigNode,
8081
"libtorch": LibTorchConfigNode,
@@ -179,6 +180,14 @@ def child_constructor(self):
179180
return ImportantConfigNode
180181

181182

183+
class DynamoConfigNode(TreeConfigNode):
184+
def init2(self, node_name):
185+
self.props["is_dynamo"] = node_name
186+
187+
def child_constructor(self):
188+
return ImportantConfigNode
189+
190+
182191
class ParallelNativeConfigNode(TreeConfigNode):
183192
def modify_label(self, label):
184193
return "PARALLELNATIVE=" + str(label)

‎.circleci/cimodel/data/pytorch_build_definitions.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -240,6 +240,7 @@ def instantiate_configs(only_slow_gradcheck):
240240
is_xla = fc.find_prop("is_xla") or False
241241
is_asan = fc.find_prop("is_asan") or False
242242
is_crossref = fc.find_prop("is_crossref") or False
243+
is_dynamo = fc.find_prop("is_dynamo") or False
243244
is_onnx = fc.find_prop("is_onnx") or False
244245
is_pure_torch = fc.find_prop("is_pure_torch") or False
245246
is_vulkan = fc.find_prop("is_vulkan") or False
@@ -286,6 +287,9 @@ def instantiate_configs(only_slow_gradcheck):
286287
if is_crossref:
287288
parms_list_ignored_for_docker_image.append("crossref")
288289

290+
if is_dynamo:
291+
parms_list_ignored_for_docker_image.append("dynamo")
292+
289293
if is_onnx:
290294
parms_list.append("onnx")
291295
python_version = fc.find_prop("pyver")

‎.circleci/cimodel/data/simple/ios_definitions.py

Lines changed: 21 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
from cimodel.data.simple.util.versions import MultiPartVersion
2+
from cimodel.data.simple.util.branch_filters import gen_filter_dict_exclude
23
import cimodel.lib.miniutils as miniutils
34

45
XCODE_VERSION = MultiPartVersion([12, 5, 1])
@@ -11,7 +12,7 @@ def __init__(self, name, custom_build_name=""):
1112

1213
def render(self):
1314
extra_parts = [self.custom_build_name] if len(self.custom_build_name) > 0 else []
14-
return "_".join([self.name] + extra_parts)
15+
return "-".join([self.name] + extra_parts).replace("_", "-")
1516

1617

1718
def get_platform(arch_variant_name):
@@ -25,30 +26,25 @@ def __init__(self, xcode_version, arch_variant, is_org_member_context=True, extr
2526
self.is_org_member_context = is_org_member_context
2627
self.extra_props = extra_props
2728

28-
def gen_name_parts(self, with_version_dots):
29-
30-
version_parts = self.xcode_version.render_dots_or_parts(with_version_dots)
31-
build_variant_suffix = "_".join([self.arch_variant.render(), "build"])
32-
29+
def gen_name_parts(self):
30+
version_parts = self.xcode_version.render_dots_or_parts("-")
31+
build_variant_suffix = self.arch_variant.render()
3332
return [
34-
"pytorch",
3533
"ios",
3634
] + version_parts + [
3735
build_variant_suffix,
3836
]
3937

4038
def gen_job_name(self):
41-
return "_".join(self.gen_name_parts(False))
39+
return "-".join(self.gen_name_parts())
4240

4341
def gen_tree(self):
44-
4542
platform_name = get_platform(self.arch_variant.name)
46-
4743
props_dict = {
48-
"build_environment": "-".join(self.gen_name_parts(True)),
44+
"name": self.gen_job_name(),
45+
"build_environment": self.gen_job_name(),
4946
"ios_arch": self.arch_variant.name,
5047
"ios_platform": platform_name,
51-
"name": self.gen_job_name(),
5248
}
5349

5450
if self.is_org_member_context:
@@ -57,30 +53,28 @@ def gen_tree(self):
5753
if self.extra_props:
5854
props_dict.update(self.extra_props)
5955

56+
props_dict["filters"] = gen_filter_dict_exclude()
57+
6058
return [{"pytorch_ios_build": props_dict}]
6159

6260

6361
WORKFLOW_DATA = [
6462
IOSJob(XCODE_VERSION, ArchVariant("x86_64"), is_org_member_context=False, extra_props={
6563
"lite_interpreter": miniutils.quote(str(int(True)))}),
66-
IOSJob(XCODE_VERSION, ArchVariant("x86_64", "full_jit"), is_org_member_context=False, extra_props={
67-
"lite_interpreter": miniutils.quote(str(int(False)))}),
68-
IOSJob(XCODE_VERSION, ArchVariant("arm64"), extra_props={
69-
"lite_interpreter": miniutils.quote(str(int(True)))}),
70-
IOSJob(XCODE_VERSION, ArchVariant("arm64", "metal"), extra_props={
71-
"use_metal": miniutils.quote(str(int(True))),
72-
"lite_interpreter": miniutils.quote(str(int(True)))}),
73-
IOSJob(XCODE_VERSION, ArchVariant("arm64", "full_jit"), extra_props={
74-
"lite_interpreter": miniutils.quote(str(int(False)))}),
75-
IOSJob(XCODE_VERSION, ArchVariant("arm64", "custom"), extra_props={
76-
"op_list": "mobilenetv2.yaml",
77-
"lite_interpreter": miniutils.quote(str(int(True)))}),
64+
# IOSJob(XCODE_VERSION, ArchVariant("arm64"), extra_props={
65+
# "lite_interpreter": miniutils.quote(str(int(True)))}),
66+
# IOSJob(XCODE_VERSION, ArchVariant("arm64", "metal"), extra_props={
67+
# "use_metal": miniutils.quote(str(int(True))),
68+
# "lite_interpreter": miniutils.quote(str(int(True)))}),
69+
# IOSJob(XCODE_VERSION, ArchVariant("arm64", "custom-ops"), extra_props={
70+
# "op_list": "mobilenetv2.yaml",
71+
# "lite_interpreter": miniutils.quote(str(int(True)))}),
7872
IOSJob(XCODE_VERSION, ArchVariant("x86_64", "coreml"), is_org_member_context=False, extra_props={
7973
"use_coreml": miniutils.quote(str(int(True))),
8074
"lite_interpreter": miniutils.quote(str(int(True)))}),
81-
IOSJob(XCODE_VERSION, ArchVariant("arm64", "coreml"), extra_props={
82-
"use_coreml": miniutils.quote(str(int(True))),
83-
"lite_interpreter": miniutils.quote(str(int(True)))}),
75+
# IOSJob(XCODE_VERSION, ArchVariant("arm64", "coreml"), extra_props={
76+
# "use_coreml": miniutils.quote(str(int(True))),
77+
# "lite_interpreter": miniutils.quote(str(int(True)))}),
8478
]
8579

8680

0 commit comments

Comments
 (0)
0