8000 Support for `uint16`, `uint32`, and `uint64` · Issue #58734 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

Support for uint16, uint32, and uint64 #58734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Tracked by #58743
pmeier opened this issue May 21, 2021 · 36 comments
Open
Tracked by #58743

Support for uint16, uint32, and uint64 #58734

pmeier opened this issue May 21, 2021 · 36 comments
Labels
ezyang's list Stuff ezyang doesn't want to lose feature A request for a proper, new feature. module: python array api Issues related to the Python Array API oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@pmeier
Copy link
Collaborator
pmeier commented May 21, 2021 •
edited by pytorch-bot bot
Loading

The array API specification stipulates the data types that we need to support to be compliant. Currently we are missing support for uint16, uint32, and uint64.

cc @mruberry @rgommers @asmeurer @leofang @AnirudhDagar @asi1024 @emcastillo @kmaehashi @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 @gchanan @soumith @ngimel
@pmeier pmeier added the module: python array api Issues related to the Python Array API label May 21, 2021
@pmeier pmeier mentioned this issue May 21, 2021
Open
29 tasks
@H-Huang H-Huang added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label May 22, 2021
@fmassa fmassa mentioned this issue Aug 27, 2021
Closed
@rgommers
Copy link
Collaborator

Someone just asked about other uint types on Slack. And from pytorch/vision#4326 (comment):

This is because PyTorch doesn't (yet) support uint16, and is also a problem when reading PNG images of type uint16.

16-bit image support making uint16 the most interesting one of the missing dtypes had been my guess before.

There are no plans to work on this issue currently I think, unless more demand materializes.

@NicolasHug
Copy link
Member
NicolasHug commented Nov 13, 2021 •
edited
Loading

There are no plans to work on this issue currently I think, unless more demand materializes.

I'll add one :)

Another tangible need for uint16 support in torchvision is pytorch/vision#4731

We added support for native 16 bits png decoding in torchvision, but we can't make this API public for now, because we output int32 tensors and this wouldn't be compatible with the rest of our transforms.
It'd be great if we could make it public because Pillow 16 bits png support is fairly limited.

@kernelmethod
Copy link
kernelmethod commented Feb 20, 2022 •
edited
Loading

Bumping this.

My research collaborators and I are working on some cryptographic applications where we could really use uint32 / uint64. Some operations on Z_{2^64} that we'd like to calculate with secure multi-party computation, e.g. comparison, would be a lot more straightforward to implement with unsigned integers as the underlying dtype.

@honno honno mentioned this issue Mar 30, 2022
Merged
@mworchel mworchel mentioned this issue Dec 10, 2022
Closed
@a-gn
Copy link
a-gn commented Jan 18, 2023

We have a GIS pipeline that uses image transforms from multiple projects, some of which only support uint, but uint8 leads to too much loss of color information. We could really use uint16 support.

@neelnanda-io
Copy link

I would appreciate uint16 support! I'm trying to do NLP stuff with a large dataset of tokens between 0 and 51000, and it's annoying to consume double the storage to keep them as int32s (I'm currently storing them as uint16 via HuggingFace, but I need to load them as NumPy and manually convert them)

@oliver-batchelor
Copy link

I'm doing work on HDR imaging and we read images from the camera as 16-bit unsigned. It's possible to work around it by using other frameworks but it would be really useful.

@VladShtompel
Copy link

I'm doing work on HDR imaging and we read images from the camera as 16-bit unsigned. It's possible to work around it by using other frameworks but it would be really useful.

This is exactly the issue me and my team are faced with right now.

@StrongChris
Copy link

I'm doing work with DICOM data that is often 10 or even 14 unsigned bits. A uint16 would be very nice for these! My work is focused on speed so using the smallest possible datatype would be very appreciated.

@StrongChris StrongChris mentioned this issue Apr 3, 2023
Open
@ezyang
Copy link
Contributor
ezyang commented Apr 22, 2023

We should add these dtypes, and then build out support via PT2. We probably aren't going to add kernels for everything but Triton makes it very easy to JIT compile these operations.

@NicolasHug
Copy link
Member

@ezyang would Triton be able to enable CPU support?

@ezyang
Copy link
Contributor
ezyang commented Apr 24, 2023

Not Triton per se, but we have a CPU inductor backend, so the answer is yes!

@soulitzer
Copy link
Contributor

From Triage review: We still need some limited eager support, e.g. factory functions, conversion functions. Also consideration with autocast? (maybe not too bad?)

@rgommers rgommers mentioned this issue Jun 11, 2023
Open
@vadimkantorov vadimkantorov mentioned this issue Jul 30, 2023
Open
@vadimkantorov
Copy link
Contributor

Also, bit ops are only well-defined/standardized in CPUs for unsigned dtypes if I understand well: #105465

@vadimkantorov
Copy link
Contributor

uint16 would also be useful for interop with opencv (CV_16U dtype)

@ClaudiaComito ClaudiaComito mentioned this issue Aug 21, 2023
Open
vivekkhandelwal1 added a commit to vivekkhandelwal1/torch-mlir that referenced this issue Nov 21, 2023
@vivekkhandelwal1
[MLIR][TORCH] Add support for unsigned integer types
ddb574e 
Refer: pytorch/pytorch#58734
@ezyang ezyang mentioned this issue Jan 2, 2024
Closed
ezyang added a commit that referenced this issue Jan 3, 2024
@ezyang
Update base for Update on "Add unsigned integer dtypes to PyTorch"
94d2239 
The dtypes are very useless right now (not even fill works), but it makes torch.uint16, uint32 and uint64 available as a dtype.

Towards #58734

Signed-off-by: Edward Z. Yang <ezyangmeta.com>

[ghstack-poisoned]
ezyang added a commit that referenced this issue Jan 3, 2024
@ezyang
Update on "Add unsigned integer dtypes to PyTorch"
ced08ca 
The dtypes are very useless right now (not even fill works), but it makes torch.uint16, uint32 and uint64 available as a dtype.

Towards #58734

Signed-off-by: Edward Z. Yang <ezyangmeta.com>

[ghstack-poisoned]
ezyang added a commit that referenced this issue Jan 4, 2024
@ezyang
Update base for Update on "Add unsigned integer dtypes to PyTorch"
56648ba 
The dtypes are very useless right now (not even fill works), but it makes torch.uint16, uint32 and uint64 available as a dtype.

Towards #58734

Signed-off-by: Edward Z. Yang <ezyangmeta.com>

[ghstack-poisoned]
ezyang added a commit that referenced this issue Jan 4, 2024
@ezyang
Update on "Add unsigned integer dtypes to PyTorch"
9f6ac04 
The dtypes are very useless right now (not even fill works), but it makes torch.uint16, uint32 and uint64 available as a dtype.

Towards #58734

Signed-off-by: Edward Z. Yang <ezyangmeta.com>

[ghstack-poisoned]
ezyang added a commit that referenced this issue Jan 5, 2024
@ezyang
Update base for Update on "Add unsigned integer dtypes to PyTorch"
75f12f2 
The dtypes are very useless right now (not even fill works), but it makes torch.uint16, uint32 and uint64 available as a dtype.

Towards #58734

Signed-off-by: Edward Z. Yang <ezyangmeta.com>

[ghstack-poisoned]
ezyang added a commit that referenced this issue Jan 5, 2024
@ezyang
Update on "Add unsigned integer dtypes to PyTorch"
d893654 
The dtypes are very useless right now (not even fill works), but it makes torch.uint16, uint32 and uint64 available as a dtype.

Towards #58734

Signed-off-by: Edward Z. Yang <ezyangmeta.com>

[ghstack-poisoned]
pytorchmergebot pushed a commit that referenced this issue Jan 7, 2024
@ezyang @pytorchmergebot
Add unsigned integer dtypes to PyTorch (#116594)
8ddac14 
The dtypes are very useless right now (not even fill works), but it makes torch.uint16, uint32 and uint64 available as a dtype.

Towards #58734

Signed-off-by: Edward Z. Yang <ezyang@meta.com>
Pull Request resolved: #116594
Approved by: https://github.com/albanD
ghstack dependencies: #116698, #116693
@ionutmodo
Copy link
ionutmodo commented Mar 6, 2024 •
edited
Loading

Hi guys! I would like to add another usage for uint16 on GPUs, which can be used in designing efficient adaptive sparse optimizers.

thewtex added a commit to thewtex/ITK-Wasm that referenced this issue Apr 18, 2024
@thewtex
feat(downsample): additional pixel types for downsample-label-image
ffb6e2c 
When working with torch, the output is often float32. Torch does not
have good support for conversion to uint types:

  pytorch/pytorch#58734

Support float32 and the signed integer types for convenience.
@Felix-Petersen Felix-Petersen mentioned this issue Jun 26, 2024
Open
@Alarmod Alarmod mentioned this issue Aug 23, 2024
Closed
2 tasks
@tylerjereddy tylerjereddy mentioned this issue Sep 7, 2024
Merged
@rbelew
Copy link
rbelew commented Sep 10, 2024

ANother uint64 use case arises trying to embed hashes from spaCy's token hashing function. Spacy uses hashing on texts to get unique ids (cf. SO )

>>> import spacy
>>> nlp = spacy.load('en')
>>> text = "here is some test text"
>>> doc = nlp(text)
>>> [token.norm for token in doc]
[411390626470654571, 3411606890003347522, 7000492816108906599, 1618900948208871284, 15099781594404091470]

@vadimkantorov
Copy link
Contributor
vadimkantorov commented Sep 11, 2024 •
edited
Loading

Another uint16 usecase is for natural representing UTF-16 code points... e.g. as one of ways of interop/representing arrays of stings as tensors:

@vadimkantorov
Copy link
Contributor
vadimkantorov commented Sep 11, 2024 •
edited
Loading

And another usecase for unsigned dtypes is bitshifts, as bitshift for signed dtypes (if signed bit is set) is theoretically implementation-defined or undefined

@oliver-batchelor
Copy link

@vadimkantorov @rbelew In case you didn't realise torch >= 2.3 has all the unsigned int types.

@vadimkantorov
Copy link
Contributor

Yeah, I saw the support, but I didn't realize which ops are enabled for them. E.g. is bitshift enabled?

@ezyang
Copy link
Contributor
ezyang commented Sep 12, 2024

@malfet just added bitshift. Op support is on a per request basis: we will add it if requested

@vadimkantorov
Copy link
Contributor

just added bitshift.

If you are meaning the #135525, it only added and/or/xor, but maybe bitshifts were added in some other PRs...

@FL33TW00D
Copy link

Echoing the comment from @neelnanda-io, would be great to add uint16 support for Embedding indices. A good chunk of vocabs are smaller than 65535 :)

@ezyang
Copy link
Contributor
ezyang commented Oct 12, 2024

for embedding I think this involves beefing up index_select

@vadimkantorov
Copy link
Contributor
vadimkantorov commented Oct 12, 2024 •
edited
Loading

related on supporting natively shorted bitwidth indexes:

(might also be useful for the sparse tensors, wherever indexes are used to also support shorter dtypes including unsigned, and not only int64)

and also related (on efficient support of shorter-width dtypes for Embedding - but back then in the signed context):

@vadimkantorov
Copy link
Contributor

@malfet just added bitshift.

@ezyang it appears that not yet:

@malfet
Copy link
Contributor
malfet commented Oct 28, 2024 •
edited
Loading

@malfet just added bitshift.

@ezyang it appears that not yet:

[Edit] You are right, it's not there yet, let me propose a PR that extends bitshifts to those...

@mdhaber mdhaber mentioned this issue Jan 29, 2025
Open
@ev-br
Copy link
Collaborator
ev-br commented Feb 26, 2025

As a data point, we tried enabling unsigned integers in the Array API test suite, in data-apis/array-api-compat#253

That results in 70-odd failures, the majority of which are of the *_cpu is not implemented for uint16 variety.
F80F

@ev-br ev-br mentioned this issue Feb 26, 2025
Closed
3 tasks
@mdhaber
Copy link
mdhaber commented Apr 15, 2025 •
edited
Loading

I also ran accross those *_cpu is not implemented for uint* errors (#58734 (comment)) when testing marray (array API masked arrays) with PyTorch as the backend (mdhaber/marray#110).

I thought I'd also report:

import torch as xp
x = xp.asarray([1, 2, 3], dtype=xp.uint64)
sentinel = xp.iinfo(xp.uint64).max // 2 + 1
x[0] = sentinel - 1  # OK
x[0] = xp.asarray(sentinel, dtype=x.dtype)  # OK
x[0] = sentinel  # RuntimeError: Overflow when unpacking long

Since operations are added by request (#58734 (comment)), I'd like to request the following. These are chosen because they are defined by the standard yet array-api-compat cannot (or does not) compensate for them:

import operator as op
# from array_api_compat import torch as xp
import torch as xp

x = xp.asarray(1, dtype=xp.uint16)
A = xp.asarray([[1]], dtype=xp.uint16)
i = xp.asarray([True])

abs(x) # "abs_cpu" not implemented for 'UInt16'
x + x  # "add_stub" not implemented for 'UInt16'
xp.arange(10, dtype=xp.uint16)  # "arange_cpu" not implemented for 'UInt16'
~x  # "bitwise_not_cpu" not implemented for 'UInt16'
x // x  # "div_floor_cpu" not implemented for 'UInt16'
x[i]  # "index_cpu" not implemented for 'UInt16'
x > x  # "gt_cpu" not implemented for 'UInt16'
x >= x  # "ge_cpu" not implemented for 'UInt16'
x < x  # "lt_cpu" not implemented for 'UInt16'
x <= x  # "le_cpu" not implemented for 'UInt16'
x << x  # "lshift_cpu" not implemented for 'UInt16'
A[i] = 2  # "masked_fill" not implemented for 'UInt16'
xp.maximum(x, x)  # "maximum_cpu" not implemented for 'UInt16'
xp.minimum(x, x)  # "minimum_cpu" not implemented for 'UInt16'
-x  # "neg_cpu" not implemented for 'UInt16'
xp.nextafter(x, x)  # "nextafter_cpu" not implemented for 'UInt16'
x % x  # "remainder_cpu" not implemented for 'UInt16'
x >> x  # "rshift_cpu" not implemented for 'UInt16'
xp.sign(x)  # "sign_cpu" not implemented for 'UInt16'
xp.tril(A)  # "tril" not implemented for 'UInt16'
xp.triu(A)  # "triu" not implemented for 'UInt16'
xp.where(i, x, x)  # "where_cpu" not implemented for 'UInt16'

That's:

  • abs_cpu
  • add_stub
  • arange_cpu
  • bitwise_not_cpu
  • div_floor_cpu
  • index_cpu
  • gt_cpu
  • ge_cpu
  • lt_cpu
  • le_cpu
  • lshift_cpu
  • masked_fill
  • maximum_cpu
  • minimum_cpu
  • neg_cpu
  • nextafter_cpu
  • remainder_cpu
  • rshift_cpu
  • sign_cpu
  • tril_cpu
  • triu_cpu
  • where_cpu

@vboussot vboussot mentioned this issue Apr 16, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ezyang's list Stuff ezyang doesn't want to lose feature A request for a proper, new feature. module: python array api Issues related to the Python Array API oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
0