Fix dataframe arithmetics for columns having several value buffers (column size is more than 2 Gb) #6724
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #6724 +/- ##
========================================
Coverage 68.89% 68.89%
========================================
Files 1216 1216
Lines 250915 251067 +152
Branches 26259 26259
========================================
+ Hits 172857 172966 +109
- Misses 71238 71287 +49
+ Partials 6820 6814 -6
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
Thanks @asmirnov82! Can you merge this into https://github.com/JakeRadMSFT/machinelearning/tree/u/jakerad/generic-math as well? It's possible this was fixed by other changes in the generic math branch. Can you make sure it meets your needs/add/update tests? |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
| for (int b = 0; b < left.Buffers.Count; b++) | ||
| { | ||
| var span = left.Buffers[b].ReadOnlySpan; | ||
| var otherSpan = right.Buffers[b].ReadOnlySpan; | ||
| for (int i = 0; i < span.Length; i++) | ||
| { | ||
| ret[i] = (span[i] == otherSpan[i]); | ||
| ret[index++] = (span[i] == otherSpan[i]); |
There was a problem hiding this comment.
Wow - this was really broken.
There was a problem hiding this comment.
Thanks for catching this.
Hello Jake, currently I am on vacation, will do this, when I return |
Awesome! I merged in all your other changes into the branch. We're working on creating better branching and process for handling changes between current train (ML.NET 3.0) and next release train (ML.NET 4.0). Hopefully we'll have it figured out by the time you return! Thanks again for all these contributions!!! |
|
Merged into Generic Math branch! |
Arithmetics for columns containg multiple buffers (actualy with size more than 2 Gb) was incorrect. Buffer number was not taken into account