fix a few performance drop in some matrix size per data type #2375
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Wang,Long <long1.wang@intel.com>
|
This is applied to all CPU types, right? You mentioned that this improves performance on AVX-512 CPUs, but what about AVX2 CPUs? (Haswell, Zen2, etc.) |
|
The gains seem small on my hardware (but at least I do not see a slowdown), would you happen to have a few numbers from your 16-thread case ? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Wang,Long long1.wang@intel.com
Add this patch to fix a few performance drop in some matrix size. It was caused by the inappropriate GEMM_PREFERED_SIZE value(=32), which set the imbalance workload split per thread.
For example 1280x1280 single precision matrix in 16 threads. It was split into 8 threads with 96 width single precision(32 bits)size and 8 threads with 64 width. But for avx512 alignment, the 16 threads with 80 width single precision(32 bits)size is also acceptable and more important in such case is that the workload is also balanced in each thread.