8000 (5.x) Merge 4.x by asmorkalov · Pull Request #24486 · opencv/opencv · GitHub
[go: up one dir, main page]

Skip to content

(5.x) Merge 4.x #24486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 45 commits into from
Nov 8, 2023
Merged

(5.x) Merge 4.x #24486

merged 45 commits into from
Nov 8, 2023

Conversation

asmorkalov
Copy link
Contributor

OpenCV Contrib: opencv/opencv_contrib#3587
Opencv Extra: opencv/opencv_extra#1119

#21066 from andrewerf:21052-openvino-native-onnx
#23879 from asmorkalov/release_3.4.20
#23894 from kallaballa:blobFromImagesWithParams
#23929 from CNClareChen:4.x
#24092 from Aser-Abdelfatah:GSoC_Support_GatherElements_ONNX
#24168 from autoantwort:missing-include-3
#24322 from Abdurrahheem:ash/dev_einsum_ellips
#24434 from Kumataro:fixWarningForGCC13
#24363 from cudawarped:videoio_ffmpeg_add_stream_encapsulation
#24367 from alexlyulkov:al/fixed-cumsum-inplace-flag
#24371 from hanliutong:clean-up
#24372 from Kumataro:fix24369
#24382 from sisakat:cuda-compile-multicore
#24405 from kochanczyk:4.x
#24409 from fengyuentau:norm_kernel
#24412 from vrabaud:inter_area1
#24417 from asmorkalov:as/resize_interarea_perf
#24420 from vpisarev:fp16bf16_arithm
#24422 from mshabunin:update-ade-012d
#24425 from fengyuentau:fix_timvx_test
#24427 from fengyuentau:gather_elements_fp16
#24429 from vrabaud:inter_area1
#24438 from vrabaud:avif_check
#24440 from COOLIRON2311:4.x
#24441 from vrabaud:avif_check
#24451 from eplankin:3.4
#24454 from komakai:refactorObjcRange
#24461 from fengyuentau:tracker_vit_backend_target
#24468 from asmorkalov:as/python_ctor_docs
#24478 from CCInc:mingw_fix

Previous "Merge 4.x": #24416

asmorkalov and others added 30 commits June 27, 2023 14:29
Ensure interarea algorithm usage in resize perfomance test
…ace-flag

Fixed CumSum layer inplace flag opencv#24367

When exclusive is false:
dst[i] = dst[i-1] + src[i]
When exclusive is true:
dst[i] = dst[i-1] + src[i-1]
So CumSum layer can be inplace only when exclusive flag is false.
…herElements_ONNX

GSoC Add ONNX Support for GatherElements opencv#24092

Merge with: opencv/opencv_extra#1082
Adds support to the ONNX operator GatherElements [operator docs](https://github.com/onnx/onnx/blob/main/docs/Operators.md#GatherElements)
Added tests to opencv_extra at pull request opencv/opencv_extra#1082

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
…FP16, NEON_BF16) (opencv#24420)

* added more or less cross-platform (based on POSIX signal() semantics) method to detect various NEON extensions, such as FP16 SIMD arithmetics, BF16 SIMD arithmetics, SIMD dotprod etc. It could be propagated to other instruction sets if necessary.

* hopefully fixed compile errors

* continue to fix CI

* another attempt to fix build on Linux aarch64

* * reverted to the original method to detect special arm neon instructions without signal()
* renamed FP16_SIMD & BF16_SIMD to NEON_FP16 and NEON_BF16, respectively

* removed extra whitespaces
Supporting protobuf v22 and later(with abseil-cpp/C++17) opencv#24372

fix opencv#24369
related opencv#23791

1. This patch supports external protobuf v22 and later, it required abseil-cpp and c++17.
    Even if the built-in protobuf is upgraded to v22 or later, 
    the dependency on abseil-cpp and the requirement for C++17 will continue.
2. Some test for caffe required patched protobuf, so this patch disable them.

This patch is tested by following libraries.
-  Protobuf:                    /usr/local/lib/libprotobuf.so (4.24.4)
-  abseil-cpp:                YES (20230125)

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Enable multicore CUDA compilation opencv#24382

CUDA source files are compiled single threaded. The option `--threads` was introduced in NVCC 11.2. The option specifies the number of threads to be used for compilation (see [NVIDIA NVCC Documentation](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#threads-number-t)).

With CMake 3.12 the environment variable `CMAKE_BUILD_PARALLEL_LEVEL` was introduced (see [CMake Documentation](https://cmake.org/cmake/help/latest/envvar/CMAKE_BUILD_PARALLEL_LEVEL.html)). This variable is used to set the NVCC `--threads` option.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Speed up line merging in INTER_AREA opencv#24412

This provides a 10 to 20% speed-up.

Related perf test fix: opencv#24417
This is a split of opencv#23525 that will be updated to only deal with column merging.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
This pointer is called unconditionally in BarcodeImpl::initDecode
assuming the size of the image is outside the specified bounds.  This
seems to not cause problems on optimized builds, I assume because the
optimizer sees through the processImageScale call to see that it can be
reduced to a resize call.  Leaving it as is relies on undefined
behavior.

This was the least invasive change I could make, however, it might be
worthwhile to pull up the logic for a resize so that a SuperScale does
not need to be allocated, which seems to be the most common case.
Unconditionally create SuperScale in BarcodeDetector to avoid null deref
…onnx

Native ONNX to Inference Engine backend opencv#21066

Resolves opencv#21052

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
- [x] The PR is proposed to proper branch
- [x] There is reference to original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
- [ ] The feature is well documented and sample code can be built with the project CMake
Clean up the obsolete API of Universal Intrinsic
dnn: fix HAVE_TIMVX macro definition in dnn test opencv#24425

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
* Optimize some function with lasx.

Optimize some function with lasx. opencv#23929

This patch optimizes some lasx functions and reduces the runtime of opencv_test_core from 662,238ms to 633603ms on the 3A5000 platform.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Pertaining Issue: opencv#5697

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
The API will soon be made no_discard.
Check the return value of avifDecoderSetIOMemory.
tutorial_py_fourier_transform wrong division operator fix
Make sure AVIF decoder is destroyed in case of failure
Abdurrahheem and others added 14 commits October 24, 2023 16:47
Ellipses supported added for Einsum Layer opencv#24322

This PR added addresses issues not covered in opencv#24037. Namely these are: 
Test case for this patch is in this PR [opencv#1106](opencv/opencv_extra#1106) in opencv extra

Added: 
 - [x] Broadcasting reduction "...ii ->...I"
 - [x] Add lazy shape deduction. "...ij, ...jk->...ik"
 
 Features to add: 
- [ ] Add implicit output computation support. "bij,bjk ->" (output subscripts should be "bik")
- [ ] Add support for CUDA backend 
- [ ] BatchWiseMultiply optimize
- [ ] Performance test

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
…ream_encapsulation

videoio: Add raw encoded video stream muxing to cv::VideoWriter with CAP_FFMPEG opencv#24363

Allow raw encoded video streams (e.g. h264[5]) to be encapsulated by `cv::VideoWriter` to video containers (e.g. mp4/mkv).

Operates in a similar way to opencv#15290 where encapsulation is enabled by setting the `VideoWriterProperties::VIDEOWRITER_PROP_RAW_VIDEO` flag when constructing `cv::VideoWriter` e.g.
```
VideoWriter container(fileNameOut, api, fourcc, fps, { width, height }, { VideoWriterProperties::VIDEOWRITER_PROP_RAW_VIDEO, 1 });
```
and each raw encoded frame is passed as single row of a `CV_8U` `cv::Mat`.

The main reason for this PR is to allow `cudacodec::VideoWriter` to output its encoded streams to a suitable container, see opencv/opencv_contrib#3569.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
* fix: supress GCC13 warnings

* fix for review and compile-warning on MacOS
…target

Video tracking (dnn): set backend and target for TrackerVit opencv#24461

Resolves opencv#24460

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Refactor ObjectiveC Range class opencv#24454

### Pull Request Readiness Checklist

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch

Fix for build issue in opencv#24405
Extend the signature of imdecodemulti() opencv#24405

(Edited after addressing Reviewers' comments.)

Add an argument to `imdecodemulti()` to enable optional selection of pages of multi-page images.

Be default, all pages are decoded. If used, the additional argument may specify a continuous selection of pages to decode.


### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [X] I agree to contribute to the project under Apache 2 License.
- [X] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
dnn: add shared fastNorm kernel for mvn, instance norm and layer norm opencv#24409

Relates opencv#24378 (comment)

TODO:

- [x] add fastNorm
- [x] refactor layer norm with fastNorm
- [x] refactor mvn with fastNorm
- [ ] add onnx mvn in importer (in a new PR?)
- [ ] refactor instance norm with fastNorm (in another PR opencv#24378, need to merge this one first though)

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fixed Python signatures in Doxygen documentation.
Fix MinGW build issue due to obsensor
@asmorkalov asmorkalov changed the base branch from 4.x to 5.x November 3, 2023 06:00
@asmorkalov asmorkalov changed the title (5.x) Merge 4.x WIP: (5.x) Merge 4.x Nov 3, 2023
@asmorkalov asmorkalov changed the title WIP: (5.x) Merge 4.x (5.x) Merge 4.x Nov 7, 2023
@asmorkalov
Copy link
Contributor Author

@opencv-alalek the PR is ready for review.

Copy link
Contributor
@opencv-alalek opencv-alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are test failures in CUDA and OpenCL DNN tests.
(Looks like CPU implementation result is changed)

test_layer({2700, 1, 2914}, {2700, 1, 81}, 2);
}

INSTANTIATE_TEST_CASE_P(/**/, Layer_Slice, dnnBackendsAndTargets(false, false));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be:

INSTANTIATE_TEST_CASE_P(/**/, Layer_Slice, dnnBackendsAndTargets(/* withInferenceEngine = */false, /* obsolete_withHalide = */false));

(keep 5.x version)

@asmorkalov
Copy link
Contributor Author

@opencv-alalek I added extra branch to FastNeuralStyle_eccv16 test for CUDA. 4.x and 5.x use different networks and thresholds are not compatible. I suspect the same issue as in #24359

Copy link
Contributor
@opencv-alalek opencv-alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

OpenCL test failure is not a regression of this patch - there is failed nightly build with the same: https://pullrequest.opencv.org/buildbot/builders/5_x-lin64/builds/100293

@asmorkalov asmorkalov merged commit 34f34f6 into opencv:5.x Nov 8, 2023
@asmorkalov asmorkalov mentioned this pull request Jan 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0