8000 refactor face detection and recognition by UnaNancyOwen · Pull Request #21565 · opencv/opencv · GitHub
[go: up one dir, main page]
Skip to content

refactor face detection and recognition #21565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 49 commits into from
Closed

refactor face detection and recognition #21565

wants to merge 49 commits into from

Conversation

UnaNancyOwen
Copy link
Contributor
@UnaNancyOwen UnaNancyOwen commented Feb 4, 2022 •
edited
Loading

This pull request is proposal implementation of high level api for face detection and recognition.
Please see #20874 for this proposal deteal.

  • Re-implementation face detection (cv::dnn::FaceDetectionModel_YN) and recognition (cv::dnn::FaceRecognitionModel_SF) to high level api extended from cv::dnn::Model.
  • Add support for OpenCV Face Detector to face detection (cv::dnn::FaceDetectionModel_SSD).
  • Change the output format to be user-friendly from cv::FaceDetectorYN style.
    (Separate results to bounding box, confidence, and landmark.)
  • Prepare sample programs and tutorials.
  • Add test code.
  • Remove cv::FaceDetectorYN and cv::FaceRecognizerSF from objdetect module.
    (Note: I think the CI for iOS will pass once this task is complete.)
  • Update branch after merge Use YuNet of fixed input shape to fix not-supported-dynamic-zero-shape for FaceDetectorYN #21607 and update 5.x branch
  • Fix model file name (yunet-202202.onnx)

Users can use face detection and recognition in the following samples with this proposal.
It is as easy to use with unified usability as the other high level apis.

What do you think this proposal? Thanks,

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@UnaNancyOwen
Copy link
Contributor Author
UnaNancyOwen commented Feb 14, 2022 •
edited
Loading

This error occurs in OpenCV 5.x, but not occurs in OpenCV 4.x.
However, the inference will be executed correctly and face is detected correctly in both versions.
This error would be a separate problem from this pull request.

OpenCV 4.x OpenCV 5.x
cv::FaceDetectionModel_YN
(new from this pull request)		 - occurs
cv::FaceDetectorYN
(old)		 not occurs occurs 
[ERROR:0@109.908] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 243 [ 0 0 0 51 ]
[ERROR:0@109.908] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 250 [ 0 0 0 34 ]
[ERROR:0@109.908] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 257 [ 0 0 0 34 ]
[ERROR:0@109.908] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 264 [ 0 0 0 51 ]
[ERROR:0@109.909] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 297 [ 0 -4 ]

@UnaNancyOwen
Copy link
Contributor Author

This error occurs in OpenCV 5.x, but not occurs in OpenCV 4.x. However, the inference will be executed correctly and face is detected correctly in both versions. This error would be a separate problem from this pull request.

OpenCV 4.x OpenCV 5.x
cv::FaceDetectionModel_YN
(new from this pull request) - occurs
cv::FaceDetectorYN
(old) not occurs occurs

[ERROR:0@109.908] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 243 [ 0 0 0 51 ]
[ERROR:0@109.908] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 250 [ 0 0 0 34 ]
[ERROR:0@109.908] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 257 [ 0 0 0 34 ]
[ERROR:0@109.908] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 264 [ 0 0 0 51 ]
[ERROR:0@109.909] global /build/precommit_linux64/5.x/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2516) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 297 [ 0 -4 ]

It is fixed by #21607 .

@alalek
Copy link
Member
alalek commented Feb 14, 2022

but not occurs in OpenCV 4.x.

This is not correct. There is the same problem in 4.x.
They are just messages without triggered error exceptions (disabled).

@UnaNancyOwen
Copy link
Contributor Author

This is not correct. There is the same problem in 4.x.
They are just messages without triggered error exceptions (disabled).

Thanks, I understood it.

@UnaNancyOwen UnaNancyOwen changed the title [wip] refactor face detection and recognition refactor face detection and recognition Feb 15, 2022
@UnaNancyOwen
Copy link
Contributor Author

BTW, 5.x branch doesn't have updates from 4.x since December 2021

@alalek Thanks, I will operate to update the branch of this pull request when the 5.x branch is updated. However, please marge this pull requests #21607 (it is target to 4.x) before update 5.x branch.

sturkmen72
sturkmen72 reviewed Feb 19, 2022
View reviewed changes
@UnaNancyOwen
Copy link
Contributor Author

@fengyuentau yunet-202202.onnx requere to fix input size to 120x160?

@fengyuentau
Copy link
Member

@fengyuentau yunet-202202.onnx requere to fix input size to 120x160?

Yes for ONNX and no for OpenCV DNN. OpenCV DNN does not support ONNX models of dynamic input shape, but it runs the graph on the actual input shape.

@UnaNancyOwen
Copy link
Contributor Author

@fengyuentau Thanks, I have updated the model, but the test fails. If you notice anything, please give a comment.

@UnaNancyOwen
Copy link
Contributor Author

@fengyuentau In the 5.x/HEAD, cv::FaceDetectorYN (objdetect api) with yunet_120x160.onnx does not work correctly regardless of this pull request? Please confirm.

@fengyuentau
Copy link
Member
fengyuentau commented Feb 25, 2022 •
edited
Loading

I tested the latest 5.x and the unit tests of cv::FaceDetectorYN passed. BTW, latest 5.x is using yunet_120x160:

opencv/modules/objdetect/test/test_face.cpp

Line 81 in aa50552

std::string model = findDataFile("dnn/onnx/models/yunet-202202.onnx", false);

opencv/modules/objdetect/test/test_face.cpp

Line 181 in aa50552

std::string detect_model = findDataFile("dnn/onnx/models/yunet-202202.onnx", false);

@UnaNancyOwen
Copy link
Contributor Author

@fengyuentau Sorry, It was my mistake and misunderstanding. I have solved this problem. Thanks,

@UnaNancyOwen
Copy link
Contributor Author

@alalek @fengyuentau All tests are now green. Please review it.

@asmorkalov asmorkalov closed this Jul 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sturkmen72 sturkmen72 sturkmen72 left review comments

@fengyuentau fengyuentau fengyuentau left review comments

@vpisarev vpisarev Awaiting requested review from vpisarev

Assignees
No one assigned
Labels
category: dnn RFC
Projects
None yet
Milestone
5.0-alpha
Development

Successfully merging this pull request may close these issues.
5 participants
@UnaNancyOwen @alalek @fengyuentau @sturkmen72 @asmorkalov
0