perf(face-swapper): optimize CoreML session configuration#1656
Open
laurigates wants to merge 1 commit intohacksider:mainfrom
Open
perf(face-swapper): optimize CoreML session configuration#1656laurigates wants to merge 1 commit intohacksider:mainfrom
laurigates wants to merge 1 commit intohacksider:mainfrom
Conversation
- Enable RequireStaticShapes (inswapper inputs are fixed-size 1x3x128x128 face + 1x512 embedding), allowing compile-time optimization of the CoreML graph - Add ModelCacheDirectory to persist compiled CoreML models to ~/.cache/deep-live-cam/coreml/, avoiding 30-60s recompilation on every application restart - Remove invalid MaximumCacheSize option (not a CoreML EP parameter) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Reviewer's guide (collapsed on small PRs)Reviewer's GuideOptimizes the CoreMLExecutionProvider configuration for the face swapper on Apple Silicon by enabling static shapes, adding a persistent CoreML model cache directory, and removing an unsupported session option, while keeping non-CoreML providers unchanged. Sequence diagram for CoreML face swapper initialization with model cachesequenceDiagram
participant App
participant FaceSwapper as get_face_swapper
participant ORT as OrtInferenceSession
participant CoreMLExecutionProvider
participant FileSystem as CoreMLModelCacheDirectory
App->>FaceSwapper: get_face_swapper()
FaceSwapper->>FileSystem: os.makedirs(coreml_cache_dir, exist_ok=True)
FaceSwapper->>ORT: Create session with CoreMLExecutionProvider
activate ORT
ORT->>CoreMLExecutionProvider: Initialize with RequireStaticShapes=1
CoreMLExecutionProvider->>FileSystem: Check for compiled model in cache
alt First startup (no cached model)
CoreMLExecutionProvider->>CoreMLExecutionProvider: Compile ONNX to MLProgram
CoreMLExecutionProvider->>FileSystem: Persist compiled model to cache
else Subsequent startups (cached model exists)
CoreMLExecutionProvider->>FileSystem: Load compiled model from cache
end
ORT-->>FaceSwapper: Initialized inference session
deactivate ORT
FaceSwapper-->>App: Face swapper instance ready
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- Consider moving the cache directory creation (
coreml_cache_dircalculation andos.makedirs) outside the providers loop so it only runs once perget_face_swapper()call instead of once per provider. - Using
~/.cache/...directly may not be ideal on macOS; you might want to respect OS-specific cache locations (e.g.~/Library/CachesorXDG_CACHE_HOME) to better align with user expectations and system conventions.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider moving the cache directory creation (`coreml_cache_dir` calculation and `os.makedirs`) outside the providers loop so it only runs once per `get_face_swapper()` call instead of once per provider.
- Using `~/.cache/...` directly may not be ideal on macOS; you might want to respect OS-specific cache locations (e.g. `~/Library/Caches` or `XDG_CACHE_HOME`) to better align with user expectations and system conventions.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The CoreML ExecutionProvider configuration for the face swapper model can be improved for Apple Silicon:
RequireStaticShapes: 1— The inswapper model has fixed-size inputs (1x3x128x128 face crop + 1x512 embedding). Enabling static shapes allows the CoreML compiler to optimize the graph at compile time rather than handling dynamic shapes at runtime.ModelCacheDirectory— Persists the compiled CoreML model to~/.cache/deep-live-cam/coreml/. Without this, CoreML recompiles the ONNX → MLProgram conversion on every application startup, which takes 30-60 seconds on first run. With caching, subsequent startups load the pre-compiled model in ~1 second.Remove
MaximumCacheSize— This is not a valid CoreML EP session option and was silently ignored.Changes
modules/processors/frame/face_swapper.py: Update CoreML provider config inget_face_swapper()Testing
CoreMLExecutionProvider)Generated with Claude Code
Summary by Sourcery
Optimize CoreML execution provider configuration for the face swapper model on Apple Silicon to improve startup performance and compilation behavior.
New Features:
Enhancements: