8000 ImageFilter jerky scrolling behavior on IOS · Issue #144213 · flutter/flutter · GitHub
[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ImageFilter jerky scrolling behavior on IOS #144213

Closed
snoopdoggy322 opened this issue Feb 27, 2024 · 11 comments · Fixed by flutter/engine#51395
Closed

ImageFilter jerky scrolling behavior on IOS #144213

snoopdoggy322 opened this issue Feb 27, 2024 · 11 comments · Fixed by flutter/engine#51395
Assignees
@jonahwilliams
Labels
c: regression It was better in the past than it is now c: rendering UI glitches reported at the engine/skia or impeller rendering level e: impeller Impeller rendering backend issues and features requests engine flutter/engine repository. See also e: labels. found in release: 3.19 Found to occur in 3.19 found in release: 3.20 Found to occur in 3.20 has reproducible steps The issue has been confirmed reproducible and is ready to work on P1 High-priority issues at the top of the work list platform-ios iOS applications specifically team-engine Owned by Engine team

Comments

@snoopdoggy322
Copy link
snoopdoggy322 commented Feb 27, 2024 •
edited
Loading

Steps to reproduce

  1. run code sample on ios device/simulator
  2. scroll

Expected results

normall scrolling

Actual results

jerky scroll

Code sample

Code sample 
import 'dart:ui';

import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      home: MyHomePage(title: 'Scrolling ios blur 3.19.1'),
    );
  }
}

class MyHomePage extends StatelessWidget {
  const MyHomePage({
    super.key,
    required this.title,
  });

  final String title;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      extendBody: true,
      extendBodyBehindAppBar: true,
      appBar: AppBar(
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        title: Text(title),
      ),
      body: ListView.separated(
        itemCount: 20,
        itemBuilder: (_, i) => ClipRect(
          child: ImageFiltered(
            imageFilter: ImageFilter.blur(sigmaX: 25, sigmaY: 25),
            child: Image.network(
              'https://cdn.pixabay.com/photo/2018/03/31/19/29/schnitzel-3279045_1280.jpg',
              height: 200,
              fit: BoxFit.cover,
            ),
          ),
        ),
        separatorBuilder: (_, __) => const SizedBox(height: 80),
      ),
    );
  }
}

Screenshots or Video

without clipping
2024-02-27.11.37.28.mov
with clipping
2024-02-27.11.38.06.mov

Logs

Logs 
[Paste your logs here]

Flutter Doctor output

Doctor output 
[✓] Flutter (Channel stable, 3.19.1, on macOS 14.0 23A344 darwin-arm64, locale ru-RU)
    • Flutter version 3.19.1 on channel stable at /Users/vlad/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision abb292a07e (6 дней назад), 2024-02-20 14:35:05 -0800
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0-rc3)
    • Android SDK at /Users/vlad/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0-rc3
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15C500b
    • CocoaPods version 1.14.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio 2.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] Connected device (4 available)
    • iPhone (Vladislav) (mobile) • 00008101-000550840EE2001E            • ios            • iOS 17.3.1 21D61
    • iPhone 15 Pro Max (mobile)  • 7B2109E5-68D8-4D3B-A77B-D1E2F1A30AA8 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-17-0
      (simulator)
    • macOS (desktop)             • macos                                • darwin-arm64   • macOS 14.0 23A344 darwin-arm64
    • Chrome (web)                • chrome                               • web-javascript • Google Chrome 122.0.6261.69

[✓] Network resources
    • All expected network resources are available.

• No issues found!
< 8000 div class="avatar-parent-child TimelineItem-avatar d-none d-md-block"> @snoopdoggy322
Copy link
Author

If you use BackdropFilter instead of ImageFiltered during layout, the result is the same

@dam-ease dam-ease added the in triage Presently being triaged by the triage team label Feb 27, 2024
@dam-ease
Copy link
dam-ease commented Feb 27, 2024 •
edited
Loading

Thanks for the report @snoopdoggy322.

I can reproduce this on the latest stable channel, on master, the jerk with and without Clipping isn't as pronounced as on stable but it also occurs on the latest master channel.

Without Clipping

stable master
Simulator.Screen.Recording.-.iPhone.15.-.2024-02-27.at.10.36.21.mp4
Simulator.Screen.Recording.-.iPhone.15.-.2024-02-27.at.10.32.38.mp4

With Clipping

stable master
Simulator.Screen.Recording.-.iPhone.15.-.2024-02-27.at.10.35.47.mp4
Simulator.Screen.Recording.-.iPhone.15.-.2024-02-27.at.10.33.19.mp4

This issue is also peculiar to impeller as without impeller, I'm unable to reproduce this. This might be similar/related to #143947.

Screen Record

Simulator.Screen.Recording.-.iPhone.15.-.2024-02-27.at.10.38.38.mp4

Additional Note from Triage: Tested this on version 3.16.9 and I'm unable to reproduce both with and without Clipping.

stable, master flutter doctor -v 

[!] Flutter (Channel stable, 3.19.1, on macOS 14.2.1 23C71 darwin-arm64, locale
    en-NG)
    • Flutter version 3.19.1 on channel stable at
      /Users/damilolaalimi/sdks/flutter
    ! Warning: `dart` on your path resolves to
      /opt/homebrew/Cellar/dart/3.1.5/libexec/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/damilolaalimi/sdks/flutter.
      Consider adding /Users/damilolaalimi/sdks/flutter/bin to the front of your
      path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision abb292a07e (6 days ago), 2024-02-20 14:35:05 -0800
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/damilolaalimi/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/damilolaalimi/Library/Android/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15C500b
    • CocoaPods version 1.14.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      17.0.6+0-17.0.6b802.4-9586694)

[!] Android Studio (version unknown)
    • Android Studio at /Users/damilolaalimi/Downloads/Android Studio
      Preview.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    ✗ Unable to determine Android Studio version.
    • Java version OpenJDK Runtime Environment (build
      17.0.7+0-17.0.7b1000.6-10550314)

[✓] VS Code (version 1.86.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.78.0

[✓] VS Code (version 1.83.1)
    • VS Code at /Users/damilolaalimi/Downloads/Visual Studio Code.app/Contents
    • Flutter extension version 3.78.0

[✓] Connected device (5 available)
    • sdk gphone64 arm64 (mobile) • emulator-5554                        •
      android-arm64  • Android 14 (API 34) (emulator)
    • Damilola’s iPhone (mobile)  • 00008110-001964480AE1801E            • ios
      • iOS 17.1.1 21B91
    • iPhone 15 (mobile)          • DBCC5388-DA57-470E-8F0E-477EA0AA3C3A • ios
      • com.apple.CoreSimulator.SimRuntime.iOS-17-2 (simulator)
    • macOS (desktop)             • macos                                •
      darwin-arm64   • macOS 14.2.1 23C71 darwin-arm64
    • Chrome (web)                • chrome                               •
      web-javascript • Google Chrome 122.0.6261.69

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 2 categories.

[!] Flutter (Channel master, 3.20.0-13.0.pre.47, on macOS 14.2.1 23C71 darwin-arm64, locale en-NG)
    • Flutter version 3.20.0-13.0.pre.47 on channel master at /Users/damilolaalimi/fvm/versions/master
    ! Warning: `dart` on your path resolves to /opt/homebrew/Cellar/dart/3.1.5/libexec/bin/dart, which is not inside your current Flutter SDK checkout at /Users/damilolaalimi/fvm/versions/master. Consider adding /Users/damilolaalimi/fvm/versions/master/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 331769f397 (3 hours ago), 2024-02-26 23:50:40 -0500
    • Engine revision 0bc21ea7bc
    • Dart version 3.4.0 (build 3.4.0-179.0.dev)
    • DevTools version 2.33.0-dev.11
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/damilolaalimi/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/damilolaalimi/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15C500b
    • CocoaPods version 1.14.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[!] Android Studio (version unknown)
    • Android Studio at /Users/damilolaalimi/Downloads/Android Studio Preview.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    ✗ Unable to determine Android Studio version.
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] VS Code (version 1.86.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.78.0

[✓] VS Code (version 1.83.1)
    • VS Code at /Users/damilolaalimi/Downloads/Visual Studio Code.app/Contents
    • Flutter extension version 3.78.0

[✓] Connected device (6 available)
    • sdk gphone64 arm64 (mobile)     • emulator-5554                        • android-arm64  • Android 14 (API 34) (emulator)
    • Damilola’s iPhone (mobile)      • 00008110-001964480AE1801E            • ios            • iOS 17.1.1 21B91
    • iPhone 15 (mobile)              • DBCC5388-DA57-470E-8F0E-477EA0AA3C3A • ios            • com.apple.CoreSimulator.SimRuntime.iOS-17-2 (simulator)
    • macOS (desktop)                 • macos                                • darwin-arm64   • macOS 14.2.1 23C71 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad                • darwin         • macOS 14.2.1 23C71 darwin-arm64
    • Chrome (web)                    • chrome                               • web-javascript • Google Chrome 122.0.6261.69

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 2 categories.

@dam-ease dam-ease added platform-ios iOS applications specifically engine flutter/engine repository. See also e: labels. c: rendering UI glitches reported at the engine/skia or impeller rendering level has reproducible steps The issue has been confirmed reproducible and is ready to work on e: impeller Impeller rendering backend issues and features requests team-engine Owned by Engine team found in release: 3.19 Found to occur in 3.19 found in release: 3.20 Found to occur in 3.20 c: regression It was better in the past than it is now and removed in triage Presently being triaged by the triage team labels Feb 27, 2024
@jonahwilliams jonahwilliams self-assigned this Mar 4, 2024
@jonahwilliams jonahwilliams added P1 High-priority issues at the top of the work list triaged-engine Triaged by Engine team labels Mar 4, 2024
@jonahwilliams
Copy link
Member

I can reproduce this. If the blur sigma is turned down to low value like 1, then the artifacting is mostly present around the border. Maybe be an alignment issue if the expanded padding for the blur isn't a whole number?

RPReplay_Final1709782752.mp4

@jonahwilliams jonahwilliams removed their assignment Mar 7, 2024
@jonahwilliams jonahwilliams added P2 Important issues not at the top of the work list and removed P1 High-priority issues at the top of the work list labels Mar 7, 2024
@jonahwilliams
Copy link
Member

I think is is actually related to an issue with the sampling mode, if I change the blur to use decal sampling the jerkiness seems to go away ( of course it looks different though)

@jason-simmons
Copy link
Member

This renders without flickering if I remove the subpass_coverage = Rect::RoundOut(subpass_coverage.value()) rounding in EntityPass::GetEntityForElement

(see https://github.com/flutter/engine/blob/main/impeller/entity/entity_pass.cc#L620)

@flar

@jonahwilliams jonahwilliams mentioned this issue Mar 13, 2024
Open
@flar
Copy link
Contributor
flar commented Mar 13, 2024

The underlying problem is that our default filter modes are clamp, which can make sense when you are 8000 applying a filter to a fixed image like a photo that has a distinct edge. In a purely theoretical view, the scrolled contents do have a physical opaque edge, but they are rendered on a non-pixel boundary thereby causing their edges to be fuzzy due to antialiasing. Skia has gone to great lengths to recognize the registration of the logical bounds of the content against the pixel grid so that they can apply the clamp sampling to an estimation of what the opaque edge should have looked like, but I do not think Impeller has that same capability.

The removal of the RoundOut simply allows the contents to be rendered to a "RoundIn" size instead due to the following line which does an implicit unmentioned round operation in the ISize constructor (that happens to be "truncate"), and that artificially makes the edge solid again by clipping off the slightly un-solid edges pixels of the AA sampled content.

The implicit RoundIn behavior that used to happen might hide these effects, but at the cost of accuracy.

In particular, that suggestion may work for a photograph that has little high frequency detail near the edge, but if the content had, say, a single pixel border that was a strikingly contrasting color then that would flash as well due to its contribution growing and shrinking as it scrolls across un-snapped pixels and getting clipped to a RoundIn operation.

We shouldn't break layer clipping as a temporary solution to a lack of treatment of non-integer edge modes in the blur filters.

@jonahwilliams
Copy link
Member

I'm not sure that assessment is accurate. Why does it matter if the coverage of an entity has a fractional value? AFAIK this code is determining the size of the texture to render into and not how it composites with the parent.

@flar
Copy link
Contributor
flar commented Mar 13, 2024

It's the tile mode of "clamp". It says that for content near the edges where you need to sample beyond the edge, what should you return as the sample. For "decal" you sample transparency, but for "clamp" you sample the edge pixel out to infinity. If the edge pixel is constantly changing in opacity as you scroll due to sub-pixel rendering then the intensity of what you sample out to infinity will be constantly changing.

@flar
Copy link
Contributor
flar commented Mar 13, 2024

After a discussion with the Skia folks, the application of clamp tile mode to content that has sub-pixel bounds is not something that they've ever been able to reliably accomplish and the latest implementation simply has a very targeted "round-in", but only when considering content to be supplied to the input of a blur filter with clamp mode. Even that solution has its issues if there is high-frequency pixel deltas near the edge of such an image.

@jonahwilliams jonahwilliams removed the P2 Important issues not at the top of the work list label Mar 13, 2024
@jonahwilliams jonahwilliams self-assigned this Mar 13, 2024
@flutter-tr 8000 iage-bot flutter-triage-bot bot removed the triaged-engine Triaged by Engine team label Mar 14, 2024
@flutter-triage-bot
Copy link

This issue is missing a priority label. Please set a priority label when adding the triaged-engine label.

@jonahwilliams jonahwilliams added the P1 High-priority issues at the top of the work list label Mar 14, 2024
@gaaclarke gaaclarke mentioned this issue Mar 14, 2024
Closed
This was referenced Mar 14, 2024
Merged
Closed
auto-submit bot pushed a commit to flutter/engine that referenced this issue Mar 14, 2024
@jonahwilliams
[Impeller] revert round out. (#51395)
622b372 
This isn't exactly the right fix, but it is a straight revert to fix flutter/flutter#144213
@jonahwilliams jonahwilliams mentioned this issue Mar 14, 2024
Closed
@github-actions GitHub Actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jonahwilliams jonahwilliams

Labels
c: regression It was better in the past than it is now c: rendering UI glitches reported at the engine/skia or impeller rendering level e: impeller Impeller rendering backend issues and features requests engine flutter/engine repository. See also e: labels. found in release: 3.19 Found to occur in 3.19 found in release: 3.20 Found to occur in 3.20 has reproducible steps The issue has been confirmed reproducible and is ready to work on P1 High-priority issues at the top of the work list platform-ios iOS applications specifically team-engine Owned by Engine team
Projects
Skia GPU on iOS Removal Blockers
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Impeller] revert round out. jonahwilliams/engine
5 participants
@jonahwilliams @jason-simmons @snoopdoggy322 @flar @dam-ease
0