8000 SF 5.2 doctrine/persistence · Issue #38694 · symfony/symfony · GitHub
[go: up one dir, main page]

Skip to content

SF 5.2 doctrine/persistence #38694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ghost opened this issue Oct 23, 2020 · 18 comments
Closed

SF 5.2 doctrine/persistence #38694

ghost opened this issue Oct 23, 2020 · 18 comments

Comments

@ghost
Copy link
ghost commented Oct 23, 2020

Symfony version(s) affected: 5.2

Description
Hi,

Just to point out that with each update of 5.2 via composer, I am forced after to redo a composer req doctrine/persistence, otherwise I have errors with the default user registration system. The login is fine, but each time it has an impact on the registration and the lost password.

How to reproduce
composer update
I suspect updating dbal

Possible Solution
composer req doctrine/persistence

Additional context
If needed at the next update

@ghost ghost added the Bug label Oct 23, 2020
@Nyholm
Copy link
Member
Nyholm commented Oct 23, 2020

Hey.
I don't really understand this issue. Does your app use doctrine/persistence? if so you should have it in your composer.json.

Are you saying that you get an error from a symfony class because you are missing doctrine/persistence?

@ghost
Copy link
Author
ghost commented Oct 23, 2020

Error list:
vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/AnnotationDriver.php line 178
Can use "yield from" only with arrays and Traversables
Warning: Invalid argument supplied for foreach() resetpasswordrequest
When execute composer req doctrine/persistence, all errors disappear.
My composer :

{
    "type": "project",
    "license": "proprietary",
    "minimum-stability": "beta",
    "require": {
        "php": ">=7.4",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "composer/package-versions-deprecated": "^1.11",
        "doctrine/doctrine-bundle": "^2.1",
        "doctrine/doctrine-migrations-bundle": "^3.0",
        "doctrine/orm": "^2.7",
        "doctrine/persistence": "^2.0",
        "predis/predis": "^1.1",
        "sensio/framework-extra-bundle": "^5.6",
        "symfony/apache-pack": "^1.0",
        "symfony/console": "5.2.*",
        "symfony/dotenv": "5.2.*",
        "symfony/flex": "^1.3.1",
        "symfony/form": "5.2.*",
        "symfony/framework-bundle": "5.2.*",
        "symfony/mailer": "5.2.*",
        "symfony/security-bundle": "5.2.*",
        "symfony/translation": "5.2.*",
        "symfony/validator": "5.2.*",
        "symfony/webpack-encore-bundle": "^1.7",
        "symfony/yaml": "5.2.*",
        "symfonycasts/reset-password-bundle": "^1.1",
        "symfonycasts/verify-email-bundle": "^1.0"
    },
    "require-dev": {
        "symfony/debug-bundle": "5.2.*",
        "symfony/maker-bundle": "^1.21",
        "symfony/phpunit-bridge": "^5.2@beta",
        "symfony/stopwatch": "^5.2@beta",
        "symfony/twig-bundle": "^5.2@beta",
        "symfony/web-profiler-bundle": "^5.2@beta"
    },
    "config": {
        "optimize-autoloader": true,
        "preferred-install": {
            "*": "dist"
        },
        "sort-packages": true
    },
    "autoload": {
        "psr-4": {
            "App\\": "src/"
        }
    },
    "autoload-dev": {
        "psr-4": {
            "App\\Tests\\": "tests/"
        }
    },
    "replace": {
        "paragonie/random_compat": "2.*",
        "symfony/polyfill-ctype": "*",
        "symfony/polyfill-iconv": "*",
        "symfony/polyfill-php72": "*",
        "symfony/polyfill-php71": "*",
        "symfony/polyfill-php70": "*",
        "symfony/polyfill-php56": "*"
    },
    "scripts": {
        "auto-scripts": {
            "cache:clear": "symfony-cmd",
            "assets:install %PUBLIC_DIR%": "symfony-cmd"
        },
        "post-install-cmd": [
            "@auto-scripts"
        ],
        "post-update-cmd": [
            "@auto-scripts"
        ]
    },
    "conflict": {
        "symfony/symfony": "*"
    },
    "extra": {
        "symfony": {
            "allow-contrib": false,
            "require": "5.2.*"
        }
    }
}

@Nyholm
Copy link
Member
Nyholm commented Oct 23, 2020

Thank you. Can you also show me a stack trace?

@ghost
Copy link
Author
ghost commented Oct 23, 2020

I managed to reproduce one.
To reproduce it a simple composer update

Error:
Can use "yield from" only with arrays and Traversables

  at vendor/symfony/validator/Mapping/Loader/AnnotationLoader.php:114
  at Symfony\Component\Validator\Mapping\Loader\AnnotationLoader->getAnnotations()
     (vendor/symfony/validator/Mapping/Loader/AnnotationLoader.php:46)
  at Symfony\Component\Validator\Mapping\Loader\AnnotationLoader->loadClassMetadata()
     (vendor/symfony/validator/Mapping/Loader/LoaderChain.php:54)
  at Symfony\Component\Validator\Mapping\Loader\LoaderChain->loadClassMetadata()
     (vendor/symfony/validator/Mapping/Factory/LazyLoadingMetadataFactory.php:101)
  at Symfony\Component\Validator\Mapping\Factory\LazyLoadingMetadataFactory->getMetadataFor()
     (vendor/symfony/validator/Validator/RecursiveValidator.php:76)
  at Symfony\Component\Validator\Validator\RecursiveValidator->getMetadataFor()
     (vendor/symfony/validator/Validator/TraceableValidator.php:50)
  at Symfony\Component\Validator\Validator\TraceableValidator->getMetadataFor()
     (vendor/symfony/form/Extension/Validator/ValidatorTypeGuesser.php:265)
  at Symfony\Component\Form\Extension\Validator\ValidatorTypeGuesser->guess()
     (vendor/symfony/form/Extension/Validator/ValidatorTypeGuesser.php:38)
  at Symfony\Component\Form\Extension\Validator\ValidatorTypeGuesser->guessType()
     (vendor/symfony/form/FormTypeGuesserChain.php:47)
  at Symfony\Component\Form\FormTypeGuesserChain->Symfony\Component\Form\{closure}()
     (vendor/symfony/form/FormTypeGuesserChain.php:93)
  at Symfony\Component\Form\FormTypeGuesserChain->guess()
     (vendor/symfony/form/FormTypeGuesserChain.php:48)
  at Symfony\Component\Form\FormTypeGuesserChain->guessType()
     (vendor/symfony/form/FormFactory.php:84)
  at Symfony\Component\Form\FormFactory->createBuilderForProperty()
     (vendor/symfony/form/FormBuilder.php:97)
  at Symfony\Component\Form\FormBuilder->create()
     (vendor/symfony/form/FormBuilder.php:244)
  at Symfony\Component\Form\FormBuilder->resolveChildren()
     (vendor/symfony/form/FormBuilder.php:195)
  at Symfony\Component\Form\FormBuilder->getForm()
     (vendor/symfony/form/FormFactory.php:28)
  at Symfony\Component\Form\FormFactory->create()
     (vendor/symfony/framework-bundle/Controller/AbstractController.php:327)
  at Symfony\Bundle\FrameworkBundle\Controller\AbstractController->createForm()
     (src/Controller/RegistrationController.php:32)
  at App\Controller\RegistrationController->register()
     (vendor/symfony/http-kernel/HttpKernel.php:157)
  at Symfony\Component\HttpKernel\HttpKernel->handleRaw()
     (vendor/symfony/http-kernel/HttpKernel.php:79)
  at Symfony\Component\HttpKernel\HttpKernel->handle()
     (vendor/symfony/http-kernel/Kernel.php:195)
  at Symfony\Component\HttpKernel\Kernel->handle()
     (public/index.php:20)                

@ghost
Copy link
Author
ghost commented Oct 24, 2020

For completed :
Composer update today doctrine/annotations
link reset-password

Warning: Invalid argument supplied for foreach()

  at /home/public/www/stellerius/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/AnnotationDriver.php:178
  at Doctrine\Persistence\Mapping\Driver\AnnotationDriver->isTransient()
     (/home/public/www/stellerius/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/MappingDriverChain.php:134)
  at Doctrine\Persistence\Mapping\Driver\MappingDriverChain->isTransient()
     (/home/public/www/stellerius/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:374)
  at Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->isTransient()
     (/home/public/www/stellerius/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaValidator.php:100)
  at Doctrine\ORM\Tools\SchemaValidator->validateClass()
     (/home/public/www/stellerius/vendor/doctrine/doctrine-bundle/DataCollector/DoctrineDataCollector.php:79)
  at Doctrine\Bundle\DoctrineBundle\DataCollector\DoctrineDataCollector->collect()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/Profiler/Profiler.php:161)
  at Symfony\Component\HttpKernel\Profiler\Profiler->collect()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/EventListener/ProfilerListener.php:90)
  at Symfony\Component\HttpKernel\EventListener\ProfilerListener->onKernelResponse()
     (/home/public/www/stellerius/vendor/symfony/event-dispatcher/Debug/WrappedListener.php:117)
  at Symfony\Component\EventDispatcher\Debug\WrappedListener->__invoke()
     (/home/public/www/stellerius/vendor/symfony/event-dispatcher/EventDispatcher.php:230)
  at Symfony\Component\EventDispatcher\EventDispatcher->callListeners()
     (/home/public/www/stellerius/vendor/symfony/event-dispatcher/EventDispatcher.php:59)
  at Symfony\Component\EventDispatcher\EventDispatcher->dispatch()
     (/home/public/www/stellerius/vendor/symfony/event-dispatcher/Debug/TraceableEventDispatcher.php:151)
  at Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->dispatch()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/HttpKernel.php:190)
  at Symfony\Component\HttpKernel\HttpKernel->filterResponse()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/HttpKernel.php:178)
  at Symfony\Component\HttpKernel\HttpKernel->handleRaw()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/HttpKernel.php:79)
  at Symfony\Component\HttpKernel\HttpKernel->handle()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/EventListener/ErrorListener.php:60)
  at Symfony\Component\HttpKernel\EventListener\ErrorListener->onKernelException()
     (/home/public/www/stellerius/vendor/symfony/event-dispatcher/Debug/WrappedListener.php:117)
  at Symfony\Component\EventDispatcher\Debug\WrappedListener->__invoke()
     (/home/public/www/stellerius/vendor/symfony/event-dispatcher/EventDispatcher.php:230)
  at Symfony\Component\EventDispatcher\EventDispatcher->callListeners()
     (/home/public/www/stellerius/vendor/symfony/event-dispatcher/EventDispatcher.php:59)
  at Symfony\Component\EventDispatcher\EventDispatcher->dispatch()
     (/home/public/www/stellerius/vendor/symfony/event-dispatcher/Debug/TraceableEventDispatcher.php:151)
  at Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->dispatch()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/HttpKernel.php:218)
  at Symfony\Component\HttpKernel\HttpKernel->handleThrowable()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/HttpKernel.php:111)
  at Symfony\Component\HttpKernel\HttpKernel->terminateWithException()
     (/home/public/www/stellerius/vendor/symfony/http-kernel/EventListener/DebugHandlersListener.php:118)
  at Symfony\Component\HttpKernel\EventListener\DebugHandlersListener::Symfony\Component\HttpKernel\EventListener\{closure}()
     (/home/public/www/stellerius/vendor/symfony/error-handler/ErrorHandler.php:594)
  at Symfony\Component\ErrorHandler\ErrorHandler->handleException()

If it can help

@xabbuh
Copy link
Member
xabbuh commented Oct 28, 2020

Could you please create a small example application that allows to reproduce your issue?

@Nyholm
Copy link
Member
Nyholm commented Oct 28, 2020

I've seen this too on 5.1. It is related to doctrine annotations I think. But I've never been able to reproduce it. I've only seen in in production logs...

@ghost
Copy link
Author
ghost commented Oct 29, 2020

I cannot reproduce the bug on the test application. I will see to provide you with that this weekend. I have the bug on my application but not on the test. I'll do a toggle and provide you with the basics of the bugging app.

@ghost
Copy link
Author
ghost commented Oct 30, 2020

I only copy my vendor folder, the files from the Entity folder and Repository. By repeating the test, the bug reproduces well on my side. Include vendor folder in the git : https://github.com/wolpheus/sftest

Edit : Link -> reset-password

@ghost
Copy link
Author
ghost commented Oct 30, 2020

Following the various tests, to resolve the problem apparently would be to delete the vendor folder and do a composer install to put everything back. The bug no longer occurs afterwards.

Edit : 31/10 : The problem is always present.
Attach file my config phpinfo
phpinfo.zip

@ghost
Copy link
Author
ghost commented Nov 1, 2020

I think I have found, the problem comes from cleaning the cache after the passage of composer. You have to do a clean cache so that everything comes back in place. Now I don't know why.
Note: I started again from 0 still with the same concerns. However the cache folder is in full permission.
I reinstalled composer in case it came from a configuration.

@xabbuh
Copy link
Member
xabbuh commented Nov 9, 2020

@Wolpheus I am not sure if I were able to follow this completely. Do you mean that the issue is gone when you fully clear the cache of your application?

@ghost
Copy link
Author
ghost commented Nov 10, 2020

Yes, I clean the cache it works well, I do a composer update I end up with these errors, I have to reset the cache.

@xabbuh
Copy link
Member
xabbuh commented Nov 10, 2020

Looks like there is nothing to do then. Thank you for the confirmation.

@xabbuh xabbuh closed this as completed Nov 10, 2020
@Seldaek
Copy link
Member
Seldaek commented Mar 30, 2021

I am also seeing this issue on my User class, but it only happens when the cache has been fully warmed up (and somehow only on the prod machine, I can not repro this locally). If I rm -rf the cache dir then things run fine, but if I do a cache:warmup then it fails again.

I believe this started happening since I added the two custom constraint annotations.

    /**
     * @ORM\Column(type="string", length=180)
     * @EmailRequirements(groups={"registration"})
     * @UniqueUserEmail(groups={"registration"}, allowNonUniqueForInvitations=true)
     */
    private $email;

Here the stack trace:


    "message": "Can use \"yield from\" only with arrays and Traversables",
    "code": 0,
    "file": "./vendor/symfony/validator/Mapping/Loader/AnnotationLoader.php:120",
    "trace": [
        "./vendor/symfony/validator/Mapping/Loader/AnnotationLoader.php:60",
        "./vendor/symfony/validator/Mapping/Loader/LoaderChain.php:54",
        "./vendor/symfony/validator/Mapping/Factory/LazyLoadingMetadataFactory.php:101",
        "./vendor/symfony/validator/Validator/RecursiveContextualValidator.php:306",
        "./vendor/symfony/validator/Validator/RecursiveContextualValidator.php:138",
        "./vendor/symfony/form/Extension/Validator/Constraints/FormValidator.php:105",
        "./vendor/symfony/validator/Validator/RecursiveContextualValidator.php:757",
        "./vendor/symfony/validator/Validator/RecursiveContextualValidator.php:488",
        "./vendor/symfony/validator/Validator/RecursiveContextualValidator.php:313",
        "./vendor/symfony/validator/Validator/RecursiveContextualValidator.php:138",
        "./vendor/symfony/validator/Validator/RecursiveValidator.php:93",
        "./vendor/symfony/form/Extension/Validator/EventListener/ValidationListener.php:50",
        "./vendor/symfony/event-dispatcher/EventDispatcher.php:230",
        "./vendor/symfony/event-dispatcher/EventDispatcher.php:59",
        "./vendor/symfony/event-dispatcher/ImmutableEventDispatcher.php:33",
        "./vendor/symfony/form/Form.php:668",
        "./vendor/symfony/form/Extension/HttpFoundation/HttpFoundationRequestHandler.php:109",
        "./vendor/symfony/form/Form.php:490",
        "./src/UserBundle/Controller/ProfileController.php:115",
        "./vendor/symfony/http-kernel/HttpKernel.php:157",
        "./vendor/symfony/http-kernel/HttpKernel.php:79",
        "./vendor/symfony/http-kernel/Kernel.php:195",
        "./web/app.php:39"
    ]

And the info I could extract from

yield from $this->reader->getPropertyAnnotations($reflection);

  • $reflection at that point is:
object(ReflectionProperty)#1346 (2) {
  ["name"]=>
  string(5) "email"
  ["class"]=>
  string(18) "App\Entity\User"
}
  • $this->reader->getPropertyAnnotations($reflection); is null (which is indeed not array or traversable..).

So it appears the cache warmup fails to find the annotations or something, dumps a null in cache, and then the cached annotation reader returns null and that does not work.

I'm not entirely sure what else to look at here. The cache warmup process isn't something I'm very familiar with.

@Nyholm Nyholm reopened this Mar 30, 2021
@Seldaek
Copy link
Member
Seldaek commented Mar 30, 2021

Alright after a lot of trial and error commenting out random bits of code.. I narrowed it down to this:

an entity with:

/**
 * @CustomConstraint
 */
private $email;

And the smallest custom constraint which triggers the issue:

<?php

namespace App\Validator;

use Symfony\Component\Validator\Constraints\Compound;
use Symfony\Component\Validator\Constraints as Assert;
use Symfony\Component\Validator\Context\ExecutionContextInterface;

/**
 * @Annotation
 */
class CustomConstraint extends Compound
{
    protected function getConstraints(array $options): array
    {
        return [
            new Assert\Callback(function ($value, ExecutionContextInterface $execContext): void {
            })
        ];
    }
}

It appears the Assert\Callback does not play well somehow with the cache warming.

Once this is in place, running the following triggers the error (first warmup works, second run fails while loading the cache):

$ rm -rf var/cache/prod/
$ bin/console cache:warmup --env=prod -v

 // Warming up the cache for the prod environment with debug false

 [OK] Cache for the "prod" environment (debug=false) was successfully warmed.

$ bin/console cache:warmup --env=prod -v

 // Warming up the cache for the prod environment with debug false

18:50:39 CRITICAL  [app] Can use "yield from" only with arrays and Traversables ["exception" => Error { …}]
18:50:39 CRITICAL  [console] Error thrown while running command "cache:warmup --env=prod -v". Message: "Can use "yield from" only with arrays and Traversables" ["exception" => Error { …},"command" => "cache:warmup --env=prod -v","message" => "Can use "yield from" only with arrays and Traversables"]

In AnnotationLoader.php line 120:

  [Error]
  Can use "yield from" only with arrays and Traversables

I hope this helps someone else figure out what the issue is. If Callback constraints can not be used in Compound ones or maybe do not play well with the cache warmer on their own (not sure if related to Compound), then it should ideally fail earlier and more explicitly than the current runtime corruption. If they can be made to work that'd be even better.

As a workaround I guess I will convert my callback constraint to a real constraint, it was mostly laziness that lead me there. I'm curious if @Bleizstudio was also using a Callback?

@Nyholm
Copy link
Member
Nyholm commented Mar 30, 2021

Thank you @Seldaek

I created a small reproducer here: https://github.com/Nyholm/sf-issue-38694

I found the issue and added a patch. See #40645.

@ghost
Copy link
Author
ghost commented Mar 31, 2021

@Seldaek Hi,

Honestly, I have changed my code so much, done tests that I don't know what I was doing. One thing is certain is that it was indeed the User class that was impacted. Anyway congratulations on finding, I don't know enough about Symfony in its design to allow me to have a problem-solving guess. Thank you for being able to push things further :)

Nyholm added a commit that referenced this issue Mar 31, 2021
…pter (nicolas-grekas)

This PR was merged into the 4.4 branch.

Discussion
----------

[Cache] skip storing failure-to-save as misses in ArrayAdapter

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #38694
| License       | MIT
| Doc PR        | -

In addition to #40645

Commits
-------

ba66987 [Cache] skip storing failure-to-save as misses in ArrayAdapter
nicolas-grekas added a commit that referenced this issue Apr 1, 2021
This PR was squashed before being merged into the 4.4 branch.

Discussion
----------

[FrameworkBundle] Dont store cache misses on warmup

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #38694
| License       | MIT
| Doc PR        | symfony/symfony-docs#15172

When we are warming the annotation cache, we are reading all annotation into an `ArrayAdapter`. When we are done we move the values to a `PhpArrayAdapter` to store them in a file.

@Seldaek [found out](#38694 (comment)) that when you are using a custom constraint with a `Symfony\Component\Validator\Constraints\Callback`, there is a strange error like:

> Can use "yield from" only with arrays and Traversables

That is because the `Closure` in the `Symfony\Component\Validator\Constraints\Callback` cannot be serialised and saved to cache. But since the `ArrayAdapter` is also [storing misses as null](#35362), the null values are understood as real values.

When all values are moved to the `PhpArrayAdapter` and we ask the cache for a value (via Doctrine's `CacheProvider`), it will return `null` as a value instead of `false` as a cache miss. And `null` is not something one could "yield from".

Commits
-------

27a22b3 [FrameworkBundle] Dont store cache misses on warmup
symfony-splitter pushed a commit to symfony/framework-bundle that referenced this issue Apr 1, 2021
This PR was squashed before being merged into the 4.4 branch.

Discussion
----------

[FrameworkBundle] Dont store cache misses on warmup

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #38694
| License       | MIT
| Doc PR        | symfony/symfony-docs#15172

When we are warming the annotation cache, we are reading all annotation into an `ArrayAdapter`. When we are done we move the values to a `PhpArrayAdapter` to store them in a file.

@Seldaek [found out](symfony/symfony#38694 (comment)) that when you are using a custom constraint with a `Symfony\Component\Validator\Constraints\Callback`, there is a strange error like:

> Can use "yield from" only with arrays and Traversables

That is because the `Closure` in the `Symfony\Component\Validator\Constraints\Callback` cannot be serialised and saved to cache. But since the `ArrayAdapter` is also [storing misses as null](symfony/symfony#35362), the null values are understood as real values.

When all values are moved to the `PhpArrayAdapter` and we ask the cache for a value (via Doctrine's `CacheProvider`), it will return `null` as a value instead of `false` as a cache miss. And `null` is not something one could "yield from".

Commits
-------

27a22b34af [FrameworkBundle] Dont store cache misses on warmup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants
0