Try to use the APCIterator to clear the APC cache #24008

guillaumelecerf · 2017-08-28T14:35:45Z

Follow up here : #24011

Q	A
Branch?	3.3
Bug fix?	yes
New feature?	no
BC breaks?	no
Deprecations?	no
Tests pass?	yes
License	MIT

TL;DR: when APCuIterator is not available (i.e. PHP 5.6 + APCu 4.0.7), the APC cache handling is broken.

When the app initializes one APC cache, it tries to retrieve its namespace flag key.
If the app can't retrieve its flag key, it clears its namespaced cache and adds its flag key back.
But when APCuIterator is not usable, the app flushes the whole cache each time, preventing the others APC caches to retrieve their flag key, resulting in a regular full APC cache flush.

Deeper explanation:

During the cache warmup, \Symfony\Component\Cache\Adapter\AbstractAdapter::createSystemCache() is called multiple times with different namespaces, i.e:

protected function getCache_SystemService()
{
    return $this->services['cache.system'] = \Symfony\Component\Cache\Adapter\AbstractAdapter::createSystemCache('42DFrsyAcq', 0, 'i6ox8pAXPv2yMq+00ATexQ', (__DIR__.'/pools'), ${($_ = isset($this->services['monolog.logger.cache']) ? $this->services['monolog.logger.cache'] : $this->get('monolog.logger.cache', ContainerInterface::NULL_ON_INVALID_REFERENCE)) && false ?: '_'});
}

protected function getCache_AnnotationsService($lazyLoad = true)
{
    return $this->services['cache.annotations'] = \Symfony\Component\Cache\Adapter\AbstractAdapter::createSystemCache('ljDfqzQGel', 0, 'i6ox8pAXPv2yMq+00ATexQ', (__DIR__.'/pools'), ${($_ = isset($this->services['monolog.logger.cache']) ? $this->services['monolog.logger.cache'] : $this->get('monolog.logger.cache', ContainerInterface::NULL_ON_INVALID_REFERENCE)) && false ?: '_'});
}

but when the APCuIterator is not usable, AbstractAdapter does init the APC cache with this code:

private function init($namespace, $defaultLifetime, $version)
<snip>
            if (!apcu_exists($version.'@'.$namespace)) {
                $this->doClear($namespace);
                apcu_add($version.'@'.$namespace, null);
            }
<snip>
}

protected function doClear($namespace)
{
    return isset($namespace[0]) && class_exists('APCuIterator', false) && ('cli' !== PHP_SAPI || ini_get('apc.enable_cli'))
        ? apcu_delete(new \APCuIterator(sprintf('/^%s/', preg_quote($namespace, '/')), APC_ITER_KEY))
        : apcu_clear_cache();
}

Here is the callstack:

getCache_SystemService()
- AbstractAdapter::createSystemCache('42DFrsyAcq', 0, 'i6ox8pAXPv2yMq+00ATexQ')
-- !apcu_exists('i6ox8pAXPv2yMq+00ATexQ@42DFrsyAcq')
--- apcu_clear_cache()
--- apcu_add('i6ox8pAXPv2yMq+00ATexQ@42DFrsyAcq', null);

APC cache contains :
'i6ox8pAXPv2yMq+00ATexQ@42DFrsyAcq' => null

getCache_AnnotationsService()
- AbstractAdapter::createSystemCache('ljDfqzQGel', 0, 'i6ox8pAXPv2yMq+00ATexQ')
-- !apcu_exists('i6ox8pAXPv2yMq+00ATexQ@ljDfqzQGel')
--- apcu_clear_cache()
--- apcu_add('i6ox8pAXPv2yMq+00ATexQ@ljDfqzQGel', null);

APC cache contains :
'i6ox8pAXPv2yMq+00ATexQ@ljDfqzQGel' => null

Resulting in a complete cache reset between the 2 calls, making every subsequent call to apcu_exists($version.'@'.$namespace) false, and so making the whole APC cache completely useless.

nicolas-grekas · 2017-08-28T16:34:24Z

when APCuIterator doesn't exist, does APCIterator exist? can't we use it instead?

nicolas-grekas · 2017-08-28T16:37:08Z

src/Symfony/Component/Cache/Traits/ApcuTrait.php

+            return apcu_delete($keysToDelete);
+        }
+
+        return apcu_clear_cache();


would be great if this return could be first, so that we can remove one level of indentation above

guillaumelecerf · 2017-08-28T17:09:34Z

Added a case for APCIterator, will need some testing though.

nicolas-grekas · 2017-08-28T17:30:31Z

src/Symfony/Component/Cache/Traits/ApcuTrait.php

+            $keysToDelete = new \APCIterator('user', sprintf('/^%s/', preg_quote($namespace, '/')), APC_ITER_KEY);
+        } else {
+            $keysToDelete = array();
+            $cacheInfo = apcu_cache_info();


given that apcu_cache_info exists since v4, but APCIterator since v3.1.1, I think we just don't need this "else" case (and consider APCIterator in the "else" instead, without any "if")

stof · 2017-08-28T17:52:37Z

src/Symfony/Component/Cache/Traits/ApcuTrait.php

@@ -71,9 +71,26 @@ protected function doHave($id)
     */
    protected function doClear($namespace)
    {
-        return isset($namespace[0]) && class_exists('APCuIterator', false) && ('cli' !== PHP_SAPI || ini_get('apc.enable_cli'))


I think we should just remove the false from the class_exists check, to allow using the polyfill implementation of APCuIterator

The polyfill doesn't use autoload to declare the class, so false or true wouldn't change anything.
Which means installing the polyfill is another way to fix this issue immediately yes.

guillaumelecerf · 2017-08-28T18:16:37Z

On tomorrow, I will try this change : https://github.com/guillaumelecerf/symfony/commit/434fdba469340728b696778fd9bb89db99ea1229
and @stof suggestion.

This has the side effect to fix a bug resulting in a non-working APC cache: When the app initializes one APC cache, it tries to retrieve its namespace flag key. If the app can't retrieve its flag key, it clears its namespaced cache and adds its flag key back. But when APCuIterator is not usable, the app flushes the whole cache each time, preventing the others APC caches to retrieve their flag key, resulting in a regular full APC cache flush.

guillaumelecerf · 2017-08-28T20:56:42Z

Follow up here : #24011

…CuIterator everywhere (guillaumelecerf) This PR was merged into the 3.3 branch. Discussion ---------- [Cache] Always require symfony/polyfill-apcu to provide APCuIterator everywhere | Q | A | ------------- | --- | Branch? | 3.3 | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | License | MIT **TL;DR: when APCuIterator is not available (i.e. PHP 5.6 + APCu 4.0.7), the APC cache handling is broken.** When [the app initializes](https://github.com/symfony/symfony/blob/master/src/Symfony/Bundle/FrameworkBundle/Resources/config/cache.xml#L31) one of [the APC caches](https://github.com/symfony/symfony/blob/master/src/Symfony/Bundle/FrameworkBundle/Resources/config/cache.xml#L14-L28), it tries to [retrieve its namespace flag key](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Cache/Traits/ApcuTrait.php#L42). If it can't, [it clears its namespaced cache](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Cache/Traits/ApcuTrait.php#L43) and [adds its flag key back](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Cache/Traits/ApcuTrait.php#L44). But [when APCuIterator is not usable](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Cache/Traits/ApcuTrait.php#L74), **the app [flushes the whole cache each time](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Cache/Traits/ApcuTrait.php#L76)**, preventing all the APC caches to retrieve their own flag key, resulting in a **perpetual full APC cache flush**. TODO: - [x] add test => https://travis-ci.org/symfony/symfony/jobs/269383629#L3334 See #24008 Commits ------- 9d44442 Always require symfony/polyfill-apcu to provide APCuIterator everywhere

carsonbot added Status: Needs Review Bug labels Aug 28, 2017

guillaumelecerf changed the title ~~WIP: Limit APC cache clearing to the provided namespace~~ Do not wipe the whole APC cache when we only want to clear a specific namespace Aug 28, 2017

nicolas-grekas reviewed Aug 28, 2017

View reviewed changes

stof reviewed Aug 28, 2017

View reviewed changes

guillaumelecerf changed the title ~~Do not wipe the whole APC cache when we only want to clear a specific namespace~~ Try to use the APCIterator to clear the APC cache Aug 28, 2017

guillaumelecerf mentioned this pull request Aug 28, 2017

[Cache] Always require symfony/polyfill-apcu to provide APCuIterator everywhere #24011

Merged

1 task

guillaumelecerf closed this Aug 28, 2017

guillaumelecerf deleted the fix_ApcuTrait_doClear branch August 29, 2017 08:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Try to use the APCIterator to clear the APC cache #24008

Try to use the APCIterator to clear the APC cache #24008

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Try to use the APCIterator to clear the APC cache #24008

Try to use the APCIterator to clear the APC cache #24008

Uh oh!

Conversation

Uh oh!

Follow up here : #24011

Deeper explanation:

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!