8000 Command for clearing old/stale http_cache (Symfony Reverse Proxy) entries · Issue #40548 · symfony/symfony · GitHub
[go: up one dir, main page]
Skip to content

Command for clearing old/stale http_cache (Symfony Reverse Proxy) entries #40548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
PabloKowalczyk opened this issue Mar 23, 2021 · 20 comments
Closed

Command for clearing old/stale http_cache (Symfony Reverse Proxy) entries #40548

PabloKowalczyk opened this issue Mar 23, 2021 · 20 comments
Labels
Feature HttpKernel Stalled

Comments

@PabloKowalczyk
Copy link
Contributor

Description
Hello, I'm using Symfony's HTTP Cache functionality (https://symfony.com/doc/current/http_cache.html), my app is not big, but this cache generates about 50MB (and growing) in entries - I would like to reduce this number. One way is to remove whole directory, but this also removes valid entries. There is a bin/console cache:pool:prune command, but it doesn't clear http_cache directory.

What I'm suggesting here is to extend command cache:pool:prune to also prune http_cache entries or add brand new command, for example http-cache:prune - of course i can provide PR.

WDYT?
@stof
Copy link
Member
stof commented Mar 23, 2021

Extending cache:pool:prune is not the way to go, as HttpCache is not based on a PSR-6 CachePool.

@PabloKowalczyk
Copy link
Contributor Author

So, the only way is to add brand new command? I'm fine with it.

@ro0NL
Copy link
Contributor
ro0NL commented Mar 23, 2021 •
edited
Loading

perhaps related to #40218 if we do use a pool :) why not?

pruning by tag / invalidity invalidting by tag in the http_cache pool would be 💯

see also https://github.com/Toflar/psr6-symfony-http-cache-store

@PabloKowalczyk
Copy link
Contributor Author

perhaps related to #40218 if we do use a pool :) why not?

CacheKernel starts very early, before actual Kernel, without access to DI container - this can be a challenge.

see also https://github.com/Toflar/psr6-symfony-http-cache-store

Nice, thanks.

IMO new command would be best fit to just clear stale entries. It won't touch CacheKernel and can use all Symfony powers (DI).

@ro0NL
Copy link
Contributor
ro0NL commented Mar 24, 2021 •
edited
Loading

i forgot to mention, but FOSHttpCache provides the command we're looking for:
https://github.com/FriendsOfSymfony/FOSHttpCacheBundle/blob/master/src/Command/InvalidateTagCommand.php

it's the more general approach, as it's store based ; not pool based per se.

however, i still think being able to configure "store as a service" for CacheKernel is nice, and with existing Psr6Store we can glue it with cache pools. Eg. framework.http_cache.store: 'my_store_service'

IIUC FrameworkBundle can then mark a public alias (eg. http_cache.store) to be accessed in CacheKernel.

AFAIK we should avoid hitting the backend, not hitting the container per se. Correct me if im wrong tho.

@PabloKowalczyk
Copy link
Contributor Author

i forgot to mention, but FOSHttpCache provides the command we're looking for:
https://github.com/FriendsOfSymfony/FOSHttpCacheBundle/blob/master/src/Command/InvalidateTagCommand.php

If I understand correctly it is invalidating not pruning (clearing) stale entries?

Also FOSHttpCache is separate bundle, what about uses who just wants CacheKernel?

@ro0NL
Copy link
Contributor
ro0NL commented Mar 24, 2021

Right, we cannot prune by tag neither with SF cache pools nor with FOSHttpCache. Is that common though? (eg. compared to pruning everything)

Also FOSHttpCache is separate bundle, what about uses who just wants CacheKernel?

I think it's reasonable to have the https://github.com/Toflar/psr6-symfony-http-cache-store/blob/main/src/Psr6StoreInterface.php contract in core, if only for a leaser amount of deps.

The tricky part is to actually tag cache entries from eg. the response headers, and this area is exactly solved by FOSHttpCache.

So given your store is PSR6 based, you should be able to leverage cache:pool:prune already. Given your pool is configured as well.

@PabloKowalczyk
Copy link
Contributor Author

For me, it will be easier to just add command to prune entries in http_cache, without any PSR integration, which IMO only complicates things.

Adding PSR6 to CacheKernel means refactor and/or rewrite some part of CacheKernel's storage system and then adding command to prune entries.
Point of this issue can be achieved with one, I think small, CLI command. Also, not complicating things may be very welcome for core code :)

Fabien's comment for "PSR6 Store for HttpCache" PR: #20061 (comment)

HttpCache is triggered very early and does not have access to the container or anything else really. And it should stay that way to be efficient.

WDYT?

@ro0NL
Copy link
Contributor
ro0NL commented Mar 25, 2021

protected function createStore() { return new Psr6Store([...]); } in kernel, + cache pool should be all there is to it.

using filesystem is an implementation detail to me.

AFAIK we do have access to the container :)

@PabloKowalczyk
Copy link
Contributor Author

AFAIK we do have access to the container :)

According to https://github.com/symfony/symfony/blob/5.x/src/Symfony/Bundle/FrameworkBundle/HttpCache/HttpCache.php#L78 decorated Kernel is booted only on "forward". Does trying to have DI container means booting kernel even with cache hit?

For me it looks like performance degradation, I know we now have Preloading, OPCache and even JIT compiler, but not running code is faster than running code :)

@ro0NL
Copy link
Contributor
ro0NL commented Mar 25, 2021

i assume a cached compiled container yes :) i dont have numbers, so im not sure how bad the degradation would be.

@PabloKowalczyk
Copy link
Contributor Author
PabloKowalczyk commented Mar 25, 2021 •
edited
Loading

i assume a cached compiled container yes :)

Sure thing, production-ready container, but still code needs to be run :)

Why would you like to make HttpCache's store PSR6 compatible?

@ro0NL
Copy link
Contributor
ro0NL commented Mar 25, 2021 •
edited
Loading

because FOSHttpCache requires it 😅

https://github.com/FriendsOfSymfony/FOSHttpCache/blob/master/src/CacheInvalidator.php#L111

and the framework's semantic config for cache pools is 👌

@PabloKowalczyk
Copy link
Contributor Author

On the other hand, one small command will resolve my issue (and others too, I bet :) ). Is there any problem with command dedicated to pruning HttpCache's storage (and storage is also dedicated)?

@ro0NL
Copy link
Contributor
ro0NL commented Mar 25, 2021

im not sure creating CLI infra around stores is worth the effort, given we have infra for pools already.

i agree a simple command in your app may be sufficient.

@dmaicher
Copy link
Contributor

i agree a simple command in your app may be sufficient.

Well isn't it as simple as rm -rf /the/path/to/the/http_cache_files/*?

Or maybe more advanced to remove only files older than 1 day: find /the/path/to/the/http_cache_files/ -type f -mtime +1 | xargs rm -rf

@ro0NL
Copy link
Contributor
ro0NL commented Mar 25, 2021

in our case we cant afford it. Must be done by tag :) also to me rm -rf is just spartan.

@PabloKowalczyk
Copy link
Contributor Author

@dmaicher The idea is to just prune stale entries, some may have 1 day TTL, others 1 month TTL.

@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

@PabloKowalczyk
Copy link
Contributor Author

In the end I used, proposed earlier, https://github.com/Toflar/psr6-symfony-http-cache-store and it works. IMO it will nice addition to Symfony but my issue was resolved, so this ticket can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature HttpKernel Stalled
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@stof @dmaicher @ro0NL @xabbuh @PabloKowalczyk @carsonbot
0