8000 ErrorException - Notice: unserialize(): Error at offset [X] of [X+2] bytes in vendor/symfony/symfony/src/Symfony/Component/Config/Resource/FileResource.php line 78 after clearing dev cache · Issue #6203 · symfony/symfony · GitHub
[go: up one dir, main page]

Skip to content

ErrorException - Notice: unserialize(): Error at offset [X] of [X+2] bytes in vendor/symfony/symfony/src/Symfony/Component/Config/Resource/FileResource.php line 78 after clearing dev cache #6203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gergelypolonkai opened this issue Dec 6, 2012 · 36 comments
Labels

Comments

@gergelypolonkai
Copy link

Sometime after I clear the dev cache with

php app/console cache:clear --env=dev

and then try to run some other command, like assets:install, I get an error like

  Notice: unserialize(): Error at offset [97 of 99 bytes in /var/www/html/vendor/symfony/symfony/src/Symfony/Component/Config/Resource/FileResource.php line 78

Although I have xdebug installed and enabled, it doesn't generate any backtrace.

@stof
Copy link
Member
stof commented Dec 7, 2012

If you want a backtrace, run the console with the --verbose option

@gergelypolonkai
Copy link
Author

It's the same:

$ app/console cache:clear --verbose

  [ErrorException]                                                                                                                                                     
  Notice: unserialize(): Error at offset 97 of 99 bytes in /var/www/html/vendor/symfony/symfony/src/Symfony/Component/Config/Resource/FileResource.php line 78

@gergelypolonkai
Copy link
Author

If I remove the cache directory, and run cache:clear again (or cache:warmup), everything goes back to normal.

@gergelypolonkai
Copy link
Author

I have added a debug line to FileResource.php, so I could generate a backtrace. Here it is: https://gist.github.com/4275576

Actually, it tries to unserialize this string:

s:91:"/var/www/html/platea/app/cache/dev/jms_diextra/doctrine/EntityManager_50c9a1736b0f8.php";}i:4

which is clearly invalid at the given position.

@ghost-x47
Copy link

I get the same error, but it's only if i cache:clear with debug .
If i provide --no-debug to cache:clear - error disappearing.
Oh, and seems like cache:warmup is working too.
Also i have no clue how did i get it, it just suddenly showed up.

@gergelypolonkai
Copy link
Author

I have successfully captured an HTML version of the exception. It's 127kB and contains the whole backtrace. Where can I upload it?

@Elendev
Copy link
Elendev commented Dec 18, 2012

Hello, same bug here and I've also successfully captured an HTML version of the exception. It's also quite big, where can I upload it ?

Here is the first part :

ErrorException: Notice: unserialize() [function.unserialize]: Error at offset 136 of 138 bytes in /home/renaudot/public_html/frtrains/shop_app/2.0.0-rc6/vendor/symfony/symfony/src/Symfony/Component/Config/Resource/FileResource.php line 78
in /home/renaudot/public_html/frtrains/shop_app/2.0.0-rc6/vendor/symfony/symfony/src/Symfony/Component/Config/Resource/FileResource.php line 78
at ErrorHandler->handle('8', 'unserialize() [function.unserialize]: Error at offset 136 of 138 bytes', '/home/renaudot/public_html/frtrains/shop_app/2.0.0-rc6/vendor/symfony/symfony/src/Symfony/Component/Config/Resource/FileResource.php', '78', array('serialized' => 's:129:"/home/renaudot/public_html/frtrains/shop_app/2.0.0-rc6/app/cache/preprod/jms_diextra/doctrine/EntityManager_50d0bd7e3fd10.php";}i:4'))
at unserialize('s:129:"/home/renaudot/public_html/frtrains/shop_app/2.0.0-rc6/app/cache/preprod/jms_diextra/doctrine/EntityManager_50d0bd7e3fd10.php";}i:4') in /home/renaudot/public_html/frtrains/shop_app/2.0.0-rc6/vendor/symfony/symfony/src/Symfony/Component/Config/Resource/FileResource.php line 78
at FileResource->unserialize('s:129:"/home/renaudot/public_html/frtrains/shop_app/2.0.0-rc6/app/cache/preprod/jms_diextra/doctrine/EntityManager_50d0bd7e3fd10.php";}i:4')
at unserialize('a:552:{i:0;C:46:"Symfony\Component\Config\Resource\FileResource":141:{s:132:"/home/renaudot/public_html/frtrains/shop_app/2.0.0-rc6/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/FrameworkBundle.php";}i:1;C:46:"Symfony\Component\Config\Resource\FileResource":86:{s:78:"/home/renaudot/public_html/frtrains/shop_app/2.0.0-........

@mvrhov
Copy link
mvrhov commented Dec 20, 2012

github's gist?

@gergelypolonkai
Copy link
Author

The file is too large for that. When I pasted it, the chrome tab died.

@jfsimon
Copy link
Contributor
jfsimon commented Mar 1, 2013

There is an identifiable problem in cache:clear command.
As you run it with warmup option:

  • a new temporary cache directory is created
  • a new temporary kernel is booted
  • the temp cache directory is warmed up with the temp kernel
  • then the temp cache dir is renamed to replace the cleared cache dir

In order to make the new cache valid, files must be updated:

  • references to kernel class path must be replaced (fro 8000 m the temp to the real one) in serialized objects dumps
  • references to cache files path must be replaced (from the temp to the real one) everywhere it's found

One thing has been forgotten: reference to file paths are replaced in serialized object dumps too, without taking care of the \d+: prefix which contains the string lentgh.

The problem described above seems to be related to this.

@jfsimon
Copy link
Contributor
jfsimon commented Mar 1, 2013

@gergelypolonkai @ghost-x47 the #7235 PR fixes a related bug (may be the whole problem).
Would you like to test it?

@gergelypolonkai
Copy link
Author

Unfortunately I cannot reproduce the issue at will. It just pops up sometimes...

@Elendev
Copy link
Elendev commented Mar 2, 2013

Same problem here...

fabpot added a commit that referenced this issue Mar 6, 2013
This PR was squashed before being merged into the 2.1 branch (closes #7239).

Commits
-------

d1f5d25 [FrameworkBundle] Fixes invalid serialized objects in cache

Discussion
----------

[FrameworkBundle] Fixes invalid serialized objects in cache

| Q             | A
| ------------- | ---
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #6203

Fixes 2 problems:
*  malformed router chache filenames (matcher & dumper)
*  invalid cache file paths in serialized objects (.meta files)
@stefanosala
Copy link

I did not have this problem, but I have now after 5f2cea3

[2013-03-07 08:51:23] request.CRITICAL: ErrorException: Notice: unserialize(): Error at offset 156 of 118187 bytes in /var/lib/jenkins/workspace/acme/app/bootstrap.php.cache line 1316 (uncaught exception) at /var/lib/jenkins/workspace/ferrari-kopt/app/bootstrap.php.cache line 1316

@stefanosala
Copy link

This is the file that fails to be unserialized: https://gist.github.com/stewe/1fcb8e39dc657e7ec6ce

@jfsimon
Copy link
Contributor
jfsimon commented Mar 7, 2013

@stewe I'll try to fix this today

@stefanosala
Copy link

@jfsimon thanks, ping me if you need some help or more info about my environment.

@jfsimon
Copy link
Contributor
jfsimon commented Mar 7, 2013

@stewe would it be possible for you to quickly patch a file and test if it works?
You can join me with jabber on my email address.

@uwej711
Copy link
Contributor
uwej711 commented Mar 7, 2013

@jfsimon there are more numbers that need to be updated (from appDevUrlMatcher.php.meta)

a:19:{i:0;C:46:"Symfony\Component\Config\Resource\FileResource":62:{s:50:"/vagrant/symfony/app/cache/dev/assetic/routing.yml";}

You need to update the 62 to 58 too since you replaced dev_new with dev. This file still contains in my case a reference to the temporary container:

i:18;C:46:"Symfony\Component\Config\Resource\FileResource":91:{s:79:"/vagrant/symfony/app/cache/dev/appDevDebugProjectContainer__51391405d3cf0__.php";}}

Not sure wether the latter creates an issue though ...

@mpdude
Copy link
Contributor
mpdude commented Mar 8, 2013

How can I reproduce this?

@jfsimon
Copy link
Contributor
jfsimon commented Mar 8, 2013

@uwej711 thanks, I will use your example for testing.
@mpdude good question! It would be nice to reproduce this.

@mpdude
Copy link
Contributor
mpdude commented Mar 8, 2013

If you have to update the element sizes/length not only in the very place where you change the strings but also upwards along nested/outer elements, this approach has no chance. Just think of a case where you have to update strings in two places and happen to have their common length somewhere up the tree, so you have to change that number by two times the delta in size.

Without having looked at the code at all I'd go for temporary strings of the same length as the original ones instead.

Clearly all this is a Kluge Scale Level 2 thing... :-(

@fabpot
Copy link
Member
fabpot commented Mar 8, 2013

So, we have two options here:

  • unserialize, make our changes, serialise again;
  • use a string of the same size to avoid the problem in the first place.

@uwej711
Copy link
Contributor
uwej711 commented Mar 8, 2013

I think the second option might be better although that leaves us with the task to find a TLA for dev_new ...

@jfsimon
Copy link
Contributor
jfsimon commented Mar 8, 2013

Yep, the second solution is the best because it's more simple.

For the directory name, I propose something like $01, if it exists (because 2 cache:clear have been executed in parallel) then $02 is tried and so on... The number can be filled with as 0 as needed to match directory name length.

@uwej711
Copy link
Contributor
uwej711 commented Mar 8, 2013

Are you sure about the '$' ? Look's suspicious to me, does it work on all platforms? Maybe '_' instead?

@mpdude
Copy link
Contributor
mpdude commented Mar 8, 2013

Probably opening a can of worms, feel free to ignore me - question: What assumptions can we make for the cache dir at all?

Currently, if the kernel.cache_dir is set to "/some/where/X", the code assumes it may/can create new dirs at /some/where at will (it moves X to X_old, creates X_new). And in this particular case we'd need to choose a single-character name for the new temp directory (the name for _old is less important).

If all the cache stuff was shifted into the kernel.cache_dir, we could make up arbitrary names. For example, if "/some/where/X" was given, we could have "current" and "new_000" to "new_FFF" as names inside that at discretion.

Additionally, we'd only require that we have write permissions for the given dir, not for the parent as well.

I smell a BC break, though.

@jfsimon
Copy link
Contributor
jfsimon commented Mar 9, 2013

@mpdude actualy, the cache directory is not manipulated, only subdirs are cloned/renamed (those directories named with environment name). You're right for the "1 char dir name" problem, temporary directories could maybe named with just a number, without prefix.

@mpdude
Copy link
Contributor
mpdude commented Mar 9, 2013

@jfsimon Hm... In my setup (which admittedly has a tweaked directory layout) I have (for a particular environment)

public function getCacheDir() {
            return '/path/to/project/tmp/environment_name';
}

in "my" AppKernel and during the CacheClearCommand, a environment_name_new is created in that .../tmp directory. So the command assumes it may create a ..._new directory besides the cache dir, not inside it?

@jfsimon
Copy link
Contributor
jfsimon commented Mar 9, 2013

@mpdude ah, okay, my bad. I assumed /path/to/project/tmp (in your exemple) was the cache directory. It's a convention to have all env specific cache dirs in the same directory, and use this directory only for cache purpose. But you're right, it's not a requirement.

Anyway, if the cache parent directory is not writeable, the command fails before removing the cache with an exception, so it should not be a problem, what do you think?

@mpdude
Copy link
Contributor
A3E2 mpdude commented Mar 9, 2013

Right, my directory names obscured that. You should have one environment-specific directory for caching purposes only, and that is called the kernel.cache_directory.

My point was that the cache clear command works outside this directory, requiring suitable permissions on the parent of the cache dir and possibly running into naming conflicts.

If BC were not an issue, users could give a "cache base dir" we may write to. By setting kernel.cache_dir = cache_base_dir + "current", we'd stay below that base at all times and safely use new_XXX for temp dirs because we "own" that directory space.

@mpdude
Copy link
Contributor
mpdude commented Mar 9, 2013

Trying to reproduce this... Obviously, the tweaks that are currently applied against the container (.php and .xml) are not a problem and string lengths make no difference there.

With a custom project I had at hand, only the .meta files for the Routing Matcher and Generator contained the temporary ..._new directory name (before replacements, of course). That was only a single match each - a FileResource added by the AsseticBundle for the routing.yml file AsseticBundle generates to inject the asset routes for the use_controller = true case.

In PHP's serialization format, it makes a difference whether a class implements \Serializable or not. Without going into detail, it looks as if the rewrite logic currently in place suffices as long as serialized classes do not implement \Serializable, because then the string length will only appear in one place immediately preceeding the string.

As we're only talking about .meta files right now, only Config\ResourceInterface implementations should matter and we currently only have File... and DirectoryResource there. I don't really see why those should implement \Serializable at all? It got there in cbd0c3c by @stof, maybe he remembers why?

When the Resources are \Serializable, I perform console cache:clear and then run a request, I get a PHP error

Warning: Insufficient data for unserializing - 132 required, 130 present

When the Resources are not \Serializable, it works.

Am I missing something? Can you guys please check your setups and projects and see if you find anything else of relevance in the .meta files?

@fabpot What do you think of moving/shifting the cache dir as I outlined above? If we could replace the "real" for the "_new" dir with no string length changes, we would not depend on implementation details like \Serializable and had a chance of fixing all kinds of files where the strings appear in plain/unencoded form, not only the .meta files.

Disclaimer
The entire idea of fiddling around with the serialized strings is an ugly hack, makes assumptions about how the ConfigCache is implemented right now and has a good chance of missing (i. e. not re-writing) any path references that are not in the container and .meta files but in arbitrary cache files written by "userland" code. It is bound to break again in the future so we need some clever ideas how we can compile/warm the container and surrounding stuff in the right place from the start on.

@toloco
Copy link
toloco commented Mar 12, 2013

Hi guys,

I am really worried about this problem, I have 3 projects using symonfy2 (2.1 and 2.2) and all have the same problem after updating... there's any chance to solve it soon?

Regards,

@jfsimon
Copy link
Contributor
jfsimon commented Mar 12, 2013

@toloco yes, this will be fixed tomorrow, even if the fix is not the ideal one.

@stof
Copy link
Member
stof commented Mar 12, 2013

@mpdude The resources implement Serializable because it is a requirement for them to be serializable. In a clean implementation, if should even be part of the interface but this has been reverted as it was breaking stuff in AsseticBundle where the resources are implemented in a bad way and thus cannot implement Serializable cleanly (which should be fixed)

@mpdude
Copy link
Contributor
mpdude commented Mar 12, 2013

@stof Unlike Java's java.io.Serializable PHP's \Serializable has nothing to do with whether or not a class can be serialized. Any class can be serialized, either by doing/implementing nothing, having __sleep and/or __wakeup methods or implementing that interface. The difference is only how exactly serialization works and what gets serialized or not.

Thus, it is an implementation detail that should be left to the classes implementing ResourceInterface (in this case). Regarding the interface contract you are talking about, AFAIK there is no language-level way to guarantee that implementors of the interface make sure that Resources work well after being deserialized.

fabpot added a commit that referenced this issue Mar 14, 2013
This PR was merged into the 2.1 branch.

Commits
-------

f2ef6bc [FrameworkBundle] removed BC break
cc3a40e [FrameworkBundle] changed temp kernel name in cache:clear
7d87ecd [FrameworkBundle] fixed cahe:clear command's warmup

Discussion
----------

[FrameworkBundle] fixes cahe:clear command's warmup

Solution taken is to replace the last char of the cache directory name to create a temporary cache directory, this way the temporary cache path has the same length than the real one. I tested this on several projects, in dev and prod environments.

| Q             | A
| ------------- | ---
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | yes
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #6203

---------------------------------------------------------------------------

by jfsimon at 2013-03-13T12:32:25Z

@toloco @gergelypolonkai @ghost-x47 @stewe it would be great if you could test this patch on your projects and report result!

---------------------------------------------------------------------------

by toloco at 2013-03-13T12:41:47Z

Im sorry but have the same...

Notice: unserialize(): Error at offset 155 of 174227 bytes in /home/tolopalmer/Projects/shareandcoach/app/bootstrap.php.cache line 915

---------------------------------------------------------------------------

by jfsimon at 2013-03-13T12:45:04Z

@toloco could you paste the backtrace in a gist? and maybe the concerned file?

---------------------------------------------------------------------------

by stof at 2013-03-13T13:11:47Z

@jfsimon You probably have the same issue with the name of the temporary kernel class

---------------------------------------------------------------------------

by jfsimon at 2013-03-13T13:36:13Z

@stof if you're right, it's a nightmare. It must be possible to write a parser/fixer for serialized objects, don't you think?

---------------------------------------------------------------------------

by toloco at 2013-03-13T14:22:56Z

Here you are the gist with the stack and the bootstrap.php.cache file

https://gist.github.com/toloco/5152581

---------------------------------------------------------------------------

by mpdude at 2013-03-13T20:08:08Z

@jfsimon Writing such a parser is painting yourself in the corner.

Use a temp kernel class name of the same length as a quick fix.

#7230 could bring a solution because we might be able to inject a different ConfigCache factory during the command that intercepts and substitutes Resources before they get written into the meta file. Not sure if that PR has a chance of being picked though.

---------------------------------------------------------------------------

by toloco at 2013-03-14T08:19:58Z

So guys? we are blocked with this problem, can I help you? I can provide more stacks if it's needed

---------------------------------------------------------------------------

by mpdude at 2013-03-14T10:05:05Z

@toloco Could you please post the /home/tolopalmer/Projects/shareandcoach/app/cache/dev/appDevUrlMatcher.php.meta file? That's the one that is broken.

---------------------------------------------------------------------------

by jfsimon at 2013-03-14T10:15:20Z

@mpdude you can find its content in the gist https://gist.github.com/toloco/5152581 (1st file, 6th line)

---------------------------------------------------------------------------

by mpdude at 2013-03-14T10:24:55Z

@toloco That file should contain a serialized set of Resources, it's not in the Gist.

---------------------------------------------------------------------------

by jfsimon at 2013-03-14T10:33:12Z

@mpdude it's more visible in the raw file: ttps://gist.github.com/toloco/5152581/raw/48a1a823b5c8e6ba03936a52e8dc0d0ff1888f8a/Error+

---------------------------------------------------------------------------

by jfsimon at 2013-03-14T10:33:27Z

sorry: https://gist.github.com/toloco/5152581/raw/48a1a823b5c8e6ba03936a52e8dc0d0ff1888f8a/Error+

---------------------------------------------------------------------------

by toloco at 2013-03-14T10:37:09Z

https://gist.github.com/toloco/5160317 here you are the appDevUrlMatcher.php and meta

---------------------------------------------------------------------------

by jfsimon at 2013-03-14T10:51:46Z

@toloco I applied @mpdude's solution (have a temp kernel class name of the same length than the real one).
Could you test it to see if it fixes your problem?

---------------------------------------------------------------------------

by mpdude at 2013-03-14T10:58:46Z

@jfsimon Thanks!
@toloco If Jean-François' fix does not work, please make sure that the .meta file you posted was the broken one? I was able to unserialize it without problems.

---------------------------------------------------------------------------

by toloco at 2013-03-14T11:02:09Z

Man!!!! you are the fucking boss it works!!

---------------------------------------------------------------------------

by mpdude at 2013-03-14T11:04:30Z

@jfsimon you just made someone happy.

---------------------------------------------------------------------------

by jfsimon at 2013-03-14T11:12:39Z

@toloco @mpdude \o/
@fabpot fabpot closed this as completed Mar 14, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

0