8000 [Feature] App Engine Support · Issue #9238 · symfony/symfony · GitHub
[go: up one dir, main page]
Skip to content

[Feature] App Engine Support #9238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
cpubot opened this issue Oct 7, 2013 · 33 comments
Closed

[Feature] App Engine Support #9238

cpubot opened this issue Oct 7, 2013 · 33 comments
Labels
Filesystem

Comments

@cpubot
Copy link
cpubot commented Oct 7, 2013

Since it is not possible to write to the filesystem in Google App Engine, Symfony is unusable, as it has to write out its class maps / cache (and doctrine with its proxy classes) to a directory on the filesystem.

It's also possible that I missed something in the documentation that would allow me to circumvent this issue, but as of now, there doesn't seem to be a way to do this.
@Tobion
Copy link
Contributor
Tobion commented Oct 7, 2013

Do I see it correctly that one needs to use memcache for caching purposes in Google App Engine?

@cpubot
Copy link
Author
cpubot commented Oct 7, 2013

One must override getCacheDir() and getLogDir() in AppKernel to set the cache/log directories. The only way to use this with memcache would be to write a custom stream wrapper.
Additionally, in the Filesystem component, one of the methods utilized by the cache calls the app-engine-restricted tempnam function, rendering it useless.

@Baachi
Copy link
Contributor
Baachi commented Oct 8, 2013

It is possible, symfony has a warmup cache so you only need to run app/console -e=prod cache:warmup before you deploy. For logging you should use another handler.

@GromNaN
Copy link
Member
GromNaN commented Oct 8, 2013

@Baachi Unfortunately, the cache contains absolute paths.

@Baachi
Copy link
Contributor
Baachi commented Oct 8, 2013

@GromNaN Ah right, didn't see that.

@GromNaN
Copy link
Member
GromNaN commented Oct 9, 2013

Google provide a stream wrapper for Google Cloud Sorage.
https://developers.google.com/appengine/docs/php/googlestorage/

The cache directory can be overridden on the Kernel class.
http://symfony.com/doc/current/cookbook/configuration/override_dir_structure.html#override-the-cache-directory

// ...
class AppKernel extends Kernel
{
    // ...

    public function getCacheDir()
    {
        return 'gs://bucket_name/'.$this->environment.'/cache';
    }
}

@cpubot
Copy link
Author
cpubot commented Oct 9, 2013

@GromNaN I've tried this -- the local SDK doesn't allow you to access the gs buckets. When running on a deployed production version, symfony tries to output the cache to the bucket and runs into the tempnam() error I described above.

@mkoubik
Copy link
mkoubik commented Oct 13, 2013

This is not only an App Engine issue, but a serious problem for deploying to enviroments without filesystem write-access generally. I need to prepare (on a different machine) a package that needs to be ready for use in production.
I think logs can be redirected to a non-filesystem handler by monolog, but the cache is an issue. As the WarmableInterface::warmUp() method takes a directory as a parameter (instead of some kind of cache provider), all cache-using services are directly dependent on the filesystem.

@GromNaN GromNaN mentioned this issue Oct 26, 2013
Closed
@cordoval
Copy link
Contributor

there is this http://www.ideato.it/planet-ideato/symfony2-su-google-app-engine/ but not sure how they handle this issue

@fabpot
Copy link
Member
fabpot commented Oct 29, 2013

A quick list of functions used by Symfony that are permanently disabled on GAE:

  • escapeshellarg() and escapeshellcmd()
  • exec() (used by the Console, so not relevant)
  • highlight_file() (only used for debugging purpose, so not relevant)
  • proc_close(), proc_open() and proc_terminate() (used by the Process component, no way around it)
  • shell_exec() (used by the Console, so not relevant)
  • symlink() (used by the Filesystem and Finder)
  • tempnam()

@mkoubik
Copy link
mkoubik commented Oct 30, 2013

The caching issue can be solved by using stream wrapper from gaufrette (https://github.com/KnpLabs/Gaufrette) and setting cache dir to gaufrette://whatever/.... So you can store cache to db, memory, apc, etc.

@greywire
Copy link
greywire commented Nov 5, 2013

So are symfony apps just not deployable on google app engine? I am stuck on the tempnam() issue right now. This sucks!

@cordoval
Copy link
Contributor
cordoval commented Nov 5, 2013

@greywire not yet, Symfony does not run on GAE as of today. @fabpot is looking into it, I think the approach would be as follows:

1 - remove disabled functions calls from symfony/symfony
1.5 - replacement of function should sieze stream protocol interface support of other functions
2 - introduce a clear way for symfony to handle fs abstraction with gaufrette into components and bundles
3 - write documentation for using this, amongst them create a cookbook for GAE and other similar systems

@cordoval cordoval mentioned this issue Nov 5, 2013
Closed
7 tasks
@cordoval
Copy link
Contributor
cordoval commented Nov 5, 2013

probably i believe the hardest will be symlink() and proc related (I wonder how far can we go without running into proc errors on Symfony for a regular app)

@greywire
Copy link
greywire commented Nov 5, 2013

Are there modules to avoid or workarounds or is this a problem thats ingrained in the core of symfony? I might have to switch to another framework if this is something too difficult to fix in the very near term..

@cordoval
Copy link
Contributor
cordoval commented Nov 5, 2013

@greywire is ingrained 👶 but we are working on it

@greywire
Copy link
greywire commented Nov 6, 2013

Are the proc_* functions used in the core of symfony? What for? It seems like the filesystem issues could be easily fixed as mentioned with a stream wrapper. I'm looking for any quick, dirty hack or if I can eliminate some non essential functionality to get going...

@cordoval
Copy link
Contributor
cordoval commented Nov 9, 2013

ref #9483 having some problems after solving some issues with GAE

@micheleorselli micheleorselli mentioned this issue Dec 15, 2013
Closed
@ihsw
Copy link
ihsw commented Mar 19, 2014

Has there been any progress on this? I'm interested in running Symfony on GAE.

@jakzal jakzal mentioned this issue Mar 22, 2014
Closed
fabpot added a commit that referenced this issue Mar 25, 2014
@fabpot
bug #9784 [HttpFoundation] Removed ini check to make Uploadedfile wor…
88f670c 
…k on Google App Engine (micheleorselli)

This PR was submitted for the master branch but it was merged into the 2.3 branch instead (closes #9784).

Discussion
----------

[HttpFoundation] Removed ini check to make Uploadedfile work on Google App Engine

| Q             | A
| ------------- | ---
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes

related to #9238

Commits
-------

ecb6481 removed ini check to make uploadedfile work on gae
@fabpot fabpot mentioned this issue Mar 27, 2014
Closed
@msjyoo
Copy link
msjyoo commented Apr 12, 2014

+1 I am also interested in running Symfony on GAE and other cloud services. This is a huge limitation of Symfony that prevents it from being used on all Platform as a Service environments that do not provide a writeable file system.

@fabpot fabpot mentioned this issue May 12, 2014
Merged
fabpot added a commit that referenced this issue May 17, 2014
@fabpot
bug #10894 [HttpKernel] removed absolute paths from the generated con…
735e9a4 
…tainer (fabpot)

This PR was merged into the 2.3 branch.

Discussion
----------

[HttpKernel] removed absolute paths from the generated container

| Q             | A
| ------------- | ---
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | first step to resolve #6484, #3079, and #9238
| License       | MIT
| Doc PR        | n/a

This PR converts absolute paths to relative ones in the dumped container. The code is a bit "ugly", but it gets the job done and I'm not sure that there is a more elegant way without breaking everything.

Commits
-------

c1450b4 [HttpKernel] removed absolute paths from the generated container
@fabpot fabpot mentioned this issue May 26, 2014
Closed
@hollodk
Copy link
Contributor
hollodk commented Jun 10, 2014

I just found a problem with the translations bundle and google app engine, i wanted to install silex and a basic form, with default form layout.. The problem is the default form layout uses translations for the labels..

And as far i can see both in the code and in the doc, http://silex.sensiolabs.org/doc/providers/translation.html, xliff is added as a default, and that will result in a call to the tempnam() function which is quite bad.. Its called in parseFile, i have fixed the problem with just return a single xml document.. but would like to be able to disable xliff support in some way..

Can it be not default with xliff, or will it be possible to remove a loader?

@jakzal
Copy link
Contributor
jakzal commented Jun 10, 2014

@hollodk looks like GAE guys are going to implement tempnam() support (see #9442 (comment)).

@msjyoo
Copy link
msjyoo commented Jul 12, 2014

Maybe this can be used as a temporary workaround to Symfony 2 requiring write access to file system?
https://github.com/adlawson/vfs.php

@nicolas-grekas nicolas-grekas mentioned this issue Nov 30, 2014
Merged
fabpot added a commit that referenced this issue Dec 2, 2014
@fabpot
bug #12784 [DependencyInjection] make paths relative to __DIR__ in th…
b604b0a 
…e generated container (nicolas-grekas)

This PR was merged into the 2.3 branch.

Discussion
----------

[DependencyInjection] make paths relative to __DIR__ in the generated container

| Q             | A
| ------------- | ---
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #6484, #3079, partially #9238, #10894, #10999
| License       | MIT
| Doc PR        | n/a

This is an alternative approach to #10999 for removing absolute paths from the generated container:
instead of trying to fix the container file after it has been dumped, telling to the PhpDumper where its output will be written allows it to replace parts of strings by an equivalent value based on `__DIR__`.
This should be safe, thus the PR is on 2.3.

Commits
-------

edd7057 [DependencyInjection] make paths relative to __DIR__ in the generated container
@itavero
Copy link
itavero commented Feb 16, 2015

What is the current state of this feature request? Is Google App Engine supported, and if not, what needs to be done to make it Symfony 2 compatible with GAE?

@eddiejaoude
Copy link
Contributor

Unfortunately, the cache contains absolute paths.

This has now been fixed (well a couple of months ago).

@GromNaN
Copy link
Member
GromNaN commented Feb 27, 2015

GAE now supports the tempnam() and sys_get_temp_dir() functions : https://cloud.google.com/appengine/docs/php/#tempnam_and_sys_get_temp_dir_support

@jakzal
Copy link
Contributor
jakzal 9E88 commented Feb 27, 2015

Is there anything missing for GAE support?

@shouze
Copy link
shouze commented Feb 28, 2015

@jakzal yes it lacks at least some documentation (can be more general thant just GAE as Heroku or deployments on containers with ready only code can be in the scope too). I think it can work BTW even if a little tricky at the moment.

For example we can't warmup the entire sf2 cache prior to deploy an app (or I have no idea how to do that). It would be cool to ensure that sf2 cache has no writes during the entire life of a deployed application.

@xabbuh
Copy link
Member
xabbuh commented Mar 1, 2015

yes it lacks at least some documentation (can be more general thant just GAE as Heroku or deployments on containers with ready only code can be in the scope too). I think it can work BTW even if a little tricky at the moment.

@shouze Could be worth adding an article to the Deployment section of the Symfony cookbook. Do you mind opening an issue over there or maybe even better start a pull request?

@nicolas-grekas
Copy link
Member

Can someone try to setup a Symfony app on Google App Engine (and write a bit of doc about it)?
It should be doable now.
ping @zachrbrown @shouze /others

@shouze
Copy link
shouze commented Jul 30, 2015

@nicolas-grekas ok so the whole sf2 cache can be warmup from now in 1 pass? If you tell me how to accomplish that... I'm very interested!

@bangpound
Copy link

I have published two packages https://github.com/caxy/AppEngineBundle and https://github.com/caxy/AppEngineBridge and am updating the documentation on the bundle (which has some helpful features for integrating with GAE users) before this weekend. Also note that Google has published a fork of the standard edition 2.3 https://cloud.google.com/appengine/docs/php/symfony-hello-world?hl=en

@fabpot
Copy link
Member
fabpot commented Oct 5, 2015

Symfony should work fine on Google AppEngine now. It there are some problems, specific issues should be created instead of using this old and generic one.

@fabpot fabpot closed this as completed Oct 5, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Filesystem
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
0