10000 [MAILER] Let Amazon Mailer use the AWS SDK if available · Issue #33183 · symfony/symfony · GitHub
[go: up one dir, main page]
Skip to content

[MAILER] Let Amazon Mailer use the AWS SDK if available #33183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
toonvdn opened this issue Aug 15, 2019 · 11 comments
Closed

[MAILER] Let Amazon Mailer use the AWS SDK if available #33183

toonvdn opened this issue Aug 15, 2019 · 11 comments
Labels
Feature Good first issue Ideal for your first contribution! (some Symfony experience may be required) Mailer

Comments

@toonvdn
Copy link
toonvdn commented Aug 15, 2019 •
edited
Loading

A big advantage of using the AWS SDK - if installed - is that no credentials need to be recorded in the environment variables when the application is installed on AWS infrastructure, and a so-called Instance Profile is setup for the EC2 infrastructure being used.

The AWS SDK can work with credentials, but default to the Instance Profile to determine the available permissions, as set in the IAM configuration of your AWS account. When your application is heavily dependent on AWS services, all permissions are maintained in the Instance Profile instead of using a multitude of AWS keys & secrets.

A proposal is to use http://ses or smtp://ses whenever you don't wish to use AWS keys & secrets.
@fabpot
Copy link
Member
fabpot commented Aug 15, 2019

Instead of using the AWS SDK, I would prefer to do the same on our client when the project is installed on the AWS infra. Would you like to submit a PR to implement that? I suppose that the logic should not be too complex and already available in the SDK.

@toonvdn
Copy link
Author
toonvdn commented Aug 15, 2019 •
edited
Loading

On the other hand, if you are installing software on AWS infra, you almost by default install the AWS SDK, so why not use their already maintained logic in that SDK to take care of the authN mechanism? In the end, it's just referencing from within the Mailer component, making your life easier.

If the AWS SDK is not present, just default to what is in the Mailer component at this moment.
Also, if there's a future component, making use of AWS services, you wouldn't integrate the logic in there as well - meaning duplicate code in the end.

Wdyt?

@fabpot
Copy link
Member
fabpot commented Aug 15, 2019

I don't want to depend on the AWS SDK, which would break some of our integrations. For instance, you will loose the Webprofiler integration, the logging, the debugger, ...

@mdeboer
Copy link
Contributor
mdeboer commented Aug 16, 2019

But in this case, why would you? I presume the credentials aren't needed because they are provided by certain environment variables or "secrets files"?

If that's the case, changes can be made to allow for a DSN without credentials to pick them up automatically. I don't see why this would suddenly require using the whole AWS SDK as that would be a lot more work and nothing to gain except for this simple feature.

So let me know how the credentials are provided and I will make a PR 😉 I have no EC2 box running myself and little time to do so.

@xabbuh xabbuh added the Mailer label Aug 17, 2019
@fabpot fabpot added the Good first issue Ideal for your first contribution! (some Symfony experience may be required) label Aug 18, 2019
@fabpot fabpot mentioned this issue Aug 21, 2019
Merged
@connorhu
Copy link
Contributor

Here are some implmentations:
https://github.com/daniel-zahariev/php-aws-ses/blob/master/src/SimpleEmailService.php
https://github.com/okamos/php-ses/blob/master/src/ses.php

and the AWS client library:
https://github.com/aws/aws-sdk-php/blob/master/src/Ses/SesClient.php
https://github.com/aws/aws-sdk-php/blob/master/src/AwsClient.php
https://github.com/aws/aws-sdk-php/blob/master/src/AwsClientTrait.php

The implementation will use some native XML library (i don't know which one is preferred by symfony projects) and the brand new HTTPClient library.

@fabpot
Copy link
Member
fabpot commented Aug 21, 2019

@connorhu Thanks for investigating. To avoid any confusion, note that we already support SES to send emails. Here, we are trying to figure out if we can automatically get the credentials from the local instance instead of forcing the dev to define them as environment variables.

Maybe https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html could be useful.

@connorhu
Copy link
Contributor
connorhu commented Aug 21, 2019 •
edited
Loading

Oh i see! There is an Amazon Mailer Transport in a different repository! In this case this will be a simple PR.

@fabpot
Copy link
Member
fabpot commented Aug 21, 2019

It is in the same repo under src/Symfony/Component/Mailer/Bridge/Amazon/.

@connorhu
Copy link
Contributor
connorhu commented Aug 24, 2019 •
edited
Loading

I've done some research. I've made an ES2 instance setup with IAM role and a SES with verified domain.

If we call the http://169.254.169.254/latest/meta-data/iam/security-credentials/{rolename} the response is the following (content type is text/plain, but the syntax is json):

{
  "Code" : "Success",
  "LastUpdated" : "2019-08-24T13:01:24Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "{accesskey}",
  "SecretAccessKey" : "{secret}",
  "Token" : "{token}",
  "Expiration" : "2019-08-24T19:17:13Z"
}

The aws-sdk-php is make that request too, but when it does the SendEmail request it injects the security token:

X-Amz-Security-Token: {token}
Authorization: AWS4-HMAC-SHA256 Credential={accesskey}/20190824/eu-west-1/ses/aws4_request, SignedHeaders=host;x-amz-date;x-amz-security-token, Signature={signature}

Without the token the SendEmail request do fail:

Unable to send an email: The security token included in the request is invalid (code InvalidClientTokenId).

We should make several credential classes instead of "string $username string $password" arguments at the constructor of the Ses{Transport}Transport classes:
UsernamePasswordCredential and ApiTokenCredential

I'm not sure but Api and Http transport works with both method but the Smtp transport only works with UsernamePasswordCredential.

There is a tested PoC implementation:
master...connorhu:master

An unhandled situation is when the token is expire.

@fabpot
Copy link
Member
fabpot commented Aug 25, 2019

@connorhu Great work! I've just reviewed your implementation quickly and it looks good. Can you submit a pull request now so that we can review it correctly?

@connorhu connorhu mentioned this issue Aug 25, 2019
Closed
@cfoehrdes
Copy link

We are running a lot of symfony projects on AWS ECS and would love to have native role based authentication support. At the moment we use a custom transport based on the AWS SDK to solve this. What is the best way to help bringing the alternative auth mechanisms to the Mailer component? The PR #33326 looks really promising but I am not sure if I can somehow contribute the ECS part to it or if we have to create a new PR as the current one seems not being continued. Is it even possible to do the change proposed by @connorhu now that 4.4/5.0 has been released as it contains a BC break?

@jderusse jderusse mentioned this issue Mar 6, 2020
Merged
@fabpot fabpot closed this as completed May 3, 2020
fabpot added a commit that referenced this issue May 3, 2020
@fabpot
feature #35992 [Mailer] Use AsyncAws to handle SES requests (jderusse)
669b7f1 
This PR was squashed before being merged into the 5.1-dev branch.

Discussion
----------

[Mailer] Use AsyncAws to handle SES requests

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| Deprecations? | no
| Tickets       | #33183, #35468 and #35037
| License       | MIT
| Doc PR        | TODO

alternative to #33326

This PR replace the native code to call AWS SES by the new [AsyncAws](https://github.com/async-aws/aws) project maintained by @Nyholm and me.

This removes complexity of signing request, and adds new features likes:
- authentication via .aws/config.ini, Instance profile, WebIdentity (K8S service account)
- usesignature V4 (the one recommanded by the Official SDK )
- fully compatible with API (uses the official AWS SDK interface contract to generate classes)

Because it's based on `symfony/http-client`, it's fully integrable with Symfony application.

Commits
-------

2124387 [Mailer] Use AsyncAws to handle SES requests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature Good first issue Ideal for your first contribution! (some Symfony experience may be required) Mailer
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@fabpot @javiereguiluz @mdeboer @cfoehrdes @connorhu @xabbuh @toonvdn
0