8000 feature #30602 [BrowserKit] Add support for HttpClient (fabpot, THERA… · symfony/symfony@d73a53a · GitHub
[go: up one dir, main page]
Skip to content

Commit d73a53a

Browse files
fabpotfabpot
committed
feature #30602 [BrowserKit] Add support for HttpClient (fabpot, THERAGE Kévin)
This PR was merged into the 4.3-dev branch. Discussion ---------- [BrowserKit] Add support for HttpClient | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | no <!-- don't forget to update UPGRADE-*.md and src/**/CHANGELOG.md files --> | Tests pass? | yes <!-- please add some, will be required by reviewers --> | Fixed tickets | part of #30502 | License | MIT | Doc PR | not yet When combining the power of the new HttpClient component with the BrowserKit and Mime components, we can makes something really powerful... a full/better/awesome replacement for https://github.com/FriendsOfPHP/Goutte. So, this PR is about integrating the HttpClient component with BrowserKit to give users a high-level interface to ease usages in the most common use cases. Scraping websites can be done like this: ```php use Symfony\Component\BrowserKit\HttpBrowser; use Symfony\Component\HttpClient\HttpClient; $client = HttpClient::create(); $browser = new HttpBrowser($client); $browser->request('GET', 'https://example.com/'); $browser->clickLink('Log In'); $browser->submitForm('Sign In', ['username' => 'me', 'password' => 'pass']); $browser->clickLink('Subscriptions')->filter('table tr:nth-child(2) td:nth-child(2)')->each(function ($node) { echo trim($node->text())."\n"; }); ``` And voilà! Nice, isn't? Want to add HTTP cache? Sure: ```php use Symfony\Component\HttpKernel\HttpCache\Store; $client = HttpClient::create(); $store = new Store(sys_get_temp_dir().'/http-cache-store'); $browser = new HttpBrowser($client, $store); // ... ``` Want logging and debugging of HTTP Cache? Yep: ```php use Psr\Log\AbstractLogger; class EchoLogger extends AbstractLogger { public function log($level, $message, array $context = []) { echo $message."\n"; } } $browser = new HttpBrowser($client, $store, new EchoLogger()); ``` The first time you run your code, you will get an output similar to: ``` Request: GET https://twig.symfony.com/ Response: 200 https://twig.symfony.com/ Cache: GET /: miss, store Request: GET https://twig.symfony.com/doc/2.x/ Response: 200 https://twig.symfony.com/doc/2.x/ Cache: GET /doc/2.x/: miss, store ``` But then: ``` Cache: GET /: fresh Cache: GET /doc/2.x/: fresh ``` Limit is the sky here as you get the full power of all the Symfony ecosystem. Under the hood, these examples leverage HttpFoundation, HttpKernel (with HttpCache), DomCrawler, BrowserKit, CssSelector, HttpClient, Mime, ... Excited? P.S. : Tests need to wait for the HttpClient Mock class to land into master. Commits ------- b5b2a25 Add tests for HttpBrowser dd55845 [BrowserKit] added support for HttpClient
2 parents eeae257 + b5b2a25 commit d73a53a

File tree

7 files changed

+306
-71
lines changed

7 files changed

+306
-71
lines changed

‎src/Symfony/Component/BrowserKit/CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ CHANGELOG
44
4.3.0
55
-----
66

7+
* Added `HttpBrowser`, an implementation of a browser with the HttpClient component
78
* Renamed `Client` to `AbstractBrowser`
89
* Marked `Response` final.
910
* Deprecated `Response::buildHeader()`

‎src/Symfony/Component/BrowserKit/HttpBrowser.php

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\BrowserKit;
13+
14+
use Symfony\Component\HttpClient\HttpClient;
15+
use Symfony\Component\Mime\Part\AbstractPart;
16+
use Symfony\Component\Mime\Part\Multipart\FormDataPart;
17+
use Symfony\Component\Mime\Part\TextPart;
18+
use Symfony\Contracts\HttpClient\HttpClientInterface;
19+
20+
/**
21+
* An implementation of a browser using the HttpClient component
22+
* to make real HTTP requests.
23+
*
24+
* @author Fabien Potencier <fabien@symfony.com>
25+
*
26+
* @final
27+
*/
28+
class HttpBrowser extends AbstractBrowser
29+
{
30+
private $client;
31+
32+
public function __construct(HttpClientInterface $client = null, History $history = null, CookieJar $cookieJar = null)
33+
{
34+
if (!class_exists(HttpClient::class)) {
35+
throw new \LogicException(sprintf('You cannot use "%s" as the HttpClient component is not installed. Try running "composer require symfony/http-client".', __CLASS__));
36+
}
37+
38+
$this->client = $client ?? HttpClient::create();
39+
40+
parent::__construct([], $history, $cookieJar);
41+
}
42+
43+
protected function doRequest($request)
44+
{
45+
$headers = $this->getHeaders($request);
46+
$body = '';
47+
if (null !== $part = $this->getBody($request)) {
48+
$headers = array_merge($headers, $part->getPreparedHeaders()->toArray());
49+
$body = $part->bodyToIterable();
50+
}
51+
$response = $this->client->request($request->getMethod(), $request->getUri(), [
52+
'headers' => $headers,
53+
'body' => $body,
54+
'max_redirects' => 0,
55+
]);
56+
57+
return new Response($response->getContent(false), $response->getStatusCode(), $response->getHeaders(false));
58+
}
59+
60+
private function getBody(Request $request): ?AbstractPart
61+
{
62+
if (\in_array($request->getMethod(), ['GET', 'HEAD'])) {
63+
return null;
64+
}
65+
66+
if (!class_exists(AbstractPart::class)) {
67+
throw new \LogicException('You cannot pass non-empty bodies as the Mime component is not installed. Try running "composer require symfony/mime".');
68+
}
69+
70+
if (null !== $content = $request->getContent()) {
71+
return new TextPart($content, 'utf-8', 'plain', '8bit');
72+
}
73+
74+
$fields = $request->getParameters();
75+
foreach ($request->getFiles() as $name => $file) {
76+
if (!isset($file['tmp_name'])) {
77+
continue;
78+
}
79+
80+
$fields[$name] = DataPart::fromPath($file['tmp_name'], $file['name']);
81+
}
82+
83+
return new FormDataPart($fields);
84+
}
85+
86+
private function getHeaders(Request $request): array
87+
{
88+
$headers = [];
89+
foreach ($request->getServer() as $key => $value) {
90+
$key = strtolower(str_replace('_', '-', $key));
91+
$contentHeaders = ['content-length' => true, 'content-md5' => true, 'content-type' => true];
92+
if (0 === strpos($key, 'http-')) {
93+
$headers[substr($key, 5)] = $value;
94+
} elseif (isset($contentHeaders[$key])) {
95+
// CONTENT_* are not prefixed with HTTP_
96+
$headers[$key] = $value;
97+
}
98+
}
99+
$cookies = [];
100+
foreach ($this->getCookieJar()->allRawValues($request->getUri()) as $name => $value) {
101+
$cookies[] = $name.'='.$value;
102+
}
103+
if ($cookies) {
104+
$headers['cookie'] = implode('; ', $cookies);
105+
}
106+
107+
return $headers;
108+
}
109+
}

‎src/Symfony/Component/BrowserKit/README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,9 @@ BrowserKit Component
44
The BrowserKit component simulates the behavior of a web browser, allowing you
55
to make requests, click on links and submit forms programmatically.
66

7+
The component comes with a concrete implementation that uses the HttpClient
8+
component to make real HTTP requests.
9+
710
Resources
811
---------
912

0 commit comments

Comments
 (0)
0