8000 Cannot stream request using CURLOPT_READFUNCTION without chunked encoding · Issue #8165 · php/php-src · GitHub
[go: up one dir, main page]
Skip to content

Cannot stream request using CURLOPT_READFUNCTION without chunked encoding #8165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
SiebelsTim opened this issue Mar 2, 2022 · 4 comments
Open

Cannot stream request using CURLOPT_READFUNCTION without chunked encoding #8165

SiebelsTim opened this issue Mar 2, 2022 · 4 comments
Labels
Extension: curl Feature Status: Verified

Comments

@SiebelsTim
Copy link
Contributor

Description

Consider this curl code in C:

int counter = 0;
size_t read_callback(char *ptr, size_t size, size_t nmemb, void *userdata)
{
	if (counter++ == 1) {
		return 0;
	}

	ptr[0] = 'W';
	ptr[1] = 'H';
	ptr[2] = 'A';
	ptr[3] = 'T';
	return 4;
}

int main() {
	CURL* handle = curl_easy_init(); 
	CURLcode res;
	curl_easy_setopt(handle, CURLOPT_URL, "http://localhost:9090");
	curl_easy_setopt(handle, CURLOPT_READFUNCTION, read_callback);
	curl_easy_setopt(handle, CURLOPT_POST, 1);
	curl_easy_setopt(handle, CURLOPT_POSTFIELDSIZE, 4);
	res = curl_easy_perform(handle);
	curl_easy_cleanup(handle);

	return 0;
}

That results in the following HTTP request

POST / HTTP/1.1
Host: localhost:9090
Accept: */*
Content-Length: 4
Content-Type: application/x-www-form-urlencoded
WHAT

When removing the CURLOPT_POSTFIELDSIZE options, the following request is issued:

POST /iserv/helloworld HTTP/1.1
Host: localhost:982
Accept: */*
Transfer-Encoding: chunked
Content-Type: application/x-www-form-urlencoded
Expect: 100-continue

WHAT

That is analogous to its PHP implementation:

$counter = 0;
$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, 'http://localhost:9090');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_READFUNCTION, static function ($ch, $fd, $length) use (&$counter) {
        if ($counter++ > 0) {
            return "";
        }

        return "WHAT";
});

$output = curl_exec($ch);
curl_close($ch);

However, PHP does not expose the constant CURLOPT_POSTFIELDSIZE, i.e. I have no option to tell curl the length of the request. As a result, I cannot stream a request body using CURLOPT_READFUNCTION without chunked encoding.

PHP Version

8.0.12

Operating System

Ubuntu 21.10
@cmb69
Copy link
Member
cmb69 commented Mar 2, 2022

I see your point (looks more like a feature request than a bug report, though), but exposing CURLOPT_POSTFIELDSIZE could cause issues, since it is currently set automatically, if CURLOPT_POSTFIELDS is given as string.

@cmb69 cmb69 changed the title curl: Cannot stream request using CURLOPT_READFUNCTION without chunked encoding Cannot stream request using CURLOPT_READFUNCTION without chunked encoding Mar 2, 2022
@KapitanOczywisty
Copy link
KapitanOczywisty commented Mar 2, 2022 •
edited
Loading

It could work with headers: "Transfer-Encoding:", "Content-Length: {$length}"

I have also habit of setting Expect: header to remove Expect/Continue mechanics, alone it could be sufficient in this case.

@SiebelsTim
Copy link
Contributor Author

looks more like a feature request than a bug report, though

yes, I wasn't quite sure about that. I'm fine either way :)

but exposing CURLOPT_POSTFIELDSIZE could cause issues

Yep. I didn't mean to suggest to simply expose it. However, I didn't have a good alternative that incorporates with the C-like curl-API we have in PHP.

It could work with headers: "Transfer-Encoding:", "Content-Length: {$length}"

Yes, I thought about it as well.

Right now, if we pass a Content-Length header using CURLOPT_HTTPHEADER, curl ignores it and still uses chunked encoding.

However, overwriting Transfer-Encoding as you suggested seems to work.

Do you think we could set CURLOPT_POSTFIELDSIZE depending on the content-length header to make it more intuitive?

@cmb69
Copy link
Member
cmb69 commented Mar 8, 2022

Do you think we could set CURLOPT_POSTFIELDSIZE depending on the content-length header to make it more intuitive?

That would require PHP to actually parse the headers passed via CURLOPT_HTTPHEADER; not sure whether that's worth it, and that change might break some code. Maybe we should just document that users need to set Transfer-Encoding and Content-Length to avoid this issue. Or maybe we should introduce a new option (say CURLOPT_READFUNCTION_LENGTH) which allows to set the length explicitly.

It might be sensible to write to the internals mailing list to get more feedback on this issue.

@SiebelsTim SiebelsTim mentioned this issue Apr 7, 2022
Closed
@mbaynton mbaynton mentioned this issue May 16, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Extension: curl Feature Status: Verified
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@SiebelsTim @cmb69 @KapitanOczywisty
0