8000 Streamed response support by hmacr · Pull Request #11429 · appwrite/appwrite · GitHub
[go: up one dir, main page]

Skip to content

Streamed response support#11429

Open
hmacr wants to merge 1 commit into1.8.xfrom
ser-334
Open

Streamed response support#11429
hmacr wants to merge 1 commit into1.8.xfrom
ser-334

Conversation

@hmacr
Copy link
Member
@hmacr hmacr commented Mar 2, 2026

Towards SER-334

@coderabbitai
Copy link
Contributor
coderabbitai bot commented Mar 2, 2026
📝 Walkthrough

Walkthrough

The changes introduce Server-Sent Events (SSE) streaming support to the application. A streamCallback closure is added to the controller that captures streaming headers and appends data chunks to the HTTP response. When streaming is detected, the response is finalized and the execution document is populated with completion metadata, bypassing normal post-processing. The Executor class is updated with an optional streaming callback parameter that manages callback invocation—emitting headers first via a null signal, then subsequent data chunks—while maintaining backward compatibility for non-streaming code paths.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Streamed response support' directly and clearly summarizes the main change—adding streaming functionality to handle streamed responses in the execution flow.
Description check ✅ Passed The description references SER-334, which appears to be a tracking issue for this feature. While minimal, it is related to the changeset's purpose.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch ser-334

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
github-actions bot commented Mar 2, 2026

Security Scan Results for PR

Docker Image Scan Results

Package Version Vulnerability Severity
libecpg 18.1-r0 CVE-2026-2004 HIGH
libecpg 18.1-r0 CVE-2026-2005 HIGH
libecpg 18.1-r0 CVE-2026-2006 HIGH
libecpg 18.1-r0 CVE-2026-2007 HIGH
libecpg-dev 18.1-r0 CVE-2026-2004 HIGH
libecpg-dev 18.1-r0 CVE-2026-2005 HIGH
libecpg-dev 18.1-r0 CVE-2026-2006 HIGH
libecpg-dev 18.1-r0 CVE-2026-2007 HIGH
libheif 1.20.2-r1 CVE-2025-68431 HIGH
libpng 1.6.54-r0 CVE-2026-25646 HIGH
libpng-dev 1.6.54-r0 CVE-2026-25646 HIGH
libpq 18.1-r0 CVE-2026-2004 HIGH
libpq 18.1-r0 CVE-2026-2005 HIGH
libpq 18.1-r0 CVE-2026-2006 HIGH
libpq 18.1-r0 CVE-2026-2007 HIGH
libpq-dev 18.1-r0 CVE-2026-2004 HIGH
libpq-dev 18.1-r0 CVE-2026-2005 HIGH
libpq-dev 18.1-r0 CVE-2026-2006 HIGH
libpq-dev 18.1-r0 CVE-2026-2007 HIGH
postgresql18-dev 18.1-r0 CVE-2026-2004 HIGH
postgresql18-dev 18.1-r0 CVE-2026-2005 HIGH
postgresql18-dev 18.1-r0 CVE-2026-2006 HIGH
postgresql18-dev 18.1-r0 CVE-2026-2007 HIGH

Source Code Scan Results

🎉 No vulnerabilities found!

Copy link
Contributor
@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/Executor/Executor.php (1)

356-405: ⚠️ Potential issue | 🔴 Critical

Callback mode can suppress transport errors.

When callback mode is enabled, execution returns at Line 402 before curl error handling (Lines 436-441). A timeout/network failure can therefore be treated as success by the caller. Move curl error/status checks before the callback-mode return.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Executor/Executor.php` around lines 356 - 405, The early return in the
callback branch suppresses curl transport errors because curl_exec()
error/statu
8000
s checks run after the "if (isset($callback)) { curl_close($ch);
return []; }" block; move the curl error/status inspection (use curl_errno(),
curl_error(), and curl_getinfo() checks that currently run after curl_exec()) to
occur immediately after curl_exec() and before the callback-mode return so
callback mode still observes/receives transport failures; ensure you still close
the handle ($ch) and return/propagate an appropriate error/result instead of
silently returning [] in the callback branch (update the logic around
curl_exec(), the isset($callback) check, and the return path in
Executor::execute or the containing method).
🧹 Nitpick comments (1)
src/Executor/Executor.php (1)

134-138: Silence the unused-parameter warning in the wrapper callback.

At Line 134, $headers is intentionally ignored. Rename it to $_headers (or _) to avoid PHPMD noise without changing behavior.

Suggested diff
-$wrappedCallback = function (?string $data, ?array $headers) use ($callback): void {
+$wrappedCallback = function (?string $data, ?array $_headers) use ($callback): void {
     if ($data !== null) {
         $callback($data);
     }
 };
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Executor/Executor.php` around lines 134 - 138, The wrapper callback
assigned to $wrappedCallback currently accepts a second parameter named $headers
which is intentionally unused and triggers PHPMD warnings; update the parameter
name to a throwaway variable (e.g. $_headers or $_) in the anonymous function
signature used for $wrappedCallback in Executor.php so the behavior remains
unchanged but the unused-parameter warning is silenced.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@app/controllers/general.php`:
- Around line 612-621: The streaming branch currently overwrites the real
streamed status/headers and returns early, skipping accounting/retention; update
the block that runs when $streamingDetected is true to set $execution attributes
using the previously extracted streamed status/headers (do not hardcode
responseStatusCode to 200 or set responseHeaders to []), keep setting
duration/logs/errors as needed, and remove the early return so the function
continues to run the later usage metrics and retention/cleanup paths; locate and
adjust the code around $streamingDetected, $execution->setAttribute(...) and the
return true to preserve streamed metadata and allow downstream accounting to
execute.
- Around line 604-606: The code currently always passes streamCallback into
Executor::createExecution, forcing the execution into the streaming branch
(which returns []); change the call site so streamCallback is only passed when
the client explicitly requested streaming (e.g., detect Accept:
text/event-stream or a dedicated boolean like $request->get('stream', false)).
Update the call that sets responseFormat and streamCallback to conditionally
include streamCallback (and set RESPONSE_FORMAT_ARRAY_HEADERS only for
streaming), and ensure the non-stream path omits the callback so
Executor::createExecution runs the non-stream branch without returning an empty
array.

In `@src/Executor/Executor.php`:
- Around line 356-364: The handler $handleEvent currently invokes the headers
callback on the first data chunk unconditionally, causing non-SSE responses to
be treated as streaming; update $handleEvent to only fire the headers signal
when the response indicates a stream (check $responseHeaders['content-type'] for
'text/event-stream' or a configured stream marker flag) before calling
$callback(null, $responseHeaders) and setting $callbackHeadersFired; keep the
existing $callback($data, null) and strlen return behavior for all chunks, but
ensure $callbackHeadersFired is only set when the content-type/marker check
passes (and account for case-insensitive header keys).

---

Outside diff comments:
In `@src/Executor/Executor.php`:
- Around line 356-405: The early return in the callback branch suppresses curl
transport errors because curl_exec() error/status checks run after the "if
(isset($callback)) { curl_close($ch); return []; }" block; move the curl
error/status inspection (use curl_errno(), curl_error(), and curl_getinfo()
checks that currently run after curl_exec()) to occur immediately after
curl_exec() and before the callback-mode return so callback mode still
observes/receives transport failures; ensure you still close the handle ($ch)
and return/propagate an appropriate error/result instead of silently returning
[] in the callback branch (update the logic around curl_exec(), the
isset($callback) check, and the return path in Executor::execute or the
containing method).

---

Nitpick comments:
In `@src/Executor/Executor.php`:
- Around line 134-138: The wrapper callback assigned to $wrappedCallback
currently accepts a second parameter named $headers which is intentionally
unused and triggers PHPMD warnings; update the parameter name to a throwaway
variable (e.g. $_headers or $_) in the anonymous function signature used for
$wrappedCallback in Executor.php so the behavior remains unchanged but the
unused-parameter warning is silenced.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b03ad6d and 5ca8d82.

📒 Files selected for processing (2)
  • app/controllers/general.php
  • src/Executor/Executor.php

Comment on lines +604 to 606
responseFormat: Executor::RESPONSE_FORMAT_ARRAY_HEADERS,
streamCallback: $streamCallback,
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

streamCallback is always enabled, so all executions are forced into stream mode.

At Line 605 this unconditionally activates Executor::createExecution’s stream branch (which returns []). That makes non-stream fallback fragile and can produce undefined-index behavior when no stream events arrive. Gate callback usage by explicit client intent (e.g., Accept: text/event-stream) or a dedicated stream flag.

Suggested diff
+$isStreamRequested = \str_contains(
+    \strtolower($request->getHeader('accept', '')),
+    'text/event-stream'
+);
+
 $executionResponse = $executor->createExecution(
@@
-    streamCallback: $streamCallback,
+    streamCallback: $isStreamRequested ? $streamCallback : null,
 );
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/controllers/general.php` around lines 604 - 606, The code currently
always passes streamCallback into Executor::createExecution, forcing the
execution into the streaming branch (which returns []); change the call site so
streamCallback is only passed when the client explicitly requested streaming
(e.g., detect Accept: text/event-stream or a dedicated boolean like
$request->get('stream', false)). Update the call that sets responseFormat and
streamCallback to conditionally include streamCallback (and set
RESPONSE_FORMAT_ARRAY_HEADERS only for streaming), and ensure the non-stream
path omits the callback so Executor::createExecution runs the non-stream branch
without returning an empty array.

Comment on lines +612 to +621
if ($streamingDetected) {
$response->chunk('', true);

$execution->setAttribute('status', 'completed');
$execution->setAttribute('logs', '');
$execution->setAttribute('errors', '');
$execution->setAttribute('responseStatusCode', 200);
$execution->setAttribute('responseHeaders', []);
$execution->setAttribute('duration', \microtime(true) - $durationStart);
return true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Streaming branch records incorrect execution metadata and exits before accounting.

Line 618 hardcodes responseStatusCode to 200, and Line 619 drops response headers, even though Line 568 already extracts streamed status. Also, returning at Line 621 bypasses later usage metrics and retention cleanup logic in this function. Persist streamed status/headers and avoid returning before accounting paths execute.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/controllers/general.php` around lines 612 - 621, The streaming branch
currently overwrites the real streamed status/headers and returns early,
skipping accounting/retention; update the block that runs when
$streamingDetected is true to set $execution attributes using the previously
extracted streamed status/headers (do not hardcode responseStatusCode to 200 or
set responseHeaders to []), keep setting duration/logs/errors as needed, and
remove the early return so the function continues to run the later usage metrics
and retention/cleanup paths; locate and adjust the code around
$streamingDetected, $execution->setAttribute(...) and the return true to
preserve streamed metadata and allow downstream accounting to execute.

Comment on lines +356 to 364
$callbackHeadersFired = false;
$handleEvent = function ($ch, $data) use ($callback, &$callbackHeadersFired, &$responseHeaders) {
if (!$callbackHeadersFired) {
// Fire headers signal once before the first data chunk
$callback(null, $responseHeaders);
$callbackHeadersFired = true;
}
$callback($data, null);
return \strlen($data);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Header signal is emitted for any callback response, not just SSE.

Line 360 currently fires the “headers” signal on the first chunk unconditionally. Downstream logic treats that as “streaming detected”, so this can incorrectly switch non-SSE responses into the streaming path. Gate the header signal by response content type (e.g., text/event-stream) or an explicit stream marker.

Suggested diff
 $handleEvent = function ($ch, $data) use ($callback, &$callbackHeadersFired, &$responseHeaders) {
-    if (!$callbackHeadersFired) {
+    $contentType = \strtolower($responseHeaders['content-type'] ?? '');
+    $isEventStream = \str_starts_with($contentType, 'text/event-stream');
+    if ($isEventStream && !$callbackHeadersFired) {
         // Fire headers signal once before the first data chunk
         $callback(null, $responseHeaders);
         $callbackHeadersFired = true;
     }
-    $callback($data, null);
+    if ($isEventStream) {
+        $callback($data, null);
+    }
     return \strlen($data);
 };
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Executor/Executor.php` around lines 356 - 364, The handler $handleEvent
currently invokes the headers callback on the first data chunk unconditionally,
causing non-SSE responses to be treated as streaming; update $handleEvent to
only fire the headers signal when the response indicates a stream (check
$responseHeaders['content-type'] for 'text/event-stream' or a configured stream
marker flag) before calling $callback(null, $responseHeaders) and setting
$callbackHeadersFired; keep the existing $callback($data, null) and strlen
return behavior for all chunks, but ensure $callbackHeadersFired is only set
when the content-type/marker check passes (and account for case-insensitive
header keys).

@github-actions
Copy link
github-actions bot commented Mar 2, 2026

🔄 PHP-Retry Summary

Flaky tests detected across commits:

Commit 5ca8d82 - 3 flaky tests
Test Retries Total Time Details
UsageTest::testDatabaseStatsCollectionsAPI 1 10.29s Logs
RealtimeCustomClientQueryTest::testCollectionScopedDocumentsChannelWithQuery 1 30.16s Logs
RealtimeCustomClientTest::testChannelParsing 1 2.25s Logs

@github-actions
Copy link
github-actions bot commented Mar 2, 2026

✨ Benchmark results

  • Requests per second: 2,305
  • Requests with 200 status code: 414,981
  • P99 latency: 0.075588439

⚡ Benchmark Comparison

Metric This PR Latest version
RPS 2,305 1,413
200 414,981 254,449
P99 0.075588439 0.157921439

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0