Announcement date: 29 September 2026. Google Cloud introduced streaming through API Gateway in Public Preview. A gateway can forward requests and responses incrementally instead of buffering the full response. The documentation lists HTTP/2, HTTP/1.1 chunked transfer, Server-Sent Events (SSE), WebSockets and bidirectional gRPC. A relevant use is an LLM response produced over time. Source: Google Cloud release notes, 29 September 2026.

Facts that affect the design

  • Streaming mode is fixed when you create a new gateway; you cannot switch an existing gateway later. The backend must also use the appropriate protocol.
  • The default deadline for SSE and chunked responses is 15 seconds. Configure deadline explicitly: the stream ends when the deadline expires even if data is still arriving. The backend enforces its own timeout too.
  • During Public Preview, streaming-enabled gateways cannot be placed behind the external Application Load Balancer or use custom domains that depend on it. Terraform does not yet support enabling streaming.

Google documents these limitations and protocol settings in Configure streaming for LLM responses and other traffic. This is Public Preview, not general availability.

Architecture interpretation

A gateway that buffers the whole response can hide the benefit of a streaming backend. The new option makes earlier delivery of initial chunks worth evaluating, but it is not a switch that makes “AI faster”. Test time to first chunk, dropped connections, stream completion, authentication and request limits. A client may see a partial answer before an error; repeating the whole request is not always safe or cheap.

If the current API Gateway uses a custom domain and a load balancer, I would not assume a direct replacement. Compare a separate path for streaming responses with keeping the existing gateway for ordinary calls. Decide after testing realistic generation times, client behaviour and the cost of operating two paths. This is an architecture inference, not a vendor guarantee.

Question for your project: does the user need the first chunk earlier, and if so, how will you handle an interrupted stream, the deadline and the existing API address?

Primary sources: Google Cloud release notes, 29 September 2026; API Gateway configuration, deadlines and Public Preview limitations. The architecture interpretation is the author's inference.