curl -Z overlapping and truncated output on stdout #5175
Description
I've been attempting to use curl -sZ with multiple urls as queries to twitch.tv which returns several lines of json such as:
{
"streams": [
{
"_id": 1234567890,
"channel": {
"name": "foobar",
}
},
{
...
}
]
}
{
"streams": [
...
]
}All of this is done before piping the whole lot through jq and filtering as needed. The trouble is jq will randomly report errors such as:
jq: parse error: Invalid numeric literal at line 1, column 13708
If I were to look at that particular column, for example, I can clearly see the output has been truncated or interwoven somehow, e.g.
... banner":null,"{"streams": ...
I'm not sure how to prevent this, or provide a test case easily, but I couldn't find anything on google or the existing issues about potential output truncation or "weaving" when using -Z.
Without -Z it works correctly, albeit much more slowly.
curl 7.70.0-DEV (x86_64-pc-linux-gnu) libcurl/7.70.0-DEV OpenSSL/1.1.1f zlib/1.2.11 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.2.0) nghttp2/1.40.0 librtmp/2.3
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets
Linux teapot 5.5.13-arch1-1 #1 SMP PREEMPT Wed, 25 Mar 2020 16:04:40 +0000 x86_64 GNU/Linux