Read compressed responses

You turn on gzip support so the server can send a compressed body and you still read plain text. Then you turn decoding off to see the compressed bytes that came over the network.

fibre_http = "0"

Set up the project#

Start a new binary crate.

cargo new compressed
cd compressed

Put this in the [dependencies] section of Cargo.toml:

[dependencies]
fibre_http = { version = "0", features = ["gzip", "json"] }
serde_json = "1"

gzip is what lets the client decode gzip bodies. A zstd feature does the same for zstd.

The program#

httpbin.org's /gzip always answers with a gzip body. Replace src/main.rs with this:

use fibre_http::SyncClient;
use serde_json::Value;

fn main() -> Result<(), fibre_http::Error> {
    let client = SyncClient::new();
    let response = client.get("https://httpbin.org/gzip").send()?;
    println!("content encoding: {:?}", response.headers().get("content-encoding"));
    let body: Value = response.json()?;
    println!("gzipped: {}", body["gzipped"]);

    let raw = SyncClient::builder().decompress(false).build_sync()?;
    let bytes = raw.get("https://httpbin.org/gzip").send()?.bytes()?;
    println!("without decompression the body starts with {:02x?}", &bytes[..2]);
    Ok(())
}

Run it:

$ cargo run
content encoding: None
gzipped: true
without decompression the body starts with [1f, 8b]

What each part does#

With the gzip feature on, every request tells the server it accepts gzip. When a gzip body comes back, the client decodes it as it reads, so json() sees plain JSON. The Content-Encoding header is gone from the response because the body you read is no longer compressed.

decompress(false) turns decoding off for that client. The body comes back exactly as the server sent it. 1f 8b are the two bytes every gzip stream starts with.

A request that sets its own Accept-Encoding header also gets the body as the server sent it, whatever the client's setting.

Where to go next#

The usage guide covers the decoding options: https://github.com/excsn/fibre/blob/main/http/README.USAGE.md