Download a file to disk from async code

You start a download in async code, then hand the response body to a blocking thread that writes it to a file with plain std::io::copy.

fibre_http = "0"

Set up the project#

Start a new binary crate.

cargo new download
cd download

Put this in the [dependencies] section of Cargo.toml:

[dependencies]
fibre_http = "0"
tokio = { version = "1", features = ["full"] }

The program#

Writing to a file is blocking work. Doing it inside an async task holds up the other tasks on that thread. With fibre_http you can start the request in async code and read the body from a blocking thread instead.

Replace src/main.rs with this:

use fibre_http::AsyncClient;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = AsyncClient::new();
    let response = client.get("https://www.rust-lang.org/logos/rust-logo-512x512.png").send().await?.error_for_status()?;
    println!("status: {}", response.status());
    println!("content length: {:?}", response.headers().get("content-length"));

    let mut body = response.into_body().to_sync();
    let copied = tokio::task::spawn_blocking(move || {
        let mut file = std::fs::File::create("rust-logo.png")?;
        std::io::copy(&mut body, &mut file)
    })
    .await??;
    println!("wrote {copied} bytes to rust-logo.png");
    Ok(())
}

Run it:

$ cargo run
status: 200 OK
content length: Some("85848")
wrote 85848 bytes to rust-logo.png

You now have rust-logo.png in the project folder.

What each part does#

send().await returns as soon as the status and headers arrive. The body has not been read yet at that point.

into_body() takes the body out of the response. It is an async body. to_sync() turns it into a blocking one. The switch costs nothing and the bytes already received are kept.

The blocking body implements std::io::Read, so std::io::copy can stream it straight into the file. It runs inside spawn_blocking, on tokio's pool for blocking work. The whole file never has to sit in memory at once.

The ? after .await handles a panic in the blocking task. The second ? handles an error from the copy itself.

Where to go next#

The usage guide shows the other direction too, with to_async() on a body, plus reading a body in chunks and streaming a request body: https://github.com/excsn/fibre/blob/main/http/README.USAGE.md

The full API is on docs.rs at https://docs.rs/fibre_http.