# Stream an upload and read a download in chunks

**For:** Rust developers moving bodies too big to hold in memory.

You upload a file straight from disk without reading it into memory first, then read a download piece by piece as it arrives.

Before you start: you need Rust with Cargo. fibre_http comes from crates.io as `fibre_http = "0"`.

## Set up the project

Start a new binary crate.

```sh
cargo new streaming
cd streaming
```

Put this in the `[dependencies]` section of `Cargo.toml`:

```toml title="Cargo.toml"
[dependencies]
fibre_http = { version = "0", features = ["json"] }
serde_json = "1"
```

The `json` feature is only there to read httpbin's answer to the upload.

## The program

The program writes a small file to upload, sends it to httpbin.org's `/put` and then downloads 100,000 bytes from `/stream-bytes`. Replace `src/main.rs` with this:

```rust title="src/main.rs"
use std::fs::File;
use std::io::Write;

use fibre_http::SyncClient;
use serde_json::Value;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut file = File::create("upload.txt")?;
    for line in 0..2000 {
        writeln!(file, "line {line} of the upload")?;
    }
    drop(file);

    let client = SyncClient::new();

    let file = File::open("upload.txt")?;
    let length = file.metadata()?.len();
    let echo: Value = client.put("https://httpbin.org/put").body_reader(file, Some(length)).send()?.error_for_status()?.json()?;
    println!("sent {length} bytes, the server counted {}", echo["headers"]["Content-Length"]);

    let mut body = client.get("https://httpbin.org/stream-bytes/100000?chunk_size=10000").send()?.error_for_status()?.into_body();
    let mut chunks = 0;
    let mut total = 0;
    while let Some(chunk) = body.chunk()? {
        chunks += 1;
        total += chunk.len();
    }
    println!("read {total} bytes in {chunks} chunks");
    Ok(())
}
```

Run it:

```shell-session
$ cargo run
sent 46890 bytes, the server counted "46890"
read 100000 bytes in 12 chunks
```

The number of chunks changes from run to run, since it depends on how the bytes arrive over the network. The total is always 100000.

## What each part does

`body_reader` takes anything that implements `std::io::Read` and sends it as the request body while reading it. Passing the length lets the client send a `Content-Length` header, which some servers require. Pass `None` when you do not know the length and the body goes out in chunks instead.

A streamed body can only be read once. So a request with one is never retried. A 307 or 308 redirect comes back to you as the response instead of being followed.

`into_body()` hands you the response body without reading it. Each call to `chunk()` returns the next piece as it arrives and `None` at the end. A blocking body also implements `std::io::Read`, so `std::io::copy` works on it too.

If you drop a body before its end, the client closes that exchange so the connection is not left holding unread bytes.

## Where to go next

The async client has the same calls: `body_stream` sends a stream of byte chunks and `chunk().await` reads a body piece by piece. The usage guide has both: [https://github.com/excsn/fibre/blob/main/http/README.USAGE.md](https://github.com/excsn/fibre/blob/main/http/README.USAGE.md)