Resumable file uploads for business applications: large files without the failed-upload loop

Every business application eventually meets the same support ticket: a user tried to upload a large file — a tender document, a training video, a CAD export, a month of scanned invoices — and the upload failed near the end. They try again, it fails again, and the file never arrives. The problem is rarely the file. It is the assumption that an upload is one request that either succeeds or does not.

Once a file is large enough that a transfer takes minutes rather than seconds, that assumption breaks. Networks drop, laptops sleep, VPNs reconnect, and mobile connections switch towers. A resumable upload design treats interruption as normal, not exceptional: the file is split into chunks, each chunk is uploaded independently, and a broken transfer picks up exactly where it stopped.

Why a single-request upload fails at scale

A single POST with a multi-gigabyte body concentrates every risk into one fragile operation. A proxy may enforce a timeout or body-size limit long before the transfer finishes, and when the connection dies at 98 percent, the client has no way to say “keep what you already have” — the protocol never gave it that vocabulary.

Chunking changes the unit of failure: a 5 MB chunk that fails costs seconds to retry, not the whole file, and chunks can upload in parallel. The trade-off is real — you now run a small protocol instead of a single endpoint — but for any file over a few tens of megabytes, it is what makes uploads reliable.

Design the session before the first byte

A resumable upload starts with a session, not with data. The client asks the server to create an upload session for a named file: its total size, a checksum of the whole file if available, the intended content type, and the user it belongs to. The server returns a session id and a target chunk size, and records the session in a database with an expiry.

That session is what makes resumption possible: when a client reconnects, it asks which chunks are already stored and uploads only the missing ones. It is also the natural place to enforce policy before any bytes move — run the go-live checklist for business uploads at session creation, so a rejected upload fails fast instead of after gigabytes of transfer.

Choose a chunk size deliberately

Chunk size is a compromise. Small chunks (1–5 MB) resume cheaply and give smooth progress bars, but each chunk costs a request and a database round trip. Large chunks (50–100 MB) cut overhead but make each retry expensive and progress jumpy. For most business applications, 5–10 MB is a good default.

Two rules matter more than the exact number: the chunk size must be fixed for the life of a session, so the server can address chunks by index, and the final chunk may be smaller — the server must not reject a file whose size is not a multiple of the chunk size.

Verify every chunk, then verify the whole

A chunked upload without integrity checks is a corruption pipeline with extra steps. The client should send a checksum (at least SHA-256, or MD5 where a storage provider requires it) with each chunk, and the server should verify it before acknowledging that chunk. A chunk that fails its checksum is retried, not stored.

When all chunks arrive, the server assembles them and verifies the whole-file checksum against the one declared at session creation. Only then does the file become visible to the rest of the application. This two-level check catches transport corruption and misordered chunks — the same discipline that makes a secure upload pipeline trustworthy: validate at every boundary.

Completion is a transaction, not an afterthought

The most dangerous moment in a chunked upload is completion. Something must turn stored chunks into one finished file, atomically, exactly once — and if the client retries the complete call because its response was lost, the server must recognise the session as already completed instead of assembling the file twice.

Keep completed files out of the web root and serve them through an access check, the same way signed, expiring download URLs protect files on the way out.

Security: chunking multiplies the attack surface

Splitting a file into pieces does not split the risk. Every chunk endpoint still accepts untrusted bytes, so the same controls apply: authenticate the session owner on every request, enforce per-user and per-file quotas, cap concurrent sessions, and expire abandoned ones so stale chunks do not accumulate into a storage bill. A client that can open unlimited sessions and never finish them is running a slow denial-of-service against your storage budget.

Malware handling needs a decision up front: scan and quarantine before the file is usable, holding the assembled file in quarantine until the scan completes. Never expose a partially assembled or unscanned file to other users mid-upload. And if the uploads are images, validate the bytes and strip the metadata after assembly, not before.

A client that deserves the protocol

The server side is only half the work. The client should show real progress per chunk, retry failed chunks with backoff instead of surfacing an error on the first hiccup, and persist the session id in local storage so a refresh or browser restart resumes instead of restarting. A resumable protocol with a client that gives up on the first failure is just a complicated way to fail.

Keep the API surface small and version it deliberately — version the API without multiplying endpoints — because upload clients in the field are the hardest to force-upgrade.

A minimal chunked upload, sketched

The shape of the protocol is simple enough to sketch in a few lines. The client creates a session, uploads chunks by index with checksums, and completes:

// 1. Create the session
POST /uploads
{ "fileName": "tender-pack.zip", "size": 2147483648,
  "sha256": "9f2c…", "contentType": "application/zip" }
// → { "sessionId": "upl_01…", "chunkSize": 8388608 }

// 2. Upload each chunk (retryable, parallelisable)
PUT /uploads/{sessionId}/chunks/{index}
Content-MD5: <base64 checksum of this chunk>
<8 MB of bytes>

// 3. Ask what is missing, then complete
GET /uploads/{sessionId}  // → { "received": [0,1,2,4] }
POST /uploads/{sessionId}/complete
// → { "fileId": "file_01…", "status": "scanning" }

Note that completion returns “scanning”, not “ready”: the file becomes usable only after validation and scanning pass.

A checklist before you ship

• Chunk size fixed per session; final chunk may be smaller.
• Per-chunk checksums verified server-side; whole-file checksum verified on completion.
• Completion is idempotent — a retried complete call returns the same file.
• Abandoned sessions and orphaned chunks expire and are cleaned up automatically.
• Per-user quotas and concurrent-session limits are enforced at session creation.
• Assembled files stay private until validation and malware scanning pass.
• The client persists the session id and resumes after refresh, restart, or network change.
• Progress is shown per chunk, with retries hidden behind backoff, not error dialogs.

Large-file uploads feel simple until they meet a real user on a real network. Chunking, sessions, checksums, and idempotent completion are the difference between an upload that usually works and one that works when it matters. Talk to us about building upload-heavy business software, or get in touch to scope the work.