Skip to content
Commitwise
Menu
System Design

Design file upload

Design photo and file uploads for a mobile app step by step, with presigned URLs so bytes go straight to object storage, verifying what actually arrived, resumable multipart uploads that survive bad networks and app restarts, background uploads on iOS and Android, and a processing pipeline for scanning and thumbnails, with code in Kotlin, Java, TypeScript and Swift.

intermediate

The core idea with one simple example.Step by step, with more examples and exercises.A cheat sheet: key APIs, gotchas and trade-offs.Internals, trade-offs and production pitfalls.

A user attaches a 40 MB video to a support ticket on a train. The first version of the app sends it to the API server in one request; the server writes it to disk, then copies it to storage. The upload fails at 92% when the train enters a tunnel and starts again from zero. Meanwhile, API servers run out of memory because a few dozen uploads hold their request threads and buffers for minutes.

Uploading files looks like “just a POST”, but at scale it’s its own system: the bytes should bypass your servers entirely, uploads must survive bad networks and app restarts, and nothing a client sends can be trusted until it’s been checked.

New to some of these? Open what you need, or skip ahead.

Basics:What is object storage?

Object storage (Amazon S3, Google Cloud Storage, Azure Blob Storage, Cloudflare R2) stores files as objects under a key, like uploads/u42/7f3a.jpg, with practically unlimited capacity. It’s cheap, very durable, and serves files over HTTP. It isn’t a file system: you write and read whole objects (or parts of them), you don’t edit them in place.

Basics:What is a presigned URL?

A presigned URL is a temporary link to object storage, signed by your server with its secret key, that allows one specific action (for example, uploading to one key) until it expires. The client can use it directly without having any storage credentials.

Part Not started

Step 1: Requirements and estimates

Functional requirements:

  • Upload photos, videos and documents, from a few KB up to a few GB.
  • Show progress, and resume after a lost connection or after the app was closed.
  • Process files after upload: thumbnails for images, previews for videos, virus scanning for documents.
  • Files are private: only people allowed to see the ticket or chat can download them.

Non-functional requirements:

  • API servers never handle the file bytes.
  • Nothing is used (shown to others, attached to a message) until it’s been checked.
  • An upload interrupted at 92% continues from about 92%, not from zero.

Clarify: file size distribution (it decides single-request vs multipart), allowed types, who can read files (private with signed downloads vs public via CDN), retention and deletion (GDPR, user deletion), processing needs (transcoding is a large subsystem of its own), deduplication, and quotas per user. The design follows from three principles: direct-to-storage data path with the API only issuing permissions and recording metadata; untrusted until verified (the client’s declared type, size and checksum are claims); and resumability at the chunk level, with the client owning retry and the server owning state.

Daily active users:          20 million
Uploads:                     ~0.5 per user per day → 10 million / day ≈ 120 / s (peak ~500 / s)
Average size:                ~3 MB (photos), long tail of videos up to 2 GB
Ingest:                      10 million × 3 MB ≈ 30 TB / day ≈ 350 MB/s average
Storage after a year:        ~11 PB before lifecycle rules (thumbnails and previews add ~10%)

If bytes went through the API:   350 MB/s of inbound traffic, plus the same again outbound to storage,
                                 and thousands of slow requests holding connections for minutes
  • Bandwidth, not requests, is the problem: 120 uploads per second is little; 350 MB per second through your API servers is a lot.
  • Mobile uploads are slow: a 40 MB video on a weak connection can take several minutes, so anything holding a server connection that long scales badly.
  • Sizes from KB to GB; resumable across network loss and app restarts; private by default.
  • ~10M uploads/day, ~30 TB/day ingest: bandwidth-dominated, slow long-lived client connections.
  • Principles: direct-to-storage data path, client claims untrusted until verified, chunk-level resumability.
  • Probe retention/deletion, quotas, dedup, and whether transcoding is in scope.

Check this part

  1. Why shouldn't file bytes pass through the API servers?

  2. Which client-provided values can the server trust without checking?

Users upload photos, videos and documents from phones on unreliable networks, up to a few GB. Uploads must resume after failures and app restarts, never pass through API servers, be validated (type, size, malware) before use, and be processed into thumbnails or previews. Files are private by default and served through short-lived links or a CDN.

Part Not started

Step 2: Presigned uploads straight to storage

The upload happens in three steps:

  1. Ask: the app tells the API “I want to upload a 40 MB video called crash.mp4 to ticket 1042”. The API checks the user is allowed to, creates an upload record with status pending, and returns a presigned URL for one storage key.
  2. Upload: the app sends the file directly to object storage using that URL. The API isn’t involved.
  3. Confirm: when the file is in storage, the system checks what actually arrived (size, type), and only then marks the upload as ready and starts processing.

The key idea is that the API gives permission, and storage receives the bytes.

The presigned grant should be as narrow as possible: one server-generated key (never a client-chosen path), the declared content type bound into the signature, a content-length-range condition (presigned POST policies support it; for presigned PUT, signing Content-Length pins an exact size), a short expiry (minutes), and optionally a required checksum header (x-amz-checksum-sha256) so storage itself rejects corrupted bytes. Completion should be event-driven (S3 event notifications, GCS Pub/Sub notifications) with an idempotent consumer, optionally plus a client complete call for faster UX; either way the server HEADs the object and validates size, content type and checksum, then sniffs the real type from the first bytes before marking it uploaded and queueing processing. Pending records that never complete are cleaned up by a TTL job, and an object storage lifecycle rule deletes orphaned objects under the uploads prefix.

Direct upload with a presigned URL

Mobile appUpload APIUploads DBObject storageStorage eventsProcessor

Tap a box to see what it does.

@PostMapping("/uploads")
fun createUpload(@AuthenticationPrincipal user: UserId, @RequestBody req: CreateUploadRequest): UploadGrant {
    require(req.contentType in ALLOWED_TYPES) { "Unsupported type" }
    require(req.sizeBytes in 1..MAX_SIZE_BYTES) { "Too large" }
    quotas.reserve(user, req.sizeBytes)                       // fails if over quota

    val key = "uploads/${user.value}/${UUID.randomUUID()}"    // server-chosen key, never the file name
    val upload = uploads.insert(Upload(owner = user, key = key, declaredSize = req.sizeBytes,
        declaredType = req.contentType, sha256 = req.sha256, status = Status.PENDING))

    val presigned = presigner.presignPutObject { p ->
        p.signatureDuration(Duration.ofMinutes(15))
        p.putObjectRequest { o ->
            o.bucket(BUCKET).key(key)
                .contentType(req.contentType)                 // must match on upload
                .contentLength(req.sizeBytes)                 // exact size is signed
                .checksumSHA256(req.sha256)                   // storage rejects other bytes
        }
    }
    return UploadGrant(uploadId = upload.id, url = presigned.url().toString(), headers = presigned.signedHeaders())
}
@PostMapping("/uploads")
public UploadGrant createUpload(@AuthenticationPrincipal UserId user, @RequestBody CreateUploadRequest req) {
    if (!ALLOWED_TYPES.contains(req.contentType())) throw new BadRequest("Unsupported type");
    if (req.sizeBytes() <= 0 || req.sizeBytes() > MAX_SIZE_BYTES) throw new BadRequest("Too large");
    quotas.reserve(user, req.sizeBytes());

    String key = "uploads/" + user.value() + "/" + UUID.randomUUID();   // server-chosen key
    Upload upload = uploads.insert(Upload.pending(user, key, req.sizeBytes(), req.contentType(), req.sha256()));

    PresignedPutObjectRequest presigned = presigner.presignPutObject(p -> p
        .signatureDuration(Duration.ofMinutes(15))
        .putObjectRequest(o -> o.bucket(BUCKET).key(key)
            .contentType(req.contentType())
            .contentLength(req.sizeBytes())
            .checksumSHA256(req.sha256())));

    return new UploadGrant(upload.id(), presigned.url().toString(), presigned.signedHeaders());
}
app.post("/uploads", async (req, res) => {
  const { contentType, sizeBytes, sha256 } = req.body as CreateUploadRequest;
  if (!ALLOWED_TYPES.has(contentType)) return res.status(400).json({ error: "Unsupported type" });
  if (sizeBytes <= 0 || sizeBytes > MAX_SIZE_BYTES) return res.status(400).json({ error: "Too large" });
  await quotas.reserve(req.user.id, sizeBytes);

  const key = `uploads/${req.user.id}/${crypto.randomUUID()}`;    // server-chosen key
  const upload = await uploads.insert({ owner: req.user.id, key, declaredSize: sizeBytes,
    declaredType: contentType, sha256, status: "pending" });

  const url = await getSignedUrl(s3, new PutObjectCommand({
    Bucket: BUCKET, Key: key, ContentType: contentType, ContentLength: sizeBytes, ChecksumSHA256: sha256,
  }), { expiresIn: 15 * 60 });

  res.json({ uploadId: upload.id, url });
});
  • The key is chosen by the server, with a random id. A client can’t overwrite someone else’s file or pick a path like ../admin.
  • Type, size and checksum are part of the signature. If the app tries to upload something else, storage refuses it.
  • 15 minutes is enough to start the upload; a leaked URL is useless soon after.
  • The upload record starts as pending. Nothing else in the app can use it yet.
Spot the bugProfile pictures that are 2 GB 'images'

Tap the line with the bug.

@PostMapping("/uploads/{id}/complete")
fun complete(@AuthenticationPrincipal user: UserId, @PathVariable id: UploadId) {
    val upload = uploads.find(id, owner = user) ?: throw NotFound()
    uploads.update(upload.copy(
        status = Status.READY,
        size = upload.declaredSize,
        type = upload.declaredType,
    ))
    profiles.setAvatar(user, upload.key)
}
@PostMapping("/uploads/{id}/complete")
public void complete(@AuthenticationPrincipal UserId user, @PathVariable UploadId id) {
    Upload upload = uploads.find(id, user).orElseThrow(NotFound::new);
    uploads.update(upload.withStatus(Status.READY)
        .withSize(upload.declaredSize())
        .withType(upload.declaredType()));
    profiles.setAvatar(user, upload.key());
}
app.post("/uploads/:id/complete", async (req, res) => {
  const upload = await uploads.find(req.params.id, req.user.id);
  if (!upload) return res.sendStatus(404);
  await uploads.update(upload.id, {
    status: "ready",
    size: upload.declaredSize,
    type: upload.declaredType,
  });
  await profiles.setAvatar(req.user.id, upload.key);
  res.sendStatus(204);
});

  • API issues narrow grants: server-generated key, signed content type, size (exact Content-Length for PUT, content-length-range for POST policies), required checksum, ~15 min expiry.
  • Pending record + quota reservation at grant time; TTL cleanup of pending rows and lifecycle rules for orphaned objects.
  • Completion via storage events (idempotent consumer) and/or client complete; HEAD and verify size/checksum, sniff real type, then process.
  • Never use declared metadata; only processed derivatives (decoded, metadata-stripped, resized) are served.

Check this part

  1. Who chooses the storage key for an upload?

  2. When can an uploaded file be shown to other users?

  3. Why must the storage-event consumer be idempotent?

The client asks the API for an upload; the API checks permissions, quotas and declared size and type, creates a pending record, and returns a presigned URL limited to one key, a content type, a size range and a short expiry. The client uploads straight to storage. Completion comes from a storage event or a complete call, and the server verifies the stored object before marking it ready.

Part Not started

Step 3: Resumable uploads from a phone

One big request is fragile: if it breaks at 92%, all of it is lost. So large files are split into parts, say 8 MB each. Each part is uploaded separately, and storage remembers which parts it has. If the connection drops, the app asks “which parts do you have?” and sends only the rest. When all parts are there, the app asks storage to combine them into the final file.

The phone also has to keep uploading when the user leaves the app. Both platforms have tools for this:

  • iOS: a background URLSession, which lets the system continue the transfer even when the app is suspended.
  • Android: WorkManager, which runs the upload when there’s a network and retries later if it fails, even after a restart.

S3-style multipart: CreateMultipartUpload returns an upload id; each part (5 MB to 5 GB, up to 10,000 parts) gets its own presigned UploadPart URL; ListParts reports what’s stored; CompleteMultipartUpload with the part numbers and ETags assembles the object; AbortMultipartUpload (plus a lifecycle rule that aborts incomplete uploads after N days) cleans up. Parts can upload in parallel (2–4 on mobile; more saturates bad links). The client persists (uploadId, partSize, completed parts with ETags) so it survives process death. Protocols like tus standardize the same idea for servers that proxy uploads. On iOS, background URLSession only supports file-based upload tasks (not streamed bodies), so the app writes each part to a temporary file; the system schedules transfers, and the app is relaunched in the background to handle completion via handleEventsForBackgroundURLSession. On Android, a CoroutineWorker with a CONNECTED (or UNMETERED for large videos) constraint, foreground info for long uploads, and ExistingWorkPolicy.KEEP per upload id.

struct PartGrant: Decodable { let partNumber: Int; let url: URL }

final class MultipartUploader {
    private let api: UploadsApi
    private let session: URLSession            // configured as .background(withIdentifier:) in the app
    private let partSize = 8 * 1024 * 1024

    init(api: UploadsApi, session: URLSession) { self.api = api; self.session = session }

    func resume(_ state: UploadState) async throws {
        // Ask the server (which calls ListParts) what storage already has
        let done = try await api.completedParts(uploadId: state.uploadId)
        let missing = (1...state.partCount).filter { !done.contains($0) }
        let grants = try await api.partURLs(uploadId: state.uploadId, parts: missing)

        for grant in grants {
            let partFile = try state.writePart(grant.partNumber, size: partSize)   // background sessions need files
            var request = URLRequest(url: grant.url)
            request.httpMethod = "PUT"
            session.uploadTask(with: request, fromFile: partFile).resume()       // the system continues it in the background
        }
        // When all parts report success (in the session delegate), call api.complete(uploadId:)
    }
}
class UploadWorker(
    context: Context,
    params: WorkerParameters,
    private val api: UploadsApi,
    private val http: OkHttpClient,
    private val store: UploadStateStore,
) : CoroutineWorker(context, params) {

    override suspend fun doWork(): Result {
        val state = store.load(inputData.getString("uploadId")!!) ?: return Result.failure()
        setForeground(progressNotification(state))              // long-running: show progress

        return try {
            val done = api.completedParts(state.uploadId)        // what storage already has
            val missing = (1..state.partCount).filter { it !in done }
            for (grant in api.partUrls(state.uploadId, missing)) {
                val body = state.readPart(grant.partNumber).toRequestBody()
                val response = http.newCall(Request.Builder().url(grant.url).put(body).build()).execute()
                response.use { if (!it.isSuccessful) return Result.retry() }   // backoff, then resume
                store.markPartDone(state.uploadId, grant.partNumber)
            }
            api.complete(state.uploadId)
            Result.success()
        } catch (e: IOException) {
            Result.retry()
        }
    }
}

fun enqueueUpload(context: Context, uploadId: String) {
    val request = OneTimeWorkRequestBuilder<UploadWorker>()
        .setInputData(workDataOf("uploadId" to uploadId))
        .setConstraints(Constraints(requiredNetworkType = NetworkType.CONNECTED))
        .setBackoffCriteria(BackoffPolicy.EXPONENTIAL, 30, TimeUnit.SECONDS)
        .build()
    WorkManager.getInstance(context).enqueueUniqueWork("upload-$uploadId", ExistingWorkPolicy.KEEP, request)
}
// Server side: the endpoints the clients above call
@PostMapping("/uploads/{id}/parts")
public List<PartGrant> partUrls(@AuthenticationPrincipal UserId user, @PathVariable UploadId id,
                                @RequestBody List<Integer> partNumbers) {
    Upload upload = uploads.find(id, user).orElseThrow(NotFound::new);
    return partNumbers.stream().map(n -> new PartGrant(n, presigner.presignUploadPart(p -> p
            .signatureDuration(Duration.ofHours(1))
            .uploadPartRequest(r -> r.bucket(BUCKET).key(upload.key())
                .uploadId(upload.multipartId()).partNumber(n))).url().toString()))
        .toList();
}

@GetMapping("/uploads/{id}/parts")
public Set<Integer> completedParts(@AuthenticationPrincipal UserId user, @PathVariable UploadId id) {
    Upload upload = uploads.find(id, user).orElseThrow(NotFound::new);
    return s3.listPartsPaginator(r -> r.bucket(BUCKET).key(upload.key()).uploadId(upload.multipartId()))
        .parts().stream().map(Part::partNumber).collect(Collectors.toSet());
}
// Web client: upload missing parts, 3 at a time
async function resumeUpload(file: File, state: UploadState, api: UploadsApi) {
  const done = new Set(await api.completedParts(state.uploadId));
  const missing = range(1, state.partCount).filter((n) => !done.has(n));
  const grants = await api.partUrls(state.uploadId, missing);

  await runWithConcurrency(3, grants, async ({ partNumber, url }) => {
    const start = (partNumber - 1) * state.partSize;
    const blob = file.slice(start, start + state.partSize);
    const res = await fetch(url, { method: "PUT", body: blob });
    if (!res.ok) throw new Error(`Part ${partNumber} failed: ${res.status}`);
  });

  await api.complete(state.uploadId);   // server calls CompleteMultipartUpload, then verifies
}

Production puzzles

Predict the output

iOS: the app creates a background URLSession upload task with uploadTask(with:from: data) (an in-memory Data body). What happens?

Predict the output

Users start large uploads and then delete the draft, or uninstall the app. Six months later, the storage bill is much higher than the sum of all completed files. Why?

The processing pipeline

After verification, the processor works on a copy, never in place:

  • Sniff the real type from the first bytes (magic numbers), and reject mismatches.
  • Scan documents and archives for malware before anyone can download them.
  • Images: decode (proves it’s an image), apply the EXIF orientation, strip metadata (GPS!), produce sizes the clients need, in modern formats (WebP or AVIF).
  • Video: transcoding into streaming renditions (HLS) is a large system on its own; in an interview, say it’s a separate pipeline with its own queue.
  • Mark the upload ready and notify the client (push, or the client polls the upload status).

Downloads of private files use short-lived signed GET URLs (or signed CDN cookies), issued after the same permission check as the ticket or chat the file belongs to.

Exercisemedium

Duplicate uploads

Users often upload the same file twice: the same photo to two tickets, or the app retries createUpload after a timeout. How could you avoid storing the same bytes twice, and what must you be careful about?

›

The client already computes a SHA-256 checksum before uploading. What could the server do with it?

SolutionTry it yourself first
  • Retries: send an idempotency key with createUpload (for example, the local draft id). The server returns the existing pending upload instead of creating a second one.
  • Same content: before issuing a URL, look up a verified object with the same SHA-256 and size. If one exists, create a new upload record pointing to the existing object and skip the upload entirely.

Be careful:

  • Only dedupe against verified checksums computed by storage or the processor, never against a client’s claim, or a user could “claim” someone else’s file by sending its hash.
  • Count references: an object can only be deleted when no upload record points to it anymore.
  • Deduplication across users can leak information (“this exact file already exists”), so many systems only dedupe within one user’s files.
  • Multipart: create → presigned part URLs → ListParts to resume → complete with ETags; abort and lifecycle rules for incomplete uploads.
  • Client persists upload id and completed parts; 2–4 parallel parts on mobile.
  • iOS background URLSession: file-based uploads only, relaunch via handleEventsForBackgroundURLSession. Android: CoroutineWorker with network constraints, foreground info, unique work per upload, backoff.
  • Processing: sniff type, scan, decode and strip metadata, derive sizes; private downloads via short-lived signed URLs.

Check this part

  1. Why split a large upload into parts?

  2. What does a background URLSession require for uploads?

  3. Why add a lifecycle rule to abort incomplete multipart uploads?

Large files are uploaded in parts (multipart upload), each with its own presigned URL, so a failure only retries one part and an upload can resume from the parts storage already has. On the phone, uploads run as background transfers (URLSession background sessions, WorkManager), persist their state, and retry with backoff when the network returns.

Key takeaways

  • The API gives permission; object storage receives the bytes.
  • The server chooses the storage key and limits type, size and time.
  • Verify what actually arrived before using it, and serve processed versions.
  • Large uploads go in parts, so they can resume, and run as background work on the phone.
  • Direct-to-storage with narrow presigned grants and pending records.
  • Event-driven, idempotent verification; declared metadata is never trusted.
  • Multipart with client-persisted state, plus lifecycle cleanup of incomplete uploads.
  • Background transfers via URLSession background sessions and WorkManager.

Finished this page?Nice work, this page is complete!

Mark it as done to track your progress. It's saved in this browser, no account needed.

Its cards are now in your Bites review ⚡
Esc

Type to search the whole site.