SD3.4 · LESSON 4 OF 6

Files, blocks and objects

Choose representations and storage boundaries from access patterns, ownership and retention requirements.

THE IDEA

Files, blocks and objects

The three storage interfaces expose different operations. Files have paths and let you read or write part of a file. Blocks are fixed-size units at addresses. Objects are stored whole under a key: you put, get or delete them, and an update replaces the entire object. A key like logs/2025/a.txt is just a string; listing with a prefix and delimiter makes it look like folders, but there is no directory behind it, so "renaming a folder" is a copy and delete of every object.

prefix:

stored keys: logs/2025/a.txt logs/2025/b.txt logs/2026/c.txt readme.txt list(prefix "logs/", delimiter "/") objects here: none common prefixes (like sub-folders): logs/2025/, logs/2026/ there are no directories behind those prefixes: "logs/" is just the start of some keys rename logs/ → archive/ = copy + delete for each of the 3 objects = 6 operations

	file	block	object
interface	paths, open/read/write/seek	fixed-size blocks at addresses	put/get/delete whole objects by key
partial update	yes	yes (a block)	no: replace the object
rename a folder	cheap	n/a	copy + delete per object

List the same keys with different prefixes.

Predict first. Does an object key containing slashes necessarily create real directories?

Chapter context

A data model describes the facts a system owns and the relationships it must preserve. Storage choices should follow how those facts are queried, updated and retained, rather than a blanket preference for one database category.

TRY IT YOURSELF

Part 1. Do it hands-on

List prefixes and cost a rename by hand, then build a small object store.

prefix:

	file	block	object
interface	paths, open/read/write/seek	fixed-size blocks at addresses	put/get/delete whole objects by key
partial update	yes	yes (a block)	no: replace the object
rename a folder	cheap	n/a	copy + delete per object

common prefixes under logs/ objects at the top level operations to rename logs/ slashes create real directories? (yes/no)

To do: Objects: logs/2025/a.txt, logs/2025/b.txt, logs/2026/c.txt, readme.txt. Listing with prefix "logs/" and delimiter "/" groups keys into how many common prefixes (like sub-folders)?
To do: Listing with an empty prefix and delimiter "/": how many objects appear directly at the top level (not under a prefix)?
To do: There is no rename. Moving "logs/" to "archive/" means copying then deleting each of the 3 objects under it. Operations?
To do: Does an object key containing slashes necessarily create real directories? Type yes or no

Part 2. Do it in code

class ObjectStore with self.objects (dict key → bytes-like text).
put(key, data): store (replaces the whole object; there is no partial update). get(key): the data or None. delete(key).
list(prefix="", delimiter="/"): return {"keys": the sorted keys that start with the prefix and contain no delimiter after it, "prefixes": the sorted unique common prefixes (prefix up to and including the next delimiter) for the rest}.
rename_prefix(old, new): there is no rename, so copy every object under `old` to the new prefix and delete the old key; return the number of operations (copies + deletes).
is_real_directory(store, path): always False — slashes in a key do not create directories.

class ObjectStore:
    def __init__(self):
        self.objects = {}

    def put(self, key, data):
        pass

    def get(self, key):
        pass

    def delete(self, key):
        pass

    def list(self, prefix="", delimiter="/"):
        pass

    def rename_prefix(self, old, new):
        pass

    def is_real_directory(self, path):
        pass

Ctrl/⌘ + Enter runs it

Stuck, or something not working? You can come back to it any time.

CHECK YOUR UNDERSTANDING

Does an object key containing slashes necessarily create real directories?

AYesBNo

Chapter reference: math, worked example & code

This section belongs to the whole chapter, so it is the same on every lesson in it and may cover a different lesson's topic. The code for the experiment you just ran is inside the experiment itself.

Math and notation

A simple retained-payload estimate is

M=w\,s\,d\,r,

where w is writes/day, s payload bytes/write, d retained days and r full-copy count. Add indexes, metadata, compaction headroom and backups separately; replication schemes may store more than exactly r payload copies.

Worked example

Two million records per day at 500 bytes each produce one decimal GB/day of raw payload. Thirty days retained across three full copies require 2,000,000×500×30×3=90,000,000,000 bytes, or 90 decimal GB before other storage. Compressibility and index size are separate measured factors.

Reference code

The following example uses the Python standard library and does not call a remote service.

daily,size,days,copies=2_000_000,500,30,3
assert daily*size == 1_000_000_000
retained=daily*size*days*copies
assert retained == 90_000_000_000
assert retained/1e9 == 90

Python examples are provided for study and local execution; this page does not execute Python.

Common mistakes

NoSQL is not a single consistency or query model. Denormalization creates a maintenance responsibility. Payload estimates must not silently stand in for provisioned disk requirements.