The Invisible Infrastructure
Every time you search for a product on an e-commerce site, post a message on a social platform, or look up your bank balance, a database query runs. How that database is structured — how data is organised, related, and indexed — determines whether that query takes 10 milliseconds or 3 seconds, whether the answer is correct or stale, and whether the system can handle a thousand users simultaneously or a million. Database design is invisible infrastructure that shapes the experience of virtually every digital product.
Relational Databases: The Workhorse
Most data in most applications is stored in relational databases — systems organised around tables with rows and columns, connected by relationships. The concept of normalisation — organising data to reduce redundancy and improve integrity — is the foundational principle. A well-normalised database stores each piece of information once, in one place, and links it to related information via references rather than repetition.
Why does this matter in practice? Consider a database that stores customer orders. If the customer’s address is stored redundantly in every order record, and the customer moves, updating all those records consistently is complex and error-prone. If the address is stored once in a customer record and referenced from orders, changing it once changes it everywhere. This is what normalisation achieves — and the same principle applies to every relationship in a system.
Indexing and Performance
Database indexes are data structures that speed up lookups at the cost of additional storage and slightly slower writes. Without an index on a column used in queries, the database has to scan every row to find matching records — fine for small tables, catastrophically slow for tables with millions of rows. Adding the right index transforms a query that takes minutes into one that takes milliseconds.
Missing indexes are the most common cause of database performance problems in production applications. They’re easy to miss in development, where tables are small and query times are acceptable, and then suddenly critical when the application is live and data has accumulated. Good database design anticipates which columns will be searched and creates indexes accordingly.
NoSQL: When the Relational Model Doesn’t Fit
Not all data fits neatly into relational tables. Document databases (like MongoDB) store data as nested JSON-like documents, which is natural for hierarchical data that varies in structure. Graph databases (like Neo4j) store data as nodes and relationships, which is natural for highly connected data like social networks. Key-value stores (like Redis) optimise for fast lookup by a single key, natural for caching and session management.
The choice between relational and non-relational databases is driven by the shape of the data and the patterns of access, not by which is generally superior. Most sophisticated applications use multiple database types: a relational database for transactional data, a cache for frequently accessed data, a search index for full-text search, and perhaps a graph database for relationship-heavy queries.
Why Developers (and Product People) Should Care
Database design decisions made early constrain everything that comes later. Adding a feature that requires a new relationship between existing data — a common product requirement — can be trivial or enormously complex depending on how the database was originally designed. A system designed for one write pattern can perform terribly when read patterns change. Migrations of large production databases carry real risk and require careful planning.
For anyone who makes decisions about products and features, the basic questions of database design — how data relates, how it scales, where the performance bottlenecks will be — are worth understanding conceptually even without deep technical expertise. They’re the questions that determine whether a promising feature is a week of work or a major architectural project.
Watch: Related Video
Sources
- Date, C. J. (2003). An Introduction to Database Systems. Addison-Wesley.
- Kleppmann, M. (2017). Designing Data-Intensive Applications. O’Reilly Media.
- Fowler, M. (2012). NoSQL Distilled. Addison-Wesley.