Introduction to Storage Engines
Database Internals Series: Part 1
I am a Journalist-turned-Software Engineer. I love coding and the associated grind of learning every day. A firm believer in social learning, I owe my dev career to all the tech content creators I have learned from. This is my contribution back to the community.
Storage Engines
Databases are modular systems and consist of multiple parts:
Transport layer accepting requests
A query processor to determine the most efficient way to run queries
An execution engine carrying out the operations
A storage engine
A storage engine is a software component in a DBMS Architecture that is responsible for storing, retrieving, and managing data in memory or in disk.
MySQL, a DBMS, has several storage engines - including InnoDB, MyISAM, and RocksDB.
When designing systems, the choice of a database is the single most important decision you will make.
So it is important to invest time earlier in the development cycle to decide on a specific database to build confidence.
The best way to do it is to simulate the operations on multiple databases using test data. Operations that are specific to your use case. This is also the best way to find out how active the community is when encountering an issue.
Do not go by industry benchmarks as they can establish bias due to test conditions that you might never encounter. You want to simulate conditions for your use case.
Understand your use case in great detail to identify...
Schema
Potential number of clients
Possible database size
Read-to-Write ratio
With this input, you will know how easy/difficult it is for you to manage your data.
To pick the database for your use case you should be ready to make tradeoffs.
A simple illustration to help you understand tradeoffs.
