Introduction to Storage Engines

Database Internals Series: Part 1

Storage Engines

Databases are modular systems and consist of multiple parts:

  • Transport layer accepting requests

  • A query processor to determine the most efficient way to run queries

  • An execution engine carrying out the operations

  • A storage engine

A storage engine is a software component in a DBMS Architecture that is responsible for storing, retrieving, and managing data in memory or in disk.

MySQL, a DBMS, has several storage engines - including InnoDB, MyISAM, and RocksDB.

When designing systems, the choice of a database is the single most important decision you will make.

So it is important to invest time earlier in the development cycle to decide on a specific database to build confidence.

The best way to do it is to simulate the operations on multiple databases using test data. Operations that are specific to your use case. This is also the best way to find out how active the community is when encountering an issue.

Do not go by industry benchmarks as they can establish bias due to test conditions that you might never encounter. You want to simulate conditions for your use case.

Understand your use case in great detail to identify...

  1. Schema

  2. Potential number of clients

  3. Possible database size

  4. Read-to-Write ratio

With this input, you will know how easy/difficult it is for you to manage your data.

To pick the database for your use case you should be ready to make tradeoffs.

A simple illustration to help you understand tradeoffs.

Image

Did you find this article valuable?

Support Zahiruddin Tavargere by becoming a sponsor. Any amount is appreciated!