What we call a shard in this topic has many names depending on which software you’re using. It’s called a partition in Kafka, a range in CockroachDB, a region in HBase and TiDB, a vBucket in Couchbase, a vnode in Riak, a token-range in Cassandra, and a tablet in Bigtable, YugabyteDB, and ScyllaDB, to name just a few.
Some databases treat partitions and shards as two distinct concepts. For example, in PostgreSQL, partitioning is a way of splitting a large table into several files that are stored on the same machine (which has several advantages, such as making it very fast to delete an entire partition), whereas sharding splits a dataset across multiple machines. In many other systems, partitioning is just another word for sharding.
While partitioning is quite descriptive, the term sharding is perhaps surprising. According to one theory, the term arose from the online role-playing game Ultima Online, in which a magic crystal was shattered into pieces, and each of the shards refracted a copy of the game world. The term shard thus came to mean one of a set of parallel game servers, and later it was carried over to databases. Another theory is that it was originally an acronym for System for Highly Available Replicated Data—reportedly a 1980s database, details of which are lost to history.
By the way, partitioning has nothing to do with network partitions (netsplits), a type of fault in the network between nodes. We will discuss such faults in the related discussion.
Everything about replication of databases in the related discussion applies equally to replication of shards. Since the choice of sharding scheme is mostly independent of the choice of replication scheme, we will ignore replication in this topic for the sake of simplicity.