Skip to content

Commit 186cf89

Browse files
committed
docs: more architecture guide cleanups
1 parent 686d397 commit 186cf89

File tree

9 files changed

+204
-173
lines changed

9 files changed

+204
-173
lines changed

docs/architecture/server.md

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -12,50 +12,51 @@ For network protocol, the server uses the Bincode encoding that we've discussed
1212
section, sent over a TCP connection. There's no need for any further framing, since Bincode knows
1313
how many bytes to expect for each message depending on the type it's decoding into.
1414

15-
The server does not use async Rust and e.g. [Tokio](https://tokio.rs), instead opting for regular OS
16-
threads. Async Rust can significantly complicate the code, which would obscure the main concepts,
17-
and any efficiency gains would be entirely irrelevant for toyDB.
15+
The server does not use [async Rust](https://rust-lang.github.io/async-book/) and e.g.
16+
[Tokio](https://tokio.rs), instead opting for regular OS threads. Async Rust can significantly
17+
complicate the code, which would obscure the main concepts, and any efficiency gains would be
18+
entirely irrelevant for toyDB.
1819

1920
Internally in the server, messages are passed around between threads using
2021
[Crossbeam channels](https://docs.rs/crossbeam/latest/crossbeam/channel/index.html).
2122

22-
The main server loop `Server::serve` listens for inbound TCP connections on port 9705 for Raft peers
23-
and 9605 for SQL clients, and spawns threads to process them. We'll look at Raft and SQL services
24-
separately.
23+
The main server loop `Server::serve()` listens for inbound TCP connections on port 9705 for Raft
24+
peers and 9605 for SQL clients, and spawns threads to process them. We'll look at Raft and SQL
25+
services separately.
2526

2627
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L66-L110
2728

2829
## Raft Routing
2930

30-
The heart of the server is the Raft processing thread `Server::raft_route`. This is responsible
31-
for periodically ticking the Raft node via `raft::Node::tick`, stepping inbound messages from
32-
Raft peers into the node via `raft::Node::step`, and sending outbound messages to peers.
31+
The heart of the server is the Raft processing thread `Server::raft_route()`. This is responsible
32+
for periodically ticking the Raft node via `raft::Node::tick()`, stepping inbound messages from
33+
Raft peers into the node via `raft::Node::step()`, and sending outbound messages to peers.
3334

3435
It also takes inbound Raft client requests from the `sql::engine::Raft` SQL engine, steps them
35-
into the Raft node via `raft::Node::step`, and passes responses back to the appropriate client
36+
into the Raft node via `raft::Node::step()`, and passes responses back to the appropriate client
3637
as the node emits them.
3738

3839
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L169-L249
3940

40-
When the node starts up, it spawns a `Server::raft_send_peer` thread for each Raft peer to send
41+
When the node starts up, it spawns a `Server::raft_send_peer()` thread for each Raft peer to send
4142
outbound messages to them.
4243

4344
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L84-L91
4445

4546
These threads continually attempt to connect to the peer via TCP, and then read any outbound
46-
`raft::Envelope(raft::Message)` messages from `Server::raft_route` via a channel and writes the
47+
`raft::Envelope(raft::Message)` messages from `Server::raft_route()` via a channel and writes the
4748
messages into the TCP connection using Bincode:
4849

4950
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L146-L167
5051

5152
The server also continually listens for inbound Raft TCP connections from peers in
52-
`Server::raft_accept`:
53+
`Server::raft_accept()`:
5354

5455
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L112-L134
5556

56-
When an inbound connection is accepted, a `Server::raft_receive_peer` thread is spawned that reads
57+
When an inbound connection is accepted, a `Server::raft_receive_peer()` thread is spawned that reads
5758
Bincode-encoded `raft::Envelope(raft::Message)` messages from the TCP connection and sends them to
58-
`Server::raft_route` via a channel.
59+
`Server::raft_route()` via a channel.
5960

6061
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L136-L144
6162

@@ -72,13 +73,14 @@ The primary request type is `Request::Execute` which executes a SQL statement ag
7273
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L312-L337
7374

7475
The server sets up a `sql::engine::Raft` SQL engine, with a Crossbeam channel that's used to send
75-
`raft::Request` Raft client requests to `Server::raft_route` and onwards to the local `raft::Node`.
76-
It then spawns a `Server::sql_accept` thread to listen for inbound SQL client connections:
76+
`raft::Request` Raft client requests to `Server::raft_route()` and onwards to the local
77+
`raft::Node`. It then spawns a `Server::sql_accept()` thread to listen for inbound SQL client
78+
connections:
7779

7880
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L104-L106
7981

8082
When a SQL client connection is accepted, a new client session `sql::execution::Session` is set up
81-
for the client, and we spawn a `Server::sql_session` thread to serve the connection:
83+
for the client, and we spawn a `Server::sql_session()` thread to serve the connection:
8284

8385
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/server.rs#L251-L272
8486

docs/architecture/sql-data.md

Lines changed: 40 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,45 @@
11
# SQL Data Model
22

3-
The SQL data model is toyDB's representation of user data. It is made up of data types and schemas.
3+
The SQL data model represents user data in tables and rows. It is made up of data types and schemas,
4+
in the [`sql::types`](https://github.com/erikgrinaker/toydb/tree/686d3971a253bfc9facc2ba1b0e716cff5c109fb/src/sql/types)
5+
module.
46

57
## Data Types
68

7-
toyDB supports four basic scalar data types as `sql::types::DataType`: booleans, floats, integers,
9+
toyDB supports four basic scalar data types as `sql::types::DataType`: booleans, integers, floats,
810
and strings.
911

1012
https://github.com/erikgrinaker/toydb/blob/b2fe7b76ee634ca6ad31616becabfddb1c03d34b/src/sql/types/value.rs#L15-L27
1113

12-
Concrete values are represented as `sql::types::Value`, using corresponding Rust types. toyDB also
13-
supports SQL `NULL` values, i.e. unknown values, following the rules of
14+
Specific values are represented as `sql::types::Value`, using the corresponding Rust types. toyDB
15+
also supports SQL `NULL` values, i.e. unknown values, following the rules of
1416
[three-valued logic](https://en.wikipedia.org/wiki/Three-valued_logic).
1517

1618
https://github.com/erikgrinaker/toydb/blob/b2fe7b76ee634ca6ad31616becabfddb1c03d34b/src/sql/types/value.rs#L40-L64
1719

18-
The `Value` type provides basic formatting, conversion, and mathematical operations. It also
19-
specifies comparison and ordering semantics, but these are subtly different from the SQL semantics.
20-
For example, in Rust code `Value::Null == Value::Null` yields `true`, while in SQL `NULL = NULL`
21-
yields `NULL`. This mismatch is necessary for the Rust code to properly detect and process `Null`
22-
values, and the desired SQL semantics are implemented higher up in the SQL execution engine (we'll
23-
get back to this later).
20+
The `Value` type provides basic formatting, conversion, and mathematical operations.
21+
22+
https://github.com/erikgrinaker/toydb/blob/686d3971a253bfc9facc2ba1b0e716cff5c109fb/src/sql/types/value.rs#L68-L79
23+
24+
https://github.com/erikgrinaker/toydb/blob/686d3971a253bfc9facc2ba1b0e716cff5c109fb/src/sql/types/value.rs#L164-L370
25+
26+
It also specifies comparison and ordering semantics, but these are subtly different from the SQL
27+
semantics. For example, in Rust code `Value::Null == Value::Null` yields `true`, while in SQL
28+
`NULL = NULL` yields `NULL`. This mismatch is necessary for the Rust code to properly detect and
29+
process `Null` values, and the desired SQL semantics are implemented during expression evaluation
30+
which we'll cover below.
2431

2532
https://github.com/erikgrinaker/toydb/blob/b2fe7b76ee634ca6ad31616becabfddb1c03d34b/src/sql/types/value.rs#L91-L162
2633

27-
During execution, a row of values will be represented as `sql::types::Row`, with multiple rows
28-
emitted as `sql::types::Rows` row iterators:
34+
During execution, a row of values is represented as `sql::types::Row`, with multiple rows emitted
35+
via `sql::types::Rows` row iterators:
2936

3037
https://github.com/erikgrinaker/toydb/blob/b2fe7b76ee634ca6ad31616becabfddb1c03d34b/src/sql/types/value.rs#L378-L388
3138

3239
## Schemas
3340

34-
toyDB schemas support a single object: a table. There's only a single, unnamed database, and no
35-
named indexes, constraints, or other schema objects.
41+
toyDB schemas only support tables. There are no named indexes or constraints, and there's only a
42+
single unnamed database.
3643

3744
Tables are represented by `sql::types::Table`:
3845

@@ -47,42 +54,41 @@ The table name serves as a unique identifier, and can't be changed later. In fac
4754
are entirely static: they can only be created or dropped (there are no schema changes).
4855

4956
Table schemas are stored in the catalog, represented by the `sql::engine::Catalog` trait. We'll
50-
revisit the implementation of this trait in the storage section below.
57+
revisit the implementation of this trait in the SQL storage section.
5158

5259
https://github.com/erikgrinaker/toydb/blob/0839215770e31f1e693d5cccf20a68210deaaa3f/src/sql/engine/engine.rs#L60-L79
5360

54-
Table schemas are validated (e.g. during creation) via the `Table::validate()` method, which
55-
enforces invariants and internal consistency. It uses the catalog to look up information about other
56-
tables, e.g. that foreign key references point to a valid target column.
61+
Table schemas are validated when created via `Table::validate()`, which enforces invariants and
62+
internal consistency. It uses the catalog to look up information about other tables, e.g. that
63+
foreign key references point to a valid target column in a different table.
5764

5865
https://github.com/erikgrinaker/toydb/blob/c2b0f7f1d6cbf6e2cdc09fc0aec7b050e840ec21/src/sql/types/schema.rs#L98-L170
5966

60-
It also has a `Table::validate_row()` method which is used to validate that a given
61-
`sql::types::Row` conforms to the schema (e.g. that the value data types match the column data
62-
types). It uses a `sql::engine::Transaction` to look up other rows in the database, e.g. to check
63-
for primary key conflicts (we'll get back to this below).
67+
Table rows are validated via `Table::validate_row()`, which ensures that a `sql::types::Row`
68+
conforms to the schema (e.g. that value types match the column data types). It uses a
69+
`sql::engine::Transaction` to look up other rows in the database, e.g. to check for primary key
70+
conflicts (we'll get back to this later).
6471

6572
https://github.com/erikgrinaker/toydb/blob/c2b0f7f1d6cbf6e2cdc09fc0aec7b050e840ec21/src/sql/types/schema.rs#L172-L236
6673

6774
## Expressions
6875

6976
During SQL execution, we also have to model _expressions_, such as `1 + 2 * 3`. These are
70-
represented as values and operations on them. They can be nested arbitrarily as a tree to represent
71-
compound operations.
77+
represented as values and operations on them, and can be nested as a tree to represent compound
78+
operations.
7279

7380
https://github.com/erikgrinaker/toydb/blob/9419bcf6aededf0e20b4e7485e2a5fa3e975d79f/src/sql/types/expression.rs#L11-L64
7481

7582

76-
For example:
83+
For example, the expression `1 + 2 * 3` (taking [precedence](https://en.wikipedia.org/wiki/Order_of_operations)
84+
into account) is represented as:
7785

7886
```rust
79-
// 1 + 2 * 3 is represented as:
80-
//
81-
// +
82-
// / \
83-
// 1 *
84-
// / \
85-
/// 2 3
87+
// +
88+
// / \
89+
// 1 *
90+
// / \
91+
/// 2 3
8692
Expression::Add(
8793
Expression::Constant(Value::Integer(1)),
8894
Expression::Multiply(
@@ -97,8 +103,8 @@ An `Expression` can contain two kinds of values: constant values as
97103
references. The latter will fetch a `sql::types::Value` from a `sql::types::Row` at the specified
98104
index during evaluation.
99105

100-
We'll see later how the SQL parser and planner transforms text expressions like `1 + 2 * 3` into
101-
this `Expression` form, and how it resolves column names to row indexes -- e.g. `price * 0.25` to
106+
We'll see later how the SQL parser and planner transforms text expression like `1 + 2 * 3` into an
107+
`Expression`, and how it resolves column names to row indexes like `price * 0.25` to
102108
`row[3] * 0.25`.
103109

104110
Expressions are evaluated recursively via `Expression::evalute()`, given a `sql::types::Row` with

0 commit comments

Comments
 (0)