v0.5.3 - Critical bug fix

[ecton]

Fixed

• File operations are now fully persisted to disk to the best ability provided
  by each operating system. @justinj discovered that no fsync() operations
  were happening, and reported the finding. Nebari's TreeFile was using
  File::flush() instead of File::sync_data()/sync_all(). This means that it
  would be possible for an OS-level buffer to not be flushed to disk before
  Nebari reported a successful write.

  Interestingly, this change has no noticable impact on performance on Linux.
  However, on Mac OS, File::sync_data() performs a fcntl with F_FULLFSYNC,
  which has a significant impact on performance. This is the correct behavior,
  however, as without this level of guarantee, sudden power loss could result in
  data loss.

  Many people argue that using F_BARRIERFSYNC is sufficient for most people,
  but Apple's [own documentation][apple-reducing-disk-writes] states this is
  necessary:

  Only use F_FULLFSYNC when your app requires a strong expectation of data
  persistence. Note that F_FULLFSYNC represents a best-effort guarantee that
  iOS writes data to the disk, but data can still be lost in the case of
  sudden power loss.

  For now, the stance of Nebari's authors is that F_FULLFSYNC is the proper
  way to implement true ACID-compliance.

---

This discussion was created from the release v0.5.3 - Critical bug fix.