Ship-Now Security Mvp (Lrtq) And Its Lifecycle Limitations
Sources: 1 • Confidence: Medium • Updated: 2026-04-11 19:45
Key takeaways
- Oxide shipped racks using a minimum-viable encryption scheme called Low-Rent Trust Quorum (LRTQ) that generates a rack secret at setup time and distributes key shares over plain TCP, lacking remote attestation, key rotation, and sled expungement capabilities.
- TrustQuorum changes were merged and tested in both simulated multi-sled environments (A4x2 via Propolis) and on racklets, including upgrades from LRTQ to TrustQuorum and sled add/remove scenarios.
- Oxide positions itself as building a secure product rather than a dedicated security product, and interest in these security mechanisms has grown among mainstream enterprise customers.
- Oxide racks use U.2 drives for encrypted ZFS datasets and M.2 drives for sled-local metadata and installable software.
- Oxide’s rack control plane requires a mechanism to establish mutual trust among sleds within a rack.
Sections
Ship-Now Security Mvp (Lrtq) And Its Lifecycle Limitations
- Oxide shipped racks using a minimum-viable encryption scheme called Low-Rent Trust Quorum (LRTQ) that generates a rack secret at setup time and distributes key shares over plain TCP, lacking remote attestation, key rotation, and sled expungement capabilities.
- In LRTQ, the initial key-share distribution occurs over an internal rack bootstrap network that is not exposed outside the rack, and intercepting shares would require an active physical or network attack not currently in scope.
- To enable adding sleds later under a fixed rack secret, LRTQ pre-generated up to 255 secret shares and stored extra shares encrypted under the rack secret across the initial sled set for later distribution.
- The TrustQuorum design documented in RFD-238 was rewritten from scratch at least three times as it evolved.
- Despite its limitations, LRTQ provides encryption at rest intended to resist casual physical attacks.
- After racks shipped, TrustQuorum work paused because customers initially prioritized other software needs, and later resumed due to a customer requirement and accumulated LRTQ technical debt.
Real Trustquorum: Fault-Tolerant Rotation, Self-Healing Shares, And Rolling Upgrades
- TrustQuorum changes were merged and tested in both simulated multi-sled environments (A4x2 via Propolis) and on racklets, including upgrades from LRTQ to TrustQuorum and sled add/remove scenarios.
- Oxide’s secret sharing implementation can reconstruct any participant’s share value at its x-coordinate, allowing a sled that lost its share to reconstitute it from other shares.
- Real TrustQuorum keeps a trusted-dealer initial distribution but uses secure channels and is designed to tolerate asynchrony and partial failures during both initial distribution and key rotation.
- TrustQuorum was implemented as a Sans-I/O side-effect-free state machine and was described as having a TLA+ spec and property tests, enabling fuzz/property testing and a step-through simulator/debugger.
- Real TrustQuorum supports asynchronous key rotation where a sled that was offline can later learn its latest share and decrypt prior epoch secrets to rotate itself forward without requiring global atomic rotation.
- A rack upgrade requires mixed-mode operation where some sleds run the new TrustQuorum while communicating with not-yet-upgraded sleds using LRTQ.
Engineering Execution Levers: Concentrated Focus And Llm-Assisted Onboarding/Navigation
- Oxide positions itself as building a secure product rather than a dedicated security product, and interest in these security mechanisms has grown among mainstream enterprise customers.
- During reconfiguration, Oxide currently mitigates a sled going offline by rebooting that sled.
- Finch used Claude to help trace multi-layer abstractions and identify which source files to inspect while building a mental model of the codebase, including using an editor-invocation workflow to tour call sites.
- Bryan asserted that well-documented code, strong typing, and using Rust make it easier for tools to help engineers ramp up on a new source base.
- Andrew was able to focus roughly 90–95% of his time on TrustQuorum once it became a priority again, and this was described as helpful in getting the work merged.
Operational Driver And Threat-Model Boundary For Unattended Encrypted Racks
- Oxide racks use U.2 drives for encrypted ZFS datasets and M.2 drives for sled-local metadata and installable software.
- TrustQuorum is designed to use Shamir secret sharing so sled-local key shares can be exchanged among sleds to reconstruct a rack secret used to derive disk encryption keys.
- A primary driver for TrustQuorum is enabling encryption at rest while allowing a rack to reboot autonomously after outages without an on-site operator entering a password.
- Oxide’s stated threat model aims to prevent data exposure from casual physical access such as stealing a few drives or sleds, but it does not protect against theft of an entire rack.
Rack-Scale Mutual Trust Anchored In Hardware Identity
- Oxide’s rack control plane requires a mechanism to establish mutual trust among sleds within a rack.
- Each sled includes a root-of-trust chip intended to store non-extractable secrets and provide a unique device identity via a physically unclonable function (PUF).
- During manufacturing, the root of trust generates an RSA-4096 key pair, exports a CSR to be signed by an Oxide intermediate service, and stores the resulting certificate binding the baseboard serial number to the public key while the private key never leaves the chip.
Watchlist
- A remaining security gap is that secret shares stored on sled M.2 drives are currently unencrypted, allowing reconstruction of the rack secret if an attacker steals all the M.2 drives, and a planned mitigation is to require the root of trust to boot and decrypt the assigned share.
Unknowns
- What are the concrete threshold parameters (number of shares, quorum threshold) used for reconstructing the rack secret in deployed configurations, and do they vary by rack size?
- What is the exact attestation gate condition used before a sled returns its share in Real TrustQuorum (which measurements, which acceptable set, and how policy updates are handled)?
- Is the claim that M.2-stored shares are unencrypted still true in the currently shipped product version, and if so, is share encryption on the root of trust implemented yet?
- What are the quantitative performance and time impacts of key rotation (normal case) and of the fallback trial-decrypt recovery (worst case), and how do these scale with number of rotations?
- How frequently does reconfiguration trigger sled reboots in real deployments, and what is the rack-level availability impact?