Stronghold is an open-source software library that was originally built to protect IOTA Seeds, but can be used to protect any digital secret.
It is a secure database for working with cryptography, which ensures that secrets (like private keys) are never revealed - but can be used according to best practices.
It provides its own peer-to-peer communication layer, so that different apps can securely communicate using the state-of-the-art Noise Protocol over libp2p.
In April of 2021, F-Secure performed a security assessment of the core crates of IOTA Stronghold and found nothing of concern. This is not an explicit declaration of fitness or freedom of error, but it is an indicator of the high quality of the code. You may review the audit here.
If you want to get involved in discussions about this library, or you're looking for support, go to the #stronghold-discussion channel on Discord.
This documentation has six sections.
- The Overview: detailed overview of the project
- Structure: explains the layout of the individual crates and systems
- The Specification: detailed explanation of requirements and functionality
- Retrospective: a look at the evolution of this project
- Contribute: how you can participate in the Stronghold software development
- Get in touch: join the community and become part of the X-Team
We maintain a bill of materials for the upstream libraries that Stronghold consumes. You can download the latest version of that here:
We will be adding video and textual tutorials for introducing the concepts behind Stronghold.
We will be adding a number of specific How To examples that will cover common use cases, like integrations, backups etc.
IOTA Stronghold is a secure software implementation with the sole purpose of isolating digital secrets from exposure to hackers and accidental leaks. It uses encrypted snapshots that can be easily backed up and securely shared between devices. Written in stable rust, it has strong guarantees of memory safety and process integrity.
There are four main components of Stronghold:
- Client: The high-level interface to Stronghold (prefers Riker, functional integration also available)
- Engine: Combines a persistence store (Snapshot) with an in-memory state interface (Vault) and a key:value read/write (Store).
- Runtime: Is a process fork with limited permissions within which cryptographic operations take place.
- Communication: Enables Strongholds in different processes or on different devices to communicate with each other securely.
Read more about the Alpha Release.
Read more about the Beta Release.
In this section we describe the individual crates and components of Stronghold:
This is the official client layer of Stronghold. It provides a Riker actor model system for easy Interface as well as functional passthrough to Stronghold's internal actor system for integrators not using Riker.
init_stronghold_system: Initializes a new instance of the Stronghold system. Sets up the first client actor. Accepts a
ActorSystem, the first
StrongholdFlagswhich pertain to the first actor.
spawn_stronghold_actor: Spawns a new set of actors for the Stronghold system. Accepts the
Vec<u8>and the options:
switch_actor_target: Switches the actor target to another actor in the system specified by the
write_to_vault: Writes data into the Stronghold. Uses the current target actor as the client and writes to the specified location of
Locationtype. The payload must be specified as a
RecordHintcan be provided. Also accepts
VaultFlagsfor when a new Vault is created.
write_to_store: Writes data into an insecure cache. This method, accepts a
Vec<u8>and an optional
Duration. The lifetime allows the data to be deleted after the specified duration has passed. If not lifetime is specified, the data will persist until it is manually deleted or over-written. Each store is mapped to a client.
read_from_store: Reads from an insecure cache. This method, accepts a
Locationand returns the payload in the form of a
Vec<u8>. If the location does not exist, an empty vector will be returned along with an error
delete_from_store- delete data from an insecure cache. This method, accepts a
Locationand returns a
delete_data: Revokes the data from the specified location of type
Location. Revoked data is not readable and can be removed from a vault with a call to
garbage_collect. if the
should_gcflag is set to
true, this call with automatically cleanup the revoke. Otherwise, the data is just marked as revoked.
garbage_collect: Garbage collects any revokes in a Vault based on the given vault_path and the current target actor.
list_hints_and_ids: Returns a list of the available
RecordHintvalues in a vault by the given
runtime_exec: Executes a runtime command given a
Procedure. Returns a
ProcResultbased off of the
record_exists: Checks whether a record exists in the client based off of the given
vault_exists: Checks whether a vault exists in the client by
read_snapshot: Reads data from a given snapshot file. Can only read the data for a single
client_pathat a time. If the actor uses a new
client_paththe former client path may be passed into the function call to read the data into the new actor. A filename and filepath can be specified, if they aren't provided, the path defaults to
$HOME/.stronghold/snapshots/and the filename defaults to
backup.stronghold. Also requires keydata to unlock the snapshot and the keydata must implement and use
write_all_to_snapshot: Writes the entire state of the
Strongholdinto a snapshot. All Actors and their associated data is written into the specified snapshot. Requires keydata to encrypt the snapshot. The Keydata should implement and use Zeroize. If a path and filename are not provided, uses the default path
$HOME/.stronghold/snapshots/and the default filename
kill_stronghold: Used to kill a stronghold actor or clear the cache of that actor. Accepts the
client_path, and a boolean for whether or not to kill the actor. If
trueboth the internal actor and the client actor are killed. Otherwise, the cache is cleared from the client and internal actor.
SLIP10Generate: Generate a raw SLIP10 seed of the specified size (in bytes, defaults to 64 bytes/512 bits) and store it in the
SLIP10Derive: Derive a Slip10 child key from a seed or parent key. Store the output in a specified
Locationand return the corresponding
BIP39Recover: Use a BIP39 mnemonic sentence (optionally protected by a passphrase) to create or recover a BIP39 seed and store it in the output
BIP39Generate: Generate a BIP39 seed and its corresponding mnemonic sentence (optionally protected by a passphrase) and store them in the output
BIP39MnemonicSentence: Read a BIP39 seed and its corresponding mnemonic sentence (optionally protected by a passphrase) and store them in the output
Ed25519PublicKey: Derive an Ed25519 public key from the corresponding private key stored at the specified
Ed25519Sign: Use the specified Ed25519 compatible key to sign the given message. Compatible keys are any record that contain the desired key material in the first 32 bytes, in particular SLIP10 keys are compatible.
SLIP10Generate: Returns a
StatusMessageindicating the result of the request.
SLIP10Derive: Returns a
ChainCodeinside of it.
BIP39Recover: Returns a
StatusMessageindicating the result of the request. .
BIP39Generate: Returns a
StatusMessageindicating the result of the request.
BIP39MnemonicSentence: Returns the mnemonic sentence for the corresponding seed.
Ed25519PublicKey: Returns an Ed25519 public key inside of a
Ed25519Sign: Returns an Ed25519 signature inside of a
Engine is the collection of low-level crates with which application architects can build higher-level implementations of Strongholds for a variety of purposes. It is platform agnostic, in that it should run anywhere a Rust Compiler will work.
It is composed of 4 primary crates:
This crate defines and implements the encrypted offline storage format used by the Stronghold ecosystem.
The format has a header with version and magic bytes to appease applications wishing to provide file-type detection.
The data stored within a snapshot is considered opaque and uses 256 bit keys. It provides recommended ways to derive the snapshot encryption key from a user provided password. The format also allows using an authenticated data bytestring to further protect the offline snapshot files (one might consider using a secondary user password strengthened by an HSM).
The current version of the format is using X25519 together with an ephemeral key to derive a shared key for the symmetric XChaCha20 cipher and uses the Poly1305 message authentication algorithm. Future versions, when the demands for larger snapshot sizes and/or random access is desired, might consider encrypting smaller chunks (B-trees?) or similar using per chunk derived ephemeral keys.
Vault is an in-memory database specification which is designed to work without a central server. Only the user which holds the associated id and key may modify the data in a vault. Another owner can take control over the data if they know the id and the key.
Data can be added to the chain via a [DataTransaction]. The [DataTransaction] is associated to the chain through the owner’s ID and it contains its own randomly generated ID.
Records may also be revoked from the Vault through a [RevocationTransaction]. A [RevocationTransaction] is created and it references the id of a existing [DataTransaction]. The RevocationTransaction stages the associated record for deletion. The record is deleted when the DbView preforms a garbage collection and the [RevocationTransaction] is deleted along with it.
This crate contains a key/value cache for the Stronghold Engine. Data is stored in key-value pairs and an expiration timestamp can be set. The data is stored in a structured format and can be quickly retrieved at will. Along with the Vault, this crate is used to store general unencrypted data.
This crate aims to provide utilities for performing computations as securely as possible with respect to the underlying operating system.
Among the considered concepts:
- guarded memory allocations
- assists with read/write protecting sensitive data
- zeroes the allocated memory when handing it back to the operating system
- uses canary and garbage values to protect the memory pages.
- leverages NACL
libsodiumfor use on all supported platforms.
It's common to restrict the amount of memory that can a non-privileged user can lock into main memory (i.e. forbidden to be swapped out to disk).
The following limit is sufficient to make the tests pass:
ulimit -l $((1024*1024))
But it's quite likely that that command will fail because the system defaults are sometimes very strict. On Arch the file that manages those limits is limit.conf and the following addition raises the limit to sufficiently run the tests:
username hard memlock 1048576
Note also that the tests in the crate allocates a lot more memory than an application using these runtime utilities are expected to allocate: by the principle of least privilege only the necessary sensitive/cryptographic operations should be performed in the most restricted sandbox.
- encrypt/authenticate locked memory with a fast algorithm such as AES.
The primary components are:
- Guarded - A guarded type for protecting fixed-length secrets allocated on the heap.
- GuardedVec - A guarded type for protecting variable-length secrets allocated on the heap.
- Secret - A Type for guarding secrets allocated to the stack.
- ZeroingAlloc - A Zeroing Allocator which wraps the standard memory allocator. This allocator zeroes out memory when it is dropped.
This library enables Strongholds on different devices and in different networks to communicate with each other. The main basis for its functionality is the rust-libp2p library, which is a system of protocols, specifications and libraries that enable the development of peer-to-peer network applications (https://libp2p.io/).
Stronghold-communication implements the
P2PNetworkBehaviour for sending messages and reacting upon the outcome of the operation.
It combines multiple protocols of Libp2p:
- Multiplexing following the Yamux specification
- Noise: Encryption of the communication using the Noise protocol with XX-Handshake
- Multicast DNS: Enable Peer Discovery in a local network
- Identify Protocol: Receive identifying information like the
PeerIdand listening addresses when connecting to a new peer.
- Request-Response Protocol: Allows sending direct request/response messages between Peers; it expects a response for each request
Upon creating a new instance, a transport is created and upgraded, and combined with the P2PNetworkBehaviour into a ExpandedSwarm. This Swarm is returned to the caller and serves as entry-point for all communication to other peers. Additional to the Libp2p methods of the
ExpandedSwarm, it enables sending outbound messages, and manages the known peers. Incoming
P2PEvents can be handled by polling from the swarm, e.g. via the
Communication Actor is using the Riker Framwork to implement the actor pattern.
When creating a new
Communication Actor, the actor creates a
P2PNetworkBehaviour and continuously polls for events,
incoming requests are sent to the client actor that has to be provided in the
All swarm interaction, and configuration of the
Communication Actor is accomplished by sending the appropriate
CommunicationRequest to it, for each
CommunicationResults is returned to the sender, this also allows using the ask pattern.
The communication actor implements a firewall that checks the permission of each outgoing and incoming requests and drops them if the necessary permission has not been set. The required
ToPermissionVariants trait for messages can be derived with the communication-macros, this allows in case of enum Request types to accept specific variants while rejecting others.
Proc macros for Stronghold.
Common utils for stronghold libraries.
The Stronghold team endeavors to ship several production ready applications, which are more than classical "examples":
- commandline: Interact with a stronghold snapshot from the commandline. Mostly for debugging and utility, not recommended for daily use
- Desktop App - WIP: A Tauri based application used for validation of
These products are documented in their respective code repositories.
This section contains a description of the scope of the project, as well as the working documents that detail the engineering and requirements specifications.
title: Stronghold stub: stronghold document: SCOPE version: 0000 maintainer: Daniel Thompson-Yvetot <firstname.lastname@example.org> contributors: [tensorprogramming <email@example.com>, Daniel Thompson-Yvetot <firstname.lastname@example.org>] sponsors: [Navin Ramachandran <email@example.com>] licenses: ["Apache-2", "CC-BY-INTL-3.0"] updated: 2021-Apr-27
All code is licensed under the Apache-2 license, all text and images are licensed under the CC-BY-INTL-3.0 license.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.
These documents MUST use incremental numbering. New documents always start at 0000. Subsequent revisions to each RFI, RFP and RFC will have their number increased by one.
Software releases will follow strict semantic versioning.
All documents in this specification are understood to flow from this document and be bound to its licenses and language as described above.
Stronghold is a secure software implementation (often used in conjunction with - or existing purely on - specialist hardware) with the sole purpose of isolating the seed, private keys, personally identifiable information (PII) and policy records from exposure to the genuinely hostile environment of user devices. It uses snapshotting and internal mechanisms for threshold signature schemes that MAY be distributed across devices.
It is based on a suite of low-level libraries collectively called "engine" that provide tooling and algorithms to build secure systems in Rust in a way that can be embedded and deployed to cross platform devices. Engine is a collection of libraries which deal with the obfuscation and sharing of secret values both mutable and immutable between devices.
The primary task is to isolate the activity of “privileged” functions from other parts of the software stack. For example, a primary goal is to create a software enclave where private keys are used to sign messages without revealing those keys to other functions.
Additionally, a system for enabling Stronghold-based systems to securely communicate with each other shall be created such that devices on different networks can collaborate cryptographically.
Coming on the heels of the Trinity attack, it became clear that a new method for securing secrets needed to be manufactured and made available to the pantheon of IOTA Products.
- Integration with the Wallet, Nodes, Identity, Access and developer toolchains strengthens IOTA’s internal position.
- Publishing the low-level libraries will enable third-parties interested in secure rust-based systems will expand the visibility of IOTA in the security community.
- Creating and maintaining open source software, and providing educational opportunities is the core mission of the IOTA Stiftung.
- Using off-the-shelf libraries has always been a trade-off. Writing the library in Rust using as few external dependencies as possible is a good baseline. Designing the library such that cryptographic primitives can be replaced will make the library viable in the long-term.
- Enhance the security posture of critical IOTA Products
- Enhance the perception of the IF as a “security-focussed” organisation.
- Create new avenues for partnership and 3rd party implementation.
- Writing in rust gives a number of memory-safety benefits
- Fuzzing from the beginning improves confidence of software fitness
- Providing reference implementation gives assurance to integrators
- Rust is a single source of code truth is a practice that the IF is interested in.
- Helping developers new to IOTA use a secure system from the beginning is a good way to train.
- Learning about Fuzzing is useful for all developers.
A number of IOTA foundation stakeholders have been involved in the design process, ranging from Engineering to Product and developer outreach.
Stronghold itself has several core components:
There are 5 low level libraries:
- crypto (swappable crypto implementation, chacha20poly1305 & salsa20)
- primitives (shared structs and traits)
- random (secure implementation of random)
- snapshot (stateful storage management)
- vault (interaction with storage)
This work has been undertaken by an external developer in the context of an EDF grant using prior work from Daniel Thompson-Yvetot and Tensor at their security boutique "IONARY".
The high level library integrates engine.rs and iota.rs to a fully fledged secret storage and enclave based system for operations in the context of the IOTA Protocol.
Its primary purpose is to serve as the operational enclave for several IOTA Products:
This work will be undertaken in house by IOTA developers.
The Actor Model layer is a thin wrapper for message parsing and message sending that is built for interaction with the wallet and any other projects that deem the actor model suitable to their needs.
This work will be undertaken in house by IOTA developers.
There is a massive amount of prior art.
The official IF wallet, available on Android, iOS, MacOs, Windows, Linux. It uses React as a front-end language, Electron as a backend for Desktop platforms and React native as the backend for Mobile devices.
A hardware token storage system that uses two STM chips (ST31 for secure storage [presumably]) and the STM32 for actual processing.
“The CryptoCore is IOTA hardware designed for applications that need fast, dedicated proof of work and a secure memory. The device consists of an IOTA CryptoCore FPGA (ICCFPGA) module and a development board that doubles as a Raspberry Pi HAT, making it perfect for standalone applications and/or quick prototyping.“
WeChat is a chat and payment application very popular in the Chinese market. MiniPrograms run inside of the scope of the main application.
JSBox is an iOS centric system for running JS in an iOS application developed primarily for the Chinese market. It is an application on the iOS Store geared toward developers:
- Many advanced development tools: lint, prettier, diff viewer and database viewer...
- A desktop extension to write code extremely fast and comfortable
- Almost all the cool tech in iOS: Siri/Shortcuts, Today Widget, Action Extension, 3D Touch, Home Screen Shortcut...
- A lot of awesome examples for beginner”
The Kamikaze pattern uses a system of event listeners and emitters in Rust and in Webview that communicate with each other using throwaway handles. Considered by the Tauri team to be the most secure pattern possible.
Open source security chip from Google available in the Pixel 3 (and other security dongles), which enables secure booting of mobile devices and provides a “secure” keystore for Third Party apps. Please review CVE-2019-9465 for a somewhat troubling “non-disclosure”. OpenTitan is the “community” project for an open hardware “Root of Trust”.
Rust based security firmware for Nordic from Google. “Under the hood, OpenSK is written in Rust and runs on TockOS to provide better isolation and cleaner OS abstractions in support of security. Rust’s strong memory safety and zero-cost abstractions makes the code less vulnerable to logical attacks.”
“When you store a private key in the Secure Enclave, you never actually handle the key, making it difficult for the key to become compromised. Instead, you instruct the Secure Enclave to create the key, securely store it, and perform operations with it. You receive only the output of these operations, such as encrypted data or a cryptographic signature verification outcome.”
The official MacOS Application verifier and Anti-Malware service verifies integrity and developer signatures, and manages the “quarantine” flag on downloaded files.
“The Secure Element 2.0 generates a unique private key that cannot be rewritten over the lifetime of the chip. The stored private key can only be used within computations of the microchip itself. It employs a highly-secure hardware-based cryptographic key storage and cryptographic countermeasures which eliminate potential backdoors linked to software weaknesses. Thus, ensuring that the key cannot be exfiltrated. The decryption of data is only run on the chip itself and happens “off-the-bus”. Thereby, leaving an absolutely minimised attack surface for attackers trying to compromise the private key.”
this does not address concerns with the onboard RNG, the Secure Element in use is EOL.
“The Cryptosteel Capsule is the premier backup tool for autonomous offline storage of valuable data without any third-party involvement. The solid metal device, designed to survive extreme conditions, works under nearly all circumstances.”
These audio plug-in systems use digital signal processing, come with a back-end, a front-end, presets and interface with a larger system. They generally require a host. Of special interest is the architectural design pattern of LV2:
“The host program loads the plugin, and calls some initialization functions. The host can provide a list of LV2_Extension that it supports when it initializes the plugin, so the capabilities of the host are known to the plugin when it is started. Similarly, the plugin uses Turtle metadata to provide a list of capabilities to the host, so the host can accommodate those. This capability concept is very powerful, but also difficult to understand at first. ‘Atom’ messages are sent between plugin event ports, and this mechanism is used to transfer MIDI, OSC and Patch information between plugin instances.”
Here is an example of a VST Builder written in rust. Here is a solution for building a dylib for MacOS, and the accompanying “base plugin”.
Trusted Execution Environments can be considered to be a “secure zone” of a processing unit. Generally more powerful than a Secure Element, their architecture isolates processes such as boot and analyzing application integrity. Obviously there are standards and any number of vendor implementations.
Here is a collection of research about Binary Obfuscation approaches: Sean Taylor presentation at DefCon Seminal Paper on Functional Obfuscation (see Multilinear Jigsaw) Runtime Encryption (hyperion) https://nullsecurity.net/tools/cryptography.html http://phrack.org/issues/63/13.html <- Excellent Writeup This idea of finger printing the system is especially appealing. When adding more than one device with "entangled" setups; deriving multiple fingerprints or a fingerprint that runs on multiple devices might be possible.
Links from Tensor:
Smart contract wallet
See Section 6 on Identity Recovery https://blog.hashd.in/hashd-in-draft0/
Fireblocks is a multisig system. Dom has more information about them. https://www.fireblocks.com/
“Using a secure decentralized network made up of trusted people, Vault12 gives cryptocurrency owners the peace of mind that their crypto assets remain backed up, cryptographically secure but accessible regardless of threats such as attacks on centralized servers and digital impersonation.” https://vault12.com/
MesaLink implements OpenSSL C APIs with Rust FFI. If you call an exported C FFI function from Rust, it’s no different to calling that same exported C function from a different C or C++ library. Unlike Java/Go, there is zero overhead. https://mesalink.io/faq/
minisign in wasm from Rust https://wapm.io/package/jedisct1/rsign2
https://guardtime.com/mida/ https://www.riddleandcode.com/secure-element https://github.com/RiddleAndCode/secure-element-sdk/wiki/Raspberrypi-HSM https://safenetwork.tech/faq/#what-is-self-authentication https://keycard.tech/
- Having a CLI
- Having a service that can run as a daemon
- Using a remote stronghold
title: Stronghold stub: stronghold document: Engineering Specification version: 0000 maintainer: Daniel Thompson-Yvetot <firstname.lastname@example.org> contributors: [Dave de Fijter <email@example.com>, tensorprogramming <firstname.lastname@example.org>, Daniel Thompson-Yvetot <email@example.com>, Marcelo Bianchi <firstname.lastname@example.org>] sponsors: [Navin Ramachandran <email@example.com>] licenses: ["CC-BY-INTL-3.0"] updated: 2021-Apr-27
This document introduces the High-Level Specification of the Stronghold.
A Stronghold is composed of several interacting systems at a low level:
- Snapshot - box-encrypted file-based persistence layer
- Vault - a write and use protected, path-based system for storing and using secrets like private keys
- Store - a read/write key:value storage system for dynamic data
- Cache - an in-memory abstraction for vault and store
- Runtime - memory protection system for secrets
- Communication - libp2p based system for communication between strongholds
At the high level, Stronghold provides an official client for interfacing with a Stronghold snapshot and its records.
This document will detail the development of the Stronghold Engine library for IOTA's Stronghold project. It will briefly touch upon the different revisions of this project and the lessons that were learned from each revision. It will also discuss some of the rationale with regards to the implementation decisions that were made along the way. This document is meant to be a high level overview but it will contain some lower level explanations where appropriate.
Development History and Breakdown:
Stronghold Engine originally started its life as a full featured security platform. The original impetus for building the software involved the idea of a Virtual Machine/Runtime which would allow a user to store data securely. The entire state of the VM could be offloaded into a Snapshot/Image file a le smalltalk. This implementation was meant to contain a few other features:
- P2P networking layer
- Secret sharing protocols
- ASN1/X509 libraries
- hybrid logical clocks
- homomorphic cryptography
- A Cryptographic Primitives DSL (Domain Specific Language)
- CRDTs (conflict replicated data types)
The Elixir programming language was initially picked because it was a natural choice for these concepts. Elixir's macro system would allow the DSL to be as flexible and simple as needed to be. Also, OTP contains native libraries for dealing with keys and other cryptographic systems. Elixir’s actor model and the existing Erlang Virtual Machine (BEAM) could be leveraged in such a way that it could stand in for the virtual machine. Elixir also has an intuitive means of communicating with lower level languages such as Rust, C, and C++ allowing modules to be written in other languages.
Unfortunately, this revision had to be scrapped because it would have been difficult to run the daemon on mobile devices. Mobile applications are less available compared to desktop or web apps due to restrictions from their OSes. Also, Elixir has little support for Android and almost no support for iOS. Solutions such as Lumen and JInterface were considered but a choice was made to scrap the project due to its development costs.
The project went back to the drawing board and Rust was chosen as the primary language for the new revision. A prototype was quickly built out as a simple secret storage system. The system contained few of the original ideas and it offloaded the data into JSON format before encrypting it with OpenSSL. The main purpose of this build was to audit the potential of Rust in this domain. Rust was found to be well suited to the use case of this system. While pieces of the language would have made a couple of the original ideas more difficult to implement, Rust did open the door to other features which would have been harder to build with Elixir.
This second build transformed into a memory database system. This storage layer was made to be secure, transactional, and ACID based with a deduplicated and verifiable data storage memory caching system. Features included:
- DEFLATE, LZ4 and LXMA compression
- XChaCha20-Poly1305 encryption via sodiumoxide
- ZPAQ chunking for Data Deduplication
- AEAD checksum metadata system
Data could be stored in multiple formats:
- binary blobs
- persistent maps
- addressable hashing buckets
- content versioned objects
- virtual file system
Ultimately there were some problems with this revision; it was very opinionated, dependency heavy, and it ran like a full blown database solution rather than a security platform.
Background on the Final Revision:
A couple important lessons were learned from building the three different revisions:
- The storage system didn’t need to be complicated, it just needed to be secure.
- It would be better if the abstractions were not opinionated and were open to extension.
- The system should include a small dependency footprint for IoT and Embedded support.
- There was no reason to reinvent the wheel as many of the features could be implemented via existing libraries and tools.
Stronghold Engine needed to be small, extensible, and secure if it was going to fit the use case that IOTA wanted and doing things this way meant that many of the features could be generalized into interfaces. Rather than a full scale platform, Stronghold Engine would be better suited as a set of modular libraries. A final revision was mapped out as a library that was split into multiple crates:
- Primitives Crate
- Random Crate
- Crypto Crate
- Vault Crate
- Snapshot Crate
The core principle behind the primitives crate hinged on implementing a bunch of traits (interfaces) which could be used to define cryptographic primitives. Each primitive contains an info data structure for describing the constraints of the algorithm and at least one trait. These primitives range from Random Number Generators to Cipher Algorithms, Hashing Algorithms and Key Derivation Functions. In this way, a developer should be able to slot in a bit of logic and have it work with the rest of the library.
The random crate is exactly as it sounds; it uses the RNG (random number generator) traits defined in the primitives crate to implement logic for a secure random number generator. A little bit of C code was used when creating this crate because all of the major platforms already have battle tested RNG libraries. This C code is bridged with Rust using CC, a
build.rs file and Rust’s FFI (foreign function interface). Thus far, random contains logic for Windows, MacOS, iOS, Linux, and a cavalcade of BSD flavors.
The Crypto crate contains five encryption algorithms:
Poly1305 and ChaCha20 were defined first which gave way to the other three variations. The internal rules were defined using Rust macros so that they would be composable. Each of these algorithms also implements some of the traits from the primitives crate which makes them extremely easy to swap out and change should the need arise.
A fuzz client was created to match the results of the library’s XChaCha20-Poly1305 and ChaCha20-Poly1305 algorithms to libsodium’s counterparts. The fuzzer has been run with up to ten billion inputs and there hasn’t been any reported variance between the implementations. XChaCha20-Poly1305 and ChaCha20-Poly1305 were used because they also verify the other algorithms indirectly.
The Vault crate contains logic and abstractions for the storage layer of this system. Importantly, the storage layer doesn’t actually define a standard shape for storing the data, instead it defines a method of reading, writing and viewing the data and the system may use any in-memory data collection type. For instance, the Proof of Concept Command Line Tool uses a hashmap wrapped in a RwLock and an Arc as its memory based data storage and the data itself is cached as bytes in that hashmap. The secure data should not be saved in any kind of persistent database; instead persistence is achieved through snapshots as detailed below.
Vault defines a format of ordered chains where in each Record contains an ID, a Transaction, some metadata, a counter and the sealed data. Each of these chains starts out with a single Initial Transaction type that contains no data aside from the owner's ID. Every proceeding transaction must be a direct descendant of this transaction for it to be valid. Also, the counter is incremented every time an event occurs on the data. In this way, the system can determine which piece of data is the latest version while still maintaining a history of the data’s state over time.
Because the data is versioned, a chain should ideally maintain data that is related. For example, if a key is placed into the first Data Transaction of a chain, the proceeding transactions should be metadata or changes to the key. Revocation Transactions can also be created to revoke a transaction. In this way, the vault can stage a proposed deletion for some transaction before the data is deleted. When a garbage collection is preformed, the Revocation transaction and the corresponding Data Transaction are removed from the vault.
The data in the Vault can be encrypted using either symmetric encryption or asymmetric encryption. With symmetric encryption, a key is assigned to each chain and that key is needed to unlock the data. With asymmetric encryption, the key can be defined as a private key and each of the transaction’s IDs could be a public key. A secure random nonce is generated and the data is sealed using the key and the nonce. The nonce is then concatenated to the sealed bytes where it is stored in the data structure. Also, the data in each transaction is a non-descript vector of bytes. As a result of this, it is entirely possible to a complex data structure into the Transaction so long as it can be converted to a binary format.
A Base64 encoder/decoder was also created. This base64 encoder uses a url/file safe character set. Information regarding this character set can be found in RFC 4648 from the internet society. As a small side note, if an ID contains a
- character it can cause issues for the CLI. Wrapping the ID in quotes should resolve this issue though.
The Vault crate includes a fuzz client. The main purpose of this fuzzer is to test the crate and see how it holds up to random inputs and random transactions. The Fuzzer generates a key and then creates a specified amount of clients. Each client is given a unique random ID along with its own data chain. The clients perform random transactions upon their chains based on a value generated by the random number generator. Also a machine object has the ability to randomly take ownership of a foreign chain at any time. Two global storage hashmaps are created and after a specified amount of cycles, the fuzz client checks to see if they are still consistent. This fuzz client was tested for a stretch of 2 days without issue.
The final major crate of this library suite is the snapshot crate. This crate defines a method for storing the state of the system into a file format. This file can be transferred between different Stronghold Engine devices. This file format can be extended and changed as needed to make it more secure and more appropriate for the system being used. The snapshot layer currently also uses sodiumoxide’s secretstream algorithm which uses XChaCha20-Poly1305 to encrypt and decrypt the data. A user’s password is required to encrypt and decrypt the snapshot.
Data is read into the snapshot crate by way of a byte buffer. A single hexadecimal signature is written to the file’s head along with the file’s version number. A salt is generated and it is used along with the user’s inputted password to derive a unique key. The Key is used to create a header and a push stream; the header is written to the file and the push stream is used to encrypt the incoming data. The databuffer’s data is read in as 256 byte chunks and it is encrypted in the stream before it is written to the file. Decryption of the snapshot follows the opposite steps: a user supplies a password, the salt is read from the file and the password and salt are used to derive a key. The header is then read from the file and used with the key to generate a pull stream. As the data is fed through this stream and it is decrypted back into a plaintext format.
Command Line Proof of Concept:
To show off the features of this set of libraries, an MVP command line tool was created. This CLI is bare bones and based heavily off of the vault fuzz client. Its main purpose is to show off the libraries in a minimal yet meaningful manner. The structure of this application follows a kind of server/client pattern. The state of the database is maintained in a hashmap wrapped in a RwLock and an Arc which is globally available via a lazy static macro. On the frontend, there is a client which contains the client ID and a Vault structure. The Vault structure contains the client’s key and a data view so that it can interact with the data. The key implements a provider which inherits from the box provider trait and this is where the encryption algorithm is defined. The client and the backend are connected through a simple connection structure with some basic logic to access the state hashmap.
Unlike the original vault fuzz client, this application needs to upload and offload its data to and from a snapshot. To achieve this, a snapshot structure was made; it consists of the client’s id and key as well as the database’s hashmap. Each time a user runs this CLI they must submit a password to unlock the snapshot so that the state can be loaded into the application. The id and key are used to create a new client and a garbage collection operation is executed to recreate the data chain from the incoming data. This operation creates a new Initial Transaction and it iterates through each of the transactions to verify that they are owned by the owner. Any foreign data is discarded in this process.
Future Development Options:
A few of the original ideas never made it into the final revision of the Engine but this is for the best. There are still a couple ways forward for this library:
- Secondary Keys/Passwords - Currently the snapshot can only be decrypted with a single password. More passwords could be added to create a blob of permissioned data.
- Homomorphic Encryption - If the data in the snapshot and the system used a Homomorphic encryption standard; operations could be performed without decrypting the data first.
- Key and Snapshot separation - Right now in the CLI example, the secret key is encrypted with the snapshot. In the future it might be better to keep that key separate from the snapshot. An encrypted archive could be used to combine a key file with the snapshot for instance.
- Asymmetric encryption - This was mentioned above; the IDs on the data could be public keys derived from the secret key on the vault.
- Multi-bucket storage. Since each vault is a set of versioned data, the data in a vault should be related. As such it makes sense to add an extension to allow for a single user to maintain multiple vaults.
- Accommodations for more complex data structures inside of the transactions. The ability to store an entire hashmap in a single transaction is possible right now but other complex data structures could see support as well.
- Hashed Owner IDs. Currently, all transactions that are owned by a single owner contain the same owner ID. It may be beneficial to instead derive the owner ID of a transaction based off of the owner's secret key, their original ID and the counter.
Most of these concepts could be implemented as independent libraries or by extending the existing crates. An audit should also be performed to make sure that the model is completely safe from attackers.
Personal Concluding Thoughts
I found working on this project was a learning experience; it was interesting and a nice change of pace. Developing the Engine forced me to examine aspects of cryptography that I had only barely been exposed to in the past. While I have worked on Cryptocurrency platforms such as Steem, I've never worked with secure data this closely. Some of my initial assumptions were either wrong or incomplete and by the end of this development process I had a much more thorough understanding of cryptography as a whole.
I do believe that Engine is a very strong starting line for the Stronghold platform. The future developers will be able to use it effectively in their projects and I look forward to seeing how they extend it. I thank IOTA for giving me the opportunity to work on this project and I wish them luck going forward.
Thanks for thinking about contributing to the project! We have the following ways that you can contribute.
The IOTA Stronghold Initiative is a collaborative effort to help improve the developer experience.
- Quality assurance and review
- Code samples
If you'd like to get involved, join the #experience channel on Discord.
All the code is open source and hosted on GitHub where you can do the following:
- Report a bug
- Suggest a new feature
- Contribute to the documentation
This documentation is also open source and hosted on GitHub.
If you want to contribute new documentation or fix an error, see the contribution guidelines.
Helping others is an important part of any open source ecosystem.
By sharing your knowledge with others, you can provide a lot of value to the community and maybe inspire someone else to learn and contribute.
Take a look at what discussions are going on in the #stronghold-discussion channel on Discord.