BigchainDB 2017 Roadmap
Introduction
In early 2016, we released BigchainDB. Throughout the rest of the year, we iterated rapidly with early adopter users to improve the product. We crystallized our vision: we’re building a decentralized database for the internet. This manifests as database software (BigchainDB), and deployments like the IPDB public network, and more private consortia. BigchainDB is designed to merge the best of database and blockchain worlds: scale and querying from the database side, and decentralization, immutability, and assets from the blockchain side. It plays well alongside decentralized file systems (e.g. IPFS) and processing (e.g. Ethereum).
For us, 2017 is about execution: making sure that BigchainDB meets the needs of our lead users, to be able to ship their applications in production. We’re iterating with a few dozen organizations building on BigchainDB, and have rapid cycles on a small handful. Some of these users have needs that we can meet in the near term; others have needs that we continue to build for throughout the year.
2017 Summary
As in 2016, we have releases approximately every two months. This includes a 1.0 release in early June 2017.
Our 2017 roadmap has three streams: core/security, UX, and IPDB.
- Core/security is the backbone; we appreciate the importance of the data that people are putting in BigchainDB-based systems. Therefore, our main focus of the database core in 2017 is security. First, to get to “your data is safe on BigchainDB”, then to “decentralized without compromising security”.
- UX is a handful of new features on user experience, including querying and read permissions as assets.
- IPDB. Alongside core/security and UX work, we have a team dedicated to rolling out the IPDB test network and production network.
I elaborate on each stream in the following sections. After that, I describe our product management process.
Core/Security
This section details our plans for security work in the BigchainDB core in 2017.
We divide security into three parts: recovery, monitoring, and defense. Recovery is the linchpin, that if a malicious attack does bypass defenses, there is a path to retrieve the data that we care about. Monitoring gives us visibility to continually improve the system. This informs our schedule, as follows.
Our 1.0 release (early June) will have:
- Recovery: making continuous backups easy, such that “your data is safe” even if a BigchainDB instance gets hacked. We’ll be leveraging MongoDB’s tooling for backups here.
- Monitoring: improved logging and debuggability.
- Security assumption is “as good as centralized” but no more; i.e. at the same level as well-secured mainstream databases. A node can be hacked. Adding a new node increases the attack surface and is not recommended. But we’re working on making it possible to tell which node acted maliciously, along with fully-replicated logs of behavior.
- Setting expectations: well-documented guarantees on what we do and don’t provide with respect to security, stability, and correctness.
Our 1.3 release (mid December) will have:
- Defense: better walls against the most probable / highest impacting attack vectors. (We’ve already documented many of these; e.g. here.)
- Monitoring: further improved logging and debuggability.
- Security Audits: this will be done once we’ve built at the levels of monitoring, recovery, and defense. We’ll start with a crowd-sourced audit, followed by specific professional security auditors. We’ll also incorporate learnings from the IPDB test network.
- Security assumption improves because of the improvements to defense, monitoring, and audits. In particular, our 1.3 consensus will make it highly unlikely for a handful of nodes to be able to mess up the network.
- Setting expectations: our documentation will reflect the improvements to security, stability, and correctness.
I’d love to say we’re “fully secure now” but that would be naive and irresponsible. We’re very careful about the claims we make because we appreciate the importance of the data that people are trusting in BigchainDB systems.
UX
For 1.0, we plan to release these features:
- Read permissions as assets.
- Events API (using WebSockets).
- Basic querying support.
“Read permissions as assets” unlocks use cases in private personal data, licensing IP, and more. It combines the database-style read permissions with the blockchain-style assets. A perfect fit for a blockchain database!
We’ll likely add more UX features later in the year, based on customer needs.
Also, we’ll continue to refine the http API and drivers as needed, and continue to support our awesome community that is building drivers around BigchainDB (thanks all!)
IPDB
As mentioned, we have a team dedicated to rolling out the IPDB test network and production network. They are doing work on:
- Systems engineering, combining for a deployment consisting of BigchainDB, MongoDB, 3scale / nginx, Kubernetes and more;
- Security around deployment, and
- Dashboards for both application developers and node caretakers.
We’ve been ramping up IPDB test network users since October 2016. We’re targeting to give test network access to all 200+ organizations on the wait list within a couple months. Production network users will also be ramping up over the course of the year, prioritizing towards the users who need to ship.
The deployment technologies being developed for IPDB will be made available for other non-IPDB deployments too, e.g. for enterprises and private consortiums.
We hatched the IPDB Foundation for long-term governance of the network. IPDB Foundation will continue to mature in 2017: IPDB’s Caretakers will take full control, with IPDB’s Board of Directors and management continuing day-to-day work.
Process
Here’s how we arrived at the high-level plans described above.
With ascribe.io, we used a continuous-integration agile methodology — a mix of XP and scrum. We often had several releases per week, focused on improvements to the webapp UX, supported as needed by frontend and backend code.
As we shifted focus to BigchainDB, we quickly learned that BigchainDB users had different needs. They didn’t care about daily software updates. Rather, they had features that they needed in BigchainDB; and they needed to know when we planned to ship those features. Internally, we had to balance the huge number of user stories with improvements to the DB core and IPDB.
I was spread too thin to manage & prioritize stories well. So, starting in January 2017, my colleague Tim Daubenschütz became our first official product manager (PM), moving from his previous role as developer.
Tim and I tightened the product management process. Here are a few of the highlights.
We define and prioritize based on business needs, external users’ needs, and technical needs.
We’ve made releases more formal, and bi-monthly rather than daily. Each release holds some of the stories, with margin for under-estimation and surprises. We have weekly sprints and daily standups, working on stories within the release.
And, we have a 2017 roadmap, of which the early part of this blog post describes:)
Conclusion
2017 is going to be a fun year, as we see the production deployment of our users’ applications. We’re super excited about so many of these applications that are just plain awesome — helping to unlock sovereign personal data, to get creators compensated, to power global supply chains, and more. Let’s build! Let’s ship!
Yours,
Trent and the BigchainDB team
P.S. We’re looking for a few great developers.
