I’m Gajanan Chinchwadkar, the CTO of Dgraph Labs. I am excited to lead the engineering team at Dgraph and connect with all of you. After my Ph.D. in Object Oriented Databases about 25 years ago, I built query optimizers and execution engines for multiple enterprise databases to cater to data models such as Relational, Object Oriented, XML, SQL/XML, Full-text, Geo-spatial, and Temporal. I am passionate about building query engines for mix-model query processing and heterogeneous data. I foresee an exciting journey ahead of us at Dgraph and would like to share my perspective with you.
In the 1990s, the impedance mismatch problem between Object Oriented programming languages and Relational Databases motivated the development of Object Oriented Databases. There were over 20 commercial OO Database offerings in the market. Similar to the SQL standard, a standard for OO query language OQL was proposed for these databases. However, OQL couldn’t gain ground over SQL.
Then came a wave of W3C standards such as XML, XPath, XQuery, RDF, and SPARQL in the late 1990s and 2000s. Several XML database companies emerged with the support for XQuery. Relational vendors also extended their support for XML and XQuery. However, XQuery could not sway the millions of application developers away from SQL. History has proven that query language is one of the most crucial factors in the success or failure of database technology, and that is why Dgraph is uniquely positioned for success.
Dgraph has native GraphQL support, and GraphQL is already a widely accepted language by application developers. Furthermore, Dgraph is ACID compliant and the only open source distributed graph database. It enjoys a passionate community that provides valuable inputs [1] [2] and consists of avid users of both GraphQL and DQL. There are two distinct classes of applications that can be satisfied by Dgraph via these two query languages. As demonstrated by the community, the first one represents a need for a GraphQL-native graph database as an application backend. The other one represents a need for a powerful graph engine for Analytics and AI applications which DGraph can provide using DQL.
I have seen two design philosophies that drive the design of database engines. The Relational style came from Relational Databases, and the Information Retrieval style came from search engines. Most database designs are relational style. The main characteristics of this style are as follows.
- Few different access methods
- Such as full scans, index-based lookups, and index-based scans
- A query optimizer (rule/cost based)
- Selects efficient access methods for a given query and generates a query plan
- An execution engine
- Interprets the plan
- Or, in some modern databases, it executes a compiled plan
Each of these topics has been an active area of research for the last 40+ years. Each has evolved with the technological journey through centralized servers, parallel computers, clustered and distributed systems to present-day cloud computing.
On the other hand, the main characteristics of IR style design are the text indexes based on inverted lists of terms, n-grams, and/or hashes. Unlike relational databases, they can provide relevance scoring of results. They usually support word and phrase searches with advanced features such as distance, similarity, and collocation. The research and technologies in IR have also been evolving for the last several decades. However, engines based on this style either do not support SQL, or they support only a subset of SQL. A few recent NoSQL databases use this style.
To understand which style of a query engine would be suitable for a graph database, let us see a few common ways to retrieve data from a graph database.
- Find nodes and/or links based on specific attributes.
- Find paths or sub-graphs based on specific attributes of nodes and/or links.
- Search nodes and links based on specific terms/phrases in the values of their attributes.
Experience tells that Relational and IR styles are performant on (1) and (3), respectively. Imagine if we combine the two kinds of design, we will get a performant engine for (1) and (3). Further, we can enhance it for (2) by applying advanced techniques from OO and XML engines.
I believe that we can make it happen with my experience building query engines in both styles.
We recently announced a funding round with investments from Venrock and Uncorrelated Ventures, along with other investors and top Silicon Valley executives. We are rebuilding the engineering team with the highest priority to product stability and availability. At the same time, we are also digging through community posts here on discuss.dgraph.io. We have a good mental map of what it would take to support GraphQL and DQL enhancements requested by our community. We will begin the process of converting it into a concrete roadmap working closely with the community. There is no doubt that building a query engine for such a wide variety of requirements is going to be a challenging task. We are ready for this challenge!
Let me stop now. I will be here with you for more discussions. I am looking forward to working with you all.