We resolved it with adding another level. Our install is shared across 3 servers. A single node hit the limit and killed the cluster. I suppose I’m curious if there a more elegant solution to hitting this limit?
The process we took was…
- Cluster crashed
- Google searched for error with “dgraph” in the search term. No hits.
- Google searched for error without “dgraph” in search term. BadgerDB error hits on this discuss forum.
- Address the issue.
- Restart the swarm.
- Working again.
This just doesn’t seem like a production level resolution to hitting 1.1TB on a node of a predicate.
Are there plans in place to address memory size limits or predicate sharding across nodes?
Addressing this: Splitting predicates into multiple groups - #13 by eugaia, seems like it could mitigate the issue substantially.
Thanks,
Ryan