Dgraph v1.0.6 is out

mrjn · June 27, 2018, 4:22am

Dgraph v1.0.6 is out. It contains many bug fixes since v1.0.5. It is highly recommended to upgrade asap. Let us know your experiences with v1.0.6 in this thread.

Quickest way to upgrade is:

curl https://get.dgraph.io -sSf | bash

Link: Release v1.0.6 · dgraph-io/dgraph · GitHub

Changelog

Added

Support GraphQL vars as args for Regexp function. #2353
Support GraphQL vars with filters. #2359
Add JSON mutations to raw HTTP. #2396

Fixed

Fix math >= evaluation. #2365
Avoid race condition between mutation commit and predicate move. #2392
Ability to correctly distinguish float from int in JSON. #2398
Remove dummy data key. #2401
Serialize applying of Raft proposals. Concurrent application was complex and
cause of multiple bugs. #2428.
Improve Zero connections.
Fix bugs in snapshot move, refactor code and improve performance significantly. #2440, #2442
Add error handling to GetNoStore. Fixes #2373.
Fix bugs in Bulk loader. #2449
Posting List and Raft bug fixes. #2457

Changed

Pull in Badger v1.5.2.
Raft storage is now done entirely via Badger. This reduces RAM
consumption by previously used MemoryStorage. #2433
Trace how node.Run loop performs.
Allow tweaking Badger options.

Note: This change modifies some flag names. In particular, Badger options
are now exposed via flags named with --badger. prefix.

shanghai-Jerry · June 27, 2018, 5:59am

i will test the new version1.0.6

after trying to use --badger.vlog disk， i found the latency is much longer than before

maybe i should set --badger.vlog mmap, and then compare those

nikkon226 · June 27, 2018, 3:32pm

Push to docker hub please?

mrjn · June 27, 2018, 8:32pm

Expect that by end of day today.

Update: Docker v1.0.6 is out. Latest now points to v1.0.6.

See https://docs.dgraph.io/deploy for more information on how to get the latest Docker image.

mrjn · June 28, 2018, 4:01pm

Warning: Due to a change in Raft storage format, this upgrade would require you to export and re-import your data, clearing out all the WAL (zw in Zero and w in Server) directories.

Please take note of this. If you try to restart your servers with this upgrade with data in WAL from v1.0.5, they would fail with this stack trace:

github.com/dgraph-io/dgraph/raftwal.(*DiskStorage).parseIndex
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/raftwal/storage.go:160
github.com/dgraph-io/dgraph/raftwal.(*DiskStorage).seekEntry.func1
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/raftwal/storage.go:221
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*DB).View
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/transaction.go:567
github.com/dgraph-io/dgraph/raftwal.(*DiskStorage).seekEntry
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/raftwal/storage.go:210
github.com/dgraph-io/dgraph/raftwal.(*DiskStorage).FirstIndex
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/raftwal/storage.go:243
github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft.newLog
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft/log.go:53
github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft.newRaft
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft/raft.go:294
github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft.RestartNode
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft/node.go:219
github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*node).initAndStartNode
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/raft.go:474
github.com/dgraph-io/dgraph/dgraph/cmd/zero.run
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:191
github.com/dgraph-io/dgraph/dgraph/cmd/zero.init.0.func1
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:60
github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(*Command).execute
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:702
github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(*Command).ExecuteC
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:783
github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(*Command).Execute
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:736
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:43
main.main
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/main.go:24

If an export and reimport is not possible for some specific reason, reach out to us.

shanghai-Jerry · June 29, 2018, 3:12am

ok.

@mrjn i found a situation:
i set replica=3, badger.vlog:disk
i have three dgraph server, i insert data to one server, and i can get results from this server, but i cannot get any results and i found p dir has not data in another two server,

why?

it seems that group members (id=2， id=3) dont have lastUpdate.
might be i just need to wait.

xinstein · June 29, 2018, 3:14am

it seems you are to gradually migrate to json mutations?
Would you recommend users to gradually switch to json mutations and drop rdf n-quads?

shanghai-Jerry · June 29, 2018, 9:11am

that’s not a good idea to upgrade to v1.0.6.

server’s log

mrjn · June 29, 2018, 7:48pm

Are you using Docker, @shanghai-Jerry? I can’t replicate this issue at my end, but I can give you an image for you to try, and see if that solves this issue at your end.

mrjn · June 30, 2018, 2:40am

@shanghai-Jerry and others, I’ve downgraded a library that we are using to the same version as we had before and updated the binaries for v1.0.6. Can you retry and check if you still see this issue?

shanghai-Jerry · June 30, 2018, 5:05am

I am not using docker.

at the end of this topic, i posted my server configs.

and

i used this query:

{
  query(func:has(name)) {
    count(uid)
  }
}

all three servers can return the same result:

{
  "data": {
    "query": [
      {
        "count": 6903
      }
    ]
  },
  "extensions": {
    "server_latency": {
      "parsing_ns": 6480,
      "processing_ns": 10197032,
      "encoding_ns": 2797792
    },
    "txn": {
      "start_ts": 69,
      "lin_read": {
        "ids": {
          "1": 14
        }
      }
    }
  }
}

but when i used a query with specific uid:

{
  query(func:uid(0x9b6)) {
    expand(_all_) {
      
    }
  }
}

only one server can get result

shanghai-Jerry · June 30, 2018, 5:08am

ok, i will retry. but when i use p directory created by bulk loader out directory to restart server, i got too many WARNs like:

2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!

can you tell me what happened?
when i used v1.0.5, it’s normal, no WARNs

shanghai-Jerry · June 30, 2018, 11:13am

i’ve tried with v1.0.6, more details

but it still happened:

and

and when i restarted dgraph-server:

mrjn · June 30, 2018, 1:36pm

How did you load your data into the cluster? Did you start with empty directories?

How long did the cluster run before you saw these MsgVote logs?

shanghai-Jerry · June 30, 2018, 2:30pm

i used mutate operation declared in api.proto, connected server with port 9080.

first, i started with empty directories. and then when i saw these MsgVote logs, i restarted with already exist directories. i found one server was getting snapshot from another server from p directory.

final question, after about one hour , i saw these MsgVote logs.

and now i found getting snapshot again

mrjn · July 1, 2018, 1:28am

Added this warning in v1.0.6. I’m seeing some issues with replication. Working on solving those, and would be part of the next release.

Due to some issues with application of replication logs, we recommend only using --replicas=1 for this release. We’re working on solving these issues for the next release.

shanghai-Jerry · July 2, 2018, 9:19am

hi, @mrjn

i used to alter schema, but i found something in server’s log:

it seems that this log "Done schema update xxx " cannot stop, i only did alter schema operation once.

selmeci · July 3, 2018, 9:01am

Any ETA for next release? Replication is critical for our production.

MichelDiz · July 3, 2018, 3:37pm

Well, if you do not need any of the features of version 1.0.6 or anything else, you can stick to version 1.0.5 until there is a final RC to next release.

Topic		Replies	Views
Dgraph Release v22.0.2 is now Generally Available Announce dgraph	7	1465	April 4, 2023
Dgraph v1.0.7-rc3 release Dgraph	26	3195	July 27, 2018
Dgraph v1.2.6 Announce	0	421	July 31, 2020
Dgraph Release Candidate v23.0.0-rc1 is now available Announce dgraph , release	3	616	May 3, 2023
Dgraph v1.0.16 release Announce	0	586	July 11, 2019

Dgraph v1.0.6 is out

Changelog

Added

Fixed

Changed

Related Topics