Differential Dataflow for the Masses
42 points by rebanevapustus 7 months ago | 11 comments- snthpy 7 months agoAlso, I never fully grokked Differential Dataflow but I get DBSP so I would love an explanation of how Differential Dataflow differs from DBSP.
- rebanevapustus 7 months agohttps://github.com/brurucy/pydbsp/issues/6 it's been answered! (to the best of my knowledge)
- rebanevapustus 7 months ago
- snthpy 7 months agoLove this project!
Since "Differential Dataflow" has a specific meaning though and this isn't it, I wonder if a different title wouldn't be better?
"DBSP for the masses" presumes someone already knows what DBSP is so that doesn't work.
What about "Incremental Dataflow for the masses" or "Incremental Dataflow in Python"?
- rebanevapustus 7 months agoIncremental Dataflow for the Masses sounds much better :)
- rebanevapustus 7 months ago
- erichocean 7 months agoElectric Clojure v3 also implements differential dataflow. [0]
- rebanevapustus 7 months agoElectric Clojure v3 cannot be "Differential Dataflow" because it *seems* to be restricted to DAGs only. Differential Dataflow is highly expressive, it supports iteration.
- dustingetz 7 months ago(electric founder here) we are lowercase differential dataflow my friend. Fwiw our model does support cycles and iteration, we simply didn’t bother to formally reify cycles yet, because we have a functional effect system under the hood and using that to implement cycles by userland side effect does the right thing, is perfectly efficient, and has very little downside.
- rebanevapustus 7 months agoThanks for clarifying it!
I checked some of your videos, and I couldn't quite grasp how much of (capital) DDflow do you borrow.
What does "our model does support cycles and iteration" mean? Do these also have the incremental semantics that guarantee bounds over the size of updates?
- rebanevapustus 7 months ago
- dustingetz 7 months ago
- rebanevapustus 7 months ago
- keithalewis 7 months agoThis seems to make the mistake of identifying a vector with a stream.
- rebanevapustus 7 months agoIn this implementation streams are not vectors nor lists. They are ordered hash maps.
- keithalewis 7 months agoJust pointing out what the theoretical documentation says. A stream should have operations for empty, current, and next. You don't want implementation details leaking into the abstract model.
- keithalewis 7 months ago
- rebanevapustus 7 months ago