10000 refactor(cache): improve cache and integrate reagent by becomingbabyman · Pull Request #107 · homebaseio/homebase-react · GitHub
[go: up one dir, main page]

Skip to content

refactor(cache): improve cache and integrate reagent #107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Jun 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ dist/

package-lock.json
report.html
pom.xml
pom.xml.asc
*.iml
*.jar
8000 Expand Down
268 changes: 73 additions & 195 deletions README.md

Large diffs are not rendered by default.

19 changes: 15 additions & 4 deletions deps.edn
Original file line number Diff line number Diff line change
@@ -1,10 +1,21 @@
{:paths ["src"]
{:paths ["src/dev"
"src/main"
"src/test"]
:deps {thheller/shadow-cljs {:mvn/version "2.11.25"}
devcards/devcards {:mvn/version "0.2.7"}
datascript/datascript {:mvn/version "1.0.7"}
reagent/reagent {:mvn/version "1.0.0-alpha2"}
inflections/inflections {:mvn/version "0.13.2"}
binaryage/devtools {:mvn/version "1.0.2"}
homebaseio/datalog-console {:git/url "https://github.com/homebaseio/datalog-console" :sha "97d5e5eb8994124ec8dc0029b33f2e88257b39b2"}
;; homebaseio/datalog-console {:local/root "../datalog-console"}
camel-snake-kebab/camel-snake-kebab {:mvn/version "0.4.2"}}}
io.homebase/datalog-console {:mvn/version "0.2.2"}
nano-id/nano-id {:mvn/version "1.0.0"}
camel-snake-kebab/camel-snake-kebab {:mvn/version "0.4.2"}}
:aliases {:jar {:replace-deps {com.github.seancorfield/depstar {:mvn/version "2.0.216"}}
:exec-fn hf.depstar/jar
:exec-args {:jar "homebase-react.jar" :sync-pom true}}
:install {:replace-deps {slipset/deps-deploy {:mvn/version "0.1.5"}}
:exec-fn deps-deploy.deps-deploy/deploy
:exec-args {:installer :local :artifact "homebase-react.jar"}}
:deploy {:replace-deps {slipset/deps-deploy {:mvn/version "0.1.5"}}
:exec-fn deps-deploy.deps-deploy/deploy
:exec-args {:installer :remote :artifact "homebase-react.jar"}}}}
71 changes: 71 additions & 0 deletions doc/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Homebase React

[![CI](https://github.com/homebaseio/homebase-react/workflows/CI/badge.svg)](https://github.com/homebaseio/homebase-react/actions?query=workflow%3ACI)
[![CD](https://github.com/homebaseio/homebase-react/workflows/CD/badge.svg)](https://github.com/homebaseio/homebase-react/actions?query=workflow%3ACD)
[![License](https://img.shields.io/github/license/homebaseio/homebase-react.svg)](LICENSE)
[![GitHub Repo stars](https://img.shields.io/github/stars/homebaseio/homebase-react?style=social)](https://github.com/homebaseio/homebase-react)
[![Twitter Follow](https://img.shields.io/twitter/follow/homebase__io?label=Follow&style=social)](https://twitter.com/homebase__io)

> Use a datalog DB to manage react application state.

S 9E81 upported languages, frameworks and DBs:

- JS + React + Datascript
- CLJS + Reagent + Datascript
- *Datahike support coming soon*

# Javascript + React

This is the Clojure docs site. Go here for [JS + React docs](https://homebase.io/docs/homebase-react).

# ClojureScript + Reagent

Start by adding `homebase-react`.

[![Clojars Project](https://img.shields.io/clojars/v/io.homebase/homebase-react.svg)](https://clojars.org/io.homebase/homebase-react)

Optionally add `datalog-console` and its corresponding [chrome extension](https://chrome.google.com/webstore/detail/datalog-console/cfgbajnnabfanfdkhpdhndegpmepnlmb?hl=en) to inspect the DB in your browser.

[![Clojars Project](https://img.shields.io/clojars/v/io.homebase/datalog-console.svg)](https://clojars.org/io.homebase/datalog-console)

## Quick Start

```clojure
(ns homebase.dev.example.reagent.counter
(:require
[datascript.core :as d]
[homebase.reagent :as hbr]
[datalog-console.integrations.datascript :as datalog-console]))

(def db-conn (d/create-conn {}))
(d/transact! db-conn [[:db/add 1 :count 0]]) ; Transact some starting data.
(hbr/connect! db-conn) ; Connect homebase.reagent to the database.
(datalog-console/enable! {:conn db-conn}) ; Also connect the datalog-console extension for better debugging.

(defn counter []
(let [[entity] (hbr/entity db-conn 1)] ; Get a homebase.reagent/Entity. Note the use of db-conn and not @db-conn, this makes it reactive.
(fn []
[:div
"Count: " (:count @entity) ; Deref the entity just like a reagent/atom.
[:div
[:button {:on-click #(d/transact! db-conn [[:db/add 1 :count (inc (:count @entity))]])} ; Use d/transact! just like normal.
"Increment"]]])))
```

[Live demo](https://homebaseio.github.io/homebase-react/index.html#!/homebase.dev.example.reagent)

## API

- [homebase.reagent](https://cljdoc.org/d/io.homebase/homebase-react/CURRENT/api/homebase.reagent)

## Performance

Our reactive query functions like `hbr/entity` and `hbr/q` will only trigger re-renders when their results change.

In the case of `hbr/entity` we track which attributes get consumed `(:attr @entity)` and only dispatch updates when those attributes are transacted.

`hbr/q` queries rerun on every transaction. If the result is different we re-render. We're looking into differential datalog and incremental view maintenance, but for typical datasets of tens of thousands of datoms the current performance is great. DOM updates tend to be much more costly, so just rerunning the queries still performs well as long as we don't repaint the DOM.

## Alternatives

- If your prefer `d/pull` over `d/entity` take a look at [Posh](https://github.com/denistakeda/posh). It supports less of the `d/q` API, but provides more tools for tuning performance.
7 changes: 7 additions & 0 deletions doc/cljdoc.edn
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{:cljdoc.doc/tree
[["Introduction" {:file "doc/README.md"}]
["Examples" {:file "doc/examples.md"}]
["Misc" {}
["Tooling & Debugging" {:file "doc/tooling_debugging.md"}]
["Performance" {:file "doc/performance.md"}]
["Contribution" {:file "CONTRIBUTING.md"}]]]}
3 changes: 3 additions & 0 deletions doc/examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Examples

Visit the [live demos](https://homebaseio.github.io/homebase-react/index.html#!/homebase.dev.example.reagent) devcards site to see some examples.
47 changes: 47 additions & 0 deletions doc/performance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Performance

Homebase React tracks the attributes consumed in each component via `homebase.reagent/Entity` and scopes those attributes to their respective `hbr/entity` reagent atom. Re-renders are only triggered when an attribute changes.

The default caching reduces unnecessary re-renders and virtual DOM thrashing a lot. That said, it is still possible to trigger more re-renders than you might want.

## Smart Prop Drilling

One top level `hbr/entity` + prop drilling the entity it returns will cause all children to re-render on any change to the parent or their siblings.

To fix this we recommend passing ids to children, not whole entities. Instead get the entity in the child with `(hbr/entity db-conn id)`. This creates a new scope for each child so they are not affected by changes in the state of the parent or sibling components.

### Good Prop Drilling

```clojure
(defn friend [id]
(let [[user] (hbr/entity db-conn id)]
(fn []
[:div (:user/name @user)])))

(defn friends [user-id]
(let [[user] (hbr/entity db-conn user-id)]
(fn [user-id]
[:div
(for [u (:user/friends @user)]
[friend (:db/id u)])])))
```

### Bad Prop Drilling

```clojure
(defn friend [user]
[:div (:user/name @user)])

(defn friends [user-id]
(let [[user] (hbr/entity db-conn user-id)]
(fn [user-id]
[:div
(for [u (:user/friends @user)]
[friend u])])))
```

## Query performance

`hbr/q` queries rerun on every transaction. If the result is different we re-render. We're looking into differential datalog and incremental view maintenance, but for typical datasets of tens of thousands of datoms the current performance is great. DOM updates tend to be much more costly, so rerunning the queries still performs well as long as we don't repaint the DOM.

If you are seeing UI slowdowns consider virtualizing large lists and only rendering DOM nodes that fit on the screen.
39 changes: 39 additions & 0 deletions doc/tooling_debugging.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Tooling & Debugging

We've built a few tools to make debugging a bit more convenient.

## Custom chrome formatters
If you develop with [Chrome](https://www.google.com/chrome/) or a Chromium browser like Brave or Edge you'll get significantly more meaningful logs for entities `(js/console.log an-entity)` due to our use of custom chrome :formatters. These custom formatters allow us to perform lazy database queries to fetch all of an entity's attributes, including references to other entities and all reverse references to the current entity. They let you access your entire data graph from the console, with any logged out entity as an entry point.

**To enable custom chrome formatters**

**1.** Add **[binaryage/cljs-devtools](https://github.com/binaryage/cljs-devtools)** to your app. Our entity formatters implement protocols defined in cljs-devtools and need cljs-devtools to work. Plus, if you've never used cljs-devtools you're in for a treat.

**2.** Open the preferences panel in chrome devtools by clicking the cog.

![image of chrome devtools preferences button](https://github.com/homebaseio/homebase-react/blob/master/public/images/enable_chrome_formatters_1.png?raw=true)

**3.** Toggle `Enabled custom formatters` on.

![image of chrome devtools custom formatters toggle](https://github.com/homebaseio/homebase-react/blob/master/public/images/enable_chrome_formatters_2.png?raw=true)

**4.** Keep the chrome console open and refresh the page. Any logged out entities should now have the custom formatting.

![image of custom entity chrome console logs](https://github.com/homebaseio/homebase-react/blob/master/public/images/enable_chrome_formatters_3.png?raw=true)

**Live demo:** open the console while on the [todo example](https://homebaseio.github.io/homebase-react/#!/homebase.dev.example.todo) page.

**Remember**: for custom formatters to work `(js/console.log an-entity)` must be called *after* you open the chrome console. Anything logged out before you open the console will not have custom formatting applied because chrome processes those logs in the background.

## Datalog Console Extension

We also integrate with the [Datalog Console](https://github.com/homebaseio/datalog-console) extension.

![image of datalog console extension](https://github.com/homebaseio/homebase-react/blob/master/public/images/datalog_console.png?raw=true)

It's still in an early stage of development, but we seek to expose all common DB administration capabilities here and let you connect to any Datalog database that implements the console's interface.

### Using the Datalog Console

1. [Add the extension to Chrome](https://chrome.google.com/webstore/detail/datalog-console/cfgbajnnabfanfdkhpdhndegpmepnlmb)
2. Vist a page built with homebase-react [like this one](https://cljdoc.org/d/io.homebase/homebase-react/CURRENT/api/homebase.reagent), open the inspector, click the `Datalog DB` tab, and click `Load database` to try it out
61 changes: 54 additions & 7 deletions docs/0100|Overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,24 @@

[![CI](https://github.com/homebaseio/homebase-react/workflows/CI/badge.svg)](https://github.com/homebaseio/homebase-react/actions?query=workflow%3ACI)
[![CD](https://github.com/homebaseio/homebase-react/workflows/CD/badge.svg)](https://github.com/homebaseio/homebase-react/actions?query=workflow%3ACD)
[![NPM Version](https://img.shields.io/npm/v/homebase-react)](https://www.npmjs.com/package/homebase-react)
[![Bundle Size](https://img.shields.io/bundlephobia/minzip/homebase-react)](https://www.npmjs.com/package/homebase-react)
[![License](https://img.shields.io/github/license/homebaseio/homebase-react.svg)](LICENSE)
[![GitHub Repo stars](https://img.shields.io/github/stars/homebaseio/homebase-react?style=social)](https://github.com/homebaseio/homebase-react)
[![Twitter Follow](https://img.shields.io/twitter/follow/homebase__io?label=Follow&style=social)](https://twitter.com/homebase__io)

## What and Why
Supported languages, frameworks and DBs:

As data and our need to annotate and organize it grows, so does our need for supporting state in *write-heavy* applications.
- JS + React + Datascript
- CLJS + Reagent + Datascript
- *Datahike support coming soon*

To solve this problem, modern write-heavy applications such as Superhuman, Roam Research, and Facebook Messenger built their own embedded data layers to enable these more sophisticated user experiences.
# ClojureScript + Reagent

Homebase React enables developers to access the same embedded datalog database as Roam Research through React hooks. You no longer have to build out a team or learn specialized tools like Clojure in order to build a delightful write-heavy application.
This is the Javascript docs site. Go here for [CLJS + Reagent docs](https://cljdoc.org/d/io.homebase/homebase-react/CURRENT).

# Javascript + React

## Install
Start by installing `homebase-react` [![NPM Version](https://img.shields.io/npm/v/homebase-react)](https://www.npmjs.com/package/homebase-react)
[![Bundle Size](https://img.shields.io/bundlephobia/minzip/homebase-react)](https://www.npmjs.com/package/homebase-react)

```bash
# NPM
Expand All @@ -26,6 +29,50 @@ npm install homebase-react --save
yarn add homebase-react
```

Optionally install the `datalog-console` [chrome extension](https://chrome.google.com/webstore/detail/datalog-console/cfgbajnnabfanfdkhpdhndegpmepnlmb?hl=en) to inspect the `homebase-react` DB in your browser.

## Quick Start

```js
import { useCallback } from 'react'
import { HomebaseProvider, useEntity, useTransact } from 'homebase-react'

const RootComponent = () => (
// Create a DB and set some starting data
<HomebaseProvider config={{ initialData: [{ counter: { id: 1, count: 0 }}] }}>
<App/>
</HomebaseProvider>
)

const App = () => {
// Get entity id = 1
const [counter] = useEntity(1)
const [transact] = useTransact()

return (
<button onClick={() => {
// Increment and save the count
transact([{ counter: {
id: 1, count: counter.get('count') + 1
}}])
}>
{/* Render the count */}
Increment {counter.get('count')}
</button>
)
}
```

[Live demo](https://homebaseio.github.io/homebase-react/#!/homebase.dev.example.counter)

## What and Why

As data and our need to annotate and organize it grows, so does our need for supporting state in *write-heavy* applications.

To solve this problem, modern write-heavy applications such as Superhuman, Roam Research, and Facebook Messenger built their own embedded data layers to enable these more sophisticated user experiences.

Homebase React enables developers to access the same embedded datalog database as Roam Research through React hooks. You no longer have to build out a team or learn specialized tools like Clojure in order to build a delightful write-heavy application.

## Testimonials
> Homebase is executing on the vision of data usage, portability, and management we had when building Firebase. We never got there. I'm excited!
>
Expand Down
4 changes: 3 additions & 1 deletion docs/0200|Quick_Start.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ Adding `HomebaseProvider` automatically creates the database.
```js
import { HomebaseProvider } from 'homebase-react'

const config = { initialData: [{ counter: { id: 1, count: 0 }}] }

const RootComponent = () => (
<HomebaseProvider>
<HomebaseProvider config={config}>
<App/>
</HomebaseProvider>
)
Expand Down
4 changes: 2 additions & 2 deletions docs/0675|Debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ If you develop with [Chrome](https://www.google.com/chrome/) or a Chromium brows

![image of custom entity chrome console logs](https://github.com/homebaseio/homebase-react/blob/master/public/images/enable_chrome_formatters_3.png?raw=true)

**Live demo:** open the console while on the [todo example](https://homebaseio.github.io/homebase-react/#!/dev.example.todo) page.
**Live demo:** open the console while on the [todo example](https://homebaseio.github.io/homebase-react/#!/homebase.dev.example.todo) page.

**Remember**: for custom formatters to work `console.log(anEntity)` must be called *after* you open the chrome console. Anything logged out before you open the console will not have custom formatting applied because chrome processes those logs in the background.

Expand All @@ -41,7 +41,7 @@ It's still in an early stage of development, but we seek to expose all common DB
#### Using the Datalog Console

1. [Add the extension to Chrome](https://chrome.google.com/webstore/detail/datalog-console/cfgbajnnabfanfdkhpdhndegpmepnlmb)
2. Vist a page built with homebase-react [like this one](https://homebaseio.github.io/homebase-react/#!/dev.example.todo), open the inspector, click the `Datalog DB` tab, and click `Load database` to try it out
2. Visit a page built with homebase-react [like this one](https://homebaseio.github.io/homebase-react/#!/homebase.dev.example.todo), open the inspector, click the `Datalog DB` tab, and click `Load database` to try it out

### DEPRECATED `_recentlyTouchedAttributes`

Expand Down
1 change: 1 addition & 0 deletions examples/roam/scripts/convert_roam_edn/convert.clj
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
to make Datalog feel more like SQL and tabular data.

This script normalizes all attributes to the 'block' namespace."
{:no-doc true}
(:require [clojure.pprint]))

;; (def input (read-string (slurp "scripts/convert_roam_edn/datasets/hn.edn")))
Expand Down
9 changes: 6 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
"main": "./dist/js/homebase.react.js",
"private": false,
"scripts": {
"dev": "shadow-cljs watch dev & babel src/dev/example/js --out-dir src/dev/example/js_compiled --watch && kill $!",
"dev": "shadow-cljs watch dev & babel src/dev/homebase/dev/example/js --out-dir src/dev/homebase/dev/example/js_compiled --watch && kill $!",
"build": "rm -rf dist && shadow-cljs release npm && yarn bundle-ts",
"build:dev": "rm -rf dist && shadow-cljs compile npm && yarn bundle-ts",
"test:js": "yarn build && jest src/* && yarn tsd",
"test:js:dev": "yarn build:dev && jest src/* && yarn tsd",
"test:js": "yarn build && jest src/test/* && yarn tsd",
"test:js:dev": "yarn build:dev && jest src/test/* && yarn tsd",
"test:ts": "yarn bundle-ts && yarn tsd",
"test:cljs": "shadow-cljs compile test && node out/node-tests.js",
"test:cljs:watch": "shadow-cljs watch test-autorun",
Expand Down Expand Up @@ -40,6 +40,7 @@
"@babel/preset-react": "^7.12.13",
"@commitlint/cli": "^11.0.0",
"@commitlint/config-conventional": "^11.0.0",
"@peculiar/webcrypto": "^1.1.7",
"@semantic-release/changelog": "5.0.1",
"@semantic-release/commit-analyzer": "8.0.1",
"@semantic-release/git": "9.0.0",
Expand Down Expand Up @@ -67,11 +68,13 @@
"eslint-plugin-react-hooks": "^4.2.0",
"firebase": "^8.0.2",
"firebaseui": "^4.7.1",
"global-jsdom": "^8.1.0",
"highlight.js": "10.4.1",
"hoist-non-react-statics": "^3.3.0",
"husky": "5.0.0-beta.0",
"jest": "26.6.0",
"jest-performance-testing": "^1.0.0",
"jsdom": "^16.6.0",
"marked": "2.0.0",
"pinst": "2.0.0",
"prettier": "^2.2.1",
Expand Down
Loading
0