Combined: Support `Mixed` data type with collections #6613

elle-j · 2024-04-12T20:21:18Z

What, How & Why?

Note

This feature has already been reviewed. New updates to review: CHANGELOG.md.
The branch can be merged whenever we want to make a release with this new feature.

Adds support for storing lists and dictionaries (with nested collections) as the underlying value of a Mixed property.

Sets are not supported as a Mixed value.

Test overview

Click to expand

  Mixed
    Collection types
      CRUD operations
        Create and access
          List
            ✓ has all primitive types (input: JS Array)
            ✓ has all primitive types (input: Realm List)
            ✓ has all primitive types (input: Default value)
            ✓ has nested lists of all primitive types
            ✓ has nested dictionaries of all primitive types
            ✓ has mix of nested collections of all types
            ✓ inserts all primitive types via `push()`
            ✓ inserts nested lists of all primitive types via `push()`
            ✓ inserts nested dictionaries of all primitive types via `push()`
            ✓ inserts mix of nested collections of all types via `push()`
            ✓ returns different reference for each access
          Dictionary
            ✓ has all primitive types (input: JS Object)
            ✓ has all primitive types (input: JS Object w/o proto)
            ✓ has all primitive types (input: Realm Dictionary)
            ✓ has all primitive types (input: Default value)
            ✓ can use the spread of embedded Realm object
            ✓ can use the spread of custom non-Realm object
            ✓ has nested lists of all primitive types
            ✓ has nested dictionaries of all primitive types
            ✓ has mix of nested collections of all types
            ✓ inserts all primitive types via setter
            ✓ inserts nested lists of all primitive types via setter
            ✓ inserts nested dictionaries of all primitive types via setter
            ✓ inserts mix of nested collections of all types via setter
            ✓ inserts mix of nested collections of all types via `set()` overloads
            ✓ returns different reference for each access
          Results
            from List
              snapshot()
                ✓ has all primitive types
                ✓ has mix of nested collections of all types
              objects().filtered()
                ✓ has all primitive types
                ✓ has mix of nested collections of all types
            from Dictionary
              objects().filtered()
                ✓ has all primitive types
                ✓ has mix of nested collections of all types
        Update
          List
            ✓ updates top-level item via setter
            ✓ updates nested item via setter
            ✓ updates itself to a new list
            ✓ updates nested list to a new list
            ✓ does not become invalidated when updated to a new list
            - self assigns
            - self assigns nested list
          Dictionary
            ✓ updates top-level entry via setter
            ✓ updates nested entry via setter
            ✓ updates itself to a new dictionary
            ✓ updates nested dictionary to a new dictionary
            ✓ does not become invalidated when updated to a new dictionary
            - self assigns
            - self assigns nested dictionary
        Remove
          List
            ✓ removes top-level item via `remove()`
            ✓ removes nested item via `remove()`
          Dictionary
            ✓ removes top-level entry via `remove()`
            ✓ removes nested entry via `remove()`
        JS collection methods
          List
            ✓ pop()
            ✓ shift()
            ✓ unshift()
            ✓ splice()
            ✓ indexOf()
          Iterators
            ✓ values() - list
            ✓ values() - dictionary
            ✓ entries() - list
            ✓ entries() - dictionary
      Filtering
        ✓ filters by query path on list of all primitive types
        ✓ filters by query path on nested list of all primitive types
        ✓ filters by query path on dictionary of all primitive types
        ✓ filters by query path on nested dictionary of all primitive types
      Invalid operations
        ✓ throws when creating a Mixed with a set
        ✓ throws when creating a set with a list
        ✓ throws when creating a set with a dictionary
        ✓ throws when updating a list item to a set
        ✓ throws when updating a dictionary entry to a set
        ✓ throws when creating a list or dictionary with an embedded object
        ✓ throws when setting a list or dictionary item to an embedded object
        ✓ throws when setting a list or dictionary outside a transaction
        ✓ throws when setting a list item out of bounds
        ✓ throws when setting a nested list item out of bounds
        ✓ throws when assigning to list snapshot (Results)
        ✓ invalidates the list when removed
        ✓ invalidates the dictionary when removed

  Observable
    Collections in Mixed
      Collection notifications
        List
          ✓ fires when inserting, updating, and deleting at top-level
          ✓ fires when inserting, updating, and deleting in nested list
          ✓ fires when inserting, updating, and deleting in nested dictionary
          ✓ does not fire when updating object at top-level
        Dictionary
          ✓ fires when inserting, updating, and deleting at top-level
          ✓ fires when inserting, updating, and deleting in nested list
          ✓ fires when inserting, updating, and deleting in nested dictionary
          ✓ does not fire when updating object at top-level
      Object notifications
        ✓ fires when inserting, updating, and deleting in top-level list
        ✓ fires when inserting, updating, and deleting in nested list
        ✓ fires when inserting, updating, and deleting in top-level dictionary
        ✓ fires when inserting, updating, and deleting in nested dictionary
        ✓ fires when inserting, updating, and deleting in nested dictionary (using key-path)

Brief overview of usage

class CustomObject extends Realm.Object {
  value!: Realm.Types.Mixed;

  static schema: ObjectSchema = {
    name: "CustomObject",
    properties: {
      value: "mixed",
    },
  };
}

const realm = await Realm.open({ schema: [CustomObject] });

// Create an object with a dictionary value as the Mixed
// property, containing primitives and a list.
const realmObject = realm.write(() => {
  return realm.create(CustomObject, {
    value: {
      num: 1,
      string: "hello",
      bool: true,
      list: [
        {
          string: "world",
        },
      ],
    },
  });
});

// Accessing the collection value returns the managed collection.
const dictionary = realmObject.value;
expectDictionary(dictionary);
const list = dictionary.list;
expectList(list);
const leafDictionary = list[0];
expectDictionary(leafDictionary);
console.log(leafDictionary.string); // "world"

// Update the Mixed property to a list.
realm.write(() => {
  realmObject.value = [1, "hello", { newKey: "new value" }];
});

// Useful custom helper functions. (Will be provided in a future release.)
function expectList(value: unknown): asserts value is Realm.List {
  if (!(value instanceof Realm.List)) {
    throw new Error("Expected a 'Realm.List'.");
  }
}
function expectDictionary(value: unknown): asserts value is Realm.Dictionary {
  if (!(value instanceof Realm.Dictionary)) {
    throw new Error("Expected a 'Realm.Dictionary'.");
  }
}

☑️ ToDos

📝 Changelog entry
🚦 Tests

CHANGELOG.md

* Differentiate use of 'mixedToBinding()' for query arg validation. * Refactor 'mixedFromBinding()' to own function and account for List and Dictionary. * Implement setting a flat list and dictionary in Mixed. * Implement accessing a flat list and dictionary in Mixed. * Add tests for storing and accessing flat lists and dictionaries in Mixed. * Refactor helper in test to not rely on collection position. * Add tests for Set in Mixed throwing. * Add tests for updating lists and dictionaries. * Add tests for removing items in lists and dictionaries. * Add tests for filtering lists and dictionaries by path. * Throw if adding a set via property accessors. * Group tests into separate sub-suites. * Guard for embedded objects being set as Mixed value. * Add tests for embedded objects in Mixed throwing. * Add more filtering tests. * Add 'at_keys' query tests to uncomment after Core bug fix. * Add tests for inserting into lists and dictionaries. * Add tests for notifications on lists. * Add tests for notifications on dictionaries. * Add tests for notifications on object when changed prop is list in mixed. * Add tests for invalidating old list and dictionary. * Minor updates to names. * Add tests for notifications on object when changed prop is dictionary in mixed. * Add tests for creating dictionary via object without prototype. * Add tests filtering by query path using IN. * Access array of changes only 1 time in notifications tests. * Remove unnecessary type assertion. * Update schema object names in tests. * Add link to Core bug. * Add tests for default list and dictionary in schema. * Add tests for setting lists and dictionaries outside transaction. * Add tests for spreading Realm and non-Realm objects as Dictionary. * Add unit tests for 'isPOJO()'. * Point to updated Core commit and enable related tests. * Wrap chai's 'instanceOf()' in custom helper to assert type. * Update helper function name to be consistent with other helpers. * Add internal docs for 'isQueryArg'. * Rename unit test file. * Refactor notification tests into 'observable.ts'. * Refactor notification tests to use test context. * Use named import of 'ObjectSchema'. * Group CRUD tests into subsuites.

…ions (#6513) * Move geospatial helper functions to related file. * Implement setting nested lists in Mixed. * Implement setting nested dictionaries in Mixed. * Implement getting nested lists in Mixed. * Implement getting nested dictionaries in Mixed. * Test creating and accessing nested lists and dicts. * Make previous flat collections tests use the new 'expect' function. * Test that max nesting level throws. * Delegate throwing when using a Set to 'mixedToBinding()'. * Implement setting nested collections on a dictionary via setter. * Test nested collections on dictionary via setter. * Minor update to names of tests. * Combine nested and flat collections tests into same suite. * Implement setting nested collections on a list via setter. * Test nested collections on list via setter. * Refactor common test logic to helper functions. * Optimize property setter for hot-path and use friendlier err msg. * Refactor test helper function to build collections of any depth. * Implement inserting nested collections on a list via 'push()'. * Test nested collections on a list via 'push()'. * Test updating dictionary entry to nested collections via setter. * Test updating nested list/dictionary item via setter. * Test removing items from collections via 'remove()'. * Test object notifications when modifying nested collections. * Group previous notification tests into one test. * Group collection notifications tests into 'List' and 'Dictionary'. * Test collection notifications when modifying nested collections. * Remove collections from test context. * Test filtering by query path on nested collections. * Align object schema property names in tests. * Test filtering with int at_type. * Implement setting nested collections on a dictionary via 'set()' overloads. * Test JS Array method 'values()'. * Test JS Array method 'entries()'. * Implement getting nested collections on dictionary 'values()' and 'entries()'. * Test 'values()' and 'entries()' on dictionary with nested collections. * Remove unnecessary 'fromBinding()' calls. * Refactor collection helpers from 'PropertyHelpers' into the respective collection file. * Introduce list/dict sentinels to circumvent extra Core access. * Rename getter to default. * Remove redundant 'snapshotGet'. * Add abstract 'get' and 'set' to 'OrderedCollection'. * Rename the collection helpers to 'accessor'. * Move tests into subsuites. * Fix 'Results.update()'. * Support nested collections in 'pop()', 'shift()', 'unshift()', 'splice()'. * Test list 'pop()'. * Test list 'shift()'. * Test list 'unshift()'. * Test list 'splice()'. * Return 'not found' for collections searched for in 'indexOf()'. * Test ordered collection 'indexOf()'. * Support list/dict sentinels in JSI. * Test references per access. * Enable skipped tests after Core bug fix. * Point to updated Core. * Fix accessor for non-Mixed top-level collection with Mixed items. * Enable and fix previously skipped test. * Update 'mixed{}'. * Update 'mixed<>'. * Remove now-invalidated test. * Remove unused injectable from Node bindgen template. * Replace if-statements with switch. * Add explicit Results and Set accessors for Mixed. * Adapt to change in Core treating missing keys as null in queries. * Rename insertion function. * Include tests of Dictionary property type with Mixed. * Test reassigning to new collection and self-assignment. * Test mixed * Update 'mixed[]'. * Test results accessor. * Update error messages. * Make accessor helpers an object field rather than spread. * Suggestions for "nested collections in mixed" (#6566) * Fix type bundling issue * Inline functions into "create*Accessor*" functions * Refactored typeHelpers out of accessors * Remove leftover 'Symbol_for' in node-wrapper template. * Test not invalidating new collection. * Remove test for max nesting level. The max nesting level in debug in Core has been updated to be the same as for release. * Remove reliance on issue-fix in certain tests. * Add key path test for object listener on mixed field. * Use '.values()' and '.entries()' in iteration. * Update comments. * Add CHANGELOG entry. --------- Co-authored-by: Kræn Hansen <kraen.hansen@mongodb.com>

* main: (22 commits) Prepare for vNext (#6677) [12.9.0] Bump version (#6676) Fix performance tests (#6665) Combined: Support `Mixed` data type with collections (#6613) Update expected error messages (#6674) Adding --latest-local to the baas test server CLI (#6673) Upgrade to Realm Core v14.7.0 (#6663) Upgrade @trunk/launcher to v1.3.1 to support Apple's versioning scheme for macOS (#6671) Prepare for vNext (#6669) [12.8.1] Bump version (#6668) Use unreleased core (#6667) Fix realm/react changelog (#6661) Fix GHA error when publishing package release (#6660) Prepend vNext to realm/react changelog. [realm-react-0.7.0] Bump version (#6658) Update OAuth2Helper to remove accessing the stitch prefix (#6659) Fix unresolvable links in API reference docs and remove re-exports (#6646) Prepare for vNext (#6645) [12.8.0] Bump version (#6643) Upgrade to HEAD of Realm Core's master (#6637) ... # Conflicts: # CHANGELOG.md # integration-tests/tests/src/tests/mixed.ts

elle-j added the no-jira-ticket Skip checking the PR title for Jira reference label Apr 12, 2024

cla-bot bot added the cla: yes label Apr 12, 2024

github-actions bot assigned elle-j Apr 12, 2024

elle-j requested review from kneth and kraenhansen April 12, 2024 20:52

elle-j marked this pull request as ready for review April 12, 2024 20:52

elle-j marked this pull request as draft April 12, 2024 21:47

kraenhansen marked this pull request as ready for review April 15, 2024 12:23

kraenhansen reviewed Apr 15, 2024

View reviewed changes

CHANGELOG.md Show resolved Hide resolved

kraenhansen approved these changes Apr 15, 2024

View reviewed changes

kneth approved these changes Apr 16, 2024

View reviewed changes

elle-j force-pushed the lj/collections-in-mixed branch from 85ca76a to ec9f30f Compare April 26, 2024 12:58

elle-j force-pushed the lj/collections-in-mixed branch 2 times, most recently from 1dc77cf to 6ddda98 Compare May 23, 2024 08:53

elle-j and others added 9 commits May 23, 2024 11:53

Temporarily comment out failing filtering tests from Core upgrade.

9d10375

Add links to Core issue.

b799b0c

Update example in CHANGELOG.

44e1d10

Update PR link in CHANGELOG to the combined PR.

a005679

Add more links to Core issue.

99e59cd

Injecting Symbol_for (#6616)

cec5fa1

Enable tests from Core fixes.

233948d

elle-j force-pushed the lj/collections-in-mixed branch from 6ddda98 to 233948d Compare May 23, 2024 09:54

elle-j merged commit 71b3cb7 into main May 23, 2024
32 checks passed

elle-j deleted the lj/collections-in-mixed branch May 23, 2024 10:47

kraenhansen mentioned this pull request Jun 12, 2024

Refactored type and property helpers #6721

Merged

github-actions bot locked as resolved and limited conversation to collaborators Jun 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Combined: Support `Mixed` data type with collections #6613

Combined: Support `Mixed` data type with collections #6613

Combined: Support Mixed data type with collections #6613

Combined: Support Mixed data type with collections #6613

Conversation

What, How & Why?

Test overview

Brief overview of usage

☑️ ToDos

Combined: Support `Mixed` data type with collections #6613

Combined: Support `Mixed` data type with collections #6613