[RFC] On Modernizing Meteor: RPC/Publication Story #13725
Replies: 9 comments 12 replies
-
|
First, super-raw thoughts (will have to go over it on the weekend and let it grow on me): I'm not fond of the proposed API. To me it is too radically different, we don't use anything remotely similar in the entire codebase and I haven't used it in anything else so it is super alien to me (the On a higher level I'm for upgrading the method to include validation, types, etc. Even going as far as opening it to other validation libraries is great. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing! While I'm not the biggest fan of decorators, this is a great discussion. If we were creating Meteor in 2025, how should we design publications and methods? We could improve the API (maintaining backward compatibility), but I think the key priorities should be better TypeScript support and client-side type safety. 1. meteor-rpcThe meteor-rpc package by @Grubba27 has an elegant approach. export const questionsModule = createModule('questions')
.addMethod('insertQuestion', InsertQuestionInputSchema, insertQuestion)
.addMethod('removeQuestion', QuestionIdInputSchema, removeQuestion)
.addPublication('findAll', z.void(), findAll)
.buildSubmodule();Methods and publications are simple async functions with Zod schemas: export const insertQuestion = async ({ description }: InsertQuestionInput) => {
const user = await getLoggedUser();
const question = {
description,
userId: user._id,
createdAt: new Date(),
};
return Questions.insertAsync(question);
};
export const removeQuestion = async ({ questionId }: QuestionIdInput) => {
return Questions.removeAsync(questionId);
};
export const findAll = () => Questions.find();The package introduces Modules and Submodules, a new concept for Meteor apps. Its nicest feature is enabling type safety on the client-side. 2. The React Router wayWe could also adopt a React Router-inspired approach: const methods = createMethodsRouter([
{
name: "questions/insert",
schema: InsertQuestionInputSchema,
resolver: insertQuestion,
},
{
name: "questions/remove",
schema: QuestionIdInputSchema,
resolver: removeQuestion,
},
]);
export { methods };Other Great Optionszodern:relay@zodern's Relay offers a nice API and type safety on the client-side, though it requires more setup with its Babel plugin and specific folder structure. jam:method@jamauro's package feels most native to Meteor and works nicely with other jam packages. The main drawback is the lack of client-side type safety. |
Beta Was this translation helpful? Give feedback.
-
|
I'm definitely in favor of improving methods and publications. I think the specific implementation could probably be bike shed until the cows come home. :) Fwiw, I think this should take a back seat to solving Meteor's scalability concerns and reliance on oplog. Having said that, here are some thoughts:
Regarding validation, I think this should be kept agnostic as in I think it would also be neat to have auto-generated docs with Regarding your specific proposal: I think people who like classes and decorators will love it. Others will hate it. :) IMO, Meteor should stick to what I'll call a "functional light" style. Keep things simple with functions and objects. Reach for classes internally when it makes sense for performance reasons. But the public api should mostly just be functions and namespaced objects, much as it is currently.
I think this can be achieved without TS in Meteor core. As I mentioned in the forums, Svelte moved from TS to JSDoc because you can still get all the same benefits as TS without its overhead. This is the goal Meteor core should strive for imo. I think Rich Harris summarizes this best:
|
Beta Was this translation helpful? Give feedback.
-
|
My two cents.
|
Beta Was this translation helpful? Give feedback.
-
|
On point 4, I think the best approach would be to extend the existing |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @Grubba27 for bringing this up. Any discussion to improve Meteor is surely welcomed. I think
The only downside which you all agree on is client-side type safety. This is where @jamauro If you can invest sometime into making that happen, I think you're package would be the clear winner. Once it's stable then we can integrate more of the ideas you speak of @Grubba27 like
As for the proposed API, I dunno. I don't like it but I don't hate it also. My preference would be to continue with one of the current solutions and improve upon it though that doesn't mean it's a bad idea. It can be continued in a community package. Meteor comes in all flavors and sizes and I'm continuously intrigued to discover the new ways that's it's used in. |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
First, regarding decorators - I don't mind them personally, but the TypeScript situation is quite messy. Some packages require old decorators, some packages require new ones... This is a problem for the Nest JS, MikroORM, and TypeORM communities in particular. I'd avoid a decorator-based solution for that reason. I think there's a non-controversial API design somewhere between jam:method and zodern:relay. I've also been thinking about trialling a package idea that has one API to serve both REST and Meteor method endpoints. You can see the discussion Jam linked to for more about that. I stopped looking into it recently because I want to try out https: 8000 //www.npmjs.com/package/express-zod-api to see if that's a good solution (though naturally that isn't compatible with jam:easy-schema), and in turn I'm waiting to see how that supports zod 4 (which greatly improves how metadata for JSON Schema and OpenAPI works). Still, you can see there's a sort of gap between jam:method and express-zod-api in terms of API complexity because if you want to fully support all things your typical OpenAPI documentation could use, you need to add extra properties to track output schemas, examples, different mime types etc - which you can also see in fastify's API. As for zodern:relay vs jam:method, IIRC zodern:relay assumes server-side execution only, where as jam:method assumes isomorphism is OK (as it's also expected to work with jam:offline I guess). I'd be greedy and try and get the best of both worlds. I think an official Meteor approach would need to sort of combine the two approaches, while also working in conjunction with an official Vite/SWC solution. Because zodern:relay already needs a special Vite plugin to work which I see as a maintenance risk (Zodern and jorgenvatle do a lot of good work, but this is like a sort of unplanned tangling of concerns due to an exceptional build situation, I don't think it's ideal) But a build system approach isn't necessary... we could just define stubs on the client using the same signature perhaps. That could be maybe "checked" using TypeScript, e.g. something like: // /imports/client.ts
import type { serverSideMethod } from '/imports/server'
import { createStub } from 'meteor/totally-not-jam-method-or-relay'
const clientSideStub = createStub<typeof serverSideMethod>(...) |
Beta Was this translation helpful? Give feedback.
-
|
Here are some more thoughts after reading the discussion so far. My assumptions:
I think this can all be accomplished with one feature: the Meteor bundler is smart enough to not send code to the client if it’s wrapped in IMO, it would align with the Meteor philosophy of doing work for app devs so they can keep things simple and allow for a great DX with no drawbacks.
I think Vite might even make this feature easier?
No builders, custom plugins, or restrictions on code organization :) Here’s what the export const setPrivate = createMethod({
name: ’todos.setPrivate’,
schema: /* schema based on the lib you prefer */,
serverOnly: true,
async run({ _id, isPrivate }) {
return Todos.updateAsync({ _id }, { $set: { isPrivate } });
}
});
Here’s what the export const setPrivate = server(async ({ _id, isPrivate }) => {
return Todos.updateAsync({ _id }, { $set: { isPrivate } });
});
|
Beta Was this translation helpful? Give feedback.
-
|
@nachocodoner That latter part sounds quite cool. Out of curiosity, is it possible to get that level of code dead code elimination to also apply to server-only imports too? Or does that run into issues with things like side-effectual imports? Or would we need to continue using dynamic imports like (see below)? if (Meteor.isServer) {
const mod = await import('./mymod.js')
} |
Beta Was this translation helpful? Give feedback.
-
|
Yes, but instead of using import { Meteor } from 'meteor/meteor';
/* server:start */
import OpenAI from 'openai';
/* server:end */
import { get } from 'lodash-es'; |
Beta Was this translation helpful? Give feedback.
-
|
Oh nice, this looks quite promising! |
Beta Was this translation helpful? Give feedback.
-
|
It looks very similar to the use server pragma from next – I would love to see it in core too |
Beta Was this translation helpful? Give feedback.
-
|
Incidentally Solid Start also uses 'use server'. For better or worse the concept is spreading! 😅 |
Beta Was this translation helpful? Give feedback.
-
|
By the way @Grubba27 I saw in a discussion on the forums that the new Galaxy uses meteor-rpc, and on #12180 I just skimmed over the previous feedback. I think that it’s unique in that it’s the one TRPC style approach that wouldn’t actually need the bundler or a vite plugin etc to do anything special, but I know the style is different to some people’s preferences, eg Akryum mentioned preferring ESM modules to the virtual modules made by meteor-rpc (though I know the latter is more powerful and enables new kinds of interactions). I wonder if that gap in DX preference can be bridged somehow? The actual There’s also the collection-centric way of modularising methods which jam:method uses but can become a bit risky when lots of packages are monkey-patching collections (just because as a developer you might not have visibility on the entire monkey-patched API that you’ve added to your project). I think the idea of a separate “service module/api” is a bit safer but ultimately there’s something nice about the idea of declaring everything related together in one place. Maybe there’s a way to start with something like the ValidatedMethod alternatives, or meteor-rpc + non-Zod support but in the small (eg just a single method), and then easily build up to more complex things as you go. Or that can be as simple as Alryum’s suggestion of just using Also, as an aside, I noticed meteor-rpc doesn’t use return type assertions, I’m a bit rusty but if you use them you can do stuff like: const mod = createModule(…)
mod.createMethod(…) // mod’s type includes the method
// looks something like
createMethod(…): assert this is X & …But it’s perhaps important to avoid overwhelming devs with too complex types, makes it hard for advanced users to debug the underlying packages. |
Beta Was this translation helpful? Give feedback.
-
IMO, the collection-centric way offers an unmatched DX, which should be a guiding principle for Meteor generally. So I’d propose we add collection features so that monkey-patching becomes unnecessary. For example, add |
Beta Was this translation helpful? Give feedback.
-
|
I think that mental model definitely works well. If it’s built in as a feature of collections it also would give meteor a way to warn users if they’re making a duplicate method declaration on the same collection. re schema, I was thinking about that regarding simple-schema, easy-schema and Zod (pmogollons:zod-schema). It’d be good for whatever agnostic validation (and OpenAPi schema?) generation APIs were offered by meteor to support both methods and collections (and everything else eg pub sub). Related but more off topic and philisophical, I think ItaloJS on the forums (and a core team member on LinkedIn?) mentioned a similar principle for the DB layer in the future, eg being able to support different kinds of collections with different kinds of data stores behind the scenes. Which would favour a modular, pluggable approach too. So at the top level you’d have a collection with schemas and methods, then at the bottom level you’d have whatever interact with the database, change streams or message busses, etc. Meteor just needs to provide stable APIs and nice DX for package authors, and provide some defaults while making it clear that eg Zod is one simple loosely-coupled package away. (which is a pain mid Zod v3 to v4 transition but the idea would be more to reduce what package authors need to consider) |
Beta Was this translation helpful? Give feedback.
-
|
I'm trying to wrap this discussion now, and I think we can get to a consensus... Most replies and the core team seem to prefer(along with the constraints and objectives mentioned before):
Functional lightI really liked this take from @jamauro, we all agreed on it being just functions and we can call objects(modules w/functions) inside those functions. It is simple but at the same time very elegant! So... no classes and decorators, which is fine. SimplicityThis is where I think I get most annoyed/bumped about it, the current proposed APIs, it should be as simple as writing a regular export const setPrivate = createMethod({
// for me, this part here is really weird.
// why do we need a name if we declare it in the `const` part and the file probably has `todo` in its name ?!
name: ’todos.setPrivate’,
schema: /* schema based on the lib you prefer */,
serverOnly: true,
async run({ _id, isPrivate }) {
return Todos.updateAsync({ _id }, { $set: { isPrivate } });
}
});
Another thing that somewhat relates to isomorphism, should we import directly our methods to be used in the client( export const setPrivate = Meteor.method(schema, async ({ _id, isPrivate }) => {}) // isomorphic
// server only, I'm still not sure where I would leave the server part, maybe we can decide the isomorphic part later...
export const setPrivateServerOnly = server(() => Meteor.method(schema, async ({ _id, isPrivate }) => {
return Todos.updateAsync({ _id }, { $set: { isPrivate } });
}))
// server only with pragmas:
const setPrivateOnlyOnServer = async ({ _id, isPrivate }) => {
"use server"; // this would translate into a Meteor.call in the client
return Todos.updateAsync({ _id }, { $set: { isPrivate } });
}
export const setPrivate = Meteor.method(schema, setPrivateOnlyOnServer)
// client
import { setPrivate } from "path/todos/setPrivate"
setPrivate(args) // this should have intellisenseNot sure where we could have the docs part too, as another argument, or make the Isomorphic codeThis, before our bundler efforts, would make me a bit worried. How do we teach people to not leak their own code? Now, with this new bundler, we can choose how to make our code isomorphic or not. I'm in the team to make it always isomorphic unless you define where it should live and we now can discuss which API we like the most, functions, or using pragmas: const logIfInServer = () => {
if (Meteor.isServer) {
console.log("Server!")
}
// other stuff
}
// is the same as:
const logIfInServer = () => {
server(() => console.log("Server!"))
// other stuff
}
// or the same as:
// the same behavior as described in: https://nextjs.org/docs/app/api-reference/directives/use-server#using-use-server-inline
const serverLog = () => {
"use server"
console.log("Server!")
}
const logIfInServer = () => {
serverLog()
// other stuff
}All are good options, although I prefer functions and/or the good old Runtime checks(certainty?!)I think we all agree that runtime certainty(soundness) is nice, but that has a cost, where we catch the errors and what we should do with them? Do we want to catch these errors at a function level or in a hook? We would run this runtime check both on the server and client, or only once? For simplicity, I would say that the Closing thoughtsLet me know what you guys think about this, if we can get some people onboard with all of this, when 3.4 hits with a new bundler I think we can start exploring these ideas. |
Beta Was this translation helpful? Give feedback.
-
Yeah it'd be great to not need to explicitly define the name and have some Meteor magic set it automatically. As I recall, with my package, I couldn't find a clean way to do this without either doing so explicitly or attaching the methods to the collection: https://github.com/jamauro/method#attach-methods-to-its-collection-optional I'm partial to attaching directly. Then it feels just like writing regular functions. You can see it in practice here: https://github.com/jamauro/easy-todos/blob/main/imports/api/todos/methods.js and here: https://github.com/jamauro/easy-todos/blob/main/imports/api/todos/collection.js Could be introduced as a convention over configuration, opinionated approach to working with Meteor methods. Regarding your ideal syntax: export const setPrivate = Meteor.method(schema, async ({ _id, isPrivate }) => {}) It looks very similar to the upcoming Svelte I'm not sure how I feel about the syntax. It looks pretty clean because the
I'm really not a fan of having special strings like this. I think wrapping in
Hmm, isn't it a must? I think the checks would run on both client and server if isomorphic, otherwise only server. I guess there could be a config to skip client side checks as an advanced option for people that want that.
I think it'd be nice if it didn't have to be a function when using either |
Beta Was this translation helpful? Give feedback.
-
|
I also want to chime in on the On one hand, every framework now seems to have their own A consequence though is a lot of those frameworks have a lot of extra compiler magic going on that might impact extensibility (not sure if better or worse). Same issue for using decorators as compiler hints, (Though in general I like a lot of what SvelteKit does) |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
While we are taking our roadmap, we recently discussed a bit on things we wished we had on Meteor or were modernized;
Although we do not currently have the time to work on it full-time, we wanted to measure the traction this kind of modernization would have and whether it makes sense.
The first piece I wanted to have a conversation/discussion about is our RPC/Publication story.
Our Methods and Publications API has remained unchanged for 10+ years. We've tried to improve it ([My 2022 PR] (#12180) and [Florians 2020 PR] (#11039)) but have never succeeded. Meanwhile, community packages like Meteor-RPC, Relay, and Jam's packages offer good solutions.
To get started, I want to lay out my constraints and objectives:
Constraints
Objectives
rails routes, I know we can usemethod_handlers), but with knowing what the method requires as params and returns, and if the person wants, some documentation, they could have easily – similar to what C# does with their swagger(Decorators generate the docs);/api-docsand would show all RPC and Publications and their respective parameters, what they return and some extra docs that we could add there;Meteor way™️ of creating projects–gravitate towards theConvention over configurationphilosophyWith that said, with our constraints, the solution I see will be in JS, and it would be something using Classes and decorators(before bringing the torches out, hear me out)
I have made a poc here in this gist; But the TL;DR; is:
This would translate to:
Note that the
thisfrom the class refers to thethisof the meteor method(I'm using it as a building block), you can think this class as a static class;This could be valid for publications; the primary win here is the docs we would get from this, which could be very nice, we could open for extensions and middleware.
For the avarage-non meteor user, I feel that it is easier to educate, you could say that this class is similar to your regular controller from Rails/Adonis or Laravel;
What do you guys think about it? I would be very happy to hear if you have a different idea that respects those constraints and tries to accomplish most of the objectives I've mentioned.
Beta Was this translation helpful? Give feedback.
All reactions