Replies: 1 comment 3 replies
-
|
Feldera is really optimized for standing queries, which are executed repeatedly for many input changes. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks Mihai! It sounds like I should experiment with representing the varying parameters as data in a separate table and modify the original user query to join against this system table. I'll give this some thought. |
Beta Was this translation helpful? Give feedback.
-
|
@sorenbs that's exactly what I was about to suggest. The parameters would end up as data in separate a table (or tables). |
Beta Was this translation helpful? Give feedback.
-
|
that may work, but there's a caveat: you will either need to delete entries that you don't care about anymore - and then you will see the corresponding outputs also deleted, or the table holding these entries that you use as queries may grow unbounded. DBSP supports an alternative mechanism, a "half-streaming join" that could solve this use case, but we don't currently expose it to user programs in any way. If you can describe your needs in a bit more detail maybe we can work out the right API. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
How well does Feldera handle millions of queries with slightly different parameters?
Ideally I would like to use SQL to construct these queries.
The queries have to be added dynamically at runtime.
Beta Was this translation helpful? Give feedback.
All reactions