wildcard search in ?all

[sean-roberts]

So, trying to mimic some really expressive behavior that grafana uses. They allow you to track metrics with keys like: a.b.c.d.e and you can do analytics/querying based on searches of a.b.*.d.e. Note, using dots instead of slashes wouldn't change how this is implemented.

A real world scenario around this:

myapp.*.logins

where the convention used is myapp.version.logins.

That query (services.com/myapp.*.logins?all=true) allows you to get all login event metrics across all versions/deploys and can compare. A/B testing and a lot of things would be enabled by this.

I'd imagine we would just pass the raw string to the db adaptor and let them figure out how to parse/support it. For the file-db adaptor, should be a simple change of the filter to use a regex. Something along the lines of key.match(new RegExp('^' + options.pathname.replace('*','.*')))

[mxstbr]

Interesting idea, I like it! Instead of just going for starting-with filtering, this essentially adds support for all sorts of complex filters.

What about going for slashes as a delimiter of choice? /myapp/v2/logins is imo a much more readable and semantic URL than with dots, and we can probably leverage existing solutions (like express) for the pathname matching if people pass /myapp/*/logins?all=true?

I wonder if other databases support this...

[sean-roberts]

Yes, exactly. As far as the support from other db adapters, yes they most certainly do, it's just that they would need to do a string.replace('*', '{wildcard_supported_value}'). So for something like mysql it would be like string.replace('*', '%') and it should "just work"

The delimiter should be irrelevant because it's just a string key. I like the dot but it really doesn't matter for any implementation that I think is planned.

[mxstbr]

I like it, let's do it. Could you submit a PR against the flat-file-db adapter implementing this, and a PR against this repo verifying it works as expected under all circumstances and adding documentation for it in the writing-adapters.md section and the user guide?

How do we handle users doing e.g. this fetch('service.com/app/*/login/bla')? Should this throw? Save app//login/bla?

[sean-roberts]

I think it would save 'service.com/app/*/login/bla'. I think this style of querying, like before and after are only on stats queries like ?inc=false or ?all=true. And with that, the onus is on them to know that with the standard GET and POST they are incrementing - which is true for all other styles of querying.

[sean-roberts]

And yes, I will create a set of prs

[mxstbr]

The issue with saving '/app/*/login/bla' is that you then cannot request that specific record anymore, since requesting 'service.com/app/*/login/bla' will get you anything. Maybe we need a disabler for this edge case, kinda like ?inc=false, something like ?wildcard=false?

[sean-roberts]

Sure, I don't think that would be a bad option. Do you think that's a v1 thing to add or a wait until people actually have the need?

[mxstbr]

We definitely need to do this before adapters get big and we have 50 of them without this feature 😉