Proposal: Add `list` module to std for Array/List operations

[sunyeongchoi]

As modern OLAP engines (such as DuckDB, ClickHouse, BigQuery, and PostgreSQL) increasingly support complex data types like Arrays and Lists, there is a growing need for a standardized way to handle these types in PRQL.

Currently, PRQL has a dedicated text module for string manipulations. I propose introducing a list (or array) module to provide a consistent and intuitive API for common list operations, such as checking for the existence of elements.

I suggest adding a list module to the standard library with the following functions. These functions will allow users to perform membership tests on arrays easily.

function	parameters	description
has	value arr	Returns true if arr contains value
has_any	values arr	Returns true if arr contains any value from values
has_all	values arr	Returns true if arr contains all values from values

from employees
derive {
  has_title = ([title, "CEO"] | list.has title),
  has_any_role = ([title, country] | list.has_any [country, "CEO"]),
}

I am happy to work on the implementation and the necessary compiler mappings for various dialects if the community agrees with this direction. What do you think?

[vanillajonathan]

How to check if a list contains a entry that starts with or ends with something?

.NET have the Any and All methods which takes an expression as the parameter:

bool hasBanana = fruits.Any(f => f == "Banana");
bool endsWithBerry = fruits.Any(f => f.EndsWith("berry"));

[max-sixty]

Any function that changes the shape of the data needs a great deal of thought, as it affects types. Even though we don't really track the concept of a list in our existing type system, that's because it's not fully developed.

so I wouldn't start here!

[sunyeongchoi]

Thanks for the feedback, @max-sixty. I understand the perspective.

I'm curious if there are any plans to introduce list or array-related functions to the PRQL standard library in the future? It would be great to know the project's direction on this.