Support SQL window functions

[roji]

Both SQL Server and PostgreSQL seem to support window functions in a similar way - in at least some cases the syntax even seems to be identical. It could be nice to add support for this to EF Core.

A main question is how this would be expressed in LINQ, AFAIK there's no way to do these kinds of operations in memory.

NOTE: For an up-to-date design proposal for expressing window functions with LINQ, see #12747 (comment).

[Diaskhan]

1.How do u Imagine window functions ?
Have U some thoghts, code example ?

Like.addcolumn(WF,sum,[doc_id,id])

2.What if wf gonna be few ?
Because simple agrregate sum,count is not to hard to implement. But in wf we must consider all test case!

[roji]

Regarding the API, it's a good question and ideally there would be a generic (non-database!) LINQ way of expressing this.

Here's a totally incomplete API idea:

Employees.PartitionBy(emp => emp.Department, (emp, dept) => new { Id=emp.Id, Salary=emp.Salary, SalarySum=Math.Sum(dept.Select(emp => emp.Salary)) })

Basically you define two lambdas: the partition lambda (given an employee, returns the partition to which it belongs), and the projection lambda (given an employee and their department, projects with the new partition-calculated lambda). So the above would return an IEnumerable of an anonymous type containing each employee's Id, Salary and also their departments salary sum. One issue is that an in-memory implementation would recalculate the sum for each and every employee in the partition, which isn't very efficient.

This definitely does not cover everything that can be done with (PostgreSQL) window functions, but it's a start :)

Hopefully if the API is done correctly, there won't be a problematic limit on which window functions are supported.

[Diaskhan]

hmmmm. what about order by ?
postgre
SELECT depname, empno, salary, rank() OVER (PARTITION BY depname ORDER BY salary DESC) FROM empsalary;

ms sql

SELECT ROW_NUMBER() OVER(PARTITION BY PostalCode ORDER BY SalesYTD DESC) AS "Row Number",   
    p.LastName, s.SalesYTD, a.PostalCode  

[pmiddleton]

@roji - That syntax is very similar to what I'm working on for Pivot #11963.

Once I get that working maybe it won't be too bad to follow the same patter to get this added in. Of course I say that having look at this for 3 minutes :)

[roji]

@pmiddleton oh, interesting, I don't know SQL Server's PIVOT functionality at all, is there a lot of overlap? I wonder if it's worth looking at the two together...

[roji]

@Diaskhan I haven't looked into it yet, but maybe simply an overload with a third lambda parameter that provides the ORDER BY clause? This definitely needs to be thought out more thoroughly than my quick API proposals here.

[pmiddleton]

@roji - There is no overlap in end user functionality, but I believe the implementation in the EF backend will be similar.

[divega]

Just thinking about this and #13805 (comment), I came to realize that many scenarios in which LINQ's GroupBy does not translate well to SQL's GROUP BY because both rows and some computation of an aggregate or raking functions need to be returned are probably equivalent to SQL window functions.

[Jonatthu]

@divega Is this something still on progress or no action will be taken?

[roji]

This feature is on the backlog, which means it's not going to be included in 3.1. Planning for the version after that hasn't started yet, so we don't know yet if this feature will be included. However, as there's relatively little interest this is unlikely to be prioritized.