forEachUniquePair : How is the pair constructed ?

[CrommVardek]

Hello,

I've been modifying some of our constraints trying to optimize the performances behind it.

Without going into too much details about the domain (I've changed it to match your doc as close as possible), I moved from a foreach + groupBy with filters and then sorting into a forEachUniquePair.

However, since I modified constraint stream logic, unit test on this constraint does not pass, more importantly, sometimes the test pass, sometimes not, I'm starting to believe it is due to a misunderstanding of how forEachUniquePair is constructed.

Let's say we have a Set of 4 shifts : (A, B, C, D). How is the the produced result of forEachUniquePair on Shifts will look like ?

Option 1 :
(A, B), (A, C), (A, D), (B, A), (B, C), (B, D), (C, A), ...

Or option 2 :
(A, B), (A, C), (A, D), (B, C), (B, D), (C, D)

Basically my question is, does the combination takes into account the "order" or not ? Because I understood it as option 1, but maybe I'm wrong...

I've used it like that :

.forEachUniquePair(
Shift.class,
equal(Shift::getEmployee),
lessThan(Shift::getShiftDateDayIndex)
)

So, in my mind, let's say all shifts are assigned to the same employee on different dates : the pair A, B that might not pass the condition lessThan(Shift::getShiftDateDayIndex) is different from pair B, A that pass the condition and can be further evaluated.

Before the constraint was written like that :

    protected Constraint shiftsTooCloseV2(ConstraintFactory factory) {
        return factory.forEach(Shift.class)
                .filter(da -> da.getEmployee().getLevel() == 3)
                .groupBy(Shift::getEmployee, toList())
                .filter((Employee, daList) ->
                        {
                            //Sort daList by ShiftDateDayIndex
                            List<Shift> sDaList = daList.stream().sorted(Comparator.comparingLong(Shift::getShiftDateDayIndex)).collect(Collectors.toList());
                            for (int i=0; i< daList.size()-1; i++) {
                                if (sDaList.get(i+1).getIsInPlanningWindow()) {
                                    long delta = sDaList.get(i+1).getShiftDateDayIndex() - sDaList.get(i).getShiftDateDayIndex();
                                    if ( delta > 1 && delta < 5) {
                                        return true;
                                    } else if (delta == 1) {
                                        if (
                                                //CustomLogic
                                        ) {
                                            return true;
                                        }
                                    }
                                }
                            }
                            return false;
                        }
                )
                .penalizeBigDecimal(HardMediumSoftBigDecimalScore.ONE_HARD,
                        (e, daList ) -> {
                            long gap = 0;
                            List<Shift> sDaList = daList.stream().sorted(Comparator.comparingLong(Shift::getShiftDateDayIndex)).collect(Collectors.toList());
                            for (int i = 0; i < daList.size() - 1; i++) {
                                if (sDaList.get(i + 1).getIsInPlanningWindow()) {
                                    long delta = sDaList.get(i + 1).getShiftDateDayIndex() - sDaList.get(i).getShiftDateDayIndex();
                                    if (delta > 0 && delta < 5) {
                                        gap = sDaList.get(i + 1).getShiftDateDayIndex() - sDaList.get(i).getShiftDateDayIndex();
                                        break;
                                    }
                                }
                            }
                            double delta = 5 - gap; //gap; //5 - gap;
                            return new BigDecimal((int)(30000 * delta * delta * delta));
                        })
                .indictWith((e, processList) -> {
                    String str = "Employee " + e.getENumber();
                    return List.of(str);
                })
                .asConstraint("shiftsTooCloseV2-3days-e-H");
 }

New way :

protected Constraint shiftsTooCloseV2(ConstraintFactory factory) {
        return factory.forEachUniquePair(
                        Shift.class,
                        equal(Shift::getEmployee),
                        lessThan(Shift::getShiftDateDayIndex)
                )
                .filter((a, b) -> a.getEmployee().getLevel() == 3 && b.getIsInPlanningWindow())
                .filter((a, b) -> {
                    long d = b.getShiftDateDayIndex() - a.getShiftDateDayIndex();
                    if (d > 1 && d < 5) {
                        return true;
                    }
                    if (d == 1) {
                        boolean sunday = b.getShiftDate().getDate().getDayOfWeek() == DayOfWeek.SUNDAY;
                        boolean excludedGroups = //Some custom conditions

                        return sunday && !(excludedGroups);
                    }
                    return false;
                })
                .penalizeBigDecimal(HardMediumSoftBigDecimalScore.ONE_HARD, (a, b) -> {
                    long gap = b.getShiftDateDayIndex() - a.getShiftDateDayIndex();
                    double delta = 5 - gap;
                    return new BigDecimal((int) (30000 * delta * delta * delta));
                })
                .indictWith((a, b) -> List.of("Employee " + a.getEmployee().getENumber()))
                .asConstraint("shiftsTooCloseV2-3days-e-H");
    }

I think my question boils down to : Are the pair produced ordered or not ?

[triceo]

Hello @CrommVardek!

Pairs are considered equal if they contain the same elements, so (A,B) and (B,A) would be considered equal and therefore only one of them would be produced.

Constraint streams respect the order of entities given to it in the planning solution. Which means your ordering issue likely comes from there - are you sure you're not using HashSet or HashMap anywhere where it matters? These have a randomized iteration order, which trickles down to the solver as well.

[CrommVardek]

Not where it matters, no.

So it means that with lessThan(Shift::getShiftDateDayIndex) joiner in forEachUniquePair will result in different behavior depending on the order of the entities...

But in my unit test, it looks like this :

    @ParameterizedTest
    @MethodSource("groupsNotSaturdaySundayShifts")
    public void ShiftsTooClose_TwoShiftsWithin3daysForANotEmployeeInSaturdaySundayGroupsShouldPenalize(Group group) {
        Employee Employee = EmployeeRosterObjectsGeneratorForTests.getEmployee(10, 3, group, anchorWeComplet);
        var shiftDate = EmployeeRosterObjectsGeneratorForTests.getshiftDate(LocalDate.of(2025, 10, 4));
        Shift da = EmployeeRosterObjectsGeneratorForTests.getShift(true, true, Employee, shiftDate);
        var shiftDate2 = EmployeeRosterObjectsGeneratorForTests.getshiftDate(LocalDate.of(2025, 10, 5));
        Shift da2 = EmployeeRosterObjectsGeneratorForTests.getShift(true, true, Employee, shiftDate2);

        var EmployeeRoster = new EmployeeRosterBuilderForTestCases()
                .withEmployee(Employee)
                .withshiftDate(shiftDate)
                .withShift(da)
                .withshiftDate(shiftDate2)
                .withShift(da2)
                .getResult();

        constraintVerifier.verifyThat(EmployeeRosteringConstraintProviderV2::ShiftsTooCloseV2)
                .givenSolution(EmployeeRoster).penalizes();
    }

"with" methods in the builder just add the entity to a list.

So the order is always the same.

[triceo]

So it means that with lessThan(Shift::getShiftDateDayIndex) joiner in forEachUniquePair will result in different behavior depending on the order of the entities...

Yes. That is why - personally - I avoid forEachUniquePair. It seems like a simplifications at first, but considering how many things actually affect the outcome of that method call, it's really not. It causes unexpected behaviors; completely correct, but unexpected.

So the order is always the same.

Strange.

If you can produce code I can run, I can look into this. But the only real explanation here is that somewhere in there, a HashSet enters the picture. Since we haven't seen this issue in tests before, and this code gets some serious everyday use, my first thought is that the problem has to be in user code.

[CrommVardek]

Thanks, I got my answer from https://discord.com/channels/1413420192213631086/1414521616955605003/1474174604753899611

It seems to depend on the @PlanningId field, and that one is generated randomly for tests. Hence why the test sometimes pass, sometimes do not pass.

[triceo]

Makes sense. I did not mention @PlanningId, because it did not cross my mind that it could be randomly generated and therefore the nondeterminism would get in that way.