Test: Memory Pressure and Resource Exhaustion Scenarios

[nickna]

Description

Test database behavior under extreme memory pressure and resource constraints to ensure graceful degradation and recovery.

Phase

Phase 2: Large-Scale Stress Testing

Epic

Related to #202

Acceptance Criteria

• Test behavior when system memory is 90%+ utilized
• Verify graceful degradation when approaching memory limits
• Test garbage collection pressure with large datasets
• Validate memory-mapped file behavior under pressure
• Ensure database remains functional after memory pressure relief

Memory Pressure Scenarios

1. GC Pressure - Force frequent garbage collection during operations
2. Memory Fragmentation - Simulate fragmented heap conditions
3. Large Object Heap - Test with vectors that trigger LOH allocation
4. Memory-Mapped File Limits - Exhaust virtual memory space
5. Thread Stack Overflow - Deep recursion in search algorithms

Test Structure

[Test]
[Category("Stress")]
[Explicit("Memory pressure test - run manually")]
public async Task Database_UnderMemoryPressure_DegradeGracefully()
{
    // Arrange
    using var memoryPressureSimulator = new MemoryPressureSimulator();
    var database = new VectorDatabase();
    
    // Populate with moderately large dataset
    PopulateDatabase(database, vectorCount: 50000);
    
    // Act: Apply severe memory pressure
    memoryPressureSimulator.ApplyPressure(MemoryPressureLevel.Severe);
    
    var operations = new List<Task>();
    
    // Try various operations under pressure
    operations.Add(TestSearchOperations(database));
    operations.Add(TestIndexBuilding(database));
    operations.Add(TestVectorAddition(database));
    
    var completedOperations = 0;
    var failedOperations = 0;
    
    foreach (var operation in operations)
    {
        try
        {
            await operation.WaitAsync(TimeSpan.FromMinutes(5));
            completedOperations++;
        }
        catch (OutOfMemoryException)
        {
            failedOperations++;
            // Expected under severe pressure
        }
    }
    
    // Assert: Some operations should complete or fail gracefully
    Assert.That(completedOperations + failedOperations, Is.EqualTo(operations.Count));
    Assert.That(database.Count, Is.GreaterThan(0)); // Database should remain intact
    
    // Relief pressure and verify recovery
    memoryPressureSimulator.RelievePressure();
    GC.Collect();
    GC.WaitForPendingFinalizers();
    
    // Should be able to perform operations normally
    var query = CreateRandomQuery();
    var results = database.Search(query, 5, SearchAlgorithm.Linear);
    Assert.That(results.Count, Is.LessThanOrEqualTo(5));
}

Monitoring Tools

• Real-time memory usage tracking
• GC generation monitoring
• LOH allocation tracking
• Virtual memory usage