Limit Spark driver memory to 512m in tests to prevent OOM

The Spark JVM defaults to grabbing ~1GB+ of heap memory. With 2 xdist
workers each spawning Spark, this exceeds the 7GB available on GitHub
Actions runners. Capping spark.driver.memory at 512m prevents the
"Cannot allocate memory" failures in test_pyspark_python_model and
the spark db_api tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: eakmanrq <6326532+eakmanrq@users.noreply.github.com>

Author: eakmanrq

tests/core/test_test.py

@@ -1722,6 +1722,7 @@ def test_pyspark_python_model(tmp_path: Path) -> None:
     spark_connection_config = SparkConnectionConfig(
         config={
             "spark.master": "local",
+            "spark.driver.memory": "512m",
             "spark.sql.warehouse.dir": f"{tmp_path}/data_dir",
             "spark.driver.extraJavaOptions": f"-Dderby.system.home={tmp_path}/derby_dir",
         },

tests/engines/spark/conftest.py

@@ -9,6 +9,7 @@ def spark_session() -> t.Generator[SparkSession, None, None]:
     session = (
         SparkSession.builder.master("local")
         .appName("SQLMesh Test")
+        .config("spark.driver.memory", "512m")
         .enableHiveSupport()
         .getOrCreate()
     )