chore(tests): add an end to end test that uses real terraform plan output #13

zpratt · 2025-11-02T16:28:06Z

chore(end-to-end-tests): add a full end to end test that uses real terraform plan output
fix: use a pinned version of terraform

…rraform plan output

e2e/terraform-integration.test.ts

Copilot

Pull Request Overview

This PR adds end-to-end integration testing for Terraform using moto (AWS API mocking) and testcontainers. The changes enable testing of the infra-diff functionality against real Terraform plan outputs in an isolated environment.

Implements Terraform integration testing with moto server running in Docker containers
Adds development container configuration for consistent local development
Configures CI pipeline to run Terraform integration tests with proper version pinning
Introduces testcontainers library to manage Docker containers programmatically

Reviewed Changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
package.json	Added testcontainers dependency and new npm script for terraform e2e tests
package-lock.json	Lock file updates for testcontainers and its transitive dependencies
features/end-to-end-with-terraform.md	Feature documentation describing the test goals and requirements
e2e/terraform-integration.test.ts	New e2e test file implementing Terraform plan generation and parsing
e2e/terraform/*.tf	Terraform fixtures for testing (provider config, backend, SQS resource)
docker-compose.yml	Docker compose file for local moto server execution
.terraform-version	Terraform version pinning file
.github/workflows/ci.yml	Added new CI job for Terraform integration tests
.github/dependabot.yml	Added Terraform dependency monitoring
.devcontainer/devcontainer.json	Development container configuration

Copilot · 2025-11-02T16:31:21Z

package.json

 		"@vercel/ncc": "^0.38.4",
 		"@vitest/ui": "^4.0.6",
 		"chance": "^1.1.13",
+		"testcontainers": "11.7.2",


The testcontainers dependency should use a caret (^) version constraint instead of an exact version pin. This aligns with the project's dependency convention seen in other devDependencies (e.g., 'vitest': '^4.0.6') and allows for automatic patch updates while maintaining compatibility.

Suggested change

"testcontainers": "11.7.2",

"testcontainers": "^11.7.2",

Copilot · 2025-11-02T16:31:22Z

e2e/terraform-integration.test.ts

+describe("E2E: Terraform Integration with moto", () => {
+	const terraformDir = path.join(process.cwd(), "e2e", "terraform");
+	const planFile = path.join(terraformDir, "plan.bin");
+	const planJsonFile = path.join(terraformDir, "plan.json");
+	let motoContainer: StartedTestContainer;
+	let motoPort: number;
+
+	beforeAll(async () => {
+		// Start moto server using testcontainers
+		console.log("Starting moto server...");
+		try {
+			motoPort = 50000;
+			motoContainer = await new GenericContainer("motoserver/moto:5.0.0")
+				.withExposedPorts({ container: 5000, host: motoPort })
+				.withStartupTimeout(120000)
+				.start();
+
+			console.log(`Moto server is ready on port ${motoPort}`);
+
+			// Initialize Terraform
+			console.log("Initializing Terraform...");
+			execSync("terraform init", { cwd: terraformDir, stdio: "pipe" });
+
+			// Generate plan
+			console.log("Generating Terraform plan...");
+			execSync(`terraform plan -out=${planFile}`, {
+				cwd: terraformDir,
+				stdio: "pipe",
+			});
+
+			// Convert plan to JSON
+			console.log("Converting plan to JSON...");
+			execSync(`terraform show -json ${planFile} > ${planJsonFile}`, {
+				cwd: terraformDir,
+				stdio: "pipe",
+				shell: "/bin/bash",
+			});
+		} catch (error) {
+			console.error("Setup failed:", error);
+			throw error;
+		}
+	}, 120000); // 2 minute timeout for setup
+


Using console.log for test output violates the operational convention stated in the coding guidelines: 'ensure all application logs are structured as JSON objects, except when logging to the console in GitHub Actions.' Since this is a test file, consider using the test framework's logging capabilities or a structured logger.

Suggested change

describe("E2E: Terraform Integration with moto", () => {

const terraformDir = path.join(process.cwd(), "e2e", "terraform");

const planFile = path.join(terraformDir, "plan.bin");

const planJsonFile = path.join(terraformDir, "plan.json");

let motoContainer: StartedTestContainer;

let motoPort: number;

beforeAll(async () => {

// Start moto server using testcontainers

console.log("Starting moto server...");

try {

motoPort = 50000;

motoContainer = await new GenericContainer("motoserver/moto:5.0.0")

.withExposedPorts({ container: 5000, host: motoPort })

.withStartupTimeout(120000)

.start();

console.log(`Moto server is ready on port ${motoPort}`);

// Initialize Terraform

console.log("Initializing Terraform...");

execSync("terraform init", { cwd: terraformDir, stdio: "pipe" });

// Generate plan

console.log("Generating Terraform plan...");

execSync(`terraform plan -out=${planFile}`, {

cwd: terraformDir,

stdio: "pipe",

});

// Convert plan to JSON

console.log("Converting plan to JSON...");

execSync(`terraform show -json ${planFile} > ${planJsonFile}`, {

cwd: terraformDir,

stdio: "pipe",

shell: "/bin/bash",

});

} catch (error) {

console.error("Setup failed:", error);

throw error;

}

}, 120000); // 2 minute timeout for setup

Copilot · 2025-11-02T16:31:22Z

e2e/terraform-integration.test.ts

+				.withStartupTimeout(120000)
+				.start();
+
+			console.log(`Moto server is ready on port ${motoPort}`);


Using console.log for test output violates the operational convention stated in the coding guidelines: 'ensure all application logs are structured as JSON objects, except when logging to the console in GitHub Actions.' Since this is a test file, consider using the test framework's logging capabilities or a structured logger.

Suggested change

console.log(`Moto server is ready on port ${motoPort}`);

console.log(JSON.stringify({ event: "moto_server_ready", port: motoPort }));

Copilot · 2025-11-02T16:31:22Z

e2e/terraform-integration.test.ts

+			console.log(`Moto server is ready on port ${motoPort}`);
+
+			// Initialize Terraform
+			console.log("Initializing Terraform...");


Using console.log for test output violates the operational convention stated in the coding guidelines: 'ensure all application logs are structured as JSON objects, except when logging to the console in GitHub Actions.' Since this is a test file, consider using the test framework's logging capabilities or a structured logger.

Suggested change

console.log("Initializing Terraform...");

console.log(JSON.stringify({ message: "Initializing Terraform..." }));

Copilot · 2025-11-02T16:31:22Z

e2e/terraform-integration.test.ts

+			execSync("terraform init", { cwd: terraformDir, stdio: "pipe" });
+
+			// Generate plan
+			console.log("Generating Terraform plan...");


Using console.log for test output violates the operational convention stated in the coding guidelines: 'ensure all application logs are structured as JSON objects, except when logging to the console in GitHub Actions.' Since this is a test file, consider using the test framework's logging capabilities or a structured logger.

Suggested change

console.log("Generating Terraform plan...");

console.log(JSON.stringify({ event: "Generating Terraform plan" }));

Copilot · 2025-11-02T16:31:24Z

e2e/terraform-integration.test.ts

+				unlinkSync(lockFile);
+			}
+		} catch (error) {
+			console.error("Failed to cleanup lock file:", error);


Using console.error violates the operational convention stated in the coding guidelines: 'ensure all application logs are structured as JSON objects, except when logging to the console in GitHub Actions.' Since this is a test file, consider using the test framework's logging capabilities or a structured logger.

Copilot · 2025-11-02T16:31:24Z

e2e/terraform-integration.test.ts

+		try {
+			if (motoContainer) {
+				await motoContainer.stop();
+				console.log("Moto container stopped");


Using console.log for test output violates the operational convention stated in the coding guidelines: 'ensure all application logs are structured as JSON objects, except when logging to the console in GitHub Actions.' Since this is a test file, consider using the test framework's logging capabilities or a structured logger.

Copilot · 2025-11-02T16:31:24Z

e2e/terraform-integration.test.ts

+				console.log("Moto container stopped");
+			}
+		} catch (error) {
+			console.error("Failed to stop moto container:", error);


Using console.error violates the operational convention stated in the coding guidelines: 'ensure all application logs are structured as JSON objects, except when logging to the console in GitHub Actions.' Since this is a test file, consider using the test framework's logging capabilities or a structured logger.

Copilot · 2025-11-02T16:31:24Z

e2e/terraform-integration.test.ts

+	beforeAll(async () => {
+		// Start moto server using testcontainers
+		console.log("Starting moto server...");
+		try {
+			motoPort = 50000;
+			motoContainer = await new GenericContainer("motoserver/moto:5.0.0")
+				.withExposedPorts({ container: 5000, host: motoPort })
+				.withStartupTimeout(120000)
+				.start();
+
+			console.log(`Moto server is ready on port ${motoPort}`);
+
+			// Initialize Terraform
+			console.log("Initializing Terraform...");
+			execSync("terraform init", { cwd: terraformDir, stdio: "pipe" });
+
+			// Generate plan
+			console.log("Generating Terraform plan...");
+			execSync(`terraform plan -out=${planFile}`, {
+				cwd: terraformDir,
+				stdio: "pipe",
+			});
+
+			// Convert plan to JSON
+			console.log("Converting plan to JSON...");
+			execSync(`terraform show -json ${planFile} > ${planJsonFile}`, {
+				cwd: terraformDir,
+				stdio: "pipe",
+				shell: "/bin/bash",
+			});
+		} catch (error) {
+			console.error("Setup failed:", error);
+			throw error;
+		}
+	}, 120000); // 2 minute timeout for setup


The beforeAll hook contains multiple complex setup steps (container startup, terraform init, plan, and JSON conversion) that violate the principle of single responsibility. Consider extracting these steps into separate, well-named helper functions to improve readability and maintainability.

Copilot · 2025-11-02T16:31:24Z

docker-compose.yml

@@ -0,0 +1,15 @@
+---
+version: '3.8'


The 'version' field in docker-compose.yml is deprecated as of Docker Compose v1.27.0+ and is no longer required. Consider removing this line as modern versions of Docker Compose ignore it.

Suggested change

version: '3.8'

Copilot

Pull Request Overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 6 comments.

Copilot · 2025-11-16T23:06:15Z

.github/workflows/ci.yml

+          node-version-file: ".nvmrc"
+      - uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
+        with:
+          terraform_version: 1.13.4


The Terraform version is hardcoded as "1.13.4" here, but it should be read from the .terraform-version file to maintain a single source of truth. Consider using terraform_version_file: ".terraform-version" instead of terraform_version: 1.13.4 to avoid version drift.

Suggested change

terraform_version: 1.13.4

terraform_version_file: ".terraform-version"

Copilot · 2025-11-16T23:06:15Z

.devcontainer/devcontainer.json

+	"features": {
+		"ghcr.io/devcontainers/features/docker-outside-of-docker:1": {},
+		"ghcr.io/devcontainers-contrib/features/terraform-asdf:2": {
+			"version": "1.13.4"


The Terraform version is duplicated here. It should be read from the .terraform-version file to maintain consistency. The devcontainer feature supports using a version file reference instead of hardcoding the version.

Suggested change

"version": "1.13.4"

"version": "file:.terraform-version"

Copilot · 2025-11-16T23:06:16Z

e2e/terraform-integration.test.ts

+			terraformContainer = await new GenericContainer(
+				`hashicorp/terraform:${terraformVersion}`,


The version string used in this template literal comes from reading the .terraform-version file, but there's no validation that the version format is correct or that the Docker image tag exists. Consider adding validation or error handling if the version is malformed or the image doesn't exist with that tag.

Copilot · 2025-11-16T23:06:16Z

e2e/terraform-integration.test.ts

+	afterAll(async () => {
+		// Cleanup
+		console.log("Cleaning up...");
+
+		// Have terraform container clean up its own files with proper permissions
+		try {
+			if (terraformContainer) {
+				console.log("Cleaning up terraform files from container...");
+				await terraformContainer.exec([
+					"sh",
+					"-c",
+					"rm -rf /terraform/.terraform /terraform/plan.bin /terraform/plan.json /terraform/.terraform.lock.hcl",
+				]);
+				console.log("Terraform files cleaned from container");
+			}
+		} catch (error) {
+			console.warn("Failed to cleanup from container:", error);
+		}
+
+		// Stop and remove terraform container
+		try {
+			if (terraformContainer) {
+				await terraformContainer.stop();
+				console.log("Terraform container stopped");
+			}
+		} catch (error) {
+			console.error("Failed to stop terraform container:", error);
+		}
+
+		// Stop and remove moto container
+		try {
+			if (motoContainer) {
+				await motoContainer.stop();
+				console.log("Moto container stopped");
+			}
+		} catch (error) {
+			console.error("Failed to stop moto container:", error);
+		}
+
+		// Stop and remove network
+		try {
+			if (network) {
+				await network.stop();
+				console.log("Network stopped");
+			}
+		} catch (error) {
+			console.error("Failed to stop network:", error);
+		}
+
+		// Remove plan files from host
+		try {
+			if (existsSync(planFile)) {
+				unlinkSync(planFile);
+			}
+			if (existsSync(planJsonFile)) {
+				unlinkSync(planJsonFile);
+			}
+		} catch (error) {
+			console.error("Failed to cleanup plan files:", error);
+		}
+
+		// Remove .terraform directory
+		try {
+			const terraformStateDir = path.join(terraformDir, ".terraform");
+			if (existsSync(terraformStateDir)) {
+				rmSync(terraformStateDir, { recursive: true, force: true });
+				console.log(".terraform directory cleaned up");
+			}
+		} catch (error) {
+			// Log error but don't fail - permissions issues in CI can be expected
+			console.warn("Failed to cleanup .terraform directory:", error);
+		}
+
+		// Remove lock file
+		try {
+			const lockFile = path.join(terraformDir, ".terraform.lock.hcl");
+			if (existsSync(lockFile)) {
+				unlinkSync(lockFile);
+			}
+		} catch (error) {
+			console.error("Failed to cleanup lock file:", error);
+		}


[nitpick] The cleanup logic in the afterAll hook has multiple try-catch blocks that swallow errors. While some failures during cleanup are acceptable (as noted in comments), the current approach makes it difficult to distinguish between expected and unexpected failures. Consider consolidating cleanup logic and using more specific error handling to ensure critical cleanup steps don't fail silently.

Copilot · 2025-11-16T23:06:16Z

features/end-to-end-with-terraform.md

@@ -0,0 +1,16 @@
+# End to End with Terraform
+
+Goal: Demonstrate how Infra Diff can be used in an end-to-end workflow with Terraform and moto to provide a fake of the AWS API. The goal is to produce a real binary terraform plan, convert it to json, and then run infra-diff against that json plan.


The feature description mentions using "moto to provide a fake of the AWS API" and creating infrastructure on AWS, but then describes using moto to "mock" the AWS API. The terminology is inconsistent. Consider using consistent terminology (either "mock" or "fake") throughout the document to avoid confusion.

Suggested change

Goal: Demonstrate how Infra Diff can be used in an end-to-end workflow with Terraform and moto to provide a fake of the AWS API. The goal is to produce a real binary terraform plan, convert it to json, and then run infra-diff against that json plan.

Goal: Demonstrate how Infra Diff can be used in an end-to-end workflow with Terraform and moto to mock the AWS API. The goal is to produce a real binary terraform plan, convert it to json, and then run infra-diff against that json plan.

Copilot · 2025-11-16T23:06:16Z

features/end-to-end-with-terraform.md

+## Requirements
+
+- Ensure the pipeline uses setup-terraform action to install terraform
+- Ensure we're testing with the latest version of terraform


The requirement states "Ensure we're testing with the latest version of terraform," but version 1.13.4 is specified. This creates ambiguity - should the version be pinned (as is done) or should it actually use the latest version? Consider clarifying this requirement to reflect the actual implementation (using a pinned version for consistency) rather than "latest."

Suggested change

- Ensure we're testing with the latest version of terraform

- Ensure we're testing with a pinned version of terraform (currently 1.13.4) for consistency and reproducibility

zpratt added 2 commits November 2, 2025 10:16

chore(end-to-end-tests): add a full end to end test that uses real te…

3c9fa30

…rraform plan output

fix: use a pinned version of terraform

8747342

Copilot AI review requested due to automatic review settings November 2, 2025 16:28

github-advanced-security bot found potential problems Nov 2, 2025

View reviewed changes

e2e/terraform-integration.test.ts Fixed Show fixed Hide fixed

e2e/terraform-integration.test.ts Fixed Show fixed Hide fixed

Copilot AI reviewed Nov 2, 2025

View reviewed changes

zpratt added 2 commits November 16, 2025 16:59

fix: run terraform from a container in e2e tests

1af27b4

fix: ensure no errors on cleanup of .terraform directory

df0e0ad

Copilot AI review requested due to automatic review settings November 16, 2025 23:03

Copilot started reviewing on behalf of zpratt November 16, 2025 23:03 View session

Copilot finished reviewing on behalf of zpratt November 16, 2025 23:05

Copilot AI reviewed Nov 16, 2025

View reviewed changes

	console.log(`Moto server is ready on port ${motoPort}`);
	console.log(JSON.stringify({ event: "moto_server_ready", port: motoPort }));

	console.log("Initializing Terraform...");
	console.log(JSON.stringify({ message: "Initializing Terraform..." }));

	console.log("Generating Terraform plan...");
	console.log(JSON.stringify({ event: "Generating Terraform plan" }));

	terraform_version: 1.13.4
	terraform_version_file: ".terraform-version"

		terraformContainer = await new GenericContainer(
		`hashicorp/terraform:${terraformVersion}`,

		@@ -0,0 +1,16 @@
		# End to End with Terraform

		Goal: Demonstrate how Infra Diff can be used in an end-to-end workflow with Terraform and moto to provide a fake of the AWS API. The goal is to produce a real binary terraform plan, convert it to json, and then run infra-diff against that json plan.

	- Ensure we're testing with the latest version of terraform
	- Ensure we're testing with a pinned version of terraform (currently 1.13.4) for consistency and reproducibility

chore(tests): add an end to end test that uses real terraform plan output #13

Are you sure you want to change the base?

chore(tests): add an end to end test that uses real terraform plan output #13

Uh oh!

Conversation

zpratt commented Nov 2, 2025

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Nov 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 16, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants