wip: feat: volume management

Author: runleveldev

create-a-container/README.md

@@ -7,13 +7,17 @@ A web application for managing LXC container creation, configuration, and lifecy
 ```mermaid
 erDiagram
     Node ||--o{ Container : "hosts"
+    Node ||--o{ Volume : "stores"
     Container ||--o{ Service : "exposes"
+    Container ||--o{ ContainerVolume : "mounts"
+    ContainerVolume }o--|| Volume : "references"
     
     Node {
         int id PK
         string name UK "Proxmox node name"
         string apiUrl "Proxmox API URL"
         boolean tlsVerify "Verify TLS certificates"
+        int placeholderCtId "VMID for volume storage"
         datetime createdAt
         datetime updatedAt
     }
@@ -34,6 +38,27 @@ erDiagram
         datetime updatedAt
     }
     
+    Volume {
+        int id PK
+        string name "User-friendly name"
+        string username "Owner username"
+        string proxmoxVolume "Proxmox reference"
+        int sizeGb "Size in GB"
+        int siteId FK "References Site"
+        int nodeId FK "References Node"
+        datetime createdAt
+        datetime updatedAt
+    }
+    
+    ContainerVolume {
+        int id PK
+        int containerId FK "References Container"
+        int volumeId FK "References Volume"
+        string mountPath "Mount point path"
+        datetime createdAt
+        datetime updatedAt
+    }
+    
     Service {
         int id PK
         int containerId FK "References Container"
@@ -51,6 +76,9 @@ erDiagram
 - `(Node.name)` - Unique
 - `(Container.hostname)` - Unique
 - `(Container.nodeId, Container.containerId)` - Unique (same VMID can exist on different nodes)
+- `(Volume.username, Volume.name, Volume.siteId)` - Unique (volume names unique per user per site)
+- `(ContainerVolume.containerId, ContainerVolume.volumeId)` - Unique (one attachment per volume per container)
+- `(ContainerVolume.containerId, ContainerVolume.mountPath)` - Unique (mount paths unique per container)
 - `(Service.externalHostname)` - Unique when type='http'
 - `(Service.type, Service.externalPort)` - Unique when type='tcp' or type='udp'
 
@@ -59,6 +87,7 @@ erDiagram
 - **User Authentication** - Proxmox VE authentication integration
 - **Container Management** - Create, list, and track LXC containers
 - **Docker/OCI Support** - Pull and deploy containers from Docker Hub, GHCR, or any OCI registry
+- **Persistent Volumes** - Named volumes that survive container deletion for data persistence
 - **Service Registry** - Track HTTP/TCP/UDP services running on containers
 - **Dynamic Nginx Config** - Generate nginx reverse proxy configurations on-demand
 - **Real-time Progress** - SSE (Server-Sent Events) for container creation progress
@@ -407,6 +436,75 @@ SELECT id, status FROM Jobs WHERE id = <ID>;
 - Add batching or file-based logs for high-volume output to reduce DB pressure
 - Implement job timeout/deadline and automatic cancellation
 
+### Volume Management Routes
+
+#### `GET /sites/:siteId/volumes` (Auth Required)
+List all volumes owned by the authenticated user in a site
+- **Returns**: HTML page with volume list
+
+#### `GET /sites/:siteId/volumes/new` (Auth Required)
+Display volume creation form
+- **Returns**: HTML page with form
+
+#### `POST /sites/:siteId/volumes` (Auth Required)
+Create a new persistent volume
+- **Body**: `{ name, nodeId }`
+  - `name`: Volume name (alphanumeric, dash, underscore only)
+  - `nodeId`: Node where the volume should be created
+- **Process**: 
+  1. Allocates disk on the node's storage
+  2. Attaches to the node's placeholder container
+  3. Creates Volume record in database
+- **Returns**: Redirect to volumes list
+
+#### `DELETE /sites/:siteId/volumes/:id` (Auth Required)
+Delete a volume permanently
+- **Path Parameter**: `id` - Volume database ID
+- **Authorization**: User can only delete their own volumes
+- **Validation**: Volume must not be attached to any container
+- **Process**:
+  1. Detaches from placeholder container
+  2. Deletes disk from Proxmox storage
+  3. Removes Volume record from database
+- **Returns**: `{ success: true, message: "Volume deleted successfully" }`
+- **Errors**:
+  - `400` - Volume is currently attached to a container
+  - `403` - User doesn't own the volume
+  - `404` - Volume not found
+
+### Volume Attachment
+
+Volumes can be attached to containers during container creation:
+
+#### During `POST /sites/:siteId/containers`
+- **Additional Body Fields**:
+  - `volumes`: Array of volume attachments
+    - `volumes[N][volumeId]`: Volume ID to attach
+    - `volumes[N][mountPath]`: Mount point inside container (e.g., `/data`)
+- **Process**:
+  1. Validates all volumes exist and are owned by user
+  2. Validates volumes are on the same node as the target container
+  3. Creates container and ContainerVolume records
+  4. Job runner moves volumes from placeholder to new container
+- **Note**: Cross-node volume attachment requires manual migration
+
+#### During `DELETE /sites/:siteId/containers/:id`
+When a container with attached volumes is deleted:
+1. All attached volumes are transferred to the placeholder container
+2. ContainerVolume records are deleted
+3. Volume records are preserved (data persists)
+4. Volumes can be reattached to new containers
+
+### Placeholder Container
+
+Each Proxmox node has a "placeholder container" for volume storage:
+
+- **Purpose**: Holds volumes not attached to user containers
+- **Auto-creation**: Created when node is registered
+- **Configuration**: Minimal Alpine, 16MB RAM, no network, protection enabled
+- **VMID**: Stored in `Node.placeholderCtId`
+- **Never started**: Exists only to own volumes
+
 ### Configuration Routes
 
 #### `GET /nginx.conf`

create-a-container/bin/create-container.js

@@ -29,7 +29,7 @@ const https = require('https');
 
 // Load models from parent directory
 const db = require(path.join(__dirname, '..', 'models'));
-const { Container, Node, Site } = db;
+const { Container, Node, Site, Volume, ContainerVolume } = db;
 
 // Load utilities
 const { parseArgs } = require(path.join(__dirname, '..', 'utils', 'cli'));
@@ -385,6 +385,99 @@ async function main() {
     await container.update({ containerId: vmid });
     console.log(`Container VMID ${vmid} stored in database`);
 
+    // Attach volumes if any were requested
+    const volumeAttachments = await ContainerVolume.findAll({
+      where: { containerId: container.id },
+      include: [{ 
+        model: Volume, 
+        as: 'volume',
+        include: [{ model: Node, as: 'node' }]
+      }]
+    });
+    
+    if (volumeAttachments.length > 0) {
+      console.log(`Attaching ${volumeAttachments.length} volume(s)...`);
+      
+      for (const attachment of volumeAttachments) {
+        const volume = attachment.volume;
+        const mountPath = attachment.mountPath;
+        
+        console.log(`  Attaching volume "${volume.name}" at ${mountPath}`);
+        
+        // Check if volume is on same node
+        if (volume.nodeId !== node.id) {
+          console.log(`    Volume is on different node (${volume.node.name}), migrating to ${node.name}...`);
+          
+          // Get the source node
+          const sourceNode = await Node.findByPk(volume.nodeId);
+          if (!sourceNode || !sourceNode.placeholderCtId) {
+            throw new Error(`Source node for volume "${volume.name}" not found or has no placeholder`);
+          }
+          
+          // Get API client for source node
+          const sourceClient = await sourceNode.api();
+          
+          // Find the volume on source placeholder
+          const sourceMp = await sourceClient.findMountPointForVolume(sourceNode.name, sourceNode.placeholderCtId, volume.proxmoxVolume);
+          if (!sourceMp) {
+            throw new Error(`Volume "${volume.name}" not found on source placeholder container`);
+          }
+          
+          // Strategy: Move volume to a temporary minimal container, migrate it, then extract
+          // For now, we'll use a simpler approach: create the volume fresh on target and warn about data loss
+          // TODO: Implement proper storage-level migration when Proxmox supports it better
+          
+          // Alternative: Use pct move command with --target-node option (requires shared storage)
+          // For local storage, we need to:
+          // 1. Create a temp container with just this volume on source
+          // 2. Migrate the temp container to target
+          // 3. Move volume from temp to target placeholder
+          // 4. Delete temp container
+          
+          // Check if target node has placeholder
+          if (!node.placeholderCtId) {
+            throw new Error(`Target node ${node.name} does not have a placeholder container configured`);
+          }
+          
+          // For MVP: Use backup/restore approach through shared storage or error out
+          // This is complex and depends on infrastructure setup
+          throw new Error(
+            `Cross-node volume migration for "${volume.name}" requires manual intervention. ` +
+            `Volume is on node "${sourceNode.name}" but container is being created on "${node.name}". ` +
+            `Please create the container on the same node as the volume, or migrate the volume manually using Proxmox.`
+          );
+        }
+        
+        // Find the mount point on the placeholder container
+        const placeholderCtId = node.placeholderCtId;
+        if (!placeholderCtId) {
+          throw new Error(`Node ${node.name} does not have a placeholder container configured`);
+        }
+        
+        const sourceMp = await client.findMountPointForVolume(node.name, placeholderCtId, volume.proxmoxVolume);
+        if (!sourceMp) {
+          throw new Error(`Volume "${volume.name}" not found on placeholder container`);
+        }
+        
+        // Find next available mount point on target container
+        const targetMp = await client.findNextMountPoint(node.name, vmid);
+        
+        // Move volume from placeholder to new container
+        console.log(`    Moving ${sourceMp} from placeholder CT ${placeholderCtId} to ${targetMp} on CT ${vmid}`);
+        const moveUpid = await client.moveVolume(node.name, placeholderCtId, sourceMp, vmid, targetMp);
+        await client.waitForTask(node.name, moveUpid);
+        
+        // Update the mount path on the target container
+        await client.updateLxcConfig(node.name, vmid, {
+          [targetMp]: `${volume.proxmoxVolume},mp=${mountPath}`
+        });
+        
+        console.log(`    Volume "${volume.name}" attached at ${mountPath}`);
+      }
+      
+      console.log('All volumes attached successfully');
+    }
+    
     // Start the container
     console.log('Starting container...');
     const startUpid = await client.startLxc(node.name, vmid);