Add limited CrossAccount support to S3 changelog stack

elastio-s3-changelog/README.md

@@ -1,7 +1,7 @@
 # S3 Changelog
 
 For large buckets, scanning all objects might take a lot of time. This CloudFormation template enables
-the Changelog feature for S3 buckets, which significantly improves the scan performance after the initial
+the Changelog feature for S3 buckets, which improves the scan performance after the initial
 scan of the entire bucket is done.
 
 This template deploys an SQS queue and an EventBridge rule which sends S3 update events to the queue.
@@ -32,7 +32,16 @@ Then, the Elastio `iscan` job reads those events to perform the scanning of new
 3. Fill in the main parameters:
     * *BucketNames* - comma-separated list of S3 bucket names;
 
-    * *ScanExistingObjects* - set to `true` if you want to perform the initial scan of all objects in the bucket(s);
+    * *CrossAccount* - set this to true if the scan happens in a different AWS account than the
+      S3 bucket (i.e. you are using a centralized scanning account). If it is not set
+      to true in the cross-account case, an error about missing AWS Lambda functions
+      will occur during stack creation.
+
+    * *ScanExistingObjects* - if set to true, the full initial scan of the bucket will be performed.
+      If set to false, only new objects in the bucket will be scanned.
+      This option can only be enabled when CrossAccount is false. If it is not set
+      to false in the cross-account case, an error about missing AWS Lambda functions
+      will occur during stack creation.
 
     * *KeyPrefixes* - (optional) comma-separated list of prefixes of objects to scan. This will be applied to all buckets.
         If you want to use different prefixes for different buckets, you need to deploy multiple S3 Changelog stacks.

elastio-s3-changelog/cloudformation-multiple-buckets-with-macros.yaml

@@ -10,6 +10,9 @@ Transform:
 Parameters:
   BucketNames:
     Type: String
+  CrossAccount:
+    Type: String
+    Default: 'false'
   ScanExistingObjects:
     Type: String
     Default: 'false'
@@ -39,6 +42,7 @@ Resources:
         Properties:
           Parameters:
             BucketName: !Ref BucketName
+            CrossAccount: !Ref CrossAccount
             ScanExistingObjects: !Ref ScanExistingObjects
             KeyPrefixes: !Ref KeyPrefixes
             DisableEventCollection: !Ref DisableEventCollection

elastio-s3-changelog/cloudformation-multiple-buckets.yaml

@@ -12,13 +12,26 @@ Parameters:
       Comma-delimited list of S3 bucket names.
       Example: examplebucket1, examplebucket2
 
+  CrossAccount:
+    Type: String
+    AllowedValues: ['true', 'false']
+    Default: 'false'
+    Description: >
+      Set this to true if the scan happens in a different AWS account than the
+      S3 bucket (i.e. you are using a centralized scanning account). If it is not set
+      to true in the cross-account case, an error about missing AWS Lambda functions
+      will occur during stack creation.
+
   ScanExistingObjects:
     Type: String
     AllowedValues: ['true', 'false']
     Default: 'false'
     Description: >
       If set to true, the full initial scan of the bucket will be performed.
       If set to false, only new objects in the bucket will be scanned.
+      This option can only be enabled when CrossAccount is false. If it is not set
+      to false in the cross-account case, an error about missing AWS Lambda functions
+      will occur during stack creation.
 
   KeyPrefixes:
     Type: CommaDelimitedList
@@ -75,6 +88,7 @@ Metadata:
       - Label: { default: Main Parameters }
         Parameters:
           - BucketNames
+          - CrossAccount
           - ScanExistingObjects
           - KeyPrefixes
 
@@ -95,6 +109,7 @@ Resources:
     Properties:
       Parameters:
         BucketNames: !Join [',', !Ref BucketNames]
+        CrossAccount: !Ref CrossAccount
         ScanExistingObjects: !Ref ScanExistingObjects
         KeyPrefixes: !Join [',', !Ref KeyPrefixes]
         DisableEventCollection: !Ref DisableEventCollection

elastio-s3-changelog/cloudformation-single-bucket.yaml

@@ -9,13 +9,26 @@ Parameters:
     ConstraintDescription: Bucket name must be at least 3 characters long.
     Description: S3 bucket name.
 
+  CrossAccount:
+    Type: String
+    AllowedValues: ['true', 'false']
+    Default: 'false'
+    Description: >
+      Set this to true if the scan happens in a different AWS account than the
+      S3 bucket (i.e. you are using a centralized scanning account). If it is not set
+      to true in the cross-account case, an error about missing AWS Lambda functions
+      will occur during stack creation.
+
   ScanExistingObjects:
     Type: String
     AllowedValues: ['true', 'false']
     Default: 'false'
     Description: >
       If set to true, the full initial scan of the bucket will be performed.
       If set to false, only new objects in the bucket will be scanned.
+      This option can only be enabled when CrossAccount is false. If it is not set
+      to false in the cross-account case, an error about missing AWS Lambda functions
+      will occur during stack creation.
 
   KeyPrefixes:
     # Unfortunately, it's impossible to pass CommaDelimitedList to a nested stack,
@@ -74,6 +87,7 @@ Metadata:
       - Label: { default: Main Parameters }
         Parameters:
           - BucketName
+          - CrossAccount
           - ScanExistingObjects
           - KeyPrefixes
 
@@ -89,6 +103,7 @@ Metadata:
           - QrtsMaxDelay
 
 Conditions:
+  NotCrossAccount: !Equals [!Ref CrossAccount, 'false']
   ScanExistingObjects: !Equals [!Ref ScanExistingObjects, 'true']
   DisableEventCollection: !Equals [!Ref DisableEventCollection, 'true']
   EnableDlq: !Equals [!Ref EnableDlq, 'true']
@@ -216,6 +231,7 @@ Resources:
 
   queueRefreshSchedule:
     Type: AWS::Scheduler::Schedule
+    Condition: NotCrossAccount
     Properties:
       Description: A schedule to invoke the periodic changelog queue refresh lambda
       ScheduleExpression: cron(0/15 * * * ? *)

elastio-s3-changelog/version

@@ -1 +1 @@
-v1
+v2