add support for generic callback hooks

This patch adds support for a generic callback system into the generator
config that allows controller implementors to specify some code that
should be injected at specific named points in a template. I expect that
eventually this generic hook system will be more useful, flexible and
extensible than the hodge-podge of custom callback methods and overrides
currently in the generator config.

The `pkg/generate/config.ResourceConfig` struct now has a `Hooks` field
of type `map[string]*HookConfig`, with the map keys being named hook
points, e.g. "sdk_update_pre_build_request".

There are two ways to inject code at hook points: inline and via a
template path. The inline method uses the `HookConfig.Code` field which
should contain the Go code that gets injected at a named hook point. The
`HookConfig.TemplatePath` field is used to refer to a template file at a
specific path. The template file is searched for in any of the
TemplateSet's base template paths.

Here's an example of a generator config snippet that uses the inline
code injection method (`HookConfig.Code`) to add a piece of custom code
to be executed in the sdk_update.go.tpl right before the code in the
resource manager's `sdkUpdate` method calls the
`newUpdateRequestPayload()` function:

```yaml
resources:
  Broker:
    hooks:
      sdk_update_pre_build_request:
        code: if err := rm.requeueIfNotRunning(latest); err != nil { return nil, err }
```

Here is the snippet from the templates/pkg/resource/sdk_update.go.tpl
file that shows how we can add these generic named hooks into our
templates at various places:

```
{{- if $hookCode := .CRD.HookCode "sdk_update_pre_build_request" }}
{{ $hookCode }}
{{- end }}
```

The controller implementor need only implement the little
`requeueIfNotRunning` function, with the function signature described
in the generator config code block. We no longer need to have function
signatures match for custom callback code since the function signature
for callback code is up to the dev writing the generator.yaml config
file.

Here is an example of the `HookConfig.TemplatePath` being used to refer
to a template file containing some code that is injected at a named hook
point:

```yaml
resources:
  Broker:
    hooks:
      sdk_update_pre_build_request:
        template_path: sdk_update_pre_build_request.go.tpl
```

A controller implementor would simply need to populate a
`sdk_update_pre_build_request.go.tpl` file with the code to be included
at the hook point.

This patch introduces the following hook points in the ACK controller
resource manager code paths:

* sdk_read_one_pre_build_request
* sdk_read_many_pre_build_request
* sdk_get_attributes_pre_build_request
* sdk_create_pre_build_request
* sdk_update_pre_build_request
* sdk_delete_pre_build_request
* sdk_read_one_post_request
* sdk_read_many_post_request
* sdk_get_attributes_post_request
* sdk_create_post_request
* sdk_update_post_request
* sdk_delete_post_request
* sdk_read_one_pre_set_output
* sdk_read_many_pre_set_output
* sdk_get_attributes_pre_set_output
* sdk_create_pre_set_output
* sdk_update_pre_set_output

The "pre_build_request" hooks are called BEFORE the call to construct
the Input shape that is used in the API operation and therefore BEFORE
any call to validate that Input shape.

The "post_request" hooks are called IMMEDIATELY AFTER the API operation
aws-sdk-go client call.  These hooks will have access to a Go variable
named `resp` that refers to the aws-sdk-go client response and a Go
variable named `respErr` that refers to any error returned from the
aws-sdk-go client call.

The "pre_set_output" hooks are called BEFORE the code that processes the
Outputshape (the pkg/generate/code.SetOutput function). These hooks will
have access to a Go variable named `ko` that represents the concrete
Kubernetes CR object that will be returned from the main method
(sdkFind, sdkCreate, etc). This `ko` variable will have been defined
immediately before the "pre_set_output" hooks as a copy of the resource
that is supplied to the main method, like so:

```go
	// Merge in the information we read from the API call above to the copy of
	// the original Kubernetes object we passed to the function
	ko := r.ko.DeepCopy()
```

Author: jaypipes

pkg/generate/ack/controller.go

@@ -122,6 +122,17 @@ func Controller(
 		return nil, err
 	}
 
+	// Hook code can reference a template path, and we can look up the template
+	// in any of our base paths...
+	controllerFuncMap["Hook"] = func(r *ackmodel.CRD, hookID string) string {
+		code, err := ResourceHookCode(templateBasePaths, r, hookID)
+		if err != nil {
+			// It's a compile-time error, so just panic...
+			panic(err)
+		}
+		return code
+	}
+
 	ts := templateset.New(
 		templateBasePaths,
 		controllerIncludePaths,

pkg/generate/ack/hook.go

@@ -0,0 +1,145 @@
+// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License"). You may
+// not use this file except in compliance with the License. A copy of the
+// License is located at
+//
+//     http://aws.amazon.com/apache2.0/
+//
+// or in the "license" file accompanying this file. This file is distributed
+// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+// express or implied. See the License for the specific language governing
+// permissions and limitations under the License.
+
+package ack
+
+import (
+	"bytes"
+	"fmt"
+	"io/ioutil"
+	"path/filepath"
+	ttpl "text/template"
+
+	ackmodel "github.com/aws-controllers-k8s/code-generator/pkg/model"
+	ackutil "github.com/aws-controllers-k8s/code-generator/pkg/util"
+)
+
+/*
+The following hook points are supported in the ACK controller resource manager
+code paths:
+
+* sdk_read_one_pre_build_request
+* sdk_read_many_pre_build_request
+* sdk_get_attributes_pre_build_request
+* sdk_create_pre_build_request
+* sdk_update_pre_build_request
+* sdk_delete_pre_build_request
+* sdk_read_one_post_request
+* sdk_read_many_post_request
+* sdk_get_attributes_post_request
+* sdk_create_post_request
+* sdk_update_post_request
+* sdk_delete_post_request
+* sdk_read_one_pre_set_output
+* sdk_read_many_pre_set_output
+* sdk_get_attributes_pre_set_output
+* sdk_create_pre_set_output
+* sdk_update_pre_set_output
+
+The "pre_build_request" hooks are called BEFORE the call to construct
+the Input shape that is used in the API operation and therefore BEFORE
+any call to validate that Input shape.
+
+The "post_request" hooks are called IMMEDIATELY AFTER the API operation
+aws-sdk-go client call.  These hooks will have access to a Go variable
+named `resp` that refers to the aws-sdk-go client response and a Go
+variable named `respErr` that refers to any error returned from the
+aws-sdk-go client call.
+
+The "pre_set_output" hooks are called BEFORE the code that processes the
+Outputshape (the pkg/generate/code.SetOutput function). These hooks will
+have access to a Go variable named `ko` that represents the concrete
+Kubernetes CR object that will be returned from the main method
+(sdkFind, sdkCreate, etc). This `ko` variable will have been defined
+immediately before the "pre_set_output" hooks as a copy of the resource
+that is supplied to the main method, like so:
+
+```go
+	// Merge in the information we read from the API call above to the copy of
+	// the original Kubernetes object we passed to the function
+	ko := r.ko.DeepCopy()
+```
+*/
+
+// ResourceHookCode returns a string with custom callback code for a resource
+// and hook identifier
+func ResourceHookCode(
+	templateBasePaths []string,
+	r *ackmodel.CRD,
+	hookID string,
+) (string, error) {
+	resourceName := r.Names.Original
+	if resourceName == "" || hookID == "" {
+		return "", nil
+	}
+	c := r.Config()
+	if c == nil {
+		return "", nil
+	}
+	rConfig, ok := c.Resources[resourceName]
+	if !ok {
+		return "", nil
+	}
+	hook, ok := rConfig.Hooks[hookID]
+	if !ok {
+		return "", nil
+	}
+	if hook.Code != nil {
+		return *hook.Code, nil
+	}
+	if hook.TemplatePath == nil {
+		err := fmt.Errorf(
+			"resource %s hook config for %s is invalid. Need either code or template_path",
+			resourceName, hookID,
+		)
+		return "", err
+	}
+	for _, basePath := range templateBasePaths {
+		tplPath := filepath.Join(basePath, *hook.TemplatePath)
+		if !ackutil.FileExists(tplPath) {
+			continue
+		}
+		tplContents, err := ioutil.ReadFile(tplPath)
+		if err != nil {
+			err := fmt.Errorf(
+				"resource %s hook config for %s is invalid: error reading %s: %s",
+				resourceName, hookID, tplPath, err,
+			)
+			return "", err
+		}
+		t := ttpl.New(tplPath)
+		if t, err = t.Parse(string(tplContents)); err != nil {
+			err := fmt.Errorf(
+				"resource %s hook config for %s is invalid: error parsing %s: %s",
+				resourceName, hookID, tplPath, err,
+			)
+			return "", err
+		}
+		var b bytes.Buffer
+		// TODO(jaypipes): Instead of nil for template vars here, maybe pass in
+		// a struct of variables?
+		if err := t.Execute(&b, nil); err != nil {
+			err := fmt.Errorf(
+				"resource %s hook config for %s is invalid: error executing %s: %s",
+				resourceName, hookID, tplPath, err,
+			)
+			return "", err
+		}
+		return b.String(), nil
+	}
+	err := fmt.Errorf(
+		"resource %s hook config for %s is invalid: template_path %s not found",
+		resourceName, hookID, *hook.TemplatePath,
+	)
+	return "", err
+}

pkg/generate/ack/hook_test.go

@@ -0,0 +1,65 @@
+// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License"). You may
+// not use this file except in compliance with the License. A copy of the
+// License is located at
+//
+//     http://aws.amazon.com/apache2.0/
+//
+// or in the "license" file accompanying this file. This file is distributed
+// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+// express or implied. See the License for the specific language governing
+// permissions and limitations under the License.
+
+package ack_test
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+
+	"github.com/aws-controllers-k8s/code-generator/pkg/generate/ack"
+	"github.com/aws-controllers-k8s/code-generator/pkg/testutil"
+)
+
+func TestResourceHookCodeInline(t *testing.T) {
+	assert := assert.New(t)
+	require := require.New(t)
+	basePaths := []string{}
+	hookID := "sdk_update_pre_build_request"
+
+	g := testutil.NewGeneratorForService(t, "mq")
+
+	crd := testutil.GetCRDByName(t, g, "Broker")
+	require.NotNil(crd)
+
+	// The Broker's update operation has a special hook callback configured
+	expected := `if err := rm.requeueIfNotRunning(latest); err != nil { return nil, err }`
+	got, err := ack.ResourceHookCode(basePaths, crd, hookID)
+	assert.Nil(err)
+	assert.Equal(expected, got)
+}
+
+func TestResourceHookCodeTemplatePath(t *testing.T) {
+	assert := assert.New(t)
+	require := require.New(t)
+	wd, _ := os.Getwd()
+	basePaths := []string{
+		filepath.Join(wd, "testdata", "templates"),
+	}
+	hookID := "sdk_delete_pre_build_request"
+
+	g := testutil.NewGeneratorForService(t, "mq")
+
+	crd := testutil.GetCRDByName(t, g, "Broker")
+	require.NotNil(crd)
+
+	// The Broker's delete operation has a special hook configured to point to a template.
+	expected := "// this is my template.\n"
+	got, err := ack.ResourceHookCode(basePaths, crd, hookID)
+	assert.Nil(err)
+	assert.Equal(expected, got)
+}

pkg/generate/ack/testdata/templates/sdk_delete_pre_build_request.go.tpl

@@ -0,0 +1 @@
+// this is my template.

pkg/generate/config/resource.go

@@ -29,7 +29,10 @@ type ResourceConfig struct {
 	// Found and other common error types for primary resources, and thus we
 	// need these instructions.
 	Exceptions *ExceptionsConfig `json:"exceptions,omitempty"`
-
+	// Hooks is a map, keyed by the hook identifier, of instructions for the
+	// the code generator about a custom callback hooks that should be injected
+	// into the resource's manager or SDK binding code.
+	Hooks map[string]*HooksConfig `json:"hooks"`
 	// Renames identifies fields in Operations that should be renamed.
 	Renames *RenamesConfig `json:"renames,omitempty"`
 	// ListOperation contains instructions for the code generator to generate
@@ -71,6 +74,37 @@ type ResourceConfig struct {
 	ShortNames []string `json:"shortNames,omitempty"`
 }
 
+// HooksConfig instructs the code generator how to inject custom callback hooks
+// at various places in the resource manager and SDK linkage code.
+//
+// Example usage from the AmazonMQ generator config:
+//
+// resources:
+//   Broker:
+//     hooks:
+//       sdk_update_pre_build_request:
+//        code: if err := rm.requeueIfNotRunning(latest); err != nil { return nil, err }
+//
+// Note that the implementor of the AmazonMQ service controller for ACK should
+// ensure that there is a `requeueIfNotRunning()` method implementation in
+// `pkg/resource/broker`
+//
+// Instead of placing Go code directly into the generator.yaml file using the
+// `code` field, you can reference a template file containing Go code with the
+// `template_path` field:
+//
+// resources:
+//   Broker:
+//     hooks:
+//       sdk_update_pre_build_update_request:
+//        template_path: templates/sdk_update_pre_build_request.go.tpl
+type HooksConfig struct {
+	// Code is the Go code to be injected at the hook point
+	Code *string `json:"code,omitempty"`
+	// TemplatePath is a path to the template containing the hook code
+	TemplatePath *string `json:"template_path,omitempty"`
+}
+
 // CompareConfig informs instruct the code generator on how to compare two different
 // two objects of the same type
 type CompareConfig struct {

pkg/generate/templateset/templateset.go

@@ -22,6 +22,8 @@ import (
 	ttpl "text/template"
 
 	"github.com/pkg/errors"
+
+	ackutil "github.com/aws-controllers-k8s/code-generator/pkg/util"
 )
 
 var (
@@ -83,7 +85,7 @@ func (ts *TemplateSet) Add(
 	var foundPath string
 	for _, basePath := range ts.baseSearchPaths {
 		path := filepath.Join(basePath, templatePath)
-		if fileExists(path) {
+		if ackutil.FileExists(path) {
 			foundPath = path
 			break
 		}
@@ -116,7 +118,7 @@ func (ts *TemplateSet) joinIncludes(t *ttpl.Template) error {
 	for _, basePath := range ts.baseSearchPaths {
 		for _, includePath := range ts.includePaths {
 			tplPath := filepath.Join(basePath, includePath)
-			if !fileExists(tplPath) {
+			if !ackutil.FileExists(tplPath) {
 				continue
 			}
 			if t, err = includeTemplate(t, tplPath); err != nil {
@@ -142,7 +144,7 @@ func (ts *TemplateSet) Execute() error {
 	for _, basePath := range ts.baseSearchPaths {
 		for _, path := range ts.copyPaths {
 			copyPath := filepath.Join(basePath, path)
-			if !fileExists(copyPath) {
+			if !ackutil.FileExists(copyPath) {
 				continue
 			}
 			b, err := byteBufferFromFile(copyPath)
@@ -194,9 +196,3 @@ func includeTemplate(t *ttpl.Template, tplPath string) (*ttpl.Template, error) {
 	}
 	return t, nil
 }
-
-// fileExists returns tTrue if the supplied file path exists, false otherwise
-func fileExists(path string) bool {
-	_, err := os.Stat(path)
-	return !os.IsNotExist(err)
-}