Better error stacktraces for Cloud Errors

[hermanbanken]

Anyone is free to use our fork as long as this is not merged. Use this in your go.mod:

replace github.com/blendle/zapdriver => github.com/q42philips/zapdriver v1.3.2-pre.0

---

Stackdriver / GCP Cloud Errors is quite picky about the stacktraces it will parse. Specifically it does only parse the output of Go's runtime.Stack() which notably includes a goroutine \d [running]: part. Without those stack contents it does not parse.

This PR adapts zapdriver to include better stacktraces by default, when the error is reported to Cloud Errors (e.g. when the context and service name are set). It also uses the exception key instead of stacktrace because Cloud Error only contemplates the message, stack_trace and exception fields of the jsonPayload. The stacktrace key wasn't inspected at all, and even if it were, it contains the Zap formatted stack, which isn't compatible with Cloud Errors.

To view how this works, consider for example these different configurations:

logger, _ := NewProduction()
logger.Error("with exception", zap.Error(errors.New("internal error")), ErrorReport(runtime.Caller(0)))

logger, _ = NewProduction(WrapCore(ServiceName("service"), ReportAllErrors(true)))
logger.Error("with exception", zap.Error(errors.New("internal error")))

logger, _ = NewProduction(WrapCore(ServiceName("service"), SkipFmtStackTraces(true)))
logger.Error("without exception", zap.Error(errors.New("internal error")))

The output would be as follows:

{"severity":"ERROR","timestamp":"2023-02-24T16:52:31.025432+01:00","caller":"zapdriver/error_test.go:52","message":"with exception","exception":"internal error\n\ngoroutine 1 [running]:\ngithub.com/blendle/zapdriver.ExampleSkipFmtStackTraces()\n\tgithub.com/blendle/zapdriver/error_test.go:52 +0x102264700\ntesting.runExample()\n\ttesting/run_example.go:63 +0x102176560\ntesting.runExamples()\n\ttesting/example.go:44 +0x102172b20\ntesting.(*M).Run()\n\ttesting/testing.go:1728 +0x10217b460\nmain.main()\n\t_testmain.go:131 +0x10226aa60\nruntime.main()\n\truntime/proc.go:250 +0x1020e8c20\nruntime.goexit()\n\truntime/asm_arm64.s:1172 +0x102118b40","context":{"reportLocation":{"filePath":"github.com/blendle/zapdriver/error_test.go","lineNumber":"52","functionName":"github.com/blendle/zapdriver.ExampleSkipFmtStackTraces"}},"logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"github.com/blendle/zapdriver/error_test.go","line":"52","function":"github.com/blendle/zapdriver.ExampleSkipFmtSta
ckTraces"}}
{"severity":"ERROR","timestamp":"2023-02-24T16:52:31.025612+01:00","caller":"zapdriver/error_test.go:55","message":"with exception","exception":"internal error\n\ngoroutine 1 [running]:\ngithub.com/blendle/zapdriver.ExampleSkipFmtStackTraces()\n\tgithub.com/blendle/zapdriver/error_test.go:55 +0x102264700\ntesting.runExample()\n\ttesting/run_example.go:63 +0x102176560\ntesting.runExamples()\n\ttesting/example.go:44 +0x102172b20\ntesting.(*M).Run()\n\ttesting/testing.go:1728 +0x10217b460\nmain.main()\n\t_testmain.go:131 +0x10226aa60\nruntime.main()\n\truntime/proc.go:250 +0x1020e8c20\nruntime.goexit()\n\truntime/asm_arm64.s:1172 +0x102118b40","logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"github.com/blendle/zapdriver/error_test.go","line":"55","function":"github.com/blendle/zapdriver.ExampleSkipFmtStackTraces"},"logging.googleapis.com/labels":{},"serviceContext":{"service":"service"},"context":{"reportLocation":{"filePath":"github.com/blendle/zapdriver/error_test.go","lineNu
mber":"55","functionName":"github.com/blendle/zapdriver.ExampleSkipFmtStackTraces"}}}
{"severity":"ERROR","timestamp":"2023-02-24T16:52:31.025644+01:00","caller":"zapdriver/error_test.go:58","message":"without exception","error":"internal error","errorVerbose":"internal error\ngithub.com/blendle/zapdriver.ExampleSkipFmtStackTraces\n\tgithub.com/blendle/zapdriver/error_test.go:58\ntesting.runExample\n\ttesting/run_example.go:63\ntesting.runExamples\n\ttesting/example.go:44\ntesting.(*M).Run\n\ttesting/testing.go:1728\nmain.main\n\t_testmain.go:131\nruntime.main\n\truntime/proc.go:250\nruntime.goexit\n\truntime/asm_arm64.s:1172","logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"github.com/blendle/zapdriver/error_test.go","line":"58","function":"github.com/blendle/zapdriver.ExampleSkipFmtStackTraces"},"logging.googleapis.com/labels":{},"serviceContext":{"service":"service"},"stacktrace":"github.com/blendle/zapdriver.ExampleSkipFmtStackTraces\n\tgithub.com/blendle/zapdriver/error_test.go:58\ntesting.runExample\n\ttesting/run_example.go:63\ntesting.runExamples\n\tt
esting/example.go:44\ntesting.(*M).Run\n\ttesting/testing.go:1728\nmain.main\n\t_testmain.go:131\nruntime.main\n\truntime/proc.go:250"}

In Cloud Errors this would show as:

Any other format (like Zap's stack format) would wrongly display as such:

The inspiration from this format comes from this error report:
googleapis/google-cloud-go#1084 (comment)