diff --git a/CHANGELOG.md b/CHANGELOG.md index 13a6188..b98ec2c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,89 +1,3 @@ -# Changelog +All notable changes to this project will be documented as part of the release notes. -All notable changes to this project will be documented in this file. - -## [1.10.0] - 2023-10-31 -### Added -- `Get` method - -## [1.9.0] - 2023-09-29 -### Added -- Added BYOT strict modes in Insert Method. - -## [1.8.1] - 2023-09-08 -### Added -- Added request index in response in Insert Method. - -## [1.8.0] - 2023-09-01 -### Added -- Support for Bulk request with Continue on Error in Detokenize Method -- Support for Continue on Error in Insert Method - -## [1.7.2] - 2023-08-28 -### Added -- Support for OFF Loglevel. - -## [1.7.1] - 2023-08-22 -### Changed -- Internal Batch API with tokenization - -## [1.7.0] - 2023-08-18 -### Added -- Support for BYOT tokens in insert method -- Support for Context in insert method - -## [1.6.0] - 2023-06-09 -### Added -- `redaction` key for detokenize method for column group support. - -## [1.5.1] - 2023-03-01 -### Added -- Fix token expiry time and removal of grace period. - -## [1.5.0] - 2022-12-07 -### Added -- Upsert support for `insert` method. - - -## [1.4.0] - 2022-04-12 - -### Added -- Support for application/x-www-form-urlencoded and multipart/form-data content-type's in connections. - -## [1.3.1] - 2022-03-29 - -### Changed -- Added validation to token from TokenProvider - -### Fixed -- requestHeaders are not case insensitive - -## [1.3.0] - 2022-03-15 - -### Changed -- deprecated `IsValid` in favor of `IsExpired` - -## [1.2.0] - 2021-02-24 - -### Added -- Request ID in error logs and error responses for API Errors -- `isValid` method for validating Service Account bearer token - -## [1.1.0] - 2022-02-15 - -### Added -- Logging functionality -- `SetLogLevel` function for setting the package-level LogLevel -- `GenerateBearerTokenFromCreds` function which takes credentials as string -- `Insert` vault API -- `Detokenize` vault API -- `GetById` vault API -- `InvokeConnection` - -### Changed -- Renamed and deprecated `GenerateToken` in favor of `GenerateBearerToken` - -## [1.0.0] - 2021-08-25 - -### Added -- `GenerateToken` for Service Account Token generation +See [GitHub Releases](https://github.com/skyflowapi/skyflow-go/releases) or [pkg.go.dev](https://pkg.go.dev/github.com/skyflowapi/skyflow-go/v2) for more details on each released version. \ No newline at end of file diff --git a/README-v1.md b/README-v1.md index f49619f..fca6a6b 100644 --- a/README-v1.md +++ b/README-v1.md @@ -1,4 +1,6 @@ # Description +> **Go V2.1.0 IS NOW AVAILABLE:** A new, improved version of the Skyflow SDK is ready with flexible authentication, multi-vault support, builder patterns, and richer error diagnostics. V1 is in maintenance mode (security patches only) and will reach End of Life on October 31, 2026. We recommend upgrading to v2.1.0 — see the **[Migration Guide](docs/migrate_to_v2.md)** for step-by-step instructions. + This go SDK is designed to help developers easily implement Skyflow into their go backend. [![CI](https://img.shields.io/static/v1?label=CI&message=passing&color=green?style=plastic&logo=github)](https://github.com/skyflowapi/skyflow-go/actions) diff --git a/README.md b/README.md index 4ec83b5..d6c411f 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,9 @@ # Description +> **This is the current, recommended version of the Skyflow SDK.** V2.1.0 brings flexible auth, multi-vault support, builder patterns, native data types, and rich error diagnostics. + +> Migrating from v1? See the **[Migration Guide](./docs/migrate_to_v2.md)** for step-by-step instructions. V1 is in maintenance mode and will reach End of Life on October 31, 2026. + + The Skyflow Go SDK is designed to help with integrating Skyflow into a go backend. [![CI](https://img.shields.io/static/v1?label=CI&message=passing&color=green?style=plastic&logo=github)](https://github.com/skyflowapi/skyflow-go/actions) @@ -13,12 +18,6 @@ The Skyflow Go SDK is designed to help with integrating Skyflow into a go backen - [Install](#install) - [Requirements](#requirements) - [Configuration](#configuration) -- [Migration from v1 to v2](#migration-from-v1-and-v2) - - [Authentication options](#authentication-options) - - [Initializing the client](#initializing-the-client) - - [Request & response structure](#request--response-structure) - - [Request options](#request-options) - - [Error structure](#error-structure) - [Quickstart](#quickstart) - [Authenticate](#authenticate) - [Initialize the client](#initialize-the-client) @@ -89,264 +88,6 @@ import ( ``` Alternatively, `go get ` can also be used to download the required dependencies -## Migration from v1 and v2 -Below are the steps to migrate the go sdk from v1 to v2. - -### **Authentication options** -In V2, we have introduced multiple authentication options. -You can now provide credentials in the following ways: - -- **Passing credentials in ENV.** (`SKYFLOW_CREDENTIALS`) (**Recommended**) -- **API Key** -- **Path to your credentials JSON file** -- **Stringified JSON of your credentials** -- **Bearer token** - -These options allow you to choose the authentication method that best suits your use case. - -#### V1 (Old): Passing the token provider function below as a parameter to the Configuration. - -```go -package main - -import ( - "fmt" - saUtil "github.com/skyflowapi/skyflow-go/serviceaccount/util" -) - -var bearerToken = "" - -func GetSkyflowBearerToken() (string, error) { - - filePath := "" - if saUtil.IsExpired(bearerToken) { - newToken, err := saUtil.GenerateBearerToken(filePath) - if err != nil { - return "", err - } else { - bearerToken = newToken.AccessToken - return bearerToken, nil - } - } - return bearerToken, nil -} -``` - -#### V2(New): Passing one of the following: -```go -// Option 1: API Key (Recommended) -skyflowCredentials := common.Credentials{ApiKey: ""} // Replace with your actual API key - - // Option 2: Environment Variables -// Set SKYFLOW_CREDENTIALS in your environment - -// Option 3: Credentials File -skyflowCredentials := common.Credentials{Path: ""} // Replace with the path to credentials file - -// Option 4: Stringified JSON -skyflowCredentials := common.Credentials{CredentialsString: ""} // Replace with the credentials string - -// Option 5: Bearer Token -skyflowCredentials := common.Credentials{Token: ""} // Replace with your actual authentication token. -``` - - -### Initializing the client -In V2, we have introduced a functional options design pattern for client initialization and added support for multi-vault. This allows you to configure multiple vaults during client initialization. - -In V2, the log level is tied to each individual client instance. - -During client initialization, you can pass the following parameters: - -- `VaultID` and `VaultURL`: These values are derived from the vault ID & vault URL. -- `Env`: Specify the environment (e.g., SANDBOX or PROD). -- `Credentials`: The necessary authentication credentials. - -#### V1 (Old): -```go -import ( - Skyflow "github.com/skyflowapi/skyflow-go/skyflow/client" - "github.com/skyflowapi/skyflow-go/skyflow/common" -) - -configuration := common.Configuration { - VaultID: "", //Id of the vault that the client should connect to - VaultURL: "", //URL of the vault that the client should connect to - TokenProvider: GetToken //helper function that retrieves a Skyflow bearer token from your backend -} - -skyflowClient := Skyflow.Init(configuration) -``` - -#### V2 (New): -```go -import ( - "context" - "fmt" - "github.com/skyflowapi/skyflow-go/v2/client" - "github.com/skyflowapi/skyflow-go/v2/utils/common" - "github.com/skyflowapi/skyflow-go/v2/utils/logger" -) - -func main() { - creds := common.Credentials{Path: ""} // Replace with the path to the credentials file - vaultConfig1 := common.VaultConfig{VaultId: "", ClusterId: "", Env: common.DEV, Credentials: creds} // Replace with the Cluster and Vault ID of the first vault, Set the environment (e.g., DEV, STAGE, PROD) - var arr []common.VaultConfig - arr = append(arr, vaultConfig1) - // Create a Skyflow client and add vault configurations - skyflowClient, err := client.NewSkyflow( - client.WithVaults(arr...), // Add the first vault configuration - client.WithCredentials(common.Credentials{}), // Add the first vault configuration - client.WithLogLevel(logger.DEBUG), // Enable debugging for detailed logs - ) -} -``` - -#### Key Changes: -- `vaultUrl` replaced with `ClusterId`. -- Added environment specification (`Env`). -- Instance-specific log levels. - -### Request & response structure -In V2, we have removed the use of JSON objects from a third-party package. Instead, we have transitioned to accepting native list and map data structures. This request needs: -- **Table**: The name of the table. -- **Values**: An array list of objects containing the data to be inserted. -The response will be of type `InsertResponse` struct, which contains `InsertedFields` and `Errors`. - -#### V1 (Old) : Request Building -```go -import ( - Skyflow "github.com/skyflowapi/skyflow-go/skyflow/client" - "github.com/skyflowapi/skyflow-go/skyflow/common" -) - -//Initialize the SkyflowClient. -var records = make(map[string] interface {}) - -var record = make(map[string] interface {}) -record["table"] = "" -var fields = make(map[string] interface {}) -fields[""] = "" -record["fields"] = fields - -var recordsArray[] interface {} -recordsArray = append(recordsArray, record) - -records["records"] = recordsArray - -var upsertArray []common.UpsertOptions -var upsertOption = common.UpsertOptions{Table:"",Column:""} -upsertArray = append(upsertArray,upsertOption) - -options := common.InsertOptions { - Tokens: true //Optional, indicates whether tokens should be returned for the inserted data. This value defaults to "true". - Upsert: upsertArray //Optional, upsert support. - ContinueOnError: true // Optional, decides whether to continue if error encountered or not -} - -res, err: = skyflowClient.Insert(records, options) -``` -#### V2 (New) : Request building -```go -service, serviceError := skyflowClient.Vault("") -if serviceError != nil { - fmt.Println(serviceError) -} else { - ctx := context.TODO() - values := make([]map[string]interface{}, 0) - values = append(values, map[string]interface{}{ - "": "", // Replace with column name and value - }) - values = append(values, map[string]interface{}{ - "": "", // Replace with another column name and value - }) - tokens := make([]map[string]interface{}, 0) - tokens = append(values, map[string]interface{}{ - "": "", - }) - insert, err := service.Insert(ctx, common.InsertRequest{ - Table: "", - Values: values, - }, common.InsertOptions{ContinueOnError: false, ReturnTokens: true, TokenMode: common.ENABLE, Tokens: tokens}) - - if err != nil { - fmt.Println("Error occurred ", *err) - } else { - fmt.Println("RESPONSE:", insert) - } -} -``` -#### V1 (Old) : Response structure -```json -{ - "Records": [ - { - "table": "cards", - "fields": { - "SkyflowId": "16419435-aa63-4823-aae7-19c6a2d6a19f", - "cardNumber": "f3907186-e7e2-466f-91e5-48e12c2bcbc1", - "cvv": "1989cb56-63da-4482-a2df-1f74cd0dd1a5" - } - } - ] -} -``` -#### V2 (New) : Response structure -```json -{ - "InsertedFields": [ - { - "card_number": "5484-7829-1702-9110", - "RequestIndex": "0", - "SkyflowId": "9fac9201-7b8a-4446-93f8-5244e1213bd1", - "cardholder_name": "b2308e2a-c1f5-469b-97b7-1f193159399b" - } - ], - "Errors": [] -} -``` -### Request options -In V2, with the introduction of the Functional options design pattern has made handling optional fields in Go more efficient and straightforward. -#### V1 (Old): -```go -options := common.InsertOptions { - Tokens: true //Optional, indicates whether tokens should be returned for the inserted data. This value defaults to "true". - Upsert: upsertArray //Optional, upsert support. - ContinueOnError: true // Optional, decides whether to continue if error encountered or not -} -``` -#### V2 (New): -```go -options := common.InsertOptions{ContinueOnError: false, ReturnTokens: true, TokenMode: common.DISABLE, Upsert: ""} -``` - -#### Error structure -In V2, we have enriched the error details to provide better debugging capabilities. -The error response now includes: -- **httpStatus**: The HTTP status code. -- **grpcCode**: The gRPC code associated with the error. -- **details & message**: A detailed description of the error. -- **requestId**: A unique request identifier for easier debugging. - - -#### V1 (Old): Error structure -```json -{ - "code": "", - "description": "", -} -``` -#### V2 (New): Error structure -```js -{ - "httpStatus": "", - "grpcCode": "", - "httpCode": "", - "message": "", - "requestId": "", - "details": ["
"] -} -``` ## Quickstart Get started quickly with the essential steps: authenticate, initialize the client, and perform a basic vault operation. This section provides a minimal setup to help you integrate the SDK efficiently. diff --git a/docs/migrate_to_v2.md b/docs/migrate_to_v2.md new file mode 100644 index 0000000..86ec2bf --- /dev/null +++ b/docs/migrate_to_v2.md @@ -0,0 +1,284 @@ + +## Migration from v1 and v2 +Below are the steps to migrate the go sdk from v1 to v2. + +### **Authentication options** +In V2, we have introduced multiple authentication options. +You can now provide credentials in the following ways: + +- **Passing credentials in ENV.** (`SKYFLOW_CREDENTIALS`) (**Recommended**) +- **API Key** +- **Path to your credentials JSON file** +- **Stringified JSON of your credentials** +- **Bearer token** + +These options allow you to choose the authentication method that best suits your use case. + +#### V1 (Old): Passing the token provider function below as a parameter to the Configuration. + +```go +package main + +import ( + "fmt" + saUtil "github.com/skyflowapi/skyflow-go/serviceaccount/util" +) + +var bearerToken = "" + +func GetSkyflowBearerToken() (string, error) { + + filePath := "" + if saUtil.IsExpired(bearerToken) { + newToken, err := saUtil.GenerateBearerToken(filePath) + if err != nil { + return "", err + } else { + bearerToken = newToken.AccessToken + return bearerToken, nil + } + } + return bearerToken, nil +} +``` + +#### V2(New): Passing one of the following: +```go +// Option 1: API Key (Recommended) +skyflowCredentials := common.Credentials{ApiKey: ""} // Replace with your actual API key + + // Option 2: Environment Variables +// Set SKYFLOW_CREDENTIALS in your environment + +// Option 3: Credentials File +skyflowCredentials := common.Credentials{Path: ""} // Replace with the path to credentials file + +// Option 4: Stringified JSON +skyflowCredentials := common.Credentials{CredentialsString: ""} // Replace with the credentials string + +// Option 5: Bearer Token +skyflowCredentials := common.Credentials{Token: ""} // Replace with your actual authentication token. +``` + + +### Initializing the client +In V2, we have introduced a functional options design pattern for client initialization and added support for multi-vault. This allows you to configure multiple vaults during client initialization. + +In V2, the log level is tied to each individual client instance. + +During client initialization, you can pass the following parameters: + +- `VaultID` and `VaultURL`: These values are derived from the vault ID & vault URL. +- `Env`: Specify the environment (e.g., SANDBOX or PROD). +- `Credentials`: The necessary authentication credentials. + +#### V1 (Old): +```go +import ( + Skyflow "github.com/skyflowapi/skyflow-go/skyflow/client" + "github.com/skyflowapi/skyflow-go/skyflow/common" +) + +configuration := common.Configuration { + VaultID: "", //Id of the vault that the client should connect to + VaultURL: "", //URL of the vault that the client should connect to + TokenProvider: GetToken //helper function that retrieves a Skyflow bearer token from your backend +} + +skyflowClient := Skyflow.Init(configuration) +``` + +#### V2 (New): +```go +import ( + "context" + "fmt" + "github.com/skyflowapi/skyflow-go/v2/client" + "github.com/skyflowapi/skyflow-go/v2/utils/common" + "github.com/skyflowapi/skyflow-go/v2/utils/logger" +) + +func main() { + creds := common.Credentials{Path: ""} // Replace with the path to the credentials file + vaultConfig1 := common.VaultConfig{VaultId: "", ClusterId: "", Env: common.DEV, Credentials: creds} // Replace with the Cluster and Vault ID of the first vault, Set the environment (e.g., DEV, STAGE, PROD) + var arr []common.VaultConfig + arr = append(arr, vaultConfig1) + // Create a Skyflow client and add vault configurations + skyflowClient, err := client.NewSkyflow( + client.WithVaults(arr...), // Add the first vault configuration + client.WithCredentials(common.Credentials{}), // Add the first vault configuration + client.WithLogLevel(logger.DEBUG), // Enable debugging for detailed logs + ) +} +``` + +#### Key Changes: +- `vaultUrl` replaced with `ClusterId`. +- Added environment specification (`Env`). +- Instance-specific log levels. + +### Request & response structure +In V2, we have removed the use of JSON objects from a third-party package. Instead, we have transitioned to accepting native list and map data structures. This request needs: +- **Table**: The name of the table. +- **Values**: An array list of objects containing the data to be inserted. +The response will be of type `InsertResponse` struct, which contains `InsertedFields` and `Errors`. + +#### V1 (Old) : Request Building +```go +import ( + Skyflow "github.com/skyflowapi/skyflow-go/skyflow/client" + "github.com/skyflowapi/skyflow-go/skyflow/common" +) + +//Initialize the SkyflowClient. +var records = make(map[string] interface {}) + +var record = make(map[string] interface {}) +record["table"] = "" +var fields = make(map[string] interface {}) +fields[""] = "" +record["fields"] = fields + +var recordsArray[] interface {} +recordsArray = append(recordsArray, record) + +records["records"] = recordsArray + +var upsertArray []common.UpsertOptions +var upsertOption = common.UpsertOptions{Table:"",Column:""} +upsertArray = append(upsertArray,upsertOption) + +options := common.InsertOptions { + Tokens: true //Optional, indicates whether tokens should be returned for the inserted data. This value defaults to "true". + Upsert: upsertArray //Optional, upsert support. + ContinueOnError: true // Optional, decides whether to continue if error encountered or not +} + +res, err: = skyflowClient.Insert(records, options) +``` +#### V2 (New) : Request building +```go +service, serviceError := skyflowClient.Vault("") +if serviceError != nil { + fmt.Println(serviceError) +} else { + ctx := context.TODO() + values := make([]map[string]interface{}, 0) + values = append(values, map[string]interface{}{ + "": "", // Replace with column name and value + }) + values = append(values, map[string]interface{}{ + "": "", // Replace with another column name and value + }) + tokens := make([]map[string]interface{}, 0) + tokens = append(values, map[string]interface{}{ + "": "", + }) + insert, err := service.Insert(ctx, common.InsertRequest{ + Table: "", + Values: values, + }, common.InsertOptions{ContinueOnError: false, ReturnTokens: true, TokenMode: common.ENABLE, Tokens: tokens}) + + if err != nil { + fmt.Println("Error occurred ", *err) + } else { + fmt.Println("RESPONSE:", insert) + } +} +``` +#### V1 (Old) : Response structure +```json +{ + "Records": [ + { + "table": "cards", + "fields": { + "skyflow_id": "16419435-aa63-4823-aae7-19c6a2d6a19f", + "cardNumber": "f3907186-e7e2-466f-91e5-48e12c2bcbc1", + "cvv": "1989cb56-63da-4482-a2df-1f74cd0dd1a5" + } + } + ] +} +``` +#### V2 (New) : Response structure +```json +{ + "InsertedFields": [ + { + "card_number": "5484-7829-1702-9110", + "request_index": "0", + "skyflow_id": "9fac9201-7b8a-4446-93f8-5244e1213bd1", + "cardholder_name": "b2308e2a-c1f5-469b-97b7-1f193159399b" + } + ], + "Errors": [] +} +``` +### Request options +In V2, with the introduction of the Functional options design pattern has made handling optional fields in Go more efficient and straightforward. +#### V1 (Old): +```go +options := common.InsertOptions { + Tokens: true //Optional, indicates whether tokens should be returned for the inserted data. This value defaults to "true". + Upsert: upsertArray //Optional, upsert support. + ContinueOnError: true // Optional, decides whether to continue if error encountered or not +} +``` +#### V2 (New): +```go +options := common.InsertOptions{ContinueOnError: false, ReturnTokens: true, TokenMode: common.DISABLE, Upsert: ""} +``` + +#### Error structure +In V2, we have enriched the error details to provide better debugging capabilities. +The error response now includes: +- **httpStatus**: The HTTP status code. +- **grpcCode**: The gRPC code associated with the error. +- **details & message**: A detailed description of the error. +- **requestId**: A unique request identifier for easier debugging. + + +#### V1 (Old): Error structure +```json +{ + "code": "", + "description": "", +} +``` +#### V2 (New): Error structure +```js +{ + "httpStatus": "", + "grpcCode": "", + "httpCode": "", + "message": "", + "requestId": "", + "details": ["
"] +} +``` + +## Credential field names (v2.1+) + +The credentials JSON file field names are updated to follow camelCase conventions. Both old and new forms are permanently accepted. + +| Old form (still accepted) | New form (preferred) | +|---|---| +| `clientID` | `clientId` | +| `keyID` | `keyId` | +| `tokenURI` | `tokenUri` | + +--- + +## Response field names (v2.1+) + +Response maps now use `SkyflowId` (PascalCase). The legacy keys are still present for backward compatibility but are deprecated. + +| Deprecated (still returned) | Preferred | +|---|---| +| `skyflow_id` | `SkyflowId` | +| `request_index` | `RequestIndex` | + +--- + +For the full list of changes see [CHANGELOG.md](../CHANGELOG.md).