diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/cloud-services-connections/_index.md b/content/en/docs/PlaidCloud/Analyze/connectors/cloud-services-connections/_index.md new file mode 100644 index 00000000..3fa03d10 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/cloud-services-connections/_index.md @@ -0,0 +1,11 @@ +--- +title: Cloud Service Connections +slug: cloud-service-connections +description: Cloud Service connection requirements vary depending on the service. See how to configure the individual connections. +date: 2022-01-25T07:39:49 +--- + + +PlaidCloud provides a direct service connections for services that don't use REST or JSON-RPC requests. + +The individual service guides will help provide the specific setup necessary to connect. diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/cloud-services-connections/quandl.md b/content/en/docs/PlaidCloud/Analyze/connectors/cloud-services-connections/quandl.md new file mode 100644 index 00000000..dcd8d807 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/cloud-services-connections/quandl.md @@ -0,0 +1,20 @@ +--- +title: Quandl Connector +slug: quandl +description: Connecting to Quandl Cloud Service +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Connection Documentation +Quandl is now Nasdaq Data Link. The documentation is [here](https://docs.data.nasdaq.com/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Service Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/_index.md b/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/_index.md new file mode 100644 index 00000000..aa2cc9a6 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/_index.md @@ -0,0 +1,11 @@ +--- +title: Team Collaboration Connections +slug: team-collaboration-connections +description: Team Collaboration connection requirements vary depending on the service. See how to configure the individual connections. +date: 2022-01-25T07:39:49 +--- + + +PlaidCloud provides a direct connections for team collaboration services. + +The individual service guides will help provide the specific setup necessary to connect. \ No newline at end of file diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/slack.md b/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/slack.md new file mode 100644 index 00000000..7fb4269b --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/slack.md @@ -0,0 +1,20 @@ +--- +title: Slack Connector +slug: slack +description: Connecting to Slack for Team Collaboration +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Connection Documentation +Slack Admin documentation is [here](https://slack.com/help). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Slack Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/teams.md b/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/teams.md new file mode 100644 index 00000000..8d16d270 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/collaboration-connections/teams.md @@ -0,0 +1,20 @@ +--- +title: Microsoft Teams Connector +slug: teams +description: Connecting to Microsoft Teams for Team Collaboration +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Connection Documentation +Microsoft Teams Admin documentation is [here](https://learn.microsoft.com/en-us/microsoftteams/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Teams Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/data-connections.md b/content/en/docs/PlaidCloud/Analyze/connectors/data-connections.md deleted file mode 100644 index c9af33a7..00000000 --- a/content/en/docs/PlaidCloud/Analyze/connectors/data-connections.md +++ /dev/null @@ -1,243 +0,0 @@ ---- -title: Data Connections -slug: data-connections -description: Use this table reference for more information on external system connections and databases -date: 2022-01-25T07:39:49 ---- - - -## Description - -PlaidCloud connects to external systems by using various data connections directly or through PlaidLink agents. - -For more details on each data connection type, please navigate to the specific data connection documentation. - -## Relational Databases - - -### Greenplum - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | greenplum | - -### Microsoft SQL Server - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | sqlserver | - -### MySQL - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | mysql | - -### ODBC - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | odbc | - -### Oracle - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | oracle | - -### Postgres - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | postgres | - -### Amazon Redshift - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | redshift | - -### SAP HANA - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | hana | - -### Exasol - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | exasol | - -### IBM DB2 - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | db2 | - -### Informix - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | informix | - -## Hadoop Based Databases - -### Hive - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | hive | - -### Presto - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | presto | - -### Spark - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Database | -| **Reference** | spark | - -## Team Collaboration Tools - -### Microsoft Teams - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Notification | -| **Reference** | teams | - -### Slack - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Notification | -| **Reference** | slack | - - -## Cloud Services - -### OAuth Connection - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | oAuth | -| **Reference** | oauth | - -### Quandl - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Quandl | -| **Reference** | quandl | - -## Google Related - -### Google Big Query - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Google Big Query | -| **Reference** | gbq | - -### Google Spreadsheet - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Google Spreadsheet | -| **Reference** | gspread | - - -## Oracle EBS Related - -Oracle EBS utilizes the standard Oracle database connection specified above. This connection provides the connectivity -to query, load, and execute PL/SQL programs in Oracle. - -If the EBS instance has the REST API interface available, this can be accessed using the same approach as Oracle Cloud described below. - - -## Oracle Cloud Related - -Oracle Cloud utilizes standard RESTful requests to perform queries, data loading, and other operations. A REST connection using OAuth2 -tokens is used for these interactions. This uses the standard oAuth connection specified above. - - -## Salesforce Related - -Salesforce utilizes standard RESTful requests to perform all operations. A REST connection using OAuth2 -tokens is used for these interactions. This uses the Salesforce specific connection type. - -## Workday Related - -Workday utilizes standard RESTful requests to perform all operations. A REST connection using OAuth2 -tokens is used for these interactions. This uses the standard oAuth connection specified above. - - -## JD Edwards Legacy Version Related - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | JD Edwards Legacy | -| **Reference** | jde_legacy | - - -## JD Edwards Related - -JD Edwards utilizes the standard Oracle database connection specified above. This connection provides the connectivity -to query, load, and execute PL/SQL programs in Oracle. - - -## Infor Related - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | Infor | -| **Reference** | infor | - - -## SAP Related - -### SAP Analytics Cloud - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | SAP Analytics Cloud | -| **Reference** | sap_sac | - -### SAP ECC - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | SAP ECC | -| **Reference** | sap_ecc | - -### SAP Profitability and Cost Management (PCM) - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | SAP PCM | -| **Reference** | sap_pcm | - -### SAP Profitability and Performance Management (PaPM) - -| Parameter | Value | -|-----------|-------| -| **Connection Type** | SAP PaPM | -| **Reference** | sap_papm | diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/_index.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/_index.md new file mode 100644 index 00000000..a4e05547 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/_index.md @@ -0,0 +1,11 @@ +--- +title: Database and Data Lake Connections +slug: database-connections +description: Database and Data Lake connections vary by service. Each connector will have specific security and access requirements in order for PlaidCloud to connect. +date: 2022-01-25T07:39:49 +--- + + +PlaidCloud enables connection directly to databases, data lakes, query engines, and lakehouses. Connections can also utilize a PlaidLink agent if services are behind a firewall. + +Since the terms of database, lakehouse, query engine, and potentially others are used to refer to data that can be queried using a connection, we generally treat all of these as "Databases" despite a wide variety of underlying technology that performs the underlying query. diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/amazon-athena.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/amazon-athena.md new file mode 100644 index 00000000..48014539 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/amazon-athena.md @@ -0,0 +1,20 @@ +--- +title: Amazon Athena +slug: amazon-athena +description: Connecting to Amazon Athena Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Amazon Athena documentation is [here](https://docs.aws.amazon.com/athena/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/amazon-redshift.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/amazon-redshift.md new file mode 100644 index 00000000..1c9852e7 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/amazon-redshift.md @@ -0,0 +1,20 @@ +--- +title: Amazon Redshift +slug: amazon-redshift +description: Connecting to Amazon Redshift Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Amazon Redshift has several guides related to use located [here](https://docs.aws.amazon.com/redshift/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/azure-databricks.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/azure-databricks.md new file mode 100644 index 00000000..b28677d3 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/azure-databricks.md @@ -0,0 +1,31 @@ +--- +title: Azure Databricks +slug: azure-databricks +description: Connecting to Azure Databricks Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Azure Databricks documentation is [here](https://learn.microsoft.com/en-us/azure/databricks/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +In order to obtain the connection credentials necessary for PlaidCloud to communicate with a Databricks warehouse, follow the steps below: + 1. Open the Databricks console + 2. Under the User Settings in the upper right, select "Settings" + 3. Navigate to the "Developers" section + 4. Generate an Access Token with a sufficient lifespan specified + 5. Navigate to the "SQL Warehouses" area + 6. Select the warehouse required for connecting + 7. Capture the connection details including host, and http path + 8. Navigate to the warehouse data area + 9. Capture the initial catalog and initial schema information + +With the information above, the connection form can be completed and tested with the Databricks warehouse + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/databend.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/databend.md new file mode 100644 index 00000000..9957bfd0 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/databend.md @@ -0,0 +1,20 @@ +--- +title: Databend +slug: databend +description: Connecting to Databend Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Databend documentation is [here](https://docs.databend.com/guides/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/doris.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/doris.md new file mode 100644 index 00000000..2acbe8cc --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/doris.md @@ -0,0 +1,22 @@ +--- +title: Apache Doris +slug: doris +description: Connecting to Apache Doris Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Apache Doris documentation is [here](https://doris.apache.org/docs/4.x/gettingStarted/what-is-apache-doris). + +The Apache project homepage for Apache Doris is [here](https://doris.apache.org/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/exasol.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/exasol.md new file mode 100644 index 00000000..d3c7210c --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/exasol.md @@ -0,0 +1,20 @@ +--- +title: Exasol +slug: Exasol +description: Connecting to Exasol Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Exasol documentation is [here](https://docs.exasol.com/home.htm). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/greenplum.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/greenplum.md new file mode 100644 index 00000000..c388718b --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/greenplum.md @@ -0,0 +1,20 @@ +--- +title: Greenplum +slug: greenplum +description: Connecting to Greenplum Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Greenplum documentation is located [here](https://techdocs.broadcom.com/us/en/vmware-tanzu/data-solutions/tanzu-greenplum/7/greenplum-database/landing-index.html). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/hive.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/hive.md new file mode 100644 index 00000000..65633c6a --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/hive.md @@ -0,0 +1,20 @@ +--- +title: Apache Hive +slug: hive +description: Connecting to Apache Hive Data Lake +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Apache Hive documentation is located [here](https://hive.apache.org/docs/latest/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Data Lake Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/ibm-db2.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/ibm-db2.md new file mode 100644 index 00000000..064db1c4 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/ibm-db2.md @@ -0,0 +1,20 @@ +--- +title: IBM DB2 +slug: ibm-db2 +description: Connecting to IBM DB2 Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The IBM DB2 documentation is located [here](https://www.ibm.com/support/pages/db2-database-product-documentation). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/informix.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/informix.md new file mode 100644 index 00000000..9ceb95e9 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/informix.md @@ -0,0 +1,20 @@ +--- +title: IBM Informix +slug: informix +description: Connecting to IBM Informix Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +IBM Informix documentation is located [here](https://www.ibm.com/docs/ar/informix-servers/14.10.0?). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/microsoft-fabric.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/microsoft-fabric.md new file mode 100644 index 00000000..c7e38af0 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/microsoft-fabric.md @@ -0,0 +1,20 @@ +--- +title: Microsoft Fabric +slug: microsoft-fabric +description: Connecting to Microsoft Fabric Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Microsoft Fabric documentation is located [here](https://learn.microsoft.com/en-us/fabric/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/microsoft-sql-server.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/microsoft-sql-server.md new file mode 100644 index 00000000..987d3d44 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/microsoft-sql-server.md @@ -0,0 +1,20 @@ +--- +title: Microsoft SQL Server +slug: microsoft-sql-server +description: Connecting to Microsoft SQL Server Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Microsoft SQL Server documentation is located [here](https://learn.microsoft.com/en-us/sql/sql-server/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/mysql.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/mysql.md new file mode 100644 index 00000000..b910d823 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/mysql.md @@ -0,0 +1,20 @@ +--- +title: MySQL +slug: mysql +description: Connecting to MySQL Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +MySQL documentation is located [here](https://dev.mysql.com/doc/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/odbc.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/odbc.md new file mode 100644 index 00000000..261fee58 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/odbc.md @@ -0,0 +1,20 @@ +--- +title: ODBC +slug: odbc +description: Connecting to a Database using ODBC +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +Using the ODBC connector will require configuration specific to the database. While ODBC is a generic connection type, each database may implement some specific configurations. Please refer to the ODBC documentation for the target database. + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/oracle.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/oracle.md new file mode 100644 index 00000000..58a7b652 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/oracle.md @@ -0,0 +1,20 @@ +--- +title: Oracle +slug: oracle +description: Connecting to Oracle Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Oracle database documentation is located [here](https://docs.oracle.com/en/database/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/plaidcloud-lakehouse.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/plaidcloud-lakehouse.md new file mode 100644 index 00000000..127b3d16 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/plaidcloud-lakehouse.md @@ -0,0 +1,20 @@ +--- +title: PlaidCloud Lakehouse +slug: plaidcloud-lakehouse +description: Connecting to PlaidCloud Lakehouse +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +There is very little configuration necessary for using the built-in PlaidCloud Lakehouse. The documentation for the service is [here](https://docs.plaidcloud.com/docs/plaidcloud/analyze/dw/getting-started/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Lakehouse Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/postgres.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/postgres.md new file mode 100644 index 00000000..8cb4fa81 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/postgres.md @@ -0,0 +1,20 @@ +--- +title: PostgreSQL +slug: postgres +description: Connecting to PostgreSQL (Postgres) Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +PostreSQL documentation is located [here](https://www.postgresql.org/docs/) + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/presto.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/presto.md new file mode 100644 index 00000000..47f6a262 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/presto.md @@ -0,0 +1,20 @@ +--- +title: Presto +slug: presto +description: Connecting to Presto Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Presto documentation is [here](https://prestodb.io/docs/current/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/sap-hana.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/sap-hana.md new file mode 100644 index 00000000..9cef6e38 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/sap-hana.md @@ -0,0 +1,20 @@ +--- +title: SAP HANA +slug: sap-hana +description: Connecting to SAP HANA Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The SAP HANA documentation is [here](https://help.sap.com/docs/SAP_HANA_PLATFORM). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/snowflake.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/snowflake.md new file mode 100644 index 00000000..6ed72f8b --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/snowflake.md @@ -0,0 +1,20 @@ +--- +title: Snowflake +slug: snowflake +description: Connecting to Snowflake Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Snowflake documentation is [here](https://docs.snowflake.com/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/spark.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/spark.md new file mode 100644 index 00000000..55599948 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/spark.md @@ -0,0 +1,22 @@ +--- +title: Apache Spark +slug: spark +description: Connecting to Apache Spark Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Apache Spark documentation is [here](https://spark.apache.org/documentation.html). + +The Apache project is [here](https://spark.apache.org/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/starrocks.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/starrocks.md new file mode 100644 index 00000000..98422456 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/starrocks.md @@ -0,0 +1,20 @@ +--- +title: StarRocks +slug: starrocks +description: Connecting to StarRocks Database +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +StarRocks documentation is [here](https://docs.starrocks.io/docs/introduction/StarRocks_intro/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/trino.md b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/trino.md new file mode 100644 index 00000000..a723c34e --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/database-connections/trino.md @@ -0,0 +1,20 @@ +--- +title: Trino +slug: trino +description: Connecting to Trino Query Engine +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Trino documentation is [here](https://trino.io/docs/current/index.html). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Database Connector +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/_index.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/_index.md new file mode 100644 index 00000000..fd8ec0e9 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/_index.md @@ -0,0 +1,11 @@ +--- +title: ERP System Connections +slug: erp-connections +description: ERP connection requirements vary depending on the system. See how to configure the individual connections. +date: 2022-01-25T07:39:49 +--- + + +PlaidCloud provides a direct connections for ERP systems. + +The individual service guides will help provide the specific setup necessary to connect. \ No newline at end of file diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/infor.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/infor.md new file mode 100644 index 00000000..bf5ff330 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/infor.md @@ -0,0 +1,20 @@ +--- +title: Infor Connector +slug: infor +description: Connecting to Infor ERP System +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Infor documentation is [here](https://docs.infor.com/en-us). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create ERP Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/jde-legacy.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/jde-legacy.md new file mode 100644 index 00000000..378cec43 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/jde-legacy.md @@ -0,0 +1,20 @@ +--- +title: JD Edwards (Legacy) Connector +slug: jde-legacy +description: Connecting to JD Edwards Legacy ERP System +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The JDE documentation is [here](https://www.oracle.com/technical-resources/documentation/jd-edwards-enterpriseone.html). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create ERP Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/oracle-ebs.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/oracle-ebs.md new file mode 100644 index 00000000..590cedca --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/oracle-ebs.md @@ -0,0 +1,22 @@ +--- +title: Oracle EBS Connector +slug: oracle-ebs +description: Connecting to Oracle EBS ERP System +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Oracle EBS documentation is [here](https://docs.oracle.com/cd/E51111_01/current/html/docset.html). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Oracle EBS utilizes the standard Oracle database connection. This connection provides the connectivity to query, load, and execute PL/SQL programs in Oracle. + +If the EBS instance has the REST API interface available, this can be accessed using the same approach as Oracle Cloud REST connection too. + +## Create ERP Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/oracle-fusion.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/oracle-fusion.md new file mode 100644 index 00000000..739ce141 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/oracle-fusion.md @@ -0,0 +1,20 @@ +--- +title: Oracle Fusion Connector +slug: oracle-fusion +description: Connecting to Oracle Fusion ERP System +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The Oracle Fusion applications documentation is [here](https://www.oracle.com/middleware/technologies/fusion-apps-doc.html). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create ERP Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-ecc.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-ecc.md new file mode 100644 index 00000000..26dd91a0 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-ecc.md @@ -0,0 +1,20 @@ +--- +title: SAP ECC Connector +slug: sap-ecc +description: Connecting to SAP ECC ERP System +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +SAP has removed all ECC documentation and currently only provides documentation for [S/4HANA](https://help.sap.com/docs/SAP_S4HANA_ON-PREMISE). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create ERP Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-papm.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-papm.md new file mode 100644 index 00000000..89287c27 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-papm.md @@ -0,0 +1,20 @@ +--- +title: SAP Profitability and Performance Management (PaPM) Connector +slug: sap-papm +description: Connecting to SAP PaPM System +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The SAP PaPM documentation is [here](https://help.sap.com/docs/SAP_PROFITABILITY_PERFORMANCE_MANAGEMENT). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create PaPM Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-pcm.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-pcm.md new file mode 100644 index 00000000..81f93dc4 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-pcm.md @@ -0,0 +1,20 @@ +--- +title: SAP Profitability and Cost Management (PCM) Connector +slug: sap-pcm +description: Connecting to SAP Profitability and Cost Managmenet System +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The SAP PCM legacy documentation is [here](https://help.sap.com/docs/SAP_PROFITABILITY_AND_COST_MANAGEMENT). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create PCM Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-s4.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-s4.md new file mode 100644 index 00000000..2dddcb29 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-s4.md @@ -0,0 +1,20 @@ +--- +title: SAP S/4HANA Connector +slug: sap-s4 +description: Connecting to SAP S/4HANA ERP System +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The documentation for SAP S/4HANA is [here](https://help.sap.com/docs/SAP_S4HANA_ON-PREMISE). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create ERP Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-sac.md b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-sac.md new file mode 100644 index 00000000..823d7688 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/erp-connections/sap-sac.md @@ -0,0 +1,20 @@ +--- +title: SAP Analytics Cloud Connector +slug: sap-sac +description: Connecting to SAP Analytics Cloud (SAC) +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Upstream Documentation +The SAP Analytics Cloud documentation is [here](https://help.sap.com/docs/SAP_ANALYTICS_CLOUD). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create SAC Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/_index.md b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/_index.md new file mode 100644 index 00000000..c16f6373 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/_index.md @@ -0,0 +1,11 @@ +--- +title: Git Repository Connections +slug: git-connections +description: Git connection requirements vary depending on the Git service. See how to configure the individual connections. +date: 2022-01-25T07:39:49 +--- + + +PlaidCloud provides a direct connections for Git repositories. + +The individual service guides will help provide the specific setup necessary to connect. \ No newline at end of file diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/azure-repos.md b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/azure-repos.md new file mode 100644 index 00000000..6a4ae3dd --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/azure-repos.md @@ -0,0 +1,20 @@ +--- +title: Azure Repos Repository Connector +slug: azure-repos +description: Connecting to Azure Repos +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Service Documentation +The Azure Repos service documentation is [here](content/en/docs/PlaidCloud/Analyze/connectors/git-connections/codecommit.md). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Git Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/bitbucket.md b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/bitbucket.md new file mode 100644 index 00000000..2b616291 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/bitbucket.md @@ -0,0 +1,20 @@ +--- +title: BitBucket Repository Connector +slug: bitbucket +description: Connecting to BitBucket +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Service Documentation +The BitBucket service documentation is [here](https://bitbucket.org/product/guides). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Git Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/codecommit.md b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/codecommit.md new file mode 100644 index 00000000..52c1aaae --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/codecommit.md @@ -0,0 +1,20 @@ +--- +title: AWS CodeCommit Repository Connector +slug: codecommit +description: Connecting to AWS CodeCommit +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Service Documentation +The AWS CodeCommit service documentation is [here](https://docs.aws.amazon.com/codecommit/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Git Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/github.md b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/github.md new file mode 100644 index 00000000..c8db5c82 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/github.md @@ -0,0 +1,20 @@ +--- +title: GitHub Repository Connector +slug: github +description: Connecting to GitHub +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Service Documentation +The GitHub service documentation is [here](https://docs.github.com/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Git Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/gitlab.md b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/gitlab.md new file mode 100644 index 00000000..0809f2ce --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/git-connections/gitlab.md @@ -0,0 +1,20 @@ +--- +title: GitLab Repository Connector +slug: gitlab +description: Connecting to GitLab +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Service Documentation +The GitLab service documentation is [here](https://docs.gitlab.com/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Git Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/_index.md b/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/_index.md new file mode 100644 index 00000000..59c62dcf --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/_index.md @@ -0,0 +1,11 @@ +--- +title: Google Service Connections +slug: google-connections +description: Google service connection requirements vary depending on the Google service. See how to configure the individual connections. +date: 2022-01-25T07:39:49 +--- + + +PlaidCloud provides a direct connections for Google services. + +The individual service guides will help provide the specific setup necessary to connect. \ No newline at end of file diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/big-query.md b/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/big-query.md new file mode 100644 index 00000000..481fa94f --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/big-query.md @@ -0,0 +1,20 @@ +--- +title: Google BigQuery Connector +slug: google-big-query +description: Connecting to Google BigQuery +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Connection Documentation +The Google BigQuery documentation is [here](https://docs.cloud.google.com/bigquery/docs). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Query Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/gspread.md b/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/gspread.md new file mode 100644 index 00000000..37050a33 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/google-connections/gspread.md @@ -0,0 +1,20 @@ +--- +title: Google Sheets +slug: google-sheets +description: Connecting to Google Sheets +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Connection Documentation +Google Sheets is oriented more towards consumers. For technical documentation, refer to the developer documentation [here](https://developers.google.com/workspace/sheets). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Spreadsheet Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/_index.md b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/_index.md new file mode 100644 index 00000000..a55bb8ca --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/_index.md @@ -0,0 +1,11 @@ +--- +title: Open Table Format Connections +slug: open-table-connections +description: Open Table Format connection requirements vary depending on the several factors. See how to configure the individual connections. +date: 2022-01-25T07:39:49 +--- + + +PlaidCloud provides a direct connections for Open Table Formats for use with the PlaidCloud Lakehouse service. This allows for hybrid query execution without moving data. + +The individual service guides will help provide the specific setup necessary to connect. \ No newline at end of file diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/delta-lake.md b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/delta-lake.md new file mode 100644 index 00000000..156f4871 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/delta-lake.md @@ -0,0 +1,20 @@ +--- +title: Delta Lake Open Table Format (Databricks Catalog) +slug: delta-lake +description: Connecting to Delta Lake Catalogs (Databricks) +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Catalog Documentation +The Delta Lake documentation is [here](https://docs.delta.io/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Catalog Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/hive.md b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/hive.md new file mode 100644 index 00000000..c13ea231 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/hive.md @@ -0,0 +1,20 @@ +--- +title: Apache Hive Open Table Format +slug: apache-hive +description: Connecting to Apache Hive Catalogs +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Catalog Documentation +Apache Hive documentation is [here](https://hive.apache.org/docs/latest/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Catalog Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/hudi.md b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/hudi.md new file mode 100644 index 00000000..71631972 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/hudi.md @@ -0,0 +1,20 @@ +--- +title: Apache Hudi Open Table Format +slug: apache-hudi +description: Connecting to Apache Hudi Catalogs +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Catalog Documentation +Apache Hudi documentation is [here](https://hudi.apache.org/docs/overview/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Catalog Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/iceberg.md b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/iceberg.md new file mode 100644 index 00000000..8c34a651 --- /dev/null +++ b/content/en/docs/PlaidCloud/Analyze/connectors/open-table-connections/iceberg.md @@ -0,0 +1,20 @@ +--- +title: Apache Iceberg Open Table Format +slug: apache-iceberg +description: Connecting to Apache Iceberg Catalogs +weight: 1.0 +date: 2025-10-21T07:39:51 +--- + + +## Catalog Documentation +Apache Iceberg documentation is [here](https://iceberg.apache.org/docs/latest/). + +## Security Requirements +Documentation under development + +## Obtain Credentials +Documentation under development + +## Create Catalog Connection +Documentation under development diff --git a/content/en/docs/PlaidCloud/PySpark/_index.md b/content/en/docs/PlaidCloud/PySpark/_index.md new file mode 100644 index 00000000..82ec470d --- /dev/null +++ b/content/en/docs/PlaidCloud/PySpark/_index.md @@ -0,0 +1,6 @@ +--- +linktitle: PySpark and Spark Compute Clusters +title: PySpark and Spark Compute Clusters +Description: Build and deploy PySpark applications and utilize Spark compute clusters for large data analysis +weight: 6.0 +--- \ No newline at end of file diff --git a/content/en/docs/PlaidCloud/PySpark/getting_started.md b/content/en/docs/PlaidCloud/PySpark/getting_started.md new file mode 100644 index 00000000..be12c81d --- /dev/null +++ b/content/en/docs/PlaidCloud/PySpark/getting_started.md @@ -0,0 +1,17 @@ +--- +title: Getting Started with PySpark +slug: using-pyspark +description: Using PySpark in UDFs and Notebooks +date: 2022-01-25T07:39:48 +weight: 2.0 +--- + +## PySpark Documentation +PySpark is similar to using Pandas but allows for distributed compute and is not RAM bound. PySpark is available in both UDFs and Jupyter Notebooks. + +## Spark Cluster +By default, workspaces do not have the Spark cluster enabled. To activate the Spark Cluster, go to the Workspace management app and enable the "Spark Compute Cluster" service. + +Once activated, Spark jobs can be submitted to the cluster. + +The cluster can be monitored from the `spark` sub-domain for the Workspace (e.g. `https://spark.my_workspace.plaid.cloud`) \ No newline at end of file diff --git a/content/en/docs/PlaidCloud/how-to/_index.md b/content/en/docs/PlaidCloud/how-to/_index.md new file mode 100644 index 00000000..9372dc93 --- /dev/null +++ b/content/en/docs/PlaidCloud/how-to/_index.md @@ -0,0 +1,6 @@ +--- +linktitle: How To +title: How To +Description: An area dedicated to helping PlaidCloud builders solve problems in unique ways +weight: 20.0 +--- \ No newline at end of file diff --git a/content/en/docs/PlaidCloud/how-to/selecting-latest-record-in-large-history-table.md b/content/en/docs/PlaidCloud/how-to/selecting-latest-record-in-large-history-table.md new file mode 100644 index 00000000..01e3650d --- /dev/null +++ b/content/en/docs/PlaidCloud/how-to/selecting-latest-record-in-large-history-table.md @@ -0,0 +1,70 @@ +--- +linktitle: Selecting the Latest Record in a Large Version History Table +title: Selecting the Latest Record in a Large Version History Table +Description: This demonstrates the use of a window function to perform a select of the latest version of a record with a single query and no joins +weight: 1.0 +--- + +## Challenge + +A table that contains many versions of each record is available but you must use the latest version. + +## Discussion + +This problem could be solved by selecting the ID and MAX update date into a temporary table. Then that temporary table could be INNER JOINED back to the history table to obtain the result. Unfortunately, this requires two steps and storing an intermediate table that has no function other than finding the latest update. + +The more elegant solution to perform this operation in a single query uses a Window Function with sort plus a filter. + +## Solution + +### The version history table + +| employee_id | department | salary | update_date | +|-------------|------------|-----------|--------------| +| 3 | IT | 90000 | 2024-09-17 | +| 2 | HR | 85000 | 2024-09-17 | +| 5 | HR | 82000 | 2024-09-17 | +| 3 | IT | 77000 | 2023-10-01 | +| 3 | IT | 75000 | 2022-10-04 | +| 5 | IT | 72000 | 2024-07-12 | +| 2 | IT | 67000 | 2024-03-18 | +| 1 | Sales | 62000 | 2022-02-28 | +| 5 | Sales | 60000 | 2023-01-14 | +| 4 | Sales | 58000 | 2021-11-19 | + +### Step Setup +Using an extract step, create a window function expression in a column called `Rank` like: + +```python +func.rank().over(order_by=table.updated_date.desc(), partition_by=table.employee_id) +``` + +On the filter tab in the Extract step, set a filter like: + +```python +table.Rank == 1 +``` + +### The Result + +| employee_id | department | salary | update_date | Rank | +|-------------|------------|-----------|--------------|------| +| 3 | IT | 90000 | 2024-09-17 | 1 | +| 2 | HR | 85000 | 2024-09-17 | 1 | +| 5 | HR | 82000 | 2024-09-17 | 1 | +| 1 | Sales | 62000 | 2022-02-28 | 1 | +| 4 | Sales | 58000 | 2021-11-19 | 1 | + +This approach is highly efficient and allows selection of the latest record in a multi-version history table in a single step. This works by ranking each record within the `employee_id` group by the `update_date` and then only picking the first record. + +If there are multiple columns that make up the unique row key, you can add them to the `partition_by` argument as a list like: + +```python +partition_by=[table.first_column, table.second_column, table.third_column] +``` + +If you need to apply multi-column sorts you can apply that with a list of columns too like: + +```python +order_by=[table.first_column.desc(), table.second_column, table.third_column.desc()] +```