updated pyspark cheat sheet to include Snowflake example

Author: pybokeh

pyspark/PySpark_CheatSheet.ipynb

@@ -41,7 +41,7 @@
    "id": "1991eb8b-1a17-4281-babc-3221525b626b",
    "metadata": {},
    "source": [
-    "#### Read csv file without infering schema"
+    "#### Read csv file by infering or guessing schema"
    ]
   },
   {
@@ -1319,6 +1319,14 @@
     "## Connecting to Relational Databases (JDBC)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "7aaf3812-5b6b-47dc-826d-58a83ede3810",
+   "metadata": {},
+   "source": [
+    "[Link](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/from_to_dbms.html) to their documentation"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "77b439ff-808b-47f1-b64f-dc60e4d0a60a",
@@ -1753,11 +1761,96 @@
    "source": [
     "Unfortunately, the IBM Java JRE does not work with Spark 3.x and thus, we will not be able to connect to mainframe DB2 z/OS platform."
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "93e3132a-bc44-4ec4-ab60-845423e92e23",
+   "metadata": {},
+   "source": [
+    "#### Connecting to Snowflake"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1cb85857-726f-44b0-96e0-7118fb9addaa",
+   "metadata": {},
+   "source": [
+    "[Link](https://docs.snowflake.com/en/user-guide/spark-connector-install.html) to Snowflake's documentation on working with the PySpark connector"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "02ed47d5-db4a-4e64-8fcf-7ead1f22e796",
+   "metadata": {},
+   "source": [
+    "Currently have PySpark 3.2.0 installed, but Snowflake does not yet support 3.2.0.  Therefore, the code below has not yet been tested and is currently being used as a placeholder."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "50d70a07-ce07-4ae1-afc0-464db3f876f7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pyspark import SparkConf, SparkContext\n",
+    "from pyspark.sql import SQLContext"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b33039ad-a2a4-4f5f-93f8-b352022d1850",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "spark = SparkSession.builder.master(\"local[*]\").appName(\"Snowflake_JDBC\")\\\n",
+    "    .config(\"spark.jars\", \"C:\\Path_To_Snowflake_JDBC.jar\")\\\n",
+    "    .getOrCreate()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "61c85a64-3281-4eb2-b92a-f7b1430ab961",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Snowflake connection parameters\n",
+    "sfparams = {\n",
+    "  \"sfURL\" : \"<account_identifier>.snowflakecomputing.com\",\n",
+    "  \"sfUser\" : \"<user_name>\",\n",
+    "  \"sfPassword\" : \"<password>\",\n",
+    "  \"sfDatabase\" : \"<database>\",\n",
+    "  \"sfSchema\" : \"<schema>\",\n",
+    "  \"sfWarehouse\" : \"<warehouse>\"\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "61fe30c6-c039-4c77-baf1-09e3a5af219c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#read full table\n",
+    "df = spark.read.format(“snowflake”) \\\n",
+    "  .options(**sfparams) \\\n",
+    "  .option(\"dbtable\",  \"Employee\") \\\n",
+    "  .load()\n",
+    "\n",
+    "#run custom query\n",
+    "df = spark.read.format(“snowflake”) \\\n",
+    "  .options(**sfparams) \\\n",
+    "  .option(\"query\",  \"SELECT * FROM Employee\") \\\n",
+    "  .load()"
+   ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Py3.8 (pyspark_dev)",
+   "display_name": "Py3.9 (pyspark_dev)",
    "language": "python",
    "name": "pyspark_dev"
   },
@@ -1771,7 +1864,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.10"
+   "version": "3.9.8"
   }
  },
  "nbformat": 4,