diff --git a/02_activities/assignments/DC_Cohort/.Rhistory b/02_activities/assignments/DC_Cohort/.Rhistory new file mode 100644 index 000000000..e69de29bb diff --git a/02_activities/assignments/DC_Cohort/Assignment1.md b/02_activities/assignments/DC_Cohort/Assignment1.md index f78778f5b..68063968c 100644 --- a/02_activities/assignments/DC_Cohort/Assignment1.md +++ b/02_activities/assignments/DC_Cohort/Assignment1.md @@ -106,6 +106,8 @@ Please do not pick the exact same tables that I have already diagrammed. For exa - - The column names can be found in a few spots (DB Schema window in the bottom right, the Database Structure tab in the main window by expanding each table entry, at the top of the Browse Data tab in the main window) +## embedding my diagram here +![conceptual model between customer purchases and customer](farmersmarket.drawio.png) *** ## Section 2: @@ -204,6 +206,13 @@ Link if you encounter a paywall: https://archive.is/srKHV or https://web.archive Consider, for example, concepts of fariness, inequality, social structures, marginalization, intersection of technology and society, etc. + ``` Your thoughts... +one value system I can think of is the social structure of race. This can be found often when filling out legal, government, or medical forms. +Like gender mentioned in the article, often race categories that we have to select from are too 'binary'. Many people are mixed race but are +forced to choose one option to identify as in these documents which can be confusing and creating unnecessary biases when interacting with the +people/administrators that have access to these forms. For example, someones appearance may not match another peoples preconceived idea +of what they should look like. As well, I feel that it may force people to unconsciously to identify with a single category rather than embracing +who they are and their diverse background. ``` diff --git a/02_activities/assignments/DC_Cohort/Assignment2.md b/02_activities/assignments/DC_Cohort/Assignment2.md index 9b804e9ee..cf498cc01 100644 --- a/02_activities/assignments/DC_Cohort/Assignment2.md +++ b/02_activities/assignments/DC_Cohort/Assignment2.md @@ -54,7 +54,10 @@ The store wants to keep customer addresses. Propose two architectures for the CU **HINT:** search type 1 vs type 2 slowly changing dimensions. ``` -Your answer... +Your answer... + +Type 1: this architecture will overwrite data with new values. This table will store 1 row per customer with their street address, city, province, country, postal code, customerID, dateofupdate to track when it was last changed. +Type 2: this architecture will store historical data and not overwrite when a new address is added. The table will store each new address per customer per row with their street address, city, province, country, postal code, customerID, dateadded (to track when this address was entered), and activeaddress(a YES/NO or any boolean data type that marks if this is their current address to be used) ``` *** @@ -184,4 +187,11 @@ Consider, for example, concepts of labour, bias, LLM proliferation, moderating c ``` Your thoughts... +The ethical issues important to this story are: +Labour exploitation: the first part of the article touches on the forgotten contributions +of human labour and skill in the industrialized world, where we in developed countries +do not consider the human need in the product of our everyday fast products that we always +assume as been automated by technology. In the same way humans are at the root of clothing manufacturing, +humans are also at the root of machine learning, neural networks and AI, being paid very little for their irreplaceable work + ``` diff --git a/02_activities/assignments/DC_Cohort/assignment1.sql b/02_activities/assignments/DC_Cohort/assignment1.sql index c992e3205..0edfb1b3d 100644 --- a/02_activities/assignments/DC_Cohort/assignment1.sql +++ b/02_activities/assignments/DC_Cohort/assignment1.sql @@ -4,18 +4,21 @@ --SELECT /* 1. Write a query that returns everything in the customer table. */ - - +SELECT * +FROM customer /* 2. Write a query that displays all of the columns and 10 rows from the cus- tomer table, sorted by customer_last_name, then customer_first_ name. */ - - +SELECT * +FROM customer +ORDER BY customer_last_name, customer_first_name +LIMIT 10 --WHERE /* 1. Write a query that returns all customer purchases of product IDs 4 and 9. */ - - +SELECT * +FROM customer_purchases +WHERE product_id = 4 OR product_id= 9 /*2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_to_customer_per_qty), filtered by customer IDs between 8 and 10 (inclusive) using either: @@ -24,47 +27,85 @@ filtered by customer IDs between 8 and 10 (inclusive) using either: */ -- option 1 - -- option 2 - - +SELECT*, +(quantity*cost_to_customer_per_qty )AS price +FROM customer_purchases +WHERE customer_id BETWEEN 8 AND 10 +ORDER BY customer_id --CASE /* 1. Products can be sold by the individual unit or by bulk measures like lbs. or oz. Using the product table, write a query that outputs the product_id and product_name columns and add a column called prod_qty_type_condensed that displays the word “unit” if the product_qty_type is “unit,” and otherwise displays the word “bulk.” */ - - +SELECT product_id, +product_name, +CASE WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' +END as prod_qty_type_condensed +FROM product /* 2. We want to flag all of the different types of pepper products that are sold at the market. add a column to the previous query called pepper_flag that outputs a 1 if the product_name contains the word “pepper” (regardless of capitalization), and otherwise outputs 0. */ - - +SELECT product_id, +product_name, +CASE WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' +END as prod_qty_type_condensed, +CASE WHEN product_name LIKE '%pepper%' THEN '1' + ELSE '0' +END as pepper_flag +FROM product --JOIN /* 1. Write a query that INNER JOINs the vendor table to the vendor_booth_assignments table on the vendor_id field they both have in common, and sorts the result by vendor_name, then market_date. */ - - - +SELECT +vendor_name, +vendor_type, +vendor_owner_first_name, +vendor_owner_last_name, +booth_number, +market_date +FROM vendor as v +INNER JOIN vendor_booth_assignments as vba + ON v.vendor_id = vba.vendor_id +ORDER BY vendor_name, market_date /* SECTION 3 */ -- AGGREGATE /* 1. Write a query that determines how many times each vendor has rented a booth at the farmer’s market by counting the vendor booth assignments per vendor_id. */ - - +SELECT vendor_id, +COUNT(booth_number) as num_rented +FROM vendor_booth_assignments +GROUP BY vendor_id /* 2. The Farmer’s Market Customer Appreciation Committee wants to give a bumper sticker to everyone who has ever spent more than $2000 at the market. Write a query that generates a list of customers for them to give stickers to, sorted by last name, then first name. HINT: This query requires you to join two tables, use an aggregate function, and use the HAVING keyword. */ - - +SELECT +customer_first_name, --from customer +customer_last_name, -- from customer +SUM(quantity*cost_to_customer_per_qty) as total_spend -- from customer_purchases +FROM customer as c +INNER JOIN customer_purchases as cp + ON c.customer_id = cp.customer_id + GROUP BY customer_last_name, customer_first_name +HAVING total_spend >2000 + + +/*checking to see that all purchases are included for each person +SELECT customer_id, +SUM(quantity*cost_to_customer_per_qty) as total +FROM customer_purchases +WHERE customer_id = 4 +*/ --Temp Table /* 1. Insert the original vendor table into a temp.new_vendor and then add a 10th vendor: @@ -77,10 +118,16 @@ When inserting the new vendor, you need to appropriately align the columns to be -> To insert the new row use VALUES, specifying the value you want for each column: VALUES(col1,col2,col3,col4,col5) */ +CREATE TABLE temp.new_vendor AS + +SELECT* +FROM vendor +INSERT INTO temp.new_vendor VALUES +(10,'Thomass Superfood Store','Fresh Focused','Thomas','Rosenthal') --- Date +-- Date: DO NOT DO /*1. Get the customer_id, month, and year (in separate columns) of every purchase in the customer_purchases table. HINT: you might need to search for strfrtime modifers sqlite on the web to know what the modifers for month diff --git a/02_activities/assignments/DC_Cohort/assignment2.sql b/02_activities/assignments/DC_Cohort/assignment2.sql index 5ad40748a..c724aec79 100644 --- a/02_activities/assignments/DC_Cohort/assignment2.sql +++ b/02_activities/assignments/DC_Cohort/assignment2.sql @@ -20,6 +20,11 @@ The `||` values concatenate the columns into strings. Edit the appropriate columns -- you're making two edits -- and the NULL rows will be fixed. All the other rows will remain the same.) */ +SELECT +product_name || ', ' || coalesce(product_size,' ')|| ' (' || coalesce(product_qty_type,'unit') || ')' + +FROM product; + --Windowed Functions @@ -33,16 +38,34 @@ each new market date for each customer, or select only the unique market dates p HINT: One of these approaches uses ROW_NUMBER() and one uses DENSE_RANK(). */ +SELECT*, +ROW_NUMBER()OVER(PARTITION BY customer_id ORDER BY market_date ASC) as [visit_number] +FROM customer_purchases; + /* 2. Reverse the numbering of the query from a part so each customer’s most recent visit is labeled 1, then write another query that uses this one as a subquery (or temp table) and filters the results to only the customer’s most recent visit. */ +SELECT*, +ROW_NUMBER()OVER(PARTITION BY customer_id ORDER BY market_date DESC) as [visit_number] +FROM customer_purchases; +SELECT customer_id, market_date FROM ( +SELECT*, +ROW_NUMBER()OVER(PARTITION BY customer_id ORDER BY market_date DESC) as [visit_number] +FROM customer_purchases +) +WHERE visit_number = 1; /* 3. Using a COUNT() window function, include a value along with each row of the customer_purchases table that indicates how many different times that customer has purchased that product_id. */ + +SELECT customer_id, product_id, market_date, +COUNT(product_id) as times_purchased +FROM customer_purchases +GROUP BY customer_id, product_id; -- String manipulations @@ -56,11 +79,28 @@ Remove any trailing or leading whitespaces. Don't just use a case statement for | Habanero Peppers - Organic | Organic | Hint: you might need to use INSTR(product_name,'-') to find the hyphens. INSTR will help split the column. */ - +SELECT *, + CASE + WHEN + INSTR(product_name, '-') > 0 THEN + TRIM(product_name) + ELSE + NULL + END as captured +FROM product; /* 2. Filter the query to show any product_size value that contain a number with REGEXP. */ - +SELECT *, + CASE + WHEN + INSTR(product_name, '-') > 0 THEN + TRIM(product_name) + ELSE + NULL + END as captured +FROM product +WHERE product_size REGEXP '[0-9]'; -- UNION @@ -73,9 +113,46 @@ HINT: There are a possibly a few ways to do this query, but if you're struggling 3) Query the second temp table twice, once for the best day, once for the worst day, with a UNION binding them. */ - - - +--1) +DROP TABLE IF EXISTS temp.customer_purchases; +CREATE TABLE temp.customer_purchases AS +--WITH customer_purchases_total AS ( +SELECT market_date, +SUM(quantity*cost_to_customer_per_qty) as totalperday +FROM customer_purchases +GROUP BY market_date; + + +--2) best day = 1 +DROP TABLE IF EXISTS temp.customer_purchases_rankedmax; +CREATE TABLE temp.customer_purchases_rankedmax AS +WITH rankmax AS ( + + SELECT market_date, totalperday, + RANK() OVER(ORDER BY totalperday DESC) as [orderedtotal] + FROM temp.customer_purchases + ) +SELECT * FROM rankmax +WHERE orderedtotal=1; + +--3) worst day = 1 + +DROP TABLE IF EXISTS temp.customer_purchases_rankedmin; +CREATE TABLE temp.customer_purchases_rankedmin AS +WITH rankmin AS ( + + SELECT market_date, totalperday, + RANK() OVER(ORDER BY totalperday ASC) as [orderedtotal] + FROM temp.customer_purchases + ) +SELECT * FROM rankmin +WHERE orderedtotal = 1; + +DROP TABLE IF EXISTS temp.customer_purchases_ranked; +CREATE TABLE temp.customer_purchases_ranked AS +SELECT * FROM temp.customer_purchases_rankedmax +UNION +SELECT * FROM temp.customer_purchases_rankedmin; /* SECTION 3 */ -- Cross Join @@ -88,28 +165,44 @@ Remember, CROSS JOIN will explode your table rows, so CROSS JOIN should likely b Think a bit about the row counts: how many distinct vendors, product names are there (x)? How many customers are there (y). Before your final group by you should have the product of those two queries (x*y). */ - - +CREATE TABLE temp.rev AS +SELECT +v.vendor_name, +p.product_name, +(vi.original_price*5*c.customercount) AS totalmade +FROM vendor_inventory as vi +JOIN vendor v ON vi.vendor_id = v.vendor_id +JOIN product p ON vi.product_id = p.product_id +CROSS JOIN +(SELECT COUNT(*) AS customercount from customer) c --counts all rows since each customer has 1 ROW +GROUP BY v.vendor_name, p.product_name, vi.original_price; -- INSERT /*1. Create a new table "product_units". This table will contain only products where the `product_qty_type = 'unit'`. It should use all of the columns from the product table, as well as a new column for the `CURRENT_TIMESTAMP`. Name the timestamp column `snapshot_timestamp`. */ +DROP TABLE IF EXISTS product_units; +CREATE TABLE product_units AS +SELECT *, +CURRENT_TIMESTAMP as [snapshot_timestamp] +FROM product +WHERE product_qty_type = 'unit'; /*2. Using `INSERT`, add a new row to the product_units table (with an updated timestamp). This can be any product you desire (e.g. add another record for Apple Pie). */ - - +INSERT INTO product_units +VALUES(33,'Poblano Peppers - Organic','large',1,'unit',CURRENT_TIMESTAMP); -- DELETE /* 1. Delete the older record for the whatever product you added. HINT: If you don't specify a WHERE clause, you are going to have a bad time.*/ - +DELETE FROM product_units +WHERE product_id = 33; -- UPDATE /* 1.We want to add the current_quantity to the product_units table. @@ -129,5 +222,35 @@ Finally, make sure you have a WHERE statement to update the right row, When you have all of these components, you can run the update statement. */ +/* +DISCLAIMER I DID NOT DO THIS EXACTLY CORRECTLY. I GIVE UP - SPENT WAY TOO MUCH TIME TRYING TO FIGURE IT OUT, +THE FINAL TABLE OUTPUT IS CORRECT BUT TABLE NAME IS NOT. SORRY :< IM SAD + + + +*/ +ALTER TABLE product_units +ADD current_quantity INT; + +DROP TABLE IF EXISTS gut; +CREATE TABLE gut AS +SELECT*, coalesce(quantity,0) as quantityfix +FROM product_units as pu +LEFT JOIN ( +SELECT product_id, quantity, max(market_date) +FROM vendor_inventory +GROUP BY product_id) as vi +ON pu.product_id = vi.product_id; + +CREATE TABLE product_units1 AS +SELECT pu.*, +g.quantityfix +FROM product_units as pu +LEFT JOIN gut as g + ON pu.product_id = g.product_id; + +UPDATE product_units1 +SET current_quantity = quantityfix; + diff --git a/02_activities/assignments/DC_Cohort/bookstore.drawio.png b/02_activities/assignments/DC_Cohort/bookstore.drawio.png new file mode 100644 index 000000000..237177b7c Binary files /dev/null and b/02_activities/assignments/DC_Cohort/bookstore.drawio.png differ diff --git a/02_activities/assignments/DC_Cohort/farmersmarket.drawio.png b/02_activities/assignments/DC_Cohort/farmersmarket.drawio.png new file mode 100644 index 000000000..1fa42520a Binary files /dev/null and b/02_activities/assignments/DC_Cohort/farmersmarket.drawio.png differ diff --git a/04_this_cohort/live_code/DC/module_2/module_2.sqbpro b/04_this_cohort/live_code/DC/module_2/module_2.sqbpro index 40c710eeb..bdca500ba 100644 --- a/04_this_cohort/live_code/DC/module_2/module_2.sqbpro +++ b/04_this_cohort/live_code/DC/module_2/module_2.sqbpro @@ -1,4 +1,4 @@ -
/* MODULE 2 */ +
/* MODULE 2 */ /* SELECT */ @@ -26,7 +26,7 @@ FROM customer; /* 5. Add a static value in a column */ SELECT 2025 as this_year, 'October' as this_month, customer_id FROM customer -/* MODULE 2 */ +/* MODULE 2 */ /* WHERE */ /* 1. Select only customer 1 from the customer table */ @@ -76,7 +76,7 @@ FROM market_date_info WHERE market_date BETWEEN '2022-10-01' AND '2022-10-31' AND market_date = 'Wednesday'; -/* MODULE 2 */ +/* MODULE 2 */ /* CASE */ @@ -116,7 +116,7 @@ FROM customer_purchases -/* MODULE 2 */ +/* MODULE 2 */ /* DISTINCT */ @@ -157,7 +157,7 @@ FROM customer_purchases; ... AND to whom was it sold*/ SELECT DISTINCT vendor_id, product_id,customer_id FROM customer_purchases -/* MODULE 2 */ +/* MODULE 2 */ /* INNER JOIN */ @@ -206,7 +206,7 @@ INNER JOIN customer_purchases -/* MODULE 2 */ +/* MODULE 2 */ /* LEFT JOIN */ @@ -270,7 +270,7 @@ LEFT JOIN product_category AS pc ON pc.product_category_id = p.product_category_id ORDER by pc.product_category_id -/* MODULE 2 */ +/* MODULE 2 */ /* Multiple Table JOINs */ @@ -278,10 +278,38 @@ LEFT JOIN product_category AS pc (Which vendor has sold products to a customer AND which product was it AND to whom was it sold) Replace all the IDs (customer, vendor, and product) with the names instead*/ +SELECT DISTINCT +--v.vendor_id +vendor_name +--, product_id +,product_name +--,customer_id -- first/last name +,customer_first_name +,customer_last_name +FROM customer_purchases as cp +INNER JOIN customer as c + ON c.customer_id = cp.customer_id +INNER JOIN vendor as v + ON v.vendor_id = cp.vendor_id +INNER JOIN product as p + ON p.product_id = cp.product_id; -/* 2. Select product_category_name, everything from the product table, and then LEFT JOIN the customer_purchases table +/* 2. Select product_category_name, everything from the product table, +and then LEFT JOIN the customer_purchases table ... how does this LEFT JOIN affect the number of rows? -Why do we have more rows now?*/ +Why do we have more rows now?*/ + +SELECT product_category_name, p.*, cp.product_id as productid_in_purchases_table + +FROM product_category as pc +INNER JOIN product as p + ON p.product_category_id = pc.product_category_id +LEFT JOIN customer_purchases as cp + ON cp.product_id = p.product_id + +ORDER BY cp.product_id + + diff --git a/04_this_cohort/live_code/DC/module_3/module_3.sqbpro b/04_this_cohort/live_code/DC/module_3/module_3.sqbpro index d3b452b67..3636a6be1 100644 --- a/04_this_cohort/live_code/DC/module_3/module_3.sqbpro +++ b/04_this_cohort/live_code/DC/module_3/module_3.sqbpro @@ -1,124 +1,226 @@ -
/* MODULE 3 */ +
/* MODULE 3 */ /* COUNT */ /* 1. Count the number of products */ - + SELECT COUNT(product_id) as num_of_product + FROM product; /* 2. How many products per product_qty_type */ - - +SELECT product_qty_type, COUNT(product_id) as num_of_product +FROM product +GROUP BY product_qty_type; /* 3. How many products per product_qty_type and per their product_size */ +SELECT product_size +,product_qty_type +, COUNT(product_id) as num_of_product +FROM product +GROUP BY product_size, product_qty_type - +ORDER BY product_qty_type; /* COUNT DISTINCT 4. How many unique products were bought */ +SELECT COUNT(DISTINCT product_id) as bought_prods +FROM customer_purchases - -/* MODULE 3 */ +/* MODULE 3 */ /* SUM & AVG */ -/* 1. How much did customers spend each day */ +/* 1. How much did customers spend each (per) day */ +SELECT +market_date +,customer_id +,SUM(quantity*cost_to_customer_per_qty) as total_spend +FROM customer_purchases +GROUP BY market_date,customer_id; /* 2. How much does each customer spend on average */ +SELECT +customer_first_name +,customer_last_name +,ROUND(AVG(quantity*cost_to_customer_per_qty),2) as avg_spend + +FROM customer_purchases as cp +INNER JOIN customer as c + ON c.customer_id = cp.customer_id +GROUP BY c.customer_id -/* MODULE 3 */ + +/* MODULE 3 */ /* MIN & MAX */ /* 1. What is the most expensive product ...pay attention to how it doesn't handle ties very well */ +SELECT product_name, max(original_price) as most_expensive + +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id; /* 2. Prove that max is working */ +SELECT DISTINCT +product_name, +original_price +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id; /* 3. Find the minimum price per each product_qty_type */ +SELECT product_name +,product_qty_type +,min(original_price) +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id +GROUP BY product_qty_type; /* 4. Prove that min is working */ +SELECT DISTINCT product_name +,product_qty_type +--,min(original_price) +,original_price - +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id; /* 5. Min/max on a string ... not particularly useful? */ +SELECT max(product_name) +FROM product - -/* MODULE 3 */ +/* MODULE 3 */ /* Arithmitic */ /* 1. power, pi(), ceiling, division, integer division, etc */ -SELECT +SELECT power(4,2), pi(); +SELECT 10.0 / 3.0 as division, +CAST(10.0 as INT) / CAST(3.0 as INT) as integer_division; /* 2. Every even vendor_id with modulo */ - - +SELECT * FROM vendor +WHERE vendor_id % 2 = 0; /* 3. What about every third? */ +SELECT * FROM vendor +WHERE vendor_id % 3 = 0; -/* MODULE 3 */ +/* MODULE 3 */ /* HAVING */ /* 1. How much did a customer spend on each day? -Filter to customer_id between 1 and 5 and total_cost > 50 +Filter to customer_id between 1 and 5 and total_spend > 50 ... What order of execution occurs?*/ - +SELECT +market_date +,customer_id +,SUM(quantity*cost_to_customer_per_qty) as total_spend +FROM customer_purchases +WHERE customer_id BETWEEN 1 AND 5 + +GROUP BY market_date, customer_id +HAVING total_spend > 50; /* 2. How many products were bought? Filter to number of purchases between 300 and 500 */ +SELECT count(product_id) as num_of_prod, product_id +FROM customer_purchases +GROUP BY product_id +HAVING count(product_id) BETWEEN 300 AND 500 + -/* MODULE 3 */ +/* MODULE 3 */ /* Subquery FROM */ /*1. Simple subquery in a FROM statement, e.g. for inflation ...we could imagine joining this to a more complex query perhaps */ +SELECT DISTINCT +product_id, inflation - +FROM ( + SELECT product_id, cost_to_customer_per_qty, + CASE WHEN cost_to_customer_per_qty < '1.00' THEN cost_to_customer_per_qty*5 + ELSE cost_to_customer_per_qty END AS inflation + + FROM customer_purchases +); /* 2. What is the single item that has been bought in the greatest quantity?*/ +--outer query +SELECT product_name, MAX(quantity_purchased) +FROM product AS p +INNER JOIN ( +-- inner query + SELECT product_id + ,count(quantity) as quantity_purchased -/* MODULE 3 */ + FROM customer_purchases + GROUP BY product_id +) AS x ON p.product_id = x.product_id + +/* MODULE 3 */ /* Subquery WHERE */ /* 1. How much did each customer spend at each vendor for each day at the market WHEN IT RAINS */ +SELECT +market_date, +customer_id, +vendor_id, +SUM(quantity*cost_to_customer_per_qty) as total_spend +FROM customer_purchases +-- filter by rain_flag +-- "what dates was it raining" +WHERE market_date IN ( + SELECT market_date + FROM market_date_info + WHERE market_rain_flag = 1 +) -/* 2. What is the name of the vendor who sells pie */ - -/* MODULE 3 */ -/* Common Table Expression (CTE) */ - +GROUP BY market_date, vendor_id, customer_id; -/* 1. Calculate sales per vendor per day */ -SELECT +/* 2. What is the name of the vendor who sells pie */ +SELECT DISTINCT vendor_name +FROM customer_purchases as cp +INNER JOIN vendor as v + ON cp.vendor_id = v.vendor_id + +WHERE product_id IN ( + SELECT product_id + FROM product + WHERE product_name LIKE '%pie%' +) -/* ... re-aggregate the daily sales for each WEEK instead now */ -/* MODULE 3 */ +/* MODULE 3 */ /* Temp Tables */ @@ -134,28 +236,76 @@ DROP TABLE IF EXISTS temp.new_vendor_inventory; CREATE TABLE temp.new_vendor_inventory AS -- definition of the table +SELECT *, +original_price*5 as inflation +FROM vendor_inventory; +/* 2. put the previous table into another temp table, e.g. as temp.new_new_vendor_inventory */ +DROP TABLE IF EXISTS temp.new_new_vendor_inventory; +CREATE TABLE temp.new_new_vendor_inventory AS +SELECT * +,inflation*2 as super_inflation +FROM temp.new_vendor_inventory +/* MODULE 3 */ +/* Common Table Expression (CTE) */ -/* 2. put the previous table into another temp table, e.g. as temp.new_new_vendor_inventory */ +/* 1. Calculate sales per vendor per day */ +WITH vendor_daily_sales AS ( + SELECT + md.market_date + ,market_day + ,market_week + ,market_year + ,vendor_name + ,SUM(quantity*cost_to_customer_per_qty) as sales -/* MODULE 3 */ -/* Date functions */ + FROM customer_purchases cp + INNER JOIN vendor v -- we want the vendor_name + ON v.vendor_id = cp.vendor_id + INNER JOIN market_date_info md -- all the date columns + ON cp.market_date = md.market_date + GROUP BY md.market_date, v.vendor_id + +) + + +/* ... re-aggregate the daily sales for each WEEK instead now */ -/* 1. now */ SELECT +market_year +,market_week +,vendor_name +,SUM(sales) as sales +FROM vendor_daily_sales +GROUP BY market_year,market_week, vendor_name -/* 2. strftime */ +/* MODULE 3 */ +/* Date functions */ -/* 3. adding dates, e.g. last date of the month */ +/* 1. now */ +SELECT DISTINCT +DATE('now') as [now] +,DATETIME('now') [rightnow] +,DATE() as[todaysdate] + +/* 2. strftime */ +,strftime('%Y/%m','now') as [1] +,strftime('%Y-%m-%d','now','+50 days') as the_future +,market_date +,strftime('%m-%d-%Y',market_date,'+50 days','-1 year') as the_past + +/* 3. adding dates, e.g. last date of the month */ +--last date of last month +,DATE(market_date,'start of month','-1 day') as end_of_prev_month /* 4. difference between dates, @@ -163,4 +313,10 @@ SELECT b. number of YEARS between now and market_date c. number of HOURS bewtween now and market_date */ + ,market_date + ,julianday('now') - julianday(market_date) as datesbetweenmktdate -- number of days between now and each market date +,(julianday('now') - julianday(market_date)) / 365.25 +,(julianday('now') - julianday(market_date)) * 24 + + FROM market_date_info diff --git a/04_this_cohort/live_code/DC/module_4/module_4.sqbpro b/04_this_cohort/live_code/DC/module_4/module_4.sqbpro index b1f922c2e..70169e47f 100644 --- a/04_this_cohort/live_code/DC/module_4/module_4.sqbpro +++ b/04_this_cohort/live_code/DC/module_4/module_4.sqbpro @@ -1,22 +1,35 @@ -
/* MODULE 4 */ +
/* MODULE 4 */ /* NULL Management */ /* 1. IFNULL: Missing product_size, missing product_qty_type */ - - +SELECT*, +IFNULL(product_size,'Unknown') as new_product_size, +--replace with another COLUMN +IFNULL(product_size, product_qty_type) as silly_replacement /* 2. Coalesce */ +coalesce(product_size, product_qty_type,'missing') as more_replacements, -- if the first value is null, then the second value +coalesce(product_qty_type,product_size,'missing') as more_replacements +,IFNULL(IFNULL(product_size,product_qty_type),'missing') as equiv_replacements -- if the first value is null, then the second value, ... if the second value is null, then "missing" +FROM product; /* 3. NULLIF finding values in the product_size column that are "blank" strings and setting them to NULL if they are blank */ - +SELECT* +--,IFNULL(product_size,'Unknown') +--,NULLIF(product_size,'') --when product_size is blank, return NULL +,coalesce(NULLIF(product_size,''),'Unknown')replaced_values +FROM product; /* 4. NULLIF filtering which rows are null or blank */ - +SELECT* +FROM product +WHERE NULLIF(product_size,'')IS NULL +--theres more code here that I did not capture, rewatch video /* MODULE 4 */ /* NULLIF Budget (example from the slides) */ @@ -36,14 +49,14 @@ CREATE TEMP TABLE IF NOT EXISTS temp.budgets ( dept STRING ,current_year INT ,previous_year INT -); +) INSERT INTO temp.budgets VALUES ('software',1000,1000) , ('candles',NULL,500) , ('coffee', 400, 200) -, ('pencils',0, 50); +, ('pencils',0, 50) /*examine each of these columns */ @@ -59,24 +72,93 @@ FROM budgets https://learn.microsoft.com/en-us/sql/t-sql/language-elements/nullif-transact-sql?view=sql-server-ver17 */ -/* MODULE 4 */ -/* Windowed functions: row_number */ +/* MODULE 4 THIS IS COPIED FROM CODE SHARE, COMPARE WITH ABOVE TO CHECK FOR CHANGES*/ +/* NULLIF Budget (example from the slides) */ +/* The following example creates a budgets table to show a department (dept) +...its current budget (current_year) and its previous budget (previous_year). -/* 1. What product is the highest price per vendor */ +For the current year, NULL is used for departments with budgets that have not changed from the previous year, +and 0 is used for budgets that have not yet been determined. + +To find out the average of only those departments that receive a budget and to include the budget value +from the previous year (use the previous_year value, where the current_year is NULL), +combine the NULLIF and COALESCE functions. */ + +DROP TABLE IF EXISTS temp.budgets; +CREATE TEMP TABLE IF NOT EXISTS temp.budgets ( +dept STRING +,current_year INT +,previous_year INT +); + + +INSERT INTO temp.budgets VALUES +('software',1000,1000) +, ('candles',NULL,500) +, ('coffee', 400, 200) +, ('pencils',0, 50); +/*examine each of these columns */ +SELECT +NULLIF(current_year, previous_year) +--,NULLIF(COALESCE(current_year, previous_year), 0.00) +--, +--AVG(NULLIF(COALESCE(current_year, previous_year), 0.00)) +FROM budgets -/* See how this varies from using max due to the group by + +/* more NULLIF here: +https://learn.microsoft.com/en-us/sql/t-sql/language-elements/nullif-transact-sql?view=sql-server-ver17 +*//* MODULE 4 */ +/* Windowed functions: row_number */ + + +/* 1. What product is the highest price per vendor */ +--outer QUERY +SELECT x.*, +product_name +-- inner QUERY +FROM +( + SELECT + vendor_id, + product_id, + original_price, + ROW_NUMBER() OVER(PARTITION BY vendor_id ORDER BY original_price DESC) as price_rank + -- if we add product_id we could potentially track the time where original_price changed + ,ROW_NUMBER() OVER(PARTITION BY vendor_id, product_id ORDER BY original_price DESC) + FROM vendor_inventory +)x +INNER JOIN product p + ON x.product_id=p.product_id + +WHERE x.price_rank=1 +/* See how this varies from using max due to the group by */ SELECT vendor_id, ---product_id, +product_id, MAX(original_price) FROM vendor_inventory -GROUP BY vendor_id--,product_id +GROUP BY vendor_id, product_id -*/ +/*highest single purchase in a day per customer*/ +SELECT * +FROM ( +SELECT + customer_id, + product_id, + market_date, + quantity, + quantity*cost_to_customer_per_qty as cost, + ROW_NUMBER() OVER(PARTITION BY customer_id ORDER BY quantity*cost_to_customer_per_qty DESC) as sales_rank + + FROM customer_purchases +) x +WHERE x.sales_rank=1 +ORDER BY cost desc /* MODULE 4 */ /* Windowed functions: dense_rank, rank, row_number */ @@ -104,14 +186,23 @@ VALUES (9, 165000), (10, 100000); +SELECT * +,ROW_NUMBER() OVER(ORDER BY salary DESC) as [row_number], +RANK() OVER (ORDER BY salary DESC) as [rank], +DENSE_RANK() OVER(ORDER BY salary DESC) as [dense_rank] + +FROM row_rank_dense /* MODULE 4 */ /* Windowed functions: NTILE */ -/* 1. Calculate quartile, quntiles, and percentiles from vendor daily sales */ - - +/* 1. Calculate quartile, quintiles, and percentiles from vendor daily sales */ +SELECT *, +NTILE(4) OVER(PARTITION BY vendor_name ORDER BY sales ASC) as [quartile], +NTILE(5) OVER(PARTITION BY vendor_name ORDER BY sales ASC) as [quintile], +NTILE(100) OVER(PARTITION BY vendor_name ORDER BY sales ASC) as [percentile] +FROM ( -- vendor daily sales SELECT md.market_date @@ -120,6 +211,7 @@ VALUES ,market_year ,vendor_name ,SUM(quantity*cost_to_customer_per_qty) AS sales + FROM customer_purchases AS cp JOIN market_date_info AS md @@ -127,15 +219,27 @@ VALUES JOIN vendor AS v ON v.vendor_id = cp.vendor_id - GROUP BY cp.market_date, v.vendor_id/* MODULE 4 */ + GROUP BY cp.market_date, v.vendor_id + )x/* MODULE 4 */ /* String Manipulations */ /* 1. ltrim, rtrim, trim*/ SELECT +LTRIM(' THOMAS ROSENTHAL ') as [ltrim], +RTRIM(' THOMAS ROSENTHAL ') as [rtrim], +TRIM(' THOMAS ROSENTHAL ') as [trim], +LTRIM(RTRIM(' THOMAS ROSENTHAL ')) as [both] /* 2. replace*/ - +,REPLACE('THOMAS ROSENTHAL','','WILLIAM') as full_name +,REPLACE('THOMAS ROSENTHAL','A','E') as a_to_e +,REPLACE('THOMAS ROSENTHAL','A','') as no_a +,customer_first_name +,REPLACE(customer_first_name,'a','e') as cust_a_to_e +,REPLACE(customer_first_name,'a',customer_last_name) as chaos + +FROM customer /* 3. upper, lower*/ /* 4. concat with || */ @@ -172,7 +276,31 @@ SELECT /* 1. Find the most and least expensive product by vendor with UNION (and row_number!) */ -/* MODULE 4 */ +SELECT vendor_id, product_id, original_price, rn_max +FROM( + SELECT + vendor_id, + product_id, + original_price, + ROW_NUMBER(), OVER(PARTITION BY vendor_id ORDER BY original_price DESC) as rn_max + + FROM vendor_inventory +) +WHERE rn_max=1; + +UNION --add union inbetween the 2 queries. union returned 5 rows...union all returned 6 rows (vendor #4 was duplicated) + +SELECT vendor_id, product_id, original_price, rn_min +FROM( + SELECT + vendor_id, + product_id, + original_price, + ROW_NUMBER(), OVER(PARTITION BY vendor_id ORDER BY original_price ASC) as rn_min + + FROM vendor_inventory +) +WHERE rn_min=1/* MODULE 4 */ /* UNION */ /* 1. Emulate a FULL OUTER JOIN with a UNION */ @@ -199,23 +327,65 @@ quantity INT INSERT INTO temp.store2 VALUES("tiger",2), ("dancer",7), - ("superhero", 5);/* MODULE 4 */ + ("superhero", 5); + +SELECT s1.costume, s1.quantity as store1_quantity, s2.quantity as store2_quantity +FROM store1 s1 +LEFT JOIN store2 s2 + ON s1.costume=s2.costume + +UNION + +SELECT s2.costume, s1.quantity, s2.quantity +FROM store2 s2 +LEFT JOIN store1 s1 + ON s1.costume=s2.costume +WHERE s1.csotume is NULL + +ORDER bY s1.quantity, s2.quantity/* MODULE 4 */ /* INTERSECT & EXCEPT */ /* 1. Find products that have been sold (e.g. are in customer purchases AND product) */ - +SELECT product_id +FROM customer_purchases +INTERSECT +SELECT product_id +FROM product; /* 2. Find products that have NOT been sold (e.g. are NOT in customer purchases even though in product) */ - - +--direction matters +SELECT x.product_id, product_name +FROM ( + SELECT product_id + FROM product + EXCEPT + SELECT product_id + FROM customer_purchases +) x +JOIN product p on x.product_id = p.product_id; /* 3. Directions matter... if we switch the order here: products that do not exist, because no products purchased are NOT in the product table (e.g. are NOT in product even though in customer purchases)*/ - - +--returning 0 rows +SELECT product_id +FROM customer_purchases +EXCEPT +SELECT product_id +FROM product; /* 4. We can remake the intersect with a WHERE subquery for more details ... */ - - +SELECT * +FROM product +WHERE product_id IN + ( + SELECT product_id + FROM customer_purchases + INTERSECT + SELECT product_id + FROM product + ) + +ORDER BY s1.quantity DESC, s2.quantity +-- Reference to file "/Users/dianayang/Desktop/sql/02_activities/assignments/DC_Cohort/assignment2.sql" (not supported by this version) -- diff --git a/04_this_cohort/live_code/DC/module_5/module_5.sqbpro b/04_this_cohort/live_code/DC/module_5/module_5.sqbpro index f608313e0..dc0471347 100644 --- a/04_this_cohort/live_code/DC/module_5/module_5.sqbpro +++ b/04_this_cohort/live_code/DC/module_5/module_5.sqbpro @@ -1,25 +1,36 @@ -
/* MODULE 5 */ +
/* MODULE 5 */ /* INSERT UPDATE DELETE */ DROP TABLE IF EXISTS temp.product_expanded; CREATE TEMP TABLE product_expanded AS - SELECT * FROM product; + SELECT * FROM product --SELECT * FROM product_expanded /* 1. add a product to the temp table */ - +INSERT INTO product_expanded +VALUES(24,'Almonds','1 lb',3,'lbs'); /* 2. change the product_size for THAT product */ - - - -/* 3. delete the newly added product *//* MODULE 5 */ +--UPDATE +--almonds to 1/2kg +UPDATE product_expanded +SET product_size='1/2 kg',product_qty_type='kg' +WHERE product_id=24; + +/* 3. delete the newly added product */ +DELETE FROM product_expanded; +--SELECT*FROM product_expanded --can help you determine you are looking at the right rows before running a deletion +WHERE product_id=24; -- if you remove this, all data will be removed from the table + +SELECT*FROM product_expanded/* MODULE 5 */ /* VIEW */ /* 1. Create a vendor daily sales view */ +DROP VIEW IF EXISTS vendor_daily_sales; +CREATE VIEW IF NOT EXISTS vendor_daily_sales AS SELECT md.market_date @@ -36,20 +47,37 @@ CREATE TEMP TABLE product_expanded AS INNER JOIN vendor v ON cp.vendor_id = v.vendor_id - GROUP BY cp.market_date, v.vendor_id; - + GROUP BY cp.market_date, v.vendor_id + +SELECT * FROM vendor_daily_sales /* MODULE 5 */ /* VIEW in another query */ /* 1. Transform the daily sales view into a sales by vendor per week result */ +SELECT +market_year +,market_week +,vendor_name +,SUM(sales) + -/* MODULE 5 */ +FROM vendor_daily_sales + +GROUP BY +market_year +,market_week +,vendor_name + +--ORDER BY s1.quantity DESC, s2.quantity +/* MODULE 5 */ /* UPDATE statements for view */ /* 1. SET market_date equal to today for new_customer_purchases */ +UPDATE new_customer_purchases +SET market_date = DATE('now') @@ -67,10 +95,37 @@ VALUES('....','....','....','....','8:00 AM','2:00 PM','nothing interesting','Su */ -/* MODULE 5 */ +INSERT INTO market_date_info +VALUES('2025-11-11','Tuesday','46','2025','8:00 AM','2:00 PM','nothing interesting','Winter','0','3',0,1) +/* MODULE 5 */ /* DYNAMIC VIEW */ +DROP VIEW IF EXISTS todays_vendor_daily_sales; +CREATE VIEW IF NOT EXISTS todays_vendor_daily_sales AS + + SELECT + md.market_date + ,market_day + ,market_week + ,market_year + ,vendor_name + ,SUM(quantity*cost_to_customer_per_qty) as sales + + + FROM market_date_info md + INNER JOIN + (SELECT * FROM customer_purchases + UNION + SELECT * FROM new_customer_purchases + ) cp + ON md.market_date = cp.market_date + INNER JOIN vendor v + ON cp.vendor_id = v.vendor_id + + WHERE md.market_date=DATE('now') + GROUP BY cp.market_date, v.vendor_id +ORDER BY s1.quantity DESC, s2.quantity @@ -105,7 +160,7 @@ VALUES('....','....','....','....','8:00 AM','2:00 PM','nothing interesting','Su -/* MODULE 5 */ +/* MODULE 5 */ /* CROSS JOIN */ @@ -121,9 +176,14 @@ VALUES('small'), SELECT * FROM TEMP.sizes; +SELECT product_name, product_qty_type,size +FROM product --23 rows +CROSS JOIN temp.sizes -- 3 rows +-- 3*23 = 69 rows for the cartesian product +--WHERE product_qty_type = 'unit' -- maybe makes more sense, but reduces the number of row -/* MODULE 5 */ +/* MODULE 5 */ /* SELF JOIN */ @@ -144,4 +204,9 @@ VALUES(1,'Thomas',3) ,(4, 'Jennie',3); SELECT * FROM TEMP.employees; - + +SELECT e.emp_name, m.emp_name as mgr_name +FROM temp.employees e +LEFT JOIN temp.employees m + on e.mgr_id = m.emp_id +WHERE e.emp_name='Niyaz' diff --git a/05_src/sql/farmersmarket.db b/05_src/sql/farmersmarket.db index 4720f2483..2e07b66e1 100644 Binary files a/05_src/sql/farmersmarket.db and b/05_src/sql/farmersmarket.db differ