Web Development Pathway

Some of these tutorials may use Sublime, or Atom - it is STRONGLY recommended that you use vscode. Some of these may use MongoDB. DO NOT USE MONGODB. Instead, use Prisma and either PostgreSQL or MySQL. Some of these use React/Next. We STRONGLY reccommend Vue/Nuxt over React/Next.

Breaking out of just reading tutorials https://www.codewell.cc/blog/how-to-escape-tutorial-hell-and-start-building-your-own-projects

The Odin Project https://www.theodinproject.com/paths/full-stack-javascript

CSS for absolute beginners https://www.youtube.com/watch?v=yfoY53QXEnI

Visual guide to CSS Selectors https://fffuel.co/css-selectors/

JS Crash Course https://www.youtube.com/watch?v=hdI2bqOjy3c

CSS Grid https://scrimba.com/learn/cssgrid

CSS Flexbox https://www.youtube.com/playlist?list=PLC3y8-rFHvwg6rjbiMadCILrjh7QkvzoQ

HTML for absolute beginners https://www.youtube.com/watch?v=UB1O30fR-EE

Overview of the Chrome DevTools - note that every browser engine (Firefox, Chrome/Chromium, Safari) has it's own equivalent https://developer.chrome.com/docs/devtools/

NodeJS For the API section: Express Database section: Prisma + Postgresql Testing: Vitest https://roadmap.sh/nodejs

Vue Roadmap https://roadmap.sh/vue https://vuejs.org

JS Overview https://javascript.info

JS Roadmap https://roadmap.sh/javascript

Creating a Webpage with CSS, HTML, and JS

Prerequisites:

VSCode/VSCodium (or text editor of choice)
Familiar with CLI - navigating to folders, running commands
Python (should still come preinstalled on macOS, may need to upgrade to python 3)

Serving a website/webapp

Web content is delivered to the users browser from the server via HTTP/HTTPS. We can achieve this in our local dev environment in several ways. Here we look at directly opening a file and using pythons http.server module.

Create an HTML file

Open a text editor and create a new file called index.html
Add the following HTML:

<html>
  <head>
    <title>My Page</title>
  </head>
  <body>
    <h1>Hello World!</h1>
    <p>This is my web page.</p>
  </body>
</html>

Save the file

View in browser

Open your web browser (Chrome, Firefox etc)
Go to File > Open File
Select the index.html file you just created
The web page with "Hello World!" text will open

Serve with Python

Open terminal/command prompt
Navigate to directory containing index.html
Run: python -m http.server 8000
Open browser and go to http://localhost:8000
You will see your index.html rendered
Press Ctrl+C in terminal to stop the server

Creating and styling elements

Add HTML elements

In index.html:

<div class="header">
  <h1>My Website</h1>
  <nav>
    <a href="#">Link 1</a>
    <a href="#">Link 2</a>
    <a href="#">Link 3</a>
  </nav>
</div>

<div class="content">
  <div class="card">
    <h2>Card Title</h2>
    <p>This is some card content</p>
  </div>

  <div class="card">
    <h2>Another Card</h2>
    <p>Some content for this card</p>
  </div>
</div>

CSS Tags

CSS can go in two places. <style></style> tags can contain class definitions. You can also write the rules for an element directly in the style attribute of any HTML element.

<style>
  /* this all needs to be valid CSS - also this is how you do comments in CSS */
  /* this is a css class */
  .example {
    /* this is a 'rule' */
    padding: 50px;
  }
</style>
<!-- example of an HTML comment, and the next line is an example of providing styles via style attribute -->
<p style="background-color: red; color: blue;"></p>
<div class="header">
  <h1>My Website</h1>
  <nav>
    <a href="#">Link 1</a>
    <a href="#">Link 2</a>
    <a href="#">Link 3</a>
  </nav>
</div>

<div class="content">
  <div class="card">
    <h2>Card Title</h2>
    <p>This is some card content</p>
  </div>

  <div class="card">
    <h2>Another Card</h2>
    <p>Some content for this card</p>
  </div>
</div>

Grid Layout

The grid layout allows elements to be organized in rows and columns:

.content {
  display: grid;
  grid-template-columns: 1fr 1fr;
}

Flexbox Layout

Flexbox allows responsive alignment using flex properties:

.header {
  display: flex;
  align-items: center;
}

This by default will render as a horizontal row. You can make it vertical with flex-direction: column;

Block Layout

This is the default display mode.

Block displays elements stacked vertically, at full width:

.card {
  display: block;
}

Styling and Transitions

.header {
  background: #eee;
  padding: 20px;
}

.card {
  background: white;
  border: 1px solid #ddd;
  padding: 20px;
  transition: box-shadow 0.3s;
}

.card:hover {
  box-shadow: 0 0 10px rgba(0, 0, 0, 0.2);
}

Brief discussion of JS

There will be words here that you are not familiar with - look them up or ask!

Vanilla JS

JS was created in 1995 to add dynamic interactivity to web pages
Early JS was simple scripts that manipulated the DOM (Document Object Model)
Issues with loading complex projects due to async fetching
Kinda hacky/janky, but in a lovable way

jQuery

jQuery was released in 2006 and quickly became popular
It simplified DOM manipulation, AJAX (network calls), events etc with easy-to-use methods
Still no guarantee an import will exist by the time it needs to be executed

JS Frameworks

AngularJS (2010) and other frameworks emerged to manage code complexity
React (2013) and Vue (2014) provided component architectures
Allowed building complex SPAs efficiently
Two-way data binding
Markup languages other than HTML such as Pug and JSX

Bundlers like Webpack/Rollup

As apps grew, bundling tools like Webpack were needed
They bundle JS modules and assets into optimized files
Enables features like code splitting for efficient delivery
Solve the issues of async asset loading in browsers

Meta Frameworks

Next.js (2016) and Nuxt (2017) built on React and Vue
Provide server-side rendering, routing, optimizations
Complete framework for building web applications
Deeper integration between frontend and backend code

The progression has been towards more powerful abstractions to manage complexity and enable faster development.

JSON (JS Object Notation)

JSON is a text-based format for representing structured data
Based on JS object syntax, available for all major languages
JSON uses key-value pairs, arrays, and objects to store data
Keys and strings are enclosed in double quotes
JSON is very machine AND human readable
JSON is lightweight and more compact than alternatives like XML
JSON data is transmitted as plain text, MIME type application/json
Can be converted to/from native JS objects

Node.js

Node.js allows running JS on the server
Having one language across the full stack simplifies a lot of things

// Load HTTP module
const http = require("http");

// Create server
const server = http.createServer((req, res) => {
  // Handle request
});

// Listen on port
server.listen(3000);

Express

Express is a web framework for Node.js
Makes it easier to build web apps and APIs
H3 is a more modern HTTP framework

// Load express
const express = require("express");

// Create app
const app = express();

// Handle GET request
app.get("/api/items", (req, res) => {
  // Return JSON response
  res.json([{ id: 1 }, { id: 2 }]);
});

// Listen on port
app.listen(3000);

Async/Await & Promises

The Promise API and async/await are both features in JavaScript that help handle asynchronous operations. They are used to write cleaner and more readable code when dealing with asynchronous tasks like making network requests or handling database operations.

The Promise API is a way to handle asynchronous operations using a chain of methods, such as .then() and .catch(). Here's an example:

fetch("https://api.example.com/data")
  .then(response => response.json())
  .then(data => console.log(data))
  .catch(error => console.log(error));

In this example, we use the fetch() function to make a network request to retrieve data from an API. The .then() method is used to handle the response and extract the JSON data. The second .then() method is used to log the data to the console. If an error occurs during the request, the .catch() method is used to handle and log the error.

On the other hand, async/await is a more modern approach to handle asynchronous operations in JavaScript. It allows you to write asynchronous code that looks and behaves more like synchronous code. Here's an example:

async function fetchData() {
  try {
    const response = await fetch("https://api.example.com/data");
    const data = await response.json();
    console.log(data);
  } catch (error) {
    console.log(error);
  }
}

fetchData();

In this example, we define an async function called fetchData() that uses the await keyword to pause the execution until the promise is resolved or rejected. The try block is used to wrap the code that may throw an error, and the catch block is used to handle any errors that occur during the execution.

Comparing the two approaches, async/await offers a more concise and readable way to write asynchronous code. It eliminates the need for a long chain of .then() methods, making the code easier to understand and maintain. Additionally, async/await allows you to handle errors using a more familiar try-catch syntax, which makes error handling more straightforward.

However, it's important to note that async/await is built on top of the Promise API. Under the hood, async/await still uses promises, so they are not completely different concepts but rather a different way of using promises.

Also note that you can only use the await keyword inside of an async function, although there are some special cases where await can be used at the top level of a file (i.e. not inside of an async function).

REST APIs

REST APIs provide data via HTTP endpoints
Express allows building REST APIs simply
Route parameters - extract parts of an URL to use as variables
Query parameters - https://example.com/?queryParam1=value&queryParam2=5
Body - GET requests by convention do not allow a body, must use route/query parameters

GET /api/items - get items
POST /api/item - create item
PUT /api/item/1 - update item with id 1
DELETE /api/item/1 - delete item with id 1

Network calls using fetch

Making a GET request:

async function getMovies() {
  const response = await fetch("http://example.com/movies.json");
  const data = await response.json(); // this reads the body of the response into a JSON object

  console.log(data);
}

getMovies();

Note that with GET requests, you should not provide a request body. Pass information along in the query parameters, which the server can extract and use.

async function getMovies() {

  // Add query parameters
  const queryParams = '?limit=10&genre=action';

  const response = await fetch('http://example.com/movies.json' + queryParams);

  const data = await response.json();

  console.log(data);

}

getMovies();```

Making a POST request:
``` js
async function addMovie() {
  const response = await fetch('http://example.com/movies', {
    method: 'POST',
    body: JSON.stringify({title: 'Star Wars'}) // note that the body MUST be a string
  });

  const data = await response.json();

  console.log(data);
}

addMovie();

DELETE and UPDATE requests work in the same way as POST.

Use JS to interact with users

Add button and text input

In HTML:

<button id="increment">Increment</button>

<input id="nameInput" type="text" />

<p>Counter: <span id="counter">0</span></p>

<p>Name: <span id="name"></span></p>

JS

JS will always go inside of a <script></script> tag. It can be loaded over network or be written directly inside the tag.

<script src="https://path/to/js/file"></script> will load a specific file over network.

An example inside a script tag:

<script>
  console.log("this is javascript");
</script>

Add the following to the above HTML:

// Get elements
/* these are javascript objects that represent elements in the page itself  */
const incrementBtn = document.getElementById("increment");
const nameInput = document.getElementById("nameInput");
const counterEl = document.getElementById("counter");
const nameEl = document.getElementById("name");

// Increment counter on click
let counter = 0;
/* this will run the function when the element emits the 'click' event - elements have a bunch of events they can emit */
incrementBtn.addEventListener("click", () => {
  // this is an arrow function
  counter++;
  counterEl.textContent = counter;
});

// Update name on input change
nameInput.addEventListener("input", () => {
  nameEl.textContent = nameInput.value;
});

Creating a project with a JS framework

A simple Single Page Application with Vue

Event listeners vs Two way data binding

Vanilla JavaScript Event Listeners

Add event listeners directly to DOM elements to listen for events like click, change, keyup etc.
Need to manually update state and re-render UI on each event.
Example:

// Get input element
const input = document.getElementById("input");

// Add change event listener
input.addEventListener("change", event => {
  // Manually update state
  state.inputValue = event.target.value;

  // Re-render UI with new state
  renderInput();
});

function renderInput() {
  // Update input with state.inputValue
}

Vue Two-Way Data Binding

Declaratively bind data properties to DOM with v-model directive.
Reactivity system automatically detects changes and re-renders.

Example:

// In Vue component
data() {
  return {
    inputValue: ''
  }
}```

// Template

inputValue will automatically update and re-render when input changes without needing event listener.

npm/pnpm/yarn and package management

npm

Default package manager for Node.js
Stores all versions of packages in node_modules folder
Large storage footprint due to storing multiple copies of packages
package.json lists project dependencies and metadata
Run npm install to install dependencies from package.json yarn
Alternative package manager developed by Facebook
Uses same package.json structure as npm
Installs packages faster than npm in some cases
More robust checksum security than npm
More emojis! pnpm
Alternative package manager focused on efficiency
Uses a content-addressable storage model to avoid duplicate packages
Links duplicated packages together via hard links
Smaller storage footprint than npm
Faster installs than npm
Supports npm packages and package.json file Package.json:
Defines app name, version, description
Lists project dependencies and versions
Can define scripts, config etc
Required for any Node.js project using a package manager
Running npm install or yarn will install all dependencies listed in package.json

Project setup

TODO: need to go over project setup

npm install vue@next pug typescript

App.vue

<script setup lang="ts">
  import LearningExamples from "./components/LearningExamples.vue";
</script>

<template lang="pug"> #app LearningExamples </template>

LearningExamples.vue

<script setup lang="ts">
  import { ref } from "vue";

  const counter = ref<number>(0);
  const name = ref<string>("");

  const increment = () => {
    counter.value++;
  };
</script>

<template lang="pug">
  div button(@click="increment") Increment input(v-model="name") p Counter: {{
  counter }} p Name: {{ name }}
</template>

You should be able to figure out how to use CSS to make the above examples look nice.

Using a server & database

Prisma

Prisma is an open-source ORM for Node.js and TypeScript. It provides a type-safe database client auto-generated from a data model. It also has features for managing the database schema and handling migrations.

Declarative data modeling using schema.prisma
Auto-generated and type-safe database client
Query building and validation
Database migrations
Works with all major databases

Express & H3

Express and H3 are frameworks for setting up easy REST servers. They provide support for routing and utilities for reading data out of query parameters and request bodies, managing cookies, redirects, etc. They are both used in essentially the same way, but H3 provides a composable, functional interface vs the more OOP interface of Express. H3 is used by Nuxt under the hood as part of its file-based routing system.

Both frameworks have extensive ecosystem support. H3 is aided by an easy interface for compatibility with Express oriented libraries.

H3 and Express do have some provided database adapters, but we want to leverage Prisma's additional schema management features.

Example Prisma schema

Key points:

The datasource block defines a PostgreSQL database connection.
The generator block specifies we want a Prisma Client JS library generated.
User model has name, email, and a relation to Todo items. id is generated with cuid().
Todo model has title, optional content, and a relation to User via the userId foreign key.
The @relation and @references directives establish the 1:M relationship between User and Todo.

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

generator client {
  provider        = "prisma-client-js"
}

model User {
  id    String @id @default(cuid())
  name  String
  email String @unique
  Todos Todo[] // one to many relationship: one user has many Todos
}

model Todo {
  id     String @id @default(cuid())
  title  String
  content String?
  userId String
  User   User @relation(fields: [userId], references: [id]) // other side, many to one relationship: one Todo belongs to one User
}

Corresponding class diagram

classDiagram

class User {
  +id: String
  +name: String
  +email: String
  +todos: Todo[]
}

class Todo {
  +id: String
  +title: String
  +content: String
  +userId: String
}

User "1" --o "0..*" Todo : has
Todo "0..*" --o "1" User : belongs to

Example Express implementation

// user routes elided for brevity
const express = require("express");
const { PrismaClient } = require("@prisma/client");

const prisma = new PrismaClient();
const app = express();

app.use(express.json());

// Create todo
app.post("/todos", async (req, res) => {
  const todo = await prisma.todo.create({
    data: {
      title: req.body.title,
      content: req.body.content,
      user: { connect: { id: req.body.userId } },
    },
  });
  res.json(todo);
});

// Update todo
app.put("/todos/:id", async (req, res) => {
  const todo = await prisma.todo.update({
    where: { id: req.params.id },
    data: req.body,
  });
  res.json(todo);
});

// Get all todos
app.get("/todos", async (req, res) => {
  const todos = await prisma.todo.findMany();
  res.json(todos);
});

// Get single todo
app.get("/todos/:id", async (req, res) => {
  const todo = await prisma.todo.findUnique({
    where: { id: req.params.id },
  });
  res.json(todo);
});

app.listen(3000, () => {
  console.log("Server started on http://localhost:3000");
});

Example H3 implementation

// user routes elided for brevity
import { createApp, eventHandler, toNodeListener } from "h3";
import { readBody } from "@h3/composables";
import { PrismaClient } from "@prisma/client";
import { createServer } from "node:http";
const prisma = new PrismaClient();
const app = createApp();
// Todo Routes

app.post(
  "/todos",
  eventHandler(async event => {
    const body = await readBody(event);

    return prisma.todo.create({
      data: {
        title: body.title,
        content: body.content,
        user: { connect: { id: body.userId } },
      },
    });
  })
);

app.get(
  "/todos",
  eventHandler(async () => {
    return prisma.todo.findMany();
  })
);

app.get(
  "/todos/:id",
  eventHandler(async event => {
    return prisma.todo.findUnique({
      where: { id: event.params.id },
    });
  })
);

app.put(
  "/todos/:id",
  eventHandler(async event => {
    const body = await readBody(event);

    return prisma.todo.update({
      where: { id: event.params.id },
      data: body,
    });
  })
);

app.delete(
  "/todos/:id",
  eventHandler(async event => {
    return prisma.todo.delete({
      where: { id: event.params.id },
    });
  })
);
createServer(toNodeListener(app)).listen(3000);

Metaframeworks

Metaframeworks (Next, Nuxt, SvelteKit, etc) combine a frontend framework and a backend framework in one overarching metaframework. Instead of managing a separate API and frontend, they live in the same codebase. This makes several things much more straightforward - you don't have to juggle types or utility functions that are used in both, you can do neat stuff with SSR/SSG, and some common boilerplate gets removed. For example, in Nuxt you can write an API call such that when a page using that API call is loaded, the server makes the API call before sending the page to the client - this means that the client doesn't have to wait for the page to load and then make the API call, so the user gets a better experience.

Databases

A database is a structured collection of data stored in a computer system. Databases allow you to efficiently store, organize, and query large amounts of data.

The data in a database is organized into tables, with rows representing individual records or items, and columns representing the attributes of each item. For example, a table storing customer data might have columns for name, address, phone number, etc. To retrieve or manipulate data in a database, you use a query language like SQL (Structured Query Language). SQL allows you to write queries to select specific data, filter rows, join tables, update records, and more. Non-technical users can interact with a database through a custom frontend application.

There are two main types of database models:

SQL databases are relational databases, where data is structured in relations (tables) and data has relationships between tables. SQL databases use SQL and are optimized for complex queries. Examples are MySQL, Oracle, SQL Server.
NoSQL databases have flexible, non-tabular data models. They are optimized for scalability and high performance on large volumes of data. But they sacrifice some functionality of relational SQL models. Examples are MongoDB, Cassandra, Redis.

SQL databases are better for complex queries, relationships between data, and data consistency. NoSQL is better for simplicity of design, handling big data, and flexibility. Most real-world systems use a mix of SQL and NoSQL databases as needed. For example, a SQL database to store core business data, and a NoSQL cache layer to quickly serve common queries.

Generally speaking you will end up with an SQL server for your core schema, and Redis, Memcached, or some other key-value store for your caching layer, if you even need one - in some cases you don't need a database cache, you can simply write to static files use a CDN like Cloudfront or Fastly.

You will almost never really need to reach for something like MongoDB outside of specific use cases that you will not encounter at the scale of an EPICS project or even most business applications. Remember that business people can usually speak SQL, and their reporting tooling always does.

Browser Dev Tools

On most browsers, ctrl+shift+i, or right click on a webpage and look for 'Inspect'. Overview of the most important tabs below.

Note that most frameworks have plugins for the dev tools that enable debugging of framework stuff.

There are lots of useful buttons. See here. These look a little different in Firefox vs Safari vs Chromium (Brave, Edge, Chrome, Vivaldi, etc)

Inspector

View and modify the DOM and CSS
Inspect element styles and layout
Debug accessibility issues
Change CSS styles dynamically (as in, change the appearance of a page without touching the actual source)

Console

Log messages (console.log outputs here) and debug JS
Execute JS
Interact with page scripts
Inspect variable values
View network requests/errors

Network

Monitor network requests and responses
Inspect request headers, params, response codes
Check load times and transfer sizes
Filter requests by type (XHR, JS, CSS, etc.)
View detailed timing breakdown

FilesExpand file tree

webdev_pathway.md

Latest commit

History