devops-bootcamp-02-linux/Exercises.md at main · fsiegrist/devops-bootcamp-02-linux

Exercises

Exercise 1: Linux Mint Virtual Machine

Create a Linux Mint Virtual Machine on your computer. Check the distribution, which package manager it uses (yum, apt, apt-get). Which CLI editor is configured (Nano, Vi, Vim). What software center/software manager it uses. Which shell is configured for your user.

Download and install UTM (VirtualBox does not yet run on Apple M2) from

https://mac.getutm.app/

Download linuxmint-21.1-cinnamon-64bit.iso from

https://linuxmint.com/download.php

Installation Guide:

https://linuxmint-installation-guide.readthedocs.io/en/latest/

Start UTM and create new VM based on downloaded .iso file. => Cannot start the OS

Try installing Ubuntu.

Download Ubuntu 22.04.1 LTS Server for ARM from

https://cdimage.ubuntu.com/releases/22.04/release/ubuntu-22.04.1-live-server-arm64.iso

Installation Guide (from https://docs.getutm.app/guides/ubuntu/):

Open UTM and click the “+” button to open the VM creation wizard.
Select “Virtualize”.
Select “Linux”.
Click “Browse” and select the Ubuntu Server ISO downloaded from the link above. Press “Next” to continue.
Pick the amount of RAM and CPU cores you wish to give access to the VM. Press “Next” to continue.
Specify the maximum amount of drive space to allocate. Press “Next” to continue.
If you have a directory you want to mount in the VM, you can select it here. Alternatively, you can skip this and select the directory later from the VM window’s toolbar. The shared directory will be available after installing SPICE tools (see below). Press “Next” to continue.
Press “Save” to create the VM and press the Run button to start the VM.
Go through the Ubuntu installer. If the reboot fails, you can manually quit the VM, unmount the installer ISO, and start the VM again to boot into your new installation.

If you installed Ubuntu Server, then at the end of the installation, you will not have any GUI. To install Ubuntu Desktop, log in and run:

$ sudo apt update
$ sudo apt install ubuntu-desktop
$ sudo reboot

Exercise 2: Bash Script - Install Java

Write a bash script using Vim editor that installs the latest java version and checks whether java was installed successfully by executing a java -version command.

After installation command, it checks 3 conditions:

whether java is installed at all
whether an older Java version is installed (java version lower than 11)
whether a java version of 11 or higher was installed

It prints relevant informative messages for all 3 conditions. Installation was successful if the 3rd condition is met and you have Java version 11 or higher available.

script:

#!/bin/bash

function installation_needed {
  java_version=$(java -version 2>&1 >/dev/null | grep "java version\|openjdk version" | awk '{print $3}')
  java_version_short=$(java -version 2>&1 >/dev/null | grep "java version\|openjdk version" | awk '{print substr($3,2,2)}')

  if [[ "$java_version" = "" ]]
  then
    echo "No java installation found."
    return 1
  elif [[ "$java_version_short" = "1." ]]
  then
    echo "Old java version $java_version found."
    return 1
  elif [[ "$java_version_short" -ge 11 ]]
  then
    echo "Java version $java_version found."
    return 0
  fi
}

function install_java {
  echo "Installing current JDK..."
  apt-get update
  apt-get install -y default-jre
}

function check_installation {
  installation_needed
  if [[ $? -eq 0 ]]
  then
    echo "The new java version >= 11 was successfully installed."
  else
    echo "The new Java version >= 11 could NOT be installed!"
  fi
}

function main {
  installation_needed
  if [[ $? -eq 1 ]]
  then
    install_java
    check_installation
  else
    echo "Java with version greater or equal 11 is already installed."
  fi
}

main

Execute script with sudo!

Breakdown of command that gets java version:

java -version gives you the complete version output.
> /dev/null 2>&1 addition does following things: > redirects the output of java -version command to a file /dev/null, which is a special type of file, that accepts and discards all input written to it, and with 2>&1, even the error output of the java -version command will be discarded by /dev/null file. So described together, this option takes the output, including any errors generated by the java -version command and discards it, not showing it on the command-line and silently forwarding it using | (pipe) to the next command, which is grep "java version\|openjdk version"
grep "java version\|openjdk version" simply finds a line in the output that has "java version" or "openjdk version" in it. The example line will look like this: openjdk version "11.0.16" 2022-07-19
awk '{print substr($3,2,2)} takes the line from the previous output and grabs the third section of the string "11.0.16" and from there grabs the first 2 characters, which will be "11"

*Detailed explanation of 2>&1:

Every time, we execute a program or a command, operating system opens three files: standard input, standard output, and standard error, and each file gets a file descriptor integer from the OS: 0, 1, and 2, respectively. So 2>&1 simply says redirect standard error (2) to standard output (1). The & before 1 in this case, means whatever follows is a file descriptor, not a filename.

Explanation of if else script:

In the if else checks, we check if the $java_version variable has no value at all or empty value, it means we have no java installation at all
If you have an older version of java already installed, like 1.6, 1.7, 1.8, then the value of "$java_version" will be "1." - first 2 characters. So with "$java_version" == "1.", we check whether java_version variable is "1.". This means installing latest java version was not successful, since you still have only the old version.
In case of success you should get java version which is 11 or higher ("$java_version" -ge 11), which will print success message.

Exercise 3, 4, 5: Bash Script - User Processes

1. Write a bash script using Vim editor that checks all the processes running for the current user (USER env var) and prints out the processes in console. Hint: use ps aux command and grep for the user.
1. Extend the previous script to ask for a user input for sorting the processes output either by memory or CPU consumption, and print the sorted list.
1. Extend the previous script to ask additionally for user input about how many processes to print. Hint: use head program to limit the number of outputs.

script:

#!/bin/bash

echo -n "Would you like to sort the processes output by memory or CPU? (m/c) "
read sortby
echo -n "How many results do you want to display? "
read lines

if [ "$sortby" = "m" ]
then
    ps aux --sort -%mem | grep "PID\|`whoami`" | head -n "$lines"
elif [ "$sortby" = "c" ]
then
    ps aux --sort -%cpu | grep -i "PID\|$USER" | head -n "$lines"
else
    echo "No input provided. Exiting"
fi

echo -n: -n: do not output the trailing newline ps aux --sort -%cpu: sort by cpu utilization of the process in "##.#" format. Currently, it is the CPU time used divided by the time the process has been running (cputime/realtime ratio), expressed as a percentage. ps aux --sort -%mem: sort by ratio of the process's resident set size to the physical memory on the machine, expressed as a percentage ps aux --sort -rss: sort by resident set size, the non-swapped physical memory that a task has used (in kilobytes)

Exercise 6, 7: Start NodeJs App

Write a bash script with following logic:

Install NodeJS and NPM and print out which versions were installed
Download an artifact file from the URL: https://node-envvars-artifact.s3.eu-west-2.amazonaws.com/bootcamp-node-envvars-project-1.0.0.tgz. Hint: use curl or wget
Unzip the downloaded file
Set the following needed environment variables: APP_ENV=dev, DB_USER=myuser, DB_PWD=mysecret
Change into the unzipped package directory
Run the NodeJS application by executing the following commands: npm install and node server.js

Notes:

Make sure to run the application in background so that it doesn't block the terminal session where you execute the shell script
If any of the variables is not set, the node app will print error message that env vars is not set and exit
It will give you a warning about LOG_DIR variable not set. You can ignore it for now.

Extend the script to check after running the application that the application has successfully started and prints out the application's running process and the port where it's listening.

script:

#!/bin/bash

# prepare environment, install all tools
apt-get update

echo "installing node, npm, wget, net-tools..."
apt-get install -y nodejs npm wget net-tools  
sleep 15
echo ""
echo "################"
echo ""

# display nodeJS version
node_version=$(node --version)
echo "NodeJS version $node_version installed"

# display npm version
npm_version=$(npm --version)
echo "NPM version $npm_version installed"

echo ""
echo "################"
echo ""

if [[ ! -d "package" ]]
then
  if [[ ! -f "bootcamp-node-envvars-project-1.0.0.tgz" ]]
  then
    # fetch NodeJS project archive from s3 bucket
    wget https://node-envvars-artifact.s3.eu-west-2.amazonaws.com/bootcamp-node-envvars-project-1.0.0.tgz
  fi
  # extract the project archive to ./package folder
  tar zxvf bootcamp-node-envvars-project-1.0.0.tgz
fi

# set all needed environment variables
export APP_ENV=dev
export DB_PWD=mysecret
export DB_USER=myuser 

# change to package directory
cd package

# install application dependencies
npm install

# start the nodejs application in the background
node server.js &

# display that nodejs process is running
ps aux | grep node | grep -v grep

# display that nodejs is running on port 3000
netstat -ltnp | grep :3000

Exercise 8, 9: Start NodeJs App with Service user & Log Directory

Extend the script to accept a parameter input log_directory: a directory where application will write logs.

The script will check whether the parameter value is a directory name that doesn't exist and will create the directory, if it does exist, it sets the env var LOG_DIR to the directory's absolute path before running the application, so the application can read the LOG_DIR environment variable and write its logs there.

Note:

Check the app.log file in the provided LOG_DIR directory.
This is what the output of running the application must look like:

app listening on port 3000!
-----------------------------
successfully set the following environment variables: 
APP_ENV - dev
DB_USER - myuser
DB_PWD - mysecret
-----------------------------
successfully set LOG_DIR environment variable. Writing logs into /home/fesi/Documents/nodejs/logs
-----------------------------
Logging into the app.log file
Application started successfully using all the provided environment variables
Node app will listen for any incoming connections
This is the end of log entries
-----------------------------

You've been running the application with your user. But we need to adjust that and create own service user: myapp for the application to run. So extend the script to create the user and then run the application with the service user.

script:

#!/bin/bash

# prepare environment, install all tools
apt-get update
NEW_USER=myapp

echo "install node, npm, wget, net-tools"
apt-get install -y nodejs npm wget net-tools  
sleep 15
echo ""
echo "################"
echo ""

# display nodeJS version
node_version=$(node --version)
echo "NodeJS version $node_version installed"

# display npm version
npm_version=$(npm --version)
echo "NPM version $npm_version installed"

echo ""
echo "################"
echo ""

# read user input for log directory
echo -n "Set log directory location for the application (absolute path): "
read LOG_DIRECTORY
if [ -d $LOG_DIRECTORY ]
then
  echo "$LOG_DIRECTORY already exists"
else
  mkdir -p $LOG_DIRECTORY
  echo "A new directory $LOG_DIRECTORY has been created"
fi

# create new user to run the application and make owner of log dir
useradd $NEW_USER -m
chown $NEW_USER -R $LOG_DIRECTORY

# executing the following commands as new user using 'runuser' command

# fetch NodeJS project archive from s3 bucket
runuser -l $NEW_USER -c "wget https://node-envvars-artifact.s3.eu-west-2.amazonaws.com/bootcamp-node-envvars-project-1.0.0.tgz"

# extract the project archive to ./package folder
runuser -l $NEW_USER -c "tar zxvf ./bootcamp-node-envvars-project-1.0.0.tgz"

# start the nodejs application in the background, with all needed env vars with new user myapp
runuser -l $NEW_USER -c "
    export APP_ENV=dev && 
    export DB_PWD=mysecret && 
    export DB_USER=myuser && 
    export LOG_DIR=$LOG_DIRECTORY && 
    cd package && 
    npm install && 
    node server.js &"

# display that nodejs process is running
ps aux | grep node | grep -v grep

# display that nodejs is running on port 3000
netstat -tlnp | grep :3000

Execute script with sudo command

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exercises

FilesExpand file tree

Exercises.md

Latest commit

History

Exercises.md

File metadata and controls

Exercises