diff --git a/README.md b/README.md new file mode 100644 index 0000000..e83f08f --- /dev/null +++ b/README.md @@ -0,0 +1,648 @@ +# Tiny Web Example + +# はじめに + +tiny_web_exampleはwebapiとfrontendからなる小規模な教材用アプリケーションです。 + +本書ではアプリケーションの実行に必要なパッケージのインストール手順と設定方法を記述しています。 + +テストやパッケージ作成方法については以下のディレクトリを参照してください。 + +* UnitTest: tiny_web_example/webapi/spec/README.md +* IntegrationTest: tiny_web_example/spec_integration/README.md +* RPMBuild: tiny_web_example/rpmbuild/README.md + +また以下の作業はrootユーザーで行ってください。 + +# 動作環境 + +CentOS-6.6にて動作確認を行いました。それよりも古い場合は動作確認していませんので、ご了承ください。 + +# インストール + +## epel環境の構築 + +tiny_web_exampleはmysql-serverとnginxを使用します。nginxはCentOSのBaseリポジトリには存在しないので、EPELリポジトリを利用してインストールします。 + +### epel-releaseのインストール + +``` +# rpm -ivh http://ftp.jaist.ac.jp/pub/Linux/Fedora/epel/6/x86_64/epel-release-6-8.noarch.rpm +``` + +実行結果例: +>``` +># rpm -ivh http://ftp.jaist.ac.jp/pub/Linux/Fedora/epel/6/x86_64/epel-release-6-8.noarch.rpm +>Retrieving http://ftp.jaist.ac.jp/pub/Linux/Fedora/epel/6/x86_64/epel-release-6-8.noarch.rpm +>warning: /var/tmp/rpm-tmp.AFsABI: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY +>Preparing... ########################################### [100%] +> 1:epel-release ########################################### [100%] +>``` + +### リポジトリリストの確認 + +epelリポジトリが有効になったことを確認します。 + +``` +# yum repolist +``` + +実行結果例: +>``` +># yum repolist +>Loaded plugins: fastestmirror +>Loading mirror speeds from cached hostfile +> * base: ftp.yz.yamagata-u.ac.jp +> * epel: ftp.tsukuba.wide.ad.jp +> * extras: ftp.yz.yamagata-u.ac.jp +> * updates: ftp.yz.yamagata-u.ac.jp +>repo id repo name status +>base CentOS-6.6 - Base 6,518 +>*epel Extra Packages for Enterprise Linux 6 - x86_64 11,557 +>extras CentOS-6.6 - Extras 38 +>updates CentOS-6.6 - Updates 1,134 +>repolist: 19,247 +>``` + +epelが表示されていれば、epelリポジトリは有効です。 + +## ruby環境のインストール + +tiny_web_exampleはアプリケーションの動作にruby環境が必要です。ここではrbenvを使用してrubyとbundlerのインストールを行います。 + +### rbenvのインストール +``` +# git clone https://github.com/sstephenson/rbenv.git ~/.rbenv +``` + +実行結果例: +>``` +># git clone https://github.com/sstephenson/rbenv.git ~/.rbenv +>Initialized empty Git repository in /root/.rbenv/.git/ +>remote: Counting objects: 2057, done. +>remote: Total 2057 (delta 0), reused 0 (delta 0), pack-reused 2057 +>Receiving objects: 100% (2057/2057), 344.17 KiB | 74 KiB/s, done. +>Resolving deltas: 100% (1272/1272), done. +>``` + +### rbenvの設定 +``` +# echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile +# echo 'eval "$(rbenv init -)"' >> ~/.bash_profile +# exec $SHELL -l +``` + +### ruby-buildに必要なパッケージのインストール +``` +# yum install -y gcc openssl-devel libyaml-devel libffi-devel readline-devel zlib-devel gdbm-devel ncurses-devel +``` + +実行結果例: +>``` +># yum install -y gcc openssl-devel libyaml-devel libffi-devel readline-devel zlib-devel gdbm-devel ncurses-devel +>Loaded plugins: fastestmirror +>Setting up Install Process +>Loading mirror speeds from cached hostfile +> * base: ftp.jaist.ac.jp +> * epel: ftp.jaist.ac.jp +> * extras: ftp.jaist.ac.jp +> * updates: ftp.jaist.ac.jp +>Package gcc-4.4.7-11.el6.x86_64 already installed and latest version +>Resolving Dependencies +>__(省略)__ +> +>Installed: +> gdbm-devel.x86_64 0:1.8.0-36.el6 libffi-devel.x86_64 0:3.0.5-3.2.el6 +> libyaml-devel.x86_64 0:0.1.3-4.el6_6 ncurses-devel.x86_64 0:5.7-3.20090208.el6 +> openssl-devel.x86_64 0:1.0.1e-30.el6.8 readline-devel.x86_64 0:6.0-4.el6 +> zlib-devel.x86_64 0:1.2.3-29.el6 +> +>Dependency Installed: +> keyutils-libs-devel.x86_64 0:1.4-5.el6 krb5-devel.x86_64 0:1.10.3-37.el6_6 +> libcom_err-devel.x86_64 0:1.41.12-21.el6 libselinux-devel.x86_64 0:2.0.94-5.8.el6 +> libsepol-devel.x86_64 0:2.0.41-4.el6 libyaml.x86_64 0:0.1.3-4.el6_6 +> +>Complete! +>``` + +### ruby-buildのインストール + +``` +# git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build +``` + +実行結果例: +>``` +># git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build +>Initialized empty Git repository in /root/.rbenv/plugins/ruby-build/.git/ +>remote: Counting objects: 4530, done. +>remote: Total 4530 (delta 0), reused 0 (delta 0), pack-reused 4530 +>Receiving objects: 100% (4530/4530), 847.84 KiB | 70 KiB/s, done. +>Resolving deltas: 100% (2344/2344), done. +>``` + +### rubyのインストール + +``` +# rbenv install -v 2.0.0-p598 +``` + +実行結果例: +>``` +># rbenv install -v 2.0.0-p598 +>/tmp/ruby-build.20150522151700.11775 ~ +>Downloading ruby-2.0.0-p598.tar.gz... +>HTTP/1.1 200 OK +>Content-Type: binary/octet-stream +>Content-Length: 13608640 +>Connection: keep-alive +>Date: Mon, 11 May 2015 13:57:49 GMT +>Last-Modified: Thu, 13 Nov 2014 14:55:37 GMT +>ETag: "e043a21ce0d138fd408518a80aa31bba" +>Accept-Ranges: bytes +>Server: AmazonS3 +>X-Cache: RefreshHit from cloudfront +>Via: 1.1 4565d1650806ee8cdd757034d90ec07d.cloudfront.net (CloudFront) +>X-Amz-Cf-Id: gn8DN0xoqLLljcQTklcAR2DABrq7vHhhqsmc2KrZ5Plkwtvpqz8_RA== +> +>-> http://dqw8nmjcqpjn7.cloudfront.net/4136bf7d764cbcc1c7da2824ed2826c3550f2b62af673c79ddbf9049b12095fd +>Installing ruby-2.0.0-p598... +>/tmp/ruby-build.20150522151700.11775/ruby-2.0.0-p598 /tmp/ruby-build.20150522151700.11775 ~ +>__(省略)__ +> +>Installed ruby-2.0.0-p598 to /root/.rbenv/versions/2.0.0-p598 +> +>/tmp/ruby-build.20150522152934.22095 ~ +>~ +>``` + +### rbenvの再読み込み + +``` +# rbenv rehash +``` + +### rubyの設定 + +現在インストールされているrubyのバージョン一覧を確認する +``` +# rbenv versions +``` + +実行結果例: +>``` +># rbenv versions +> 2.0.0-p598 +>``` + +### 使用するrubyのバージョンを設定する +``` +# rbenv global 2.0.0-p598 +# ruby -v +``` + +実行結果例: +>``` +># ruby -v +>ruby 2.0.0p598 (2014-11-13 revision 48408) [x86_64-linux] +>``` + +### bundlerのインストール +``` +# gem install bundler --no-ri --no-rdoc +``` + +実行結果例: +>``` +># gem install bundler --no-ri --no-rdoc +>Fetching: bundler-1.9.9.gem (100%) +>Successfully installed bundler-1.9.9 +>1 gem installed +>``` + +## Yumを使ったインストール + +すでにRPMパッケージとリポジトリを作成済みの方はYumによるインストールを推奨します。 + +### repoファイルの作成 +``` +# curl -fsSkL \ +> https://raw.githubusercontent.com/axsh/tiny_web_example/master/rpmbuild/tiny-web-example.repo \ +> -o /etc/yum.repos.d/tiny-web-example.repo +``` + +### baseurlの修正 + +baseurlを作成したリポジトリのあるサーバーのIPアドレスに変更します。 +``` +# vi /etc/yum.repos.d/tiny-web-example.repo +``` + +修正結果: +``` +[tiny-web-example] +name=tiny-web-example +baseurl=http://10.0.22.100/pub/ +enabled=1 +gpgcheck=0 +``` + +### tiny_web_exampleのインストール +``` +# yum install -y tiny-web-example +``` + +実行結果例: +>``` +># yum install -y tiny-web-example +>Loaded plugins: fastestmirror +>Setting up Install Process +>Loading mirror speeds from cached hostfile +>epel/metalink | 5.0 kB 00:00 +> * base: ftp.jaist.ac.jp +> * epel: ftp.jaist.ac.jp +> * extras: ftp.jaist.ac.jp +> * updates: ftp.jaist.ac.jp +>base | 3.7 kB 00:00 +>base/primary_db | 4.6 MB 00:00 +>epel | 4.4 kB 00:00 +>epel/primary_db | 6.5 MB 00:00 +>extras | 3.4 kB 00:00 +>extras/primary_db | 31 kB 00:00 +>tiny-web-example | 2.9 kB 00:00 +>tiny-web-example/primary_db | 3.7 kB 00:00 +>updates | 3.4 kB 00:00 +>updates/primary_db | 3.3 MB 00:00 +>Resolving Dependencies +>__(省略)__ +> +>Installed: +> tiny-web-example.x86_64 0:0.0.1-1.daily.el6 +> +>Dependency Installed: +> GeoIP.x86_64 0:1.6.5-1.el6 GeoIP-GeoLite-data.noarch 0:2015.04-2.el6 +> GeoIP-GeoLite-data-extra.noarch 0:2015.04-2.el6 compat-readline5.x86_64 0:5.2-17.1.el6 +> fontconfig.x86_64 0:2.8.0-5.el6 freetype.x86_64 0:2.3.11-15.el6_6.1 +> gd.x86_64 0:2.0.35-11.el6 geoipupdate.x86_64 0:2.2.1-2.el6 +> libX11.x86_64 0:1.6.0-2.2.el6 libX11-common.noarch 0:1.6.0-2.2.el6 +> libXau.x86_64 0:1.0.6-4.el6 libXpm.x86_64 0:3.5.10-2.el6 +> libjpeg-turbo.x86_64 0:1.2.1-3.el6_5 libpng.x86_64 2:1.2.49-1.el6_2 +> libxcb.x86_64 0:1.9.1-2.el6 libxslt.x86_64 0:1.1.26-2.el6_3.1 +> mysql.x86_64 0:5.1.73-3.el6_5 mysql-server.x86_64 0:5.1.73-3.el6_5 +> nginx.x86_64 0:1.0.15-11.el6 nginx-filesystem.noarch 0:1.0.15-11.el6 +> perl-DBD-MySQL.x86_64 0:4.013-3.el6 perl-DBI.x86_64 0:1.609-4.el6 +> ruby.x86_64 0:1.8.7.374-4.el6_6 ruby-libs.x86_64 0:1.8.7.374-4.el6_6 +> +>Complete! +>``` + +## Gitを使用したインストール + +### ソースコードの取得 + +``` +# git clone https://github.com/axsh/tiny_web_example.git +# mkdir -p /opt/axsh +# mv -i tiny_web_example /opt/axsh/tiny-web-example +``` + +実行結果例: +>``` +># git clone https://github.com/axsh/tiny_web_example.git +>Initialized empty Git repository in /home/vagrant/tiny_web_example/.git/ +>remote: Counting objects: 295, done. +>remote: Compressing objects: 100% (10/10), done. +>remote: Total 295 (delta 3), reused 0 (delta 0), pack-reused 285 +>Receiving objects: 100% (295/295), 288.04 KiB | 165 KiB/s, done. +>Resolving deltas: 100% (105/105), done. +># mkdir -p /opt/axsh +># mv -i tiny_web_example /opt/axsh +>``` + +### 依存パッケージのインストール + +``` +# yum install -y nginx mysql-server mysql-devel +``` + + +実行結果例: +>``` +># yum install -y nginx mysql-server mysql-devel +>Loaded plugins: fastestmirror +>Setting up Install Process +>Loading mirror speeds from cached hostfile +> * base: ftp.jaist.ac.jp +> * extras: ftp.jaist.ac.jp +> * updates: ftp.jaist.ac.jp +>No package nginx available. +>Resolving Dependencies +>--> Running transaction check +>___(省略)___ +> +>Installed: +> mysql-devel.x86_64 0:5.1.73-3.el6_5 mysql-server.x86_64 0:5.1.73-3.el6_5 +> +>Dependency Installed: +> keyutils-libs-devel.x86_64 0:1.4-5.el6 krb5-devel.x86_64 0:1.10.3-37.el6_6 +> libcom_err-devel.x86_64 0:1.41.12-21.el6 libselinux-devel.x86_64 0:2.0.94-5.8.el6 +> libsepol-devel.x86_64 0:2.0.41-4.el6 mysql.x86_64 0:5.1.73-3.el6_5 +> openssl-devel.x86_64 0:1.0.1e-30.el6.8 perl-DBD-MySQL.x86_64 0:4.013-3.el6 +> perl-DBI.x86_64 0:1.609-4.el6 zlib-devel.x86_64 0:1.2.3-29.el6 +> +>Complete! +>``` + +### gemパッケージのインストール + +webapiで使用するgemをインストールする +``` +# cd /opt/axsh/tiny-web-example/webapi +# bundle install +``` + +実行結果例: +>``` +># bundle install +>Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root +>will break this application for all non-root users on this machine. +>Fetching gem metadata from https://rubygems.org/......... +>Fetching version metadata from https://rubygems.org/.. +>Resolving dependencies... +>Installing rake 10.4.2 +>Installing backports 3.6.4 +>Using bundler 1.9.9 +>Installing diff-lcs 1.2.5 +>Installing fuguta 1.0.4 +>Installing get_process_mem 0.2.0 +>Installing kgio 2.9.3 +>Installing multi_json 1.11.0 +>Installing mysql2 0.3.18 +>Installing rack 1.6.1 +>Installing rack-cors 0.4.0 +>Installing rack-protection 1.5.3 +>Installing rack-test 0.6.3 +>Installing raindrops 0.13.0 +>Installing rspec-support 3.2.2 +>Installing rspec-core 3.2.3 +>Installing rspec-expectations 3.2.1 +>Installing rspec-mocks 3.2.1 +>Installing rspec 3.2.0 +>Installing sequel 4.22.0 +>Installing tilt 1.4.1 +>Installing sinatra 1.4.6 +>Installing sinatra-contrib 1.4.2 +>Installing unicorn 4.9.0 +>Installing unicorn-worker-killer 0.4.3 +>Bundle complete! 11 Gemfile dependencies, 25 gems now installed. +>Bundled gems are installed into ./vendor/bundle. +>``` + +frontendで使用するgemをインストールする +``` +# cd /opt/axsh/tiny-web-example/frontend +# bundle install +``` + +実行結果例: +>``` +># bundle install +>Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root +>will break this application for all non-root users on this machine. +>Fetching gem metadata from https://rubygems.org/.......... +>Fetching version metadata from https://rubygems.org/.. +>Resolving dependencies... +>Installing backports 3.6.4 +>Using bundler 1.9.9 +>Installing get_process_mem 0.2.0 +>Installing kgio 2.9.3 +>Installing multi_json 1.11.0 +>Installing mysql2 0.3.18 +>Installing rack 1.6.1 +>Installing rack-protection 1.5.3 +>Installing rack-test 0.6.3 +>Installing raindrops 0.13.0 +>Installing sequel 4.22.0 +>Installing tilt 1.4.1 +>Installing sinatra 1.4.6 +>Installing sinatra-contrib 1.4.2 +>Installing unicorn 4.9.0 +>Installing unicorn-worker-killer 0.4.3 +>Bundle complete! 7 Gemfile dependencies, 16 gems now installed. +>Bundled gems are installed into ./vendor/bundle. +>``` + +### 起動スクリプトの配置 +``` +# cd /opt/axsh/tiny-web-example/contrib/etc +# cp default/* /etc/default/ +# cp init/* /etc/init/ +``` + +### configファイルの配置 +``` +# mkdir -p /etc/tiny-web-example +# cp tiny-web-example/* /etc/tiny-web-example/ +``` + +### logディレクトリの作成 +``` +# mkdir -p /var/log/tiny-web-example +``` + +# 設定 + +## 起動スクリプトの設定 + +webapiの起動スクリプトを修正する +``` +# vi /etc/default/tiny-web-example-webapi +``` + +修正結果: +``` +# tiny-web-example +EXAMPLE_ROOT=/opt/axsh/tiny-web-example +PATH=/root/.rbenv/shims:$PATH + +# Commnet out to run the vdc init script. +#RUN=yes + +## rack params +RACK_ENV=development +BIND_ADDR=0.0.0.0 +PORT=8080 +UNICORN_CONF=/etc/tiny-web-example/unicorn-common.conf +``` + +frontendの起動スクリプトを修正する +``` +# vi /etc/default/tiny-web-example-webapp +``` + +修正結果: +``` +# tiny-web-example +EXAMPLE_ROOT=/opt/axsh/tiny-web-example +PATH=/root/.rbenv/shims:$PATH + +# Commnet out to run the vdc init script. +#RUN=yes + +## rack params +RACK_ENV=development +BIND_ADDR=0.0.0.0 +PORT=80 +UNICORN_CONF=/etc/tiny-web-example/unicorn-common.conf +``` + +## configファイルの設定 + +webapiとfrontendが接続するDatabaseのアドレスを記述します + +webapi.confの修正 +``` +# vi /etc/tiny-web-example/webapi.conf +``` + +修正結果: +``` +# Database connection string +database_uri 'mysql2://localhost/tiny_web_example?user=root' +``` + +webapp.ymlの修正 +``` +# vi /etc/tiny-web-example/webapp.yml +``` + +修正結果: +``` +database_uri: 'mysql2://localhost/tiny_web_example?user=root' +``` + +## mysqldの起動 +``` +# service mysqld start +``` + +実行結果例: +>``` +>Initializing MySQL database: WARNING: The host 'vagrant-centos6' could not be looked up with resolveip. +>This probably means that your libc libraries are not 100 % compatible +>with this binary MySQL version. The MySQL daemon, mysqld, should work +>normally with the exception that host name resolving will not work. +>This means that you should use IP addresses instead of hostnames +>when specifying MySQL privileges ! +>Installing MySQL system tables... +>OK +>Filling help tables... +>OK +> +>To start mysqld at boot time you have to copy +>support-files/mysql.server to the right place for your system +> +>PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER ! +>To do so, start the server, then issue the following commands: +> +>/usr/bin/mysqladmin -u root password 'new-password' +>/usr/bin/mysqladmin -u root -h vagrant-centos6 password 'new-password' +> +>Alternatively you can run: +>/usr/bin/mysql_secure_installation +> +>which will also give you the option of removing the test +>databases and anonymous user created by default. This is +>strongly recommended for production servers. +> +>See the manual for more instructions. +> +>You can start the MySQL daemon with: +>cd /usr ; /usr/bin/mysqld_safe & +> +>You can test the MySQL daemon with mysql-test-run.pl +>cd /usr/mysql-test ; perl mysql-test-run.pl +> +>Please report any problems with the /usr/bin/mysqlbug script! +> +> [ OK ] +>Starting mysqld: [ OK ] +>``` + +## DataBaseの作成 +``` +# mysqladmin create tiny_web_example +``` + +## DataBaseTableの作成 +``` +# cd /opt/axsh/tiny-web-example/webapi/ +# bundle exec rake db:up +``` + +# アプリケーションの実行 + +## webapiの起動 +``` +# initctl start tiny-web-example-webapi RUN=yes +``` + +実行結果例: +>``` +># initctl start tiny-web-example-webapi RUN=yes +>tiny-web-example-webapi start/running, process 2996 +>``` + +## frontendの起動 +``` +# initctl start tiny-web-example-webapp RUN=yes +``` + +実行結果例: +>``` +># initctl start tiny-web-example-webapp RUN=yes +>tiny-web-example-webapp start/running, process 2988 +>``` + +# アプリケーションの動作確認 + +## webapiの確認 + +POSTの確認 +``` +# curl -fs -X POST --data-urlencode display_name='webapi test' --data-urlencode comment='sample message.' http://localhost:8080/api/0.0.1/comments +``` + +実行結果例: +>``` +>{"id":1,"display_name":"webapi test","comment":"sample message.","created_at":"2015-05-21 11:18:07 UTC","updated_at":"2015-05-21 11:18:07 UTC"} +>``` + +GETの確認(list) +``` +# curl -fs -X GET http://localhost:8080/api/0.0.1/comments +``` + +実行結果例: +>``` +>[{"total":2,"results":[{"id":1,"display_name":"webapi test","comment":"sample message.","created_at":"2015-05-21 11:18:07 UTC","updated_at":"2015-05-21 11:18:07 UTC"},{"id":2,"display_name":"webapi test","comment":"sample message 2.","created_at":"2015-05-21 11:19:04 UTC","updated_at":"2015-05-21 11:19:04 UTC"}]}] +>``` + +GETの確認(show) +``` +# curl -fs -X GET http://localhost:8080/api/0.0.1/comments/1 +``` + +実行結果例: +>``` +>{"id":1,"display_name":"webapi test","comment":"sample message.","created_at":"2015-05-21 11:18:07 UTC","updated_at":"2015-05-21 11:18:07 UTC"} +>``` + +## frontendの確認 + +Webブラウザからアクセスすると以下のような画面が表示されます + +![sample_bbs](https://cloud.githubusercontent.com/assets/380254/7747443/675f3c94-fff5-11e4-9d03-4eb74a2c68e1.png) + +NameとCommentを入れて動作するか確認してみましょう + diff --git a/ciscripts/README.md b/ciscripts/README.md new file mode 100644 index 0000000..498134e --- /dev/null +++ b/ciscripts/README.md @@ -0,0 +1,41 @@ +## Image Build + +シェルスクリプトに下記内容を定義してください。 + +``` +#!/bin/bash +# +# +set -e +set -o pipefail +set -u +set -x +cd ciscripts + +WRITE_FILE=${WORKSPACE}/${BUILD_TAG} IMAGE_ID=${image_id} YUM_HOST=${yum_host} ./image-build.sh +``` + +## JenkinsCI + +シェルジョブに下記内容を定義して下さい。 +なお、`APP_IMAGE_ID`と`DB_IMAGE_ID`には、それぞれ新規作成したマシンイメージIDで置き換えて下さい。 + +``` +#!/bin/bash +# +# +set -e +set -o pipefail +set -u +set -x + +cd ciscripts +ls -l + +ipaddr="$(< /metadata/meta-data/local-ipv4)" + +APP_IMAGE_ID="wmi-********" \ + DB_IMAGE_ID="wmi-********" \ + YUM_HOST="${ipaddr}" \ + ./web3layers-ci.sh +``` diff --git a/ciscripts/create-repo.sh b/ciscripts/create-repo.sh old mode 100644 new mode 100755 diff --git a/ciscripts/image-build.sh b/ciscripts/image-build.sh new file mode 100755 index 0000000..9520c10 --- /dev/null +++ b/ciscripts/image-build.sh @@ -0,0 +1,176 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u + +# required shell params + +: "${IMAGE_ID:?"should not be empty"}" +: "${YUM_HOST:?"should not be empty"}" +: "${WRITE_FILE:?"shoud not be empty"}" + +cd ${BASH_SOURCE[0]%/*}/wakame-vdc + +# wait for the instance to be running +. ${BASH_SOURCE[0]%/*}/retry.sh + +# vifs +network_id="nw-demo1" +security_group_id="sg-cicddemo" +vifs="vifs.json" + +# instance-specific parameter +cpu_cores="1" +hypervisor="openvz" +memory_size="512" +image_id="${IMAGE_ID}" +ssh_key_id="ssh-cicddemo" + +# create an musselrc +cat < ~/.musselrc +DCMGR_HOST=10.0.2.2 +account_id=a-shpoolxx +EOS + +# create an vifs +cat < "${vifs}" +{ + "eth0":{"network":"${network_id}","security_groups":"${security_group_id}"} +} +EOS + +## create database image + +# db display name +display_name="db" + +# create an instance +instance_id="$( + mussel instance create \ + --cpu-cores "${cpu_cores}" \ + --hypervisor "${hypervisor}" \ + --image-id "${image_id}" \ + --memory-size "${memory_size}" \ + --ssh-key-id "${ssh_key_id}" \ + --vifs "${vifs}" \ + --display-name "${display_name}" \ + | egrep ^:id: | awk '{print $2}' +)" +: "${instance_id:?"should not be empty"}" +echo "${instance_id} is initializing..." + +trap 'mussel instance destroy "${instance_id}"' ERR + +# wait for the instance to be running +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] +echo "${instance_id} is running" + +# get instance ipaddr +ipaddr="$( + mussel instance show "${instance_id}" \ + | egrep :address: \ + | awk '{print $2}' \ + | tr '\n' ',' +)" +: "${ipaddr:?"should not be empty"}" +ipaddr="${ipaddr%%,}" +echo "${instance_id} ipaddr: ${ipaddr}" + +# wait for ssh to be ready +${BASH_SOURCE[0]%/*}/instance-wait4ssh.sh "${instance_id}" + +# install package +${BASH_SOURCE[0]%/*}/instance-exec.sh "${instance_id}" < ${BASH_SOURCE[0]%/*}/provision-imgdb.sh + +# instance state: running -> halted +mussel instance poweroff --force false ${instance_id} +echo "${instance_id} is halting" + +# wait for the instance to be halted +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: halted")"' ]] +echo "${instance_id} is halted" + +# instance backup +DB_IMAGE_ID="$(mussel instance backup ${instance_id} --display-name db | egrep ^:image_id: | awk '{print $2}')" +echo "database image id: ${DB_IMAGE_ID}" + +# wait for the image to be available +retry_until [[ '"$(mussel image show "${DB_IMAGE_ID}" | egrep -w "^:state: available")"' ]] +echo "${DB_IMAGE_ID} is available" + +# instance destroy +mussel instance destroy "${instance_id}" +echo "${instance_id} is deleted" + +## create app image + +# app display name +display_name="app" + +# create an instance +instance_id="$( + mussel instance create \ + --cpu-cores "${cpu_cores}" \ + --hypervisor "${hypervisor}" \ + --image-id "${image_id}" \ + --memory-size "${memory_size}" \ + --ssh-key-id "${ssh_key_id}" \ + --vifs "${vifs}" \ + --display-name "${display_name}" \ + | egrep ^:id: | awk '{print $2}' +)" +: "${instance_id:?"should not be empty"}" +echo "${instance_id} is initializing..." + +trap 'mussel instance destroy "${instance_id}"' ERR + +# wait for the instance to be running +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] +echo "${instance_id} is running" + +# get instance ipaddr +ipaddr="$( + mussel instance show "${instance_id}" \ + | egrep :address: \ + | awk '{print $2}' \ + | tr '\n' ',' +)" +: "${ipaddr:?"should not be empty"}" +ipaddr="${ipaddr%%,}" +echo "${instance_id} ipaddr: ${ipaddr}" + +# wait for ssh to be ready +${BASH_SOURCE[0]%/*}/instance-wait4ssh.sh "${instance_id}" + +# install package +${BASH_SOURCE[0]%/*}/instance-exec.sh "${instance_id}" \ + YUM_HOST="${YUM_HOST}" \ + bash -l < ${BASH_SOURCE[0]%/*}/provision-imgapp.sh + +# instance state: running -> halted +mussel instance poweroff --force false ${instance_id} +echo "${instance_id} is halting" + +# wait for the instance to be halted +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: halted")"' ]] +echo "${instance_id} is halted" + +# instance backup +APP_IMAGE_ID="$(mussel instance backup ${instance_id} --display-name app | egrep ^:image_id: | awk '{print $2}')" +echo "app image id: ${APP_IMAGE_ID}" + +# wait for the image to be available +RETRY_WAIT_SEC=180 retry_until [[ '"$(mussel image show "${APP_IMAGE_ID}" | egrep -w "^:state: available")"' ]] +echo "${DB_IMAGE_ID} is available" + +# instance destroy +mussel instance destroy "${instance_id}" +echo "${instance_id} is deleted" + +echo DB_IMAGE_ID="${DB_IMAGE_ID}" +echo APP_IMAGE_ID="${APP_IMAGE_ID}" + +echo "DB_IMAGE_ID=${DB_IMAGE_ID}" > ${WRITE_FILE} +echo "APP_IMAGE_ID=${APP_IMAGE_ID}" >> ${WRITE_FILE} diff --git a/ciscripts/integration-test.sh b/ciscripts/integration-test.sh new file mode 100755 index 0000000..036bffb --- /dev/null +++ b/ciscripts/integration-test.sh @@ -0,0 +1,17 @@ +#!/bin/bash +# +# +set -e +set -x +set -o pipefail + +cd spec_integration + +bundle install +cp config/webapi.conf.example config/webapi.conf + +if [[ -n "${APP_HOST:-""}" ]]; then + sed -i s,localhost,${APP_HOST}, config/webapi.conf +fi + +bundle exec rspec ./spec/webapi_integration_spec.rb diff --git a/ciscripts/rpm-build.sh b/ciscripts/rpm-build.sh old mode 100644 new mode 100755 diff --git a/ciscripts/unit-test.sh b/ciscripts/unit-test.sh old mode 100644 new mode 100755 diff --git a/ciscripts/wakame-vdc/.gitignore b/ciscripts/wakame-vdc/.gitignore new file mode 100644 index 0000000..74afec2 --- /dev/null +++ b/ciscripts/wakame-vdc/.gitignore @@ -0,0 +1 @@ +vifs.json diff --git a/ciscripts/wakame-vdc/README.md b/ciscripts/wakame-vdc/README.md new file mode 100644 index 0000000..31a28f5 --- /dev/null +++ b/ciscripts/wakame-vdc/README.md @@ -0,0 +1,13 @@ +## Script Chain + +1. run `setup-db.sh` + 1. run `run-db.sh` + 1. run `create-db.sh` + 2. run `provision-db.sh` +2. run `setup-app.sh` + 1. run `run-app.sh` + 1. run `create-app.sh` + 2. run `provision-app.sh` +3. run `setup-lbweb.sh` + 1. run `run-lbweb.sh` + 1. run `create-lbweb.sh` diff --git a/ciscripts/wakame-vdc/config/.gitkeep b/ciscripts/wakame-vdc/config/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/ciscripts/wakame-vdc/create-app.sh b/ciscripts/wakame-vdc/create-app.sh new file mode 100755 index 0000000..63e7318 --- /dev/null +++ b/ciscripts/wakame-vdc/create-app.sh @@ -0,0 +1,57 @@ +#!/bin/bash +# +set -e +set -o pipefail +set -u + +: "${IMAGE_ID:?"should not be empty"}" + +# vifs + +network_id="nw-demo1" +security_group_id="sg-cicddemo" +vifs="vifs.json" + +# instance-specific parameter + +cpu_cores="1" +hypervisor="openvz" +memory_size="1024" +image_id="${IMAGE_ID}" +display_name="app" +ssh_key_id="ssh-cicddemo" + +# + +if [[ -f ${BASH_SOURCE[0]%/*}/config/${display_name} ]]; then + . ${BASH_SOURCE[0]%/*}/config/${display_name} +fi + +## create an instance + +${BASH_SOURCE[0]%/*}/gen-musselrc.sh +vifs="${vifs}" network_id="${network_id}" security_group_id="${security_group_id}" \ + ${BASH_SOURCE[0]%/*}/gen-vifs.sh + +instance_id="$( + mussel instance create \ + --cpu-cores "${cpu_cores}" \ + --hypervisor "${hypervisor}" \ + --image-id "${image_id}" \ + --memory-size "${memory_size}" \ + --ssh-key-id "${ssh_key_id}" \ + --vifs "${vifs}" \ + --display-name "${display_name}" \ + | egrep ^:id: | awk '{print $2}' +)" +: "${instance_id:?"should not be empty"}" +echo "${instance_id} is initializing..." >&2 + +trap 'mussel instance destroy "${instance_id}"' ERR + +## wait for the instance to be running + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] +echo instance_id="${instance_id}" diff --git a/ciscripts/wakame-vdc/create-centos.sh b/ciscripts/wakame-vdc/create-centos.sh new file mode 100755 index 0000000..71608b6 --- /dev/null +++ b/ciscripts/wakame-vdc/create-centos.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# +set -e +set -o pipefail +set -u + +# setup vifs.json + +network_id="nw-demo1" +security_group_id="sg-cicddemo" +vifs="vifs.json" + +# instance-specific parameter + +cpu_cores="1" +hypervisor="openvz" +memory_size="1024" +image_id="wmi-centos1d64" +display_name="centos" +ssh_key_id="ssh-cicddemo" + +# + +if [[ -f ${BASH_SOURCE[0]%/*}/config/${display_name} ]]; then + . ${BASH_SOURCE[0]%/*}/config/${display_name} +fi + +## create an instance + +${BASH_SOURCE[0]%/*}/gen-musselrc.sh +vifs="${vifs}" network_id="${network_id}" security_group_id="${security_group_id}" \ + ${BASH_SOURCE[0]%/*}/gen-vifs.sh + +instance_id="$( + mussel instance create \ + --cpu-cores "${cpu_cores}" \ + --hypervisor "${hypervisor}" \ + --image-id "${image_id}" \ + --memory-size "${memory_size}" \ + --ssh-key-id "${ssh_key_id}" \ + --vifs "${vifs}" \ + --display-name "${display_name}" \ + | egrep ^:id: | awk '{print $2}' +)" +: "${instance_id:?"should not be empty"}" +echo "${instance_id} is initializing..." >&2 + +trap 'mussel instance destroy "${instance_id}"' ERR + +## wait for the instance to be running + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] +echo instance_id="${instance_id}" diff --git a/ciscripts/wakame-vdc/create-db.sh b/ciscripts/wakame-vdc/create-db.sh new file mode 100755 index 0000000..691671a --- /dev/null +++ b/ciscripts/wakame-vdc/create-db.sh @@ -0,0 +1,57 @@ +#!/bin/bash +# +set -e +set -o pipefail +set -u + +: "${IMAGE_ID:?"should not be empty"}" + +# vifs + +network_id="nw-demo1" +security_group_id="sg-cicddemo" +vifs="vifs.json" + +# instance-specific parameter + +cpu_cores="1" +hypervisor="openvz" +memory_size="512" +image_id="${IMAGE_ID}" +display_name="db" +ssh_key_id="ssh-cicddemo" + +# + +if [[ -f ${BASH_SOURCE[0]%/*}/config/${display_name} ]]; then + . ${BASH_SOURCE[0]%/*}/config/${display_name} +fi + +## create an instance + +${BASH_SOURCE[0]%/*}/gen-musselrc.sh +vifs="${vifs}" network_id="${network_id}" security_group_id="${security_group_id}" \ + ${BASH_SOURCE[0]%/*}/gen-vifs.sh + +instance_id="$( + mussel instance create \ + --cpu-cores "${cpu_cores}" \ + --hypervisor "${hypervisor}" \ + --image-id "${image_id}" \ + --memory-size "${memory_size}" \ + --ssh-key-id "${ssh_key_id}" \ + --vifs "${vifs}" \ + --display-name "${display_name}" \ + | egrep ^:id: | awk '{print $2}' +)" +: "${instance_id:?"should not be empty"}" +echo "${instance_id} is initializing..." >&2 + +trap 'mussel instance destroy "${instance_id}"' ERR + +## wait for the instance to be running + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] +echo instance_id="${instance_id}" diff --git a/ciscripts/wakame-vdc/create-imgdb.sh b/ciscripts/wakame-vdc/create-imgdb.sh new file mode 100755 index 0000000..43ae80e --- /dev/null +++ b/ciscripts/wakame-vdc/create-imgdb.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# +set -e +set -o pipefail +set -u + +# vifs + +network_id="nw-demo1" +security_group_id="sg-cicddemo" +vifs="vifs.json" + +# instance-specific parameter + +cpu_cores="1" +hypervisor="openvz" +memory_size="512" +image_id="wmi-centos1d64" +display_name="imgdb" +ssh_key_id="ssh-cicddemo" + +# + +if [[ -f ${BASH_SOURCE[0]%/*}/config/${display_name} ]]; then + . ${BASH_SOURCE[0]%/*}/config/${display_name} +fi + +## create an instance + +${BASH_SOURCE[0]%/*}/gen-musselrc.sh +vifs="${vifs}" network_id="${network_id}" security_group_id="${security_group_id}" \ + ${BASH_SOURCE[0]%/*}/gen-vifs.sh + +instance_id="$( + mussel instance create \ + --cpu-cores "${cpu_cores}" \ + --hypervisor "${hypervisor}" \ + --image-id "${image_id}" \ + --memory-size "${memory_size}" \ + --ssh-key-id "${ssh_key_id}" \ + --vifs "${vifs}" \ + --display-name "${display_name}" \ + | egrep ^:id: | awk '{print $2}' +)" +: "${instance_id:?"should not be empty"}" +echo "${instance_id} is initializing..." >&2 + +trap 'mussel instance destroy "${instance_id}"' ERR + +## wait for the instance to be running + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] +echo instance_id="${instance_id}" diff --git a/ciscripts/wakame-vdc/create-imgrbenv.sh b/ciscripts/wakame-vdc/create-imgrbenv.sh new file mode 100755 index 0000000..fb88ac5 --- /dev/null +++ b/ciscripts/wakame-vdc/create-imgrbenv.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# +set -e +set -o pipefail +set -u + +# vifs + +network_id="nw-demo1" +security_group_id="sg-cicddemo" +vifs="vifs.json" + +# instance-specific parameter + +cpu_cores="1" +hypervisor="openvz" +memory_size="1024" +image_id="wmi-centos1d64" +display_name="imgrbenv" +ssh_key_id="ssh-cicddemo" + +# + +if [[ -f ${BASH_SOURCE[0]%/*}/config/${display_name} ]]; then + . ${BASH_SOURCE[0]%/*}/config/${display_name} +fi + +## create an instance + +${BASH_SOURCE[0]%/*}/gen-musselrc.sh +vifs="${vifs}" network_id="${network_id}" security_group_id="${security_group_id}" \ + ${BASH_SOURCE[0]%/*}/gen-vifs.sh + +instance_id="$( + mussel instance create \ + --cpu-cores "${cpu_cores}" \ + --hypervisor "${hypervisor}" \ + --image-id "${image_id}" \ + --memory-size "${memory_size}" \ + --ssh-key-id "${ssh_key_id}" \ + --vifs "${vifs}" \ + --display-name "${display_name}" \ + | egrep ^:id: | awk '{print $2}' +)" +: "${instance_id:?"should not be empty"}" +echo "${instance_id} is initializing..." >&2 + +trap 'mussel instance destroy "${instance_id}"' ERR + +## wait for the instance to be running + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] +echo instance_id="${instance_id}" diff --git a/ciscripts/wakame-vdc/create-lbapp.sh b/ciscripts/wakame-vdc/create-lbapp.sh new file mode 100755 index 0000000..792b419 --- /dev/null +++ b/ciscripts/wakame-vdc/create-lbapp.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# +# Usage: +# $0 +# +set -e +set -o pipefail +set -u + +## shell params + +balance_algorithm="leastconn" +engine="haproxy" +max_connection="1000" +instance_port="8080" +instance_protocol="http" +port_maps="8080:http" +display_name="lbapp" + +# + +if [[ -f ${BASH_SOURCE[0]%/*}/config/${display_name} ]]; then + . ${BASH_SOURCE[0]%/*}/config/${display_name} +fi + +## create a load_balancer + +${BASH_SOURCE[0]%/*}/gen-musselrc.sh + +load_balancer_id="$( + mussel load_balancer create \ + --balance-algorithm "${balance_algorithm}" \ + --engine "${engine}" \ + --instance-port "${instance_port}" \ + --instance-protocol "${instance_protocol}" \ + --max-connection "${max_connection}" \ + --display-name "${display_name}" \ + $( + IFS=, + for i in ${port_maps}; do + echo --port ${i%%:*} + echo --protocol ${i##*:} + done + ) \ + | egrep ^:id: | awk '{print $2}' +)" + +: "${load_balancer_id:?"load_balancer is empty"}" +echo "${load_balancer_id} is initializing..." >&2 + +trap "mussel load_balancer destroy \"${load_balancer_id}\"" ERR + +## wait for the load_balancer to be running + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel load_balancer show "${load_balancer_id}" | egrep -w "^:state: running")"' ]] +echo load_balancer_id="${load_balancer_id}" diff --git a/ciscripts/wakame-vdc/create-lbweb.sh b/ciscripts/wakame-vdc/create-lbweb.sh new file mode 100755 index 0000000..780783a --- /dev/null +++ b/ciscripts/wakame-vdc/create-lbweb.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# +# Usage: +# $0 +# +set -e +set -o pipefail +set -u + +## shell params + +balance_algorithm="leastconn" +engine="haproxy" +max_connection="1000" +instance_port="80" +instance_protocol="http" +port_maps="80:http" +display_name="lbweb" + +# + +if [[ -f ${BASH_SOURCE[0]%/*}/config/${display_name} ]]; then + . ${BASH_SOURCE[0]%/*}/config/${display_name} +fi + +## create a load_balancer + +${BASH_SOURCE[0]%/*}/gen-musselrc.sh + +load_balancer_id="$( + mussel load_balancer create \ + --balance-algorithm "${balance_algorithm}" \ + --engine "${engine}" \ + --instance-port "${instance_port}" \ + --instance-protocol "${instance_protocol}" \ + --max-connection "${max_connection}" \ + --display-name "${display_name}" \ + $( + IFS=, + for i in ${port_maps}; do + echo --port ${i%%:*} + echo --protocol ${i##*:} + done + ) \ + | egrep ^:id: | awk '{print $2}' +)" + +: "${load_balancer_id:?"load_balancer is empty"}" +echo "${load_balancer_id} is initializing..." >&2 + +trap "mussel load_balancer destroy \"${load_balancer_id}\"" ERR + +## wait for the load_balancer to be running + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel load_balancer show "${load_balancer_id}" | egrep -w "^:state: running")"' ]] +echo load_balancer_id="${load_balancer_id}" diff --git a/ciscripts/wakame-vdc/demokeypair b/ciscripts/wakame-vdc/demokeypair new file mode 100644 index 0000000..b6a26a9 --- /dev/null +++ b/ciscripts/wakame-vdc/demokeypair @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEpAIBAAKCAQEA6KmXs11l/2WUIBDYmTB0T+BXwwCXT+RP/CTVtxq1Tnq8biwa +pDxHuYgvQWSOOH7DIZq/+GU+P69BBWAbnd1LNkWDOoMmnaIthXQBptZupYFfYiKA +Uh4UH0L0wwenifE2yV+SdWLT6FEiiQ2RTatqK1xiWSwvduWkeMA+dW1NbSk0XmEh +Z76QLsRxrs9JF4jqPXJVulzgjnD9Z5tkNY7MyfD1PNJcM2+MS8XmAApxLQLrfxEl +LZMsgvzvFVec45siOiG+VTWbGADc3lBSHIj2pt6aZDLkOhSnZegmsciVFQk1ulLF +jGjD2LoqYT5/UirpwQsElsjWTEEbBZzV10AVlwIDAQABAoIBAQCdnQ4cv1/ypXC0 +TFU/abjRx8wMWWEoCSY6TQXOtjQvByyRgiVGL2PzhxNkPGewVAeCw1/bOVLzN5lX +t+Tdi+WAzZR51hEZ5pzp9E2OJWPtkPf59h9yAdhl2SkQ2iWgaB1STAFermWZ0yUP +LXbK5B3XZA1oFWvOIwHJn4pwaGx0TpOtEjPHiEkJxj1SRAzN377Uu3SNz9UsRrfQ +3v7iLxrPvwqhXIBo1VzQIliWzH5/IQ6xAqAsMLTo0uJ+d1wkoZ6nGkjt+LYD5hyD +Ov76lOjlevkPu3BENwwt3Este2d00gOC2Qt649P/chd9B8vc4ZZ8F3bPVfmfdiJt +fYRPaF5ZAoGBAP5vEA/lWH5xN6dB7j+wFPLAP7+8H2jz4aBcDWCiDjoMA4JRvc8V +gJxaW33b8gbP39byZAfBLNWHPE+4q/95CW7TkXnzdCR9HxeKC77jnBXg6wX5zist +E/cDMPykATtMqUFf/K46lPjaUbn4gmLEkc9lS7V+ySoPMdMUG6zqQf2DAoGBAOoY +OPSHu2Y7R4V3BnzNeGz6PCrohz7IjqjSD74KhAhuFCM7w+ymDmk2xSIR4S4F7qlD +mBodXpncqxQMtkF2pRRGDefbTXW5m+lWOy/DrsYV0bqy5OqA2r7Ukj5M0o97S8D1 +vhTxwXCehx8GX8RlbybuMkfpB2NefMxakG+BAX9dAoGASPS/vk8dGOSN+L/G+Swc +VZ8aqHfg6c9Emx7KFzNgsPRQ7UVTD9YykqK2KViwBZQFszS9yhtyJ6gnexSQ/ShP +tB+mTzmny+60w6Mpywqo7v0XZxdCLs82MlYP7eF5GO/aeIx1f9/8Z37ygEjp2jhT +NwzssJYySIUi3Eufw+1IDtECgYEAr3NOJMAiTWH6neZyn1Fkg9EdDU/QJdctTQx7 +rgS1ppfSUgH2O0TOIj9hisJ50gOyN3yo4FHI2GrScimA5BmnakWDIJZ2PNjLKRxv +KcJxGJe75EE2XygKSuKJZVYwrkdLpKjKOWpkgCLgxPkDB/C6WSRH3SujVO+5e3QZ +MukulSUCgYBMtuQ6VMrlMTedLW6ryd8VYsVNZaAGuphejFCCuur13M/1wHrRUzqM +hECAngl6fus+weYMiQYx1V8oxz3tBdYO8KKG8pnQySTt5Dln19+vqH2+18RWDKtH +0rwxRJ4Rc3wKFVwK+gz6NsBvftnQAK52qWip71tPY7zt9LeWWJv08g== +-----END RSA PRIVATE KEY----- diff --git a/ciscripts/wakame-vdc/gen-musselrc.sh b/ciscripts/wakame-vdc/gen-musselrc.sh new file mode 100755 index 0000000..fdd480a --- /dev/null +++ b/ciscripts/wakame-vdc/gen-musselrc.sh @@ -0,0 +1,12 @@ +#!/bin/bash +# +set -e +set -o pipefail +set -u + +# setup musselrc + +cat < ~/.musselrc +DCMGR_HOST=10.0.2.15 +account_id=a-shpoolxx +EOS diff --git a/ciscripts/wakame-vdc/gen-vifs.sh b/ciscripts/wakame-vdc/gen-vifs.sh new file mode 100755 index 0000000..66672b4 --- /dev/null +++ b/ciscripts/wakame-vdc/gen-vifs.sh @@ -0,0 +1,15 @@ +#!/bin/bash +# +set -e +set -o pipefail +set -u + +: "${vifs:?"should not be empty"}" +: "${network_id:?"should not be empty"}" +: "${security_group_id:?"should not be empty"}" + +cat < "${vifs}" +{ + "eth0":{"network":"${network_id}","security_groups":"${security_group_id}"} +} +EOS diff --git a/ciscripts/wakame-vdc/image-destroy.sh b/ciscripts/wakame-vdc/image-destroy.sh new file mode 100755 index 0000000..16ecfaf --- /dev/null +++ b/ciscripts/wakame-vdc/image-destroy.sh @@ -0,0 +1,18 @@ +#!/bin/bash +# +# Usage: +# $0 image_id +# +set -e +set -o pipefail +set -u + +## shell params + +image_id="${1}" +: "${image_id:?"should not be empty"}" + +## main + +mussel image destroy "${image_id}" >/dev/null +echo image_id="${image_id}" diff --git a/ciscripts/wakame-vdc/instance-backup.sh b/ciscripts/wakame-vdc/instance-backup.sh new file mode 100755 index 0000000..ead5ec8 --- /dev/null +++ b/ciscripts/wakame-vdc/instance-backup.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +set -u + +## shell params + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## backup an instance + +image_id="$( + mussel instance backup "${instance_id}" \ + | egrep ^:image_id: | awk '{print $2}' +)" +: "${image_id:?"should not be empty"}" +echo "${image_id} is creating..." >&2 + +## wait for the image to be available + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel image show "${image_id}" | egrep -w "^:state: available")"' ]] +echo image_id="${image_id}" diff --git a/ciscripts/wakame-vdc/instance-commit.sh b/ciscripts/wakame-vdc/instance-commit.sh new file mode 100755 index 0000000..180b60e --- /dev/null +++ b/ciscripts/wakame-vdc/instance-commit.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +set -u + +## shell params + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## backup an instance + +eval "$( + ${BASH_SOURCE[0]%/*}/instance-stop.sh "${instance_id}" >&2 + ${BASH_SOURCE[0]%/*}/instance-backup.sh "${instance_id}" + ${BASH_SOURCE[0]%/*}/instance-start.sh "${instance_id}" >&2 +)" + +echo image_id="${image_id}" diff --git a/ciscripts/wakame-vdc/instance-exec.sh b/ciscripts/wakame-vdc/instance-exec.sh new file mode 100755 index 0000000..266317b --- /dev/null +++ b/ciscripts/wakame-vdc/instance-exec.sh @@ -0,0 +1,37 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +set -u + +## include + +function ssh() { + $(type -P ssh) -o 'StrictHostKeyChecking no' -o 'UserKnownHostsFile /dev/null' ${@} +} + +## shell params + +ssh_user="root" +ssh_key="demokeypair" + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## get the instance's ipaddress + +ipaddr="$( + mussel instance show "${instance_id}" \ + | egrep -w :address: | awk '{print $2}' +)" +: "${ipaddr:?"should not be empty"}" + +## ssh to the instance + +chmod 600 "${ssh_key}" + +shift +ssh "${ssh_user}@${ipaddr}" -i "${ssh_key}" "${@}" diff --git a/ciscripts/wakame-vdc/instance-get-ipaddr.sh b/ciscripts/wakame-vdc/instance-get-ipaddr.sh new file mode 100755 index 0000000..ed647ba --- /dev/null +++ b/ciscripts/wakame-vdc/instance-get-ipaddr.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +set -u + +## include + +## shell params + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## get the instance's ipaddress + +ipaddr="$( + mussel instance show "${instance_id}" \ + | egrep :address: \ + | awk '{print $2}' \ + | tr '\n' ',' +)" +: "${ipaddr:?"should not be empty"}" + +## show the instance + +echo ipaddr="${ipaddr%%,}" diff --git a/ciscripts/wakame-vdc/instance-get-vif.sh b/ciscripts/wakame-vdc/instance-get-vif.sh new file mode 100755 index 0000000..7bddf1d --- /dev/null +++ b/ciscripts/wakame-vdc/instance-get-vif.sh @@ -0,0 +1,28 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +#set -u + +## include + +## shell params + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## get the instance's vifs + +vif="$( + mussel instance show "${instance_id}" | egrep :vif_id: \ + | awk '{print $3}' \ + | tr '\n' ',' +)" +: "${vif:?"should not be empty"}" + +## show the instance + +echo vif="${vif%%,}" diff --git a/ciscripts/wakame-vdc/instance-kill.sh b/ciscripts/wakame-vdc/instance-kill.sh new file mode 100755 index 0000000..72a4257 --- /dev/null +++ b/ciscripts/wakame-vdc/instance-kill.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +set -u + +## shell params + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## main + +mussel instance destroy "${instance_id}" >/dev/null +echo "${instance_id} is shuttingdown..." >&2 + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: terminated")"' ]] +echo instance_id="${instance_id}" diff --git a/ciscripts/wakame-vdc/instance-start.sh b/ciscripts/wakame-vdc/instance-start.sh new file mode 100755 index 0000000..6497287 --- /dev/null +++ b/ciscripts/wakame-vdc/instance-start.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +set -u + +## shell params + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## create an instance + +mussel instance poweron "${instance_id}" >/dev/null +echo "${instance_id} is starting..." >&2 + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] +echo instance_id="${instance_id}" diff --git a/ciscripts/wakame-vdc/instance-stop.sh b/ciscripts/wakame-vdc/instance-stop.sh new file mode 100755 index 0000000..2a2690a --- /dev/null +++ b/ciscripts/wakame-vdc/instance-stop.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +set -u + +## shell params + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## create an instance + +mussel instance poweroff "${instance_id}" --force false >/dev/null +echo "${instance_id} is halting..." >&2 + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: halted")"' ]] +echo instance_id="${instance_id}" diff --git a/ciscripts/wakame-vdc/instance-wait4ssh.sh b/ciscripts/wakame-vdc/instance-wait4ssh.sh new file mode 100755 index 0000000..c6712e3 --- /dev/null +++ b/ciscripts/wakame-vdc/instance-wait4ssh.sh @@ -0,0 +1,38 @@ +#!/bin/bash +# +# Usage: +# $0 instance_id +# +set -e +set -o pipefail +set -u + +## shell params + +instance_id="${1}" +: "${instance_id:?"should not be empty"}" + +## wait... + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel instance show "${instance_id}" | egrep -w "^:state: running")"' ]] + +## get the instance's ipaddress + +ipaddr="$( + mussel instance show "${instance_id}" \ + | egrep :address: \ + | awk '{print $2}' \ + | tr '\n' ',' +)" +: "${ipaddr:?"should not be empty"}" +ipaddr="${ipaddr%%,}" + +## wait... + +{ + wait_for_network_to_be_ready "${ipaddr}" + wait_for_sshd_to_be_ready "${ipaddr}" +} >&2 +echo ipaddr="${ipaddr}" diff --git a/ciscripts/wakame-vdc/load_balancer-get-ipaddr.sh b/ciscripts/wakame-vdc/load_balancer-get-ipaddr.sh new file mode 100755 index 0000000..77535a3 --- /dev/null +++ b/ciscripts/wakame-vdc/load_balancer-get-ipaddr.sh @@ -0,0 +1,32 @@ +#!/bin/bash +# +# Usage: +# $0 load_balancer_id +# +set -e +set -o pipefail +set -u + +## include + +## shell params + +load_balancer_id="${1}" +: "${load_balancer_id:?"should not be empty"}" + +## get the load_balancer's ipaddress + +ipaddr="$( + mussel load_balancer show "${load_balancer_id}" | egrep :address: \ + | awk '{print $2}' \ + | tr '\n' ',' +)" +: "${ipaddr:?"should not be empty"}" + +## show the load_balancer + +ipaddr="${ipaddr%%,}" + +echo ipaddr="${ipaddr}" +echo ipaddr_public="${ipaddr%,*}" +echo ipaddr_managed="${ipaddr#*,}" diff --git a/ciscripts/wakame-vdc/load_balancer-kill.sh b/ciscripts/wakame-vdc/load_balancer-kill.sh new file mode 100755 index 0000000..d24219a --- /dev/null +++ b/ciscripts/wakame-vdc/load_balancer-kill.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# +# Usage: +# $0 load_balancer_id +# +set -e +set -o pipefail +set -u + +## shell params + +load_balancer_id="${1}" +: "${load_balancer_id:?"should not be empty"}" + +## main + +while read network_vif_id; do + ${BASH_SOURCE[0]%/*}/load_balancer-unregister-vif.sh "${load_balancer_id}" "${network_vif_id}" >/dev/null +done < <(mussel load_balancer show "${load_balancer_id}" | egrep network_vif_id | awk '{print $3}') + +mussel load_balancer destroy "${load_balancer_id}" >/dev/null +echo "${load_balancer_id} is shuttingdown..." >&2 + +## + +. ${BASH_SOURCE[0]%/*}/retry.sh + +retry_until [[ '"$(mussel load_balancer show "${load_balancer_id}" | egrep -w "^:state: terminated")"' ]] +echo load_balancer_id="${load_balancer_id}" diff --git a/ciscripts/wakame-vdc/load_balancer-register-instance.sh b/ciscripts/wakame-vdc/load_balancer-register-instance.sh new file mode 100755 index 0000000..d5a1167 --- /dev/null +++ b/ciscripts/wakame-vdc/load_balancer-register-instance.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# +# Usage: +# $0 load_balancer_id +# +set -e +set -o pipefail +#set -u + +## shell params + +load_balancer_id="${1}" +: "${load_balancer_id:?"should not be empty"}" +shift +instance_ids="${@:-""}" +: "${instance_ids:?"should not be empty"}" + +## register instances to the load_balancer + +instance_id= +while [[ "${1:-""}" ]]; do + instance_id="${1}" + echo "registering ${instance_id} to ${load_balancer_id}..." >&2 + eval "$(${BASH_SOURCE[0]%/*}/instance-get-vif.sh "${instance_id}")" + ${BASH_SOURCE[0]%/*}/load_balancer-register-vif.sh "${load_balancer_id}" ${vif} >/dev/null + shift +done + +echo load_balancer_id="${load_balancer_id}" diff --git a/ciscripts/wakame-vdc/load_balancer-register-vif.sh b/ciscripts/wakame-vdc/load_balancer-register-vif.sh new file mode 100755 index 0000000..5263530 --- /dev/null +++ b/ciscripts/wakame-vdc/load_balancer-register-vif.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# +# Usage: +# $0 load_balancer_id +# +set -e +set -o pipefail +#set -u + +## shell params + +load_balancer_id="${1}" +: "${load_balancer_id:?"should not be empty"}" +shift +vifs="${@:-""}" +: "${vifs:?"should not be empty"}" + +## register vifs to the load_balancer + +network_vif_id= +while [[ "${1:-""}" ]]; do + network_vif_id="${1}" + echo "registering ${network_vif_id} to ${load_balancer_id}..." >&2 + mussel load_balancer register "${load_balancer_id}" \ + --vifs "${network_vif_id}" >/dev/null + shift +done + +echo load_balancer_id="${load_balancer_id}" diff --git a/ciscripts/wakame-vdc/load_balancer-unregister-instance.sh b/ciscripts/wakame-vdc/load_balancer-unregister-instance.sh new file mode 100755 index 0000000..656831c --- /dev/null +++ b/ciscripts/wakame-vdc/load_balancer-unregister-instance.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# +# Usage: +# $0 load_balancer_id +# +set -e +set -o pipefail +#set -u + +## shell params + +load_balancer_id="${1}" +: "${load_balancer_id:?"should not be empty"}" +shift +instance_ids="${@:-""}" +: "${instance_ids:?"should not be empty"}" + +## unregister instances from the load_balancer + +instance_id= +while [[ "${1:-""}" ]]; do + instance_id="${1}" + echo "unregister ${instance_id} from ${load_balancer_id}..." >&2 + eval "$(${BASH_SOURCE[0]%/*}/instance-get-vif.sh "${instance_id}")" + ${BASH_SOURCE[0]%/*}/load_balancer-unregister-vif.sh "${load_balancer_id}" ${vif} >/dev/null + shift +done + +echo load_balancer_id="${load_balancer_id}" diff --git a/ciscripts/wakame-vdc/load_balancer-unregister-vif.sh b/ciscripts/wakame-vdc/load_balancer-unregister-vif.sh new file mode 100755 index 0000000..430ed9f --- /dev/null +++ b/ciscripts/wakame-vdc/load_balancer-unregister-vif.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# +# Usage: +# $0 load_balancer_id +# +set -e +set -o pipefail +#set -u + +## shell params + +load_balancer_id="${1}" +: "${load_balancer_id:?"should not be empty"}" +shift +vifs="${@:-""}" +: "${vifs:?"should not be empty"}" + +## unregister vifs to the load_balancer + +network_vif_id= +while [[ "${1:-""}" ]]; do + network_vif_id="${1}" + echo "ununregistering ${network_vif_id} from ${load_balancer_id}..." >&2 + mussel load_balancer unregister "${load_balancer_id}" \ + --vifs "${network_vif_id}" >/dev/null + shift +done + +echo load_balancer_id="${load_balancer_id}" diff --git a/ciscripts/wakame-vdc/provision-app.sh b/ciscripts/wakame-vdc/provision-app.sh new file mode 100755 index 0000000..8f87843 --- /dev/null +++ b/ciscripts/wakame-vdc/provision-app.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# +# requires: +# bash +# +set -e +set -o pipefail +set -u +set -x + +# required shell params + +: "${DB_HOST:?"should not be empty"}" + +# configure db host +for config in /etc/tiny-web-example/webapi.conf /etc/tiny-web-example/webapp.yml; do + sed -i "s,localhost,${DB_HOST}," "${config}" + egrep "${DB_HOST}" "${config}" +done + +# setup db +cd /opt/axsh/tiny-web-example/webapi/ +bundle exec rake db:up + +# start system jobs +initctl start tiny-web-example-webapi RUN=yes +initctl start tiny-web-example-webapp RUN=yes diff --git a/ciscripts/wakame-vdc/provision-db.sh b/ciscripts/wakame-vdc/provision-db.sh new file mode 100755 index 0000000..82a1359 --- /dev/null +++ b/ciscripts/wakame-vdc/provision-db.sh @@ -0,0 +1,14 @@ +#!/bin/bash +# +# requires: +# bash +# +set -e +set -o pipefail +set -u +set -x + +# wait for mysqld to be ready +until mysqladmin -uroot ping; do + sleep 1 +done diff --git a/ciscripts/wakame-vdc/provision-imgapp.sh b/ciscripts/wakame-vdc/provision-imgapp.sh new file mode 100755 index 0000000..b2a9348 --- /dev/null +++ b/ciscripts/wakame-vdc/provision-imgapp.sh @@ -0,0 +1,104 @@ +#!/bin/bash +# +# requires: +# bash +# +set -e +set -o pipefail +set -u +set -x + +# required shell params +: "${YUM_HOST:?"should not be empty"}" + +# git +yum install -y git + +# epel +rpm -qa epel-release* | egrep -q epel-release || { + rpm -Uvh http://ftp.jaist.ac.jp/pub/Linux/Fedora/epel/6/i386/epel-release-6-8.noarch.rpm + sed -i \ + -e 's,^#baseurl,baseurl,' \ + -e 's,^mirrorlist=,#mirrorlist=,' \ + -e 's,http://download.fedoraproject.org/pub/epel/,http://ftp.jaist.ac.jp/pub/Linux/Fedora/epel/,' \ + /etc/yum.repos.d/epel.repo /etc/yum.repos.d/epel-testing.repo + yum install -y ca-certificates +} +yum repolist + +# update libcurl +rpm -Uvh http://www.city-fan.org/ftp/contrib/yum-repo/city-fan.org-release-1-13.rhel6.noarch.rpm +sed -i "s,enabled=1,enabled=0," /etc/yum.repos.d/city-fan.org.repo +yum update -y --enablerepo=city-fan.org libcurl + +# install rbenv +git clone https://github.com/sstephenson/rbenv.git ~/.rbenv +# setup rbenv +echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile +echo 'eval "$(rbenv init -)"' >> ~/.bash_profile +exec $SHELL -l + +# install build require for ruby-build +yum install -y gcc openssl-devel libyaml-devel libffi-devel readline-devel zlib-devel gdbm-devel ncurses-devel libffi-devel + +# install ruby-build +git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build + +# install ruby +ruby_version="2.2.2" + +rbenv install -v "${ruby_version}" +rbenv rehash +rbenv versions +rbenv global "${ruby_version}" +ruby -v + +# install bundler +gem install bundler --no-ri --no-rdoc + +# install tiny-web-example.repo +cat <<-EOS > /etc/yum.repos.d/tiny-web-example.repo +[tin-web-example] +name=tiny-web-example +baseurl=http://${YUM_HOST}/pub/ +enabled=1 +gpgcheck=0 +EOS + +# show available repo list +yum repolist + +# install tiny-web-example.rpm +yum install -y tiny-web-example + +## /etc/default/tiny-web-example-webapi +cat <<-'EOS' > /etc/default/tiny-web-example-webapi + # tiny-web-example + EXAMPLE_ROOT=/opt/axsh/tiny-web-example + PATH=/root/.rbenv/shims:$PATH + + # Commnet out to run the vdc init script. + #RUN=yes + + ## rack params + RACK_ENV=development + BIND_ADDR=0.0.0.0 + PORT=8080 + UNICORN_CONF=/etc/tiny-web-example/unicorn-common.conf + EOS + +## /etc/default/tiny-web-example-webapp +cat <<-'EOS' > /etc/default/tiny-web-example-webapp + # tiny-web-example + EXAMPLE_ROOT=/opt/axsh/tiny-web-example + PATH=/root/.rbenv/shims:$PATH + + # Commnet out to run the vdc init script. + #RUN=yes + + ## rack params + RACK_ENV=development + BIND_ADDR=0.0.0.0 + PORT=80 + UNICORN_CONF=/etc/tiny-web-example/unicorn-common.conf + EOS diff --git a/ciscripts/wakame-vdc/provision-imgdb.sh b/ciscripts/wakame-vdc/provision-imgdb.sh new file mode 100755 index 0000000..a454d6b --- /dev/null +++ b/ciscripts/wakame-vdc/provision-imgdb.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# +# requires: +# bash +# mysqladmin, mysql +# +set -e +set -o pipefail +set -u +set -x + +# install mysqld +yum install -y --disablerepo=updates mysql-server + +# configure service +svc="mysqld" +chkconfig --list "${svc}" +chkconfig "${svc}" on +chkconfig --list "${svc}" + +# start mysqld service +if ! service "${svc}" status; then + service "${svc}" start +fi + +# wait for mysqld to be ready +until mysqladmin -uroot ping; do + sleep 1 +done + +# db params +dbname="tiny_web_example" +dbuser="root" +dbacl="10.0.2.%" + +# grant db +mysql -u${dbuser} mysql <> ~/.bash_profile +echo 'eval "$(rbenv init -)"' >> ~/.bash_profile +exec $SHELL -l + +# install build require for ruby-build +yum install -y gcc openssl-devel libyaml-devel libffi-devel readline-devel zlib-devel gdbm-devel ncurses-devel + +# install ruby-build +git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build + +# install ruby +ruby_version="2.0.0-p598" + +rbenv install -v "${ruby_version}" +rbenv rehash +rbenv versions +rbenv global "${ruby_version}" +ruby -v + +# install bundler +gem install bundler --no-ri --no-rdoc diff --git a/ciscripts/wakame-vdc/retry.sh b/ciscripts/wakame-vdc/retry.sh new file mode 100644 index 0000000..e2de868 --- /dev/null +++ b/ciscripts/wakame-vdc/retry.sh @@ -0,0 +1,101 @@ +# -*-Shell-script-*- +# +# requires: +# bash +# + +## retry + +function retry_until() { + local blk="$@" + + local wait_sec=${RETRY_WAIT_SEC:-120} + local sleep_sec=${RETRY_SLEEP_SEC:-3} + local tries=0 + local start_at=$(date +%s) + local chk_cmd= + + while :; do + eval "${blk}" && { + break + } || { + sleep ${sleep_sec} + } + + tries=$((${tries} + 1)) + if [[ "$(($(date +%s) - ${start_at}))" -gt "${wait_sec}" ]]; then + echo "Retry Failure: Exceed ${wait_sec} sec: Retried ${tries} times" >&2 + return 1 + fi + echo [$(date +%FT%X) "#$$"] time:${tries} "eval:${chk_cmd}" >&2 + done +} + +function retry_while() { + local blk="$@" + retry_until ! ${blk} +} + +## check + +function open_port?() { + local ipaddr=$1 protocol=$2 port=$3 + + local nc_opts="-w 3" + case ${protocol} in + tcp) ;; + udp) nc_opts="${nc_opts} -u";; + *) ;; + esac + + nc ${nc_opts} ${ipaddr} ${port} <<< "" >/dev/null +} + +function network_connection?() { + local ipaddr=$1 + ping -c 1 -W 3 ${ipaddr} +} + +## wait for *to be* + +function wait_for_network_to_be_ready() { + local ipaddr=$1 + retry_until "network_connection? ${ipaddr}" +} + +function wait_for_port_to_be_ready() { + local ipaddr=$1 protocol=$2 port=$3 + retry_until "open_port? ${ipaddr} ${protocol} ${port}" +} + +function wait_for_sshd_to_be_ready() { + local ipaddr=$1 + wait_for_port_to_be_ready ${ipaddr} tcp 22 +} + +function wait_for_httpd_to_be_ready() { + local ipaddr=$1 + wait_for_port_to_be_ready ${ipaddr} tcp 80 +} + +## wait for *not to be* + +function wait_for_network_not_to_be_ready() { + local ipaddr=$1 + retry_until "! network_connection? ${ipaddr}" +} + +function wait_for_port_not_to_be_ready() { + local ipaddr=$1 protocol=$2 port=$3 + retry_until "! open_port? ${ipaddr} ${protocol} ${port}" +} + +function wait_for_sshd_not_to_be_ready() { + local ipaddr=$1 + wait_for_port_not_to_be_ready ${ipaddr} tcp 22 +} + +function wait_for_httpd_not_to_be_ready() { + local ipaddr=$1 + wait_for_port_not_to_be_ready ${ipaddr} tcp 80 +} diff --git a/ciscripts/wakame-vdc/run-app.sh b/ciscripts/wakame-vdc/run-app.sh new file mode 100755 index 0000000..b6ff870 --- /dev/null +++ b/ciscripts/wakame-vdc/run-app.sh @@ -0,0 +1,39 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u +set -x + +# required shell params + +: "${YUM_HOST:?"should not be empty"}" +: "${DB_HOST:?"should not be empty"}" + +# + +eval "$( + ${BASH_SOURCE[0]%/*}/create-app.sh + )" +trap 'mussel instance destroy "${instance_id}"' ERR + +eval "$(${BASH_SOURCE[0]%/*}/instance-get-ipaddr.sh "${instance_id}")" + +{ + ${BASH_SOURCE[0]%/*}/instance-wait4ssh.sh "${instance_id}" + ${BASH_SOURCE[0]%/*}/instance-exec.sh "${instance_id}" \ + YUM_HOST="${YUM_HOST}" \ + DB_HOST="${DB_HOST}" \ + bash -l < ${BASH_SOURCE[0]%/*}/provision-app.sh +} >&2 + +{ + . ${BASH_SOURCE[0]%/*}/retry.sh + + wait_for_port_to_be_ready "${ipaddr}" tcp 8080 + wait_for_port_to_be_ready "${ipaddr}" tcp 80 +} >&2 + +echo instance_id="${instance_id}" +echo ipaddr="${ipaddr}" diff --git a/ciscripts/wakame-vdc/run-db.sh b/ciscripts/wakame-vdc/run-db.sh new file mode 100755 index 0000000..a20eb3b --- /dev/null +++ b/ciscripts/wakame-vdc/run-db.sh @@ -0,0 +1,22 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u +set -x + +eval "$( + ${BASH_SOURCE[0]%/*}/create-db.sh + )" +trap 'mussel instance destroy "${instance_id}"' ERR + +eval "$(${BASH_SOURCE[0]%/*}/instance-get-ipaddr.sh "${instance_id}")" + +{ + ${BASH_SOURCE[0]%/*}/instance-wait4ssh.sh "${instance_id}" + ${BASH_SOURCE[0]%/*}/instance-exec.sh "${instance_id}" < ${BASH_SOURCE[0]%/*}/provision-db.sh +} >&2 + +echo instance_id="${instance_id}" +echo ipaddr="${ipaddr}" diff --git a/ciscripts/wakame-vdc/run-imgdb.sh b/ciscripts/wakame-vdc/run-imgdb.sh new file mode 100755 index 0000000..78a33a3 --- /dev/null +++ b/ciscripts/wakame-vdc/run-imgdb.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u +set -x + +eval "$( + ${BASH_SOURCE[0]%/*}/create-imgdb.sh +)" + +trap 'mussel instance destroy "${instance_id}"' ERR + +eval "$(${BASH_SOURCE[0]%/*}/instance-get-ipaddr.sh "${instance_id}")" + +{ + ${BASH_SOURCE[0]%/*}/instance-wait4ssh.sh "${instance_id}" + ${BASH_SOURCE[0]%/*}/instance-exec.sh "${instance_id}" < ${BASH_SOURCE[0]%/*}/provision-imgdb.sh +} >&2 + +echo instance_id="${instance_id}" +echo ipaddr="${ipaddr}" diff --git a/ciscripts/wakame-vdc/run-imgrbenv.sh b/ciscripts/wakame-vdc/run-imgrbenv.sh new file mode 100755 index 0000000..875582d --- /dev/null +++ b/ciscripts/wakame-vdc/run-imgrbenv.sh @@ -0,0 +1,22 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u +set -x + +eval "$( + ${BASH_SOURCE[0]%/*}/create-imgrbenv.sh + )" +trap 'mussel instance destroy "${instance_id}"' ERR + +eval "$(${BASH_SOURCE[0]%/*}/instance-get-ipaddr.sh "${instance_id}")" + +{ + ${BASH_SOURCE[0]%/*}/instance-wait4ssh.sh "${instance_id}" + ${BASH_SOURCE[0]%/*}/instance-exec.sh "${instance_id}" < ${BASH_SOURCE[0]%/*}/provision-imgrbenv.sh +} >&2 + +echo instance_id="${instance_id}" +echo ipaddr="${ipaddr}" diff --git a/ciscripts/wakame-vdc/run-lbapp.sh b/ciscripts/wakame-vdc/run-lbapp.sh new file mode 100755 index 0000000..fe3a581 --- /dev/null +++ b/ciscripts/wakame-vdc/run-lbapp.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u +set -x + +# + +eval "$( + ${BASH_SOURCE[0]%/*}/create-lbapp.sh + )" +trap 'mussel load_balancer destroy "${load_balancer_id}"' ERR + +eval "$(${BASH_SOURCE[0]%/*}/load_balancer-get-ipaddr.sh "${load_balancer_id}")" + +{ + . ${BASH_SOURCE[0]%/*}/retry.sh + + wait_for_port_to_be_ready "${ipaddr_public}" tcp 8080 +} >&2 + +echo load_balancer_id="${load_balancer_id}" +echo ipaddr="${ipaddr}" +echo ipaddr_public="${ipaddr_public}" +echo ipaddr_managed="${ipaddr_managed}" diff --git a/ciscripts/wakame-vdc/run-lbweb.sh b/ciscripts/wakame-vdc/run-lbweb.sh new file mode 100755 index 0000000..9c72818 --- /dev/null +++ b/ciscripts/wakame-vdc/run-lbweb.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u +set -x + +# + +eval "$( + ${BASH_SOURCE[0]%/*}/create-lbweb.sh + )" +trap 'mussel load_balancer destroy "${load_balancer_id}"' ERR + +eval "$(${BASH_SOURCE[0]%/*}/load_balancer-get-ipaddr.sh "${load_balancer_id}")" + +{ + . ${BASH_SOURCE[0]%/*}/retry.sh + + wait_for_port_to_be_ready "${ipaddr_public}" tcp 80 +} >&2 + +echo load_balancer_id="${load_balancer_id}" +echo ipaddr="${ipaddr}" +echo ipaddr_public="${ipaddr_public}" +echo ipaddr_managed="${ipaddr_managed}" diff --git a/ciscripts/wakame-vdc/setup-app.sh b/ciscripts/wakame-vdc/setup-app.sh new file mode 100755 index 0000000..d763d56 --- /dev/null +++ b/ciscripts/wakame-vdc/setup-app.sh @@ -0,0 +1,20 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u + +: "${IMAGE_ID:?"should not be empty"}" +: "${YUM_HOST:?"should not be empty"}" +: "${DB_HOST:?"should not be empty"}" + +eval "$( + IMAGE_ID="${IMAGE_ID}" \ + YUM_HOST="${YUM_HOST}" \ + DB_HOST="${DB_HOST}" \ + ${BASH_SOURCE[0]%/*}/run-app.sh +)" + +echo APP_ID="${instance_id}" +echo APP_HOST="${ipaddr}" diff --git a/ciscripts/wakame-vdc/setup-db.sh b/ciscripts/wakame-vdc/setup-db.sh new file mode 100755 index 0000000..de711e4 --- /dev/null +++ b/ciscripts/wakame-vdc/setup-db.sh @@ -0,0 +1,16 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u + +: "${IMAGE_ID:?"should not be empty"}" + +eval "$( + IMAGE_ID="${IMAGE_ID}" \ + ${BASH_SOURCE[0]%/*}/run-db.sh +)" + +echo DB_ID="${instance_id}" +echo DB_HOST="${ipaddr}" diff --git a/ciscripts/wakame-vdc/setup-lbweb.sh b/ciscripts/wakame-vdc/setup-lbweb.sh new file mode 100755 index 0000000..650a456 --- /dev/null +++ b/ciscripts/wakame-vdc/setup-lbweb.sh @@ -0,0 +1,17 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u + +: "${APP_ID:?"should not be empty"}" + +eval "$( + ${BASH_SOURCE[0]%/*}/run-lbweb.sh +)" + +${BASH_SOURCE[0]%/*}/load_balancer-register-instance.sh "${load_balancer_id}" "${APP_ID}" >&2 + +echo LDWEB_ID="${load_balancer_id}" +echo LBWEB_HOST="${ipaddr_public}" diff --git a/ciscripts/wakame-vdc/smoketest-app.sh b/ciscripts/wakame-vdc/smoketest-app.sh new file mode 100755 index 0000000..b0a7ffd --- /dev/null +++ b/ciscripts/wakame-vdc/smoketest-app.sh @@ -0,0 +1,21 @@ +#!/bin/bash +# +# requires: +# bash +# +set -e +set -o pipefail +set -u +set -x + +# required shell params + +: "${APP_HOST:?"should not be empty"}" + +# + +api_url="http://${APP_HOST}:8080/api/0.0.1/comments" + +curl -fs -X POST --data-urlencode display_name='webapi test' --data-urlencode comment='sample message.' ${api_url} +curl -fs -X GET ${api_url} +curl -fs -X GET ${api_url}/1 diff --git a/ciscripts/wakame-vdc/smoketest-web.sh b/ciscripts/wakame-vdc/smoketest-web.sh new file mode 100755 index 0000000..6919fc9 --- /dev/null +++ b/ciscripts/wakame-vdc/smoketest-web.sh @@ -0,0 +1,17 @@ +#!/bin/bash +# +# requires: +# bash +# +set -e +set -o pipefail +set -u +set -x + +# required shell params + +: "${WEB_HOST:?"should not be empty"}" + +# + +curl -fs -X GET http://${WEB_HOST}/ diff --git a/ciscripts/web3layers-ci.sh b/ciscripts/web3layers-ci.sh new file mode 100755 index 0000000..839ccbb --- /dev/null +++ b/ciscripts/web3layers-ci.sh @@ -0,0 +1,61 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u + +# required shell params + +: "${YUM_HOST:?"should not be empty"}" +: "${DB_IMAGE_ID:?"should not be empty"}" +: "${APP_IMAGE_ID:?"should not be empty"}" + +# + +cd ${BASH_SOURCE[0]%/*}/wakame-vdc + +# setup instances + +eval "$( + IMAGE_ID="${DB_IMAGE_ID}" \ + ${BASH_SOURCE[0]%/*}/setup-db.sh +)" + +eval "$( + IMAGE_ID="${APP_IMAGE_ID}" \ + YUM_HOST="${YUM_HOST}" DB_HOST="${DB_HOST}" \ + ${BASH_SOURCE[0]%/*}/setup-app.sh +)" + +eval "$( + APP_ID="${APP_ID}" \ + ${BASH_SOURCE[0]%/*}/setup-lbweb.sh +)" + +# cleanup + +trap ' + mussel instance destroy "${DB_ID}" + mussel instance destroy "${APP_ID}" + ${BASH_SOURCE[0]%/*}/load_balancer-kill.sh "${LDWEB_ID}" +' ERR EXIT + +# smoketest + +## app + +if [[ -n "${JENKINS_HOME:-""}" ]]; then + # called by jenkins + ( + cd ${BASH_SOURCE[0]%/*}/../../ + APP_HOST="${APP_HOST}" ./ciscripts/integration-test.sh + ) +else + APP_HOST="${APP_HOST}" ${BASH_SOURCE[0]%/*}/smoketest-app.sh +fi + +## web + +WEB_HOST="${APP_HOST}" ${BASH_SOURCE[0]%/*}/smoketest-web.sh +WEB_HOST="${LBWEB_HOST}" ${BASH_SOURCE[0]%/*}/smoketest-web.sh diff --git a/ciscripts/web3layers-lite.sh b/ciscripts/web3layers-lite.sh new file mode 100755 index 0000000..91c0aa9 --- /dev/null +++ b/ciscripts/web3layers-lite.sh @@ -0,0 +1,54 @@ +#!/bin/bash +# +# +set -e +set -o pipefail +set -u + +# required shell params + +: "${YUM_HOST:?"should not be empty"}" +: "${DB_IMAGE_ID:?"should not be empty"}" +: "${APP_IMAGE_ID:?"should not be empty"}" + +# + +cd ${BASH_SOURCE[0]%/*}/wakame-vdc + +# setup instances + +eval "$( + IMAGE_ID="${DB_IMAGE_ID}" \ + ${BASH_SOURCE[0]%/*}/setup-db.sh +)" + +eval "$( + IMAGE_ID="${APP_IMAGE_ID}" \ + YUM_HOST="${YUM_HOST}" DB_HOST="${DB_HOST}" \ + ${BASH_SOURCE[0]%/*}/setup-app.sh +)" + +# cleanup + +trap ' + mussel instance destroy "${DB_ID}" + mussel instance destroy "${APP_ID}" +' ERR EXIT + +# smoketest + +## app + +if [[ -n "${JENKINS_HOME:-""}" ]]; then + # called by jenkins + ( + cd ${BASH_SOURCE[0]%/*}/../../ + APP_HOST="${APP_HOST}" ./ciscripts/integration-test.sh + ) +else + APP_HOST="${APP_HOST}" ${BASH_SOURCE[0]%/*}/smoketest-app.sh +fi + +## web + +WEB_HOST="${APP_HOST}" ${BASH_SOURCE[0]%/*}/smoketest-web.sh diff --git a/rpmbuild/README.md b/rpmbuild/README.md new file mode 100644 index 0000000..9601f8c --- /dev/null +++ b/rpmbuild/README.md @@ -0,0 +1,122 @@ +# RPM Build + +# はじめに + +本書ではtiny-web-exampleのRPMパッケージ作成手順を記述しています。 + +また以下の作業はrootユーザーで行ってください。 + +# 動作環境 + +CentOS-6.6にて動作確認を行いました。それよりも古い環境の場合は動作確認してませんので、ご了承下さい。 + +# 前提条件 + +トップページのREADME.md (Tiny Web Example)を読み環境構築が完了していること。 + +# rpmbuild環境の構築 + +rpmをビルドするために必要なパッケージをインストールします +``` +# yum install -y rpm-build rpmlint yum-utils +``` + +実行結果例: +>``` +># yum install -y rpm-build rpmlint yum-utils +>Loaded plugins: fastestmirror +>Setting up Install Process +>Loading mirror speeds from cached hostfile +> * base: mirror.fairway.ne.jp +> * epel: ftp.iij.ad.jp +> * extras: mirror.fairway.ne.jp +> * updates: mirror.fairway.ne.jp +>Resolving Dependencies +>__(省略)__ +> +>Installed: +> rpm-build.x86_64 0:4.8.0-38.el6_6 rpmlint.noarch 0:0.94-3.1.el6 yum-utils.noarch 0:1.1.30-30.el6 +> +>Dependency Installed: +> desktop-file-utils.x86_64 0:0.15-9.el6 elfutils.x86_64 0:0.158-3.2.el6 enchant.x86_64 1:1.5.0-4.el6 gdb.x86_64 0:7.2-75.el6 +> hunspell.x86_64 0:1.2.8-16.el6 patch.x86_64 0:2.6-6.el6 python-enchant.x86_64 0:1.3.1-5.2.el6 python-magic.x86_64 0:5.04-21.el6 +> unzip.x86_64 0:6.0-2.el6_6 +> +>Complete! +>``` + +# ソースコードの取得 +``` +# git clone https://github.com/axsh/tiny_web_example.git +``` + +実行結果例: +>``` +># git clone https://github.com/axsh/tiny_web_example.git +>Initialized empty Git repository in /root/tiny_web_example/.git/ +>remote: Counting objects: 307, done. +>remote: Compressing objects: 100% (22/22), done. +>Receiving objects: 100% (307/307), 294.71 KiB | 103 KiB/s, done. +>remote: Total 307 (delta 10), reused 0 (delta 0), pack-reused 285 +>Resolving deltas: 100% (112/112), done. +>``` + + +# 依存関係のインストール +``` +# cd tiny_web_example +# yum-builddep ./rpmbuild/SPECS/tiny-web-example.spec +``` + +実行結果例: +>``` +># yum-builddep ./rpmbuild/SPECS/tiny-web-example.spec +>Loaded plugins: fastestmirror +>Enabling epel-source repository +>Loading mirror speeds from cached hostfile +>epel-source/metalink | 5.0 kB 00:00 +> * base: mirror.fairway.ne.jp +> * epel: ftp.iij.ad.jp +> * epel-source: ftp.iij.ad.jp +> * extras: mirror.fairway.ne.jp +> * updates: mirror.fairway.ne.jp +>epel-source | 3.7 kB 00:00 +>epel-source/primary_db | 1.8 MB 00:00 +>Getting requirements for ./rpmbuild/SPECS/tiny-web-example.spec +> --> Already installed : mysql-devel-5.1.73-3.el6_5.x86_64 +>No uninstalled build requires +>``` + +# rpmbuildの実行 +``` +rpmbuild -bb ./rpmbuild/SPECS/tiny-web-example.spec +``` + +実行結果例: +>``` +># rpmbuild -bb ./rpmbuild/SPECS/tiny-web-example.spec +>Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.w7wyVS +>+ umask 022 +>+ cd /root/rpmbuild/BUILD +>+ '[' -d tiny-web-example-0.0.1 ']' +>+ git clone git://github.com/axsh/tiny_web_example.git tiny-web-example-0.0.1 +>Initialized empty Git repository in /root/rpmbuild/BUILD/tiny-web-example-0.0.1/.git/ +>remote: Counting objects: 307, done. +>remote: Compressing objects: 100% (22/22), done. +>remote: Total 307 (delta 10), reused 0 (delta 0), pack-reused 285 +>Receiving objects: 100% (307/307), 294.71 KiB | 103 KiB/s, done. +>Resolving deltas: 100% (112/112), done. +>__(省略)__ +> +>+ umask 022 +>+ cd /root/rpmbuild/BUILD +>+ cd tiny-web-example-0.0.1 +>+ RUBYDIR=/usr/bin/ruby +>+ rpmbuild/rules clean +>rm -rf /root/rpmbuild/BUILD/tiny-web-example-0.0.1/vendor/bundle +>rm -f /root/rpmbuild/BUILD/tiny-web-example-0.0.1/bundle-install-stamp +>rm -f build-stamp +>+ rm -rf /root/rpmbuild/BUILDROOT/tiny-web-example-0.0.1-1.daily.el6.x86_64 +>+ exit 0 +>``` + diff --git a/spec_integration/README.md b/spec_integration/README.md new file mode 100644 index 0000000..fec44fe --- /dev/null +++ b/spec_integration/README.md @@ -0,0 +1,78 @@ +# Integration Test + +# はじめに + +本書はIntegration testの実行方法を記述しています。 + +また以下の作業はrootユーザーで行ってください。 + +# 動作環境 + +CentOS-6.6にて動作確認を行いました。それよりも古い環境の場合は動作確認してませんので、ご了承下さい。 + +# 前提条件 + +トップページのREADME.md (Tiny Web Example)を読み環境構築が完了していること。 + +# Gemのインストール +``` +# cd /opt/axsh/tiny-web-example/spec_integration +# bundle install +``` + +実行結果例: +>``` +># bundle install +>Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all +>non-root users on this machine. +>Fetching gem metadata from https://rubygems.org/......... +>Fetching version metadata from https://rubygems.org/.. +>Resolving dependencies... +>Using bundler 1.9.9 +>Installing diff-lcs 1.2.5 +>Installing rspec-support 3.2.2 +>Installing rspec-core 3.2.3 +>Installing rspec-expectations 3.2.1 +>Installing rspec-mocks 3.2.1 +>Installing rspec 3.2.0 +>Bundle complete! 2 Gemfile dependencies, 7 gems now installed. +>Bundled gems are installed into ./vendor/bundle. +>``` + +# Configの設定 +``` +# cd /opt/axsh/tiny-web-example/spec_integration/config +# cp webapi.conf.example webapi.conf +``` + +webapiに接続するWebサーバーのIPアドレスを記述します +``` +# vi webapi.conf +``` + +修正結果: +``` +uri: 'http://localhost:8080' +``` + +# Integration Testの実行 +``` +# cd /opt/axsh/tiny-web-example/spec_integration +# bundle exec rspec ./spec/webapi_integration_spec.rb +``` + +実行結果例: +>``` +># bundle exec rspec ./spec/webapi_integration_spec.rb +> +>Webapi Integration spec +> post +> create a new comment +> get +> show list for the comments +> show detail the comment +> +>Finished in 0.04481 seconds (files took 0.15042 seconds to load) +>3 examples, 0 failures +> +>``` diff --git a/webapi/spec/README.md b/webapi/spec/README.md new file mode 100644 index 0000000..7c552c6 --- /dev/null +++ b/webapi/spec/README.md @@ -0,0 +1,40 @@ +# Unit Test + +# はじめに + +本書はUnit testの実行方法を記述しています。 + +また以下の作業はrootユーザーで行ってください。 + +# 動作環境 + +CentOS-6.6にて動作確認を行いました。それよりも古い環境の場合は動作確認してませんので、ご了承下さい。 + +# 前提条件 + +トップページのREADME.md (Tiny Web Example)を読み環境構築が完了していること。 + +# configファイルの設定 + +``` +# cd /opt/axsh/tiny-web-example/webapi/spec +# cp webapi.conf.example webapi.conf +``` + +DataBaseのIPアドレスを記述します +``` +# vi webapi.conf +``` + +修正結果: +``` +# Database connection string +database_uri 'mysql2://localhost/tiny_web_example?user=root' +``` + +# Unit Testの実行 + +``` +# cd /opt/axsh/tiny-web-example/webapi/spec +# bundle exec rspec ./comment_spec.rb +```