@making's memo

All Categories All Tags


New BOSH CLI (Beta)でBOSH Lite環境作成

Edit History

分散ソフトウェアプロビジョニングツールであるBOSHCLIがGo言語で書き直されていて、使い方が新しくなっていた。まだ正式リリース前だけど試したみた。

新BOSH CLIでBOSH Liteをセットアップしたメモ。

事前準備

  • VirtualBox 5.1.xをインストール

https://www.virtualbox.org/wiki/Downloads

  • New BOSH CLI (Beta)のインストール

https://bosh.io/docs/cli-v2.html

バージョンは0.0.147を使用した。

BOSH Directorのセットアップ

旧CLIのbosh-initに相当する作業を行う。

以前はBOSH Liteの場合、Vagrantのboxファイルが用意されていたのでvagrant upで起動したが、
新CLIではBOSH Lite用のDirectorインスタンスもbosh create-envコマンドで作成する。他のIaaSと同じようになって、セットアップ方法が統一された模様。

Directorセットアップ用のマニフェストファイルは
https://github.com/cloudfoundry/bosh-deployment
にある。

この記事では99a403d32842c5f19c1d3d06bafa494d0a463762を使用した。

git clone https://github.com/cloudfoundry/bosh-deployment ~/workspace/bosh-deployment
mkdir -p ~/deployments/vbox
cd ~/deployments/vbox
bosh create-env ~/workspace/bosh-deployment/bosh.yml \
  --state ./state.json \
  -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \
  -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \
  -o ~/workspace/bosh-deployment/bosh-lite.yml \
  -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \
  -o ~/workspace/bosh-deployment/jumpbox-user.yml \
  --vars-store ./creds.yml \
  -v director_name="Bosh Lite Director" \
  -v internal_ip=192.168.50.6 \
  -v internal_gw=192.168.50.1 \
  -v internal_cidr=192.168.50.0/24 \
  -v network_name=vboxnet0 \
  -v outbound_network_name=NatNetwork

cpiとstemcesllのダウンロードが遅くて、bosh create-envが完了するのに3時間以上かかった・・

bosh.ymlが基本的なマニフェストで、環境差分は-oで指定するopsfileで基本の設定内容を部分的に書き換えている。

ちなみにどんなマニフェストファイルが使われているかは次のようにbosh interpolateで補完結果が見れる。

bosh interpolate ~/workspace/bosh-deployment/bosh.yml \
  -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \
  -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \
  -o ~/workspace/bosh-deployment/bosh-lite.yml \
  -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \
  -o ~/workspace/bosh-deployment/jumpbox-user.yml \
  --vars-store ./creds.yml \
  -v director_name="Bosh Lite Director" \
  -v internal_ip=192.168.50.6 \
  -v internal_gw=192.168.50.1 \
  -v internal_cidr=192.168.50.0/24 \
  -v network_name=vboxnet0 \
  -v outbound_network_name=NatNetwork

ちなみにbosh create-envのログはこんな感じだった。

vbox $ bosh create-env ~/workspace/bosh-deployment/bosh.yml \
>   --state ./state.json \
>   -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \
>   -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \
>   -o ~/workspace/bosh-deployment/bosh-lite.yml \
>   -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \
>   -o ~/workspace/bosh-deployment/jumpbox-user.yml \
>   --vars-store ./creds.yml \
>   -v director_name="Bosh Lite Director" \
>   -v internal_ip=192.168.50.6 \
>   -v internal_gw=192.168.50.1 \
>   -v internal_cidr=192.168.50.0/24 \
>   -v network_name=vboxnet0 \
>   -v outbound_network_name=NatNetwork
Deployment manifest: '/Users/makit/workspace/bosh-deployment/bosh.yml'
Deployment state: './state.json'

Started validating
  Downloading release 'bosh'...

 Finished (00:26:33)
  Validating release 'bosh'... Finished (00:00:00)
  Downloading release 'bosh-virtualbox-cpi'... Finished (00:03:04)
  Validating release 'bosh-virtualbox-cpi'... Finished (00:00:01)
  Downloading release 'bosh-warden-cpi'... Finished (00:28:33)
  Validating release 'bosh-warden-cpi'... Finished (00:00:00)
  Downloading release 'os-conf'... Finished (00:00:01)
  Validating release 'os-conf'... Finished (00:00:00)
  Downloading release 'garden-runc'... Finished (00:21:48)
  Validating release 'garden-runc'... Finished (00:00:00)
  Validating cpi release... Finished (00:00:00)
  Validating deployment manifest... Finished (00:00:00)
  Downloading stemcell... Finished (02:01:38)
  Validating stemcell... Finished (00:00:01)
Finished validating (03:21:47)

Started installing CPI
  Compiling package 'golang_1.7/21609f611781e8586e713cfd7ceb389cee429c5a'... Finished (00:00:16)
  Compiling package 'virtualbox_cpi/56bb5910d296d4eec615f9a6fc02ae84a8bbc933'... Finished (00:00:13)
  Installing packages... Finished (00:00:03)
  Rendering job templates... Finished (00:00:00)
  Installing job 'virtualbox_cpi'... Finished (00:00:00)
Finished installing CPI (00:00:34)

Starting registry... Finished (00:00:00)
Uploading stemcell 'bosh-vsphere-esxi-ubuntu-trusty-go_agent/3312.15'... Finished (00:00:16)

Started deploying
  Creating VM for instance 'bosh/0' from stemcell 'sc-a59c49ce-56c9-49bf-7d14-d1c9443ab21a'... Finished (00:00:01)
  Waiting for the agent on VM 'vm-e42454bb-59ec-4721-6f1b-d2dc3ac86d4c' to be ready... Finished (00:00:41)
  Creating disk... Finished (00:00:00)
  Attaching disk 'disk-e63ec35a-cb76-44a7-4187-7b4f67033169' to VM 'vm-e42454bb-59ec-4721-6f1b-d2dc3ac86d4c'... Finished (00:00:05)
  Rendering job templates... Finished (00:00:05)
  Compiling package 'libseccomp/7a54b27a61b42980935e863d7060dc5a076b44d0'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'golang_1.7.1/91909d54d203acc915a4392b52c37716e15b5aff'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'apparmor/c8e25d84146677878c699ddc5cdd893030acb26f'... Skipped [Package already compiled] (00:00:00)
  Compiling package 's3cli/8cbc6ee1b5acaac18c63fafc5989bd6911c9be83'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'ruby/589d4b05b422ac6c92ee7094fc2a402db1f2d731'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'mysql/b7e73acc0bfe05f1c6cbfd97bf92d39b0d3155d5'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'libpq/09c8f60b87c9bd41b37b0f62159c9d77163f52b8'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'davcli/5f08f8d5ab3addd0e11171f739f072b107b30b8c'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'golang_1.7/c82ff355bb4bd412a4397dba778682293cd4f392'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'golang_1.7/21609f611781e8586e713cfd7ceb389cee429c5a'... Finished (00:00:21)
  Compiling package 'runc/68f36fbe363fefa5ec8d44b48ee30a56ac6e1e0e'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'director/edab99214539f050e0149824f5a064e7b5d57bf2'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'shadow/7a5e46357a33cafc8400a8e3e2e1f6d3a1159cb6'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'tar/f2ea61c537d8eb8cb2d691ce51e8516b28fa5bb7'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'iptables/24e83997945f8817627223c6cee78ca9064f42d5'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'health_monitor/c801fc513e73f1b55285b840e82caac24a435d8b'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'postgres-9.4/6c9e820cdfe15267c8f864f482c7fbed0943c6de'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'postgres/4b9f6514001f7c3f7d4394920d6aced9435a3bbd'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'nats/0155cf6be0305c9f98ba2e9e2503cd72da7c05c3'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'nginx/21e909d27fa69b3b2be036cdf5b8b293c6800158'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'busybox/fc652425c32d0dad62f45bca18e1899671e2e570'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'warden_cpi/29ac97b841a747dc238277ffc7d6bf59a278fa37'... Skipped [Package already compiled] (00:00:00)
  Compiling package 'virtualbox_cpi/56bb5910d296d4eec615f9a6fc02ae84a8bbc933'... Finished (00:00:18)
  Compiling package 'guardian/c4acb6073abb4e17165253935c923dfbdfbfb188'... Skipped [Package already compiled] (00:00:00)
  Updating instance 'bosh/0'... Finished (00:00:25)
  Waiting for instance 'bosh/0' to be running... Finished (00:00:10)
  Running the post-start scripts 'bosh/0'... Finished (00:00:01)
Finished deploying (00:02:13)

Stopping registry... Finished (00:00:00)
Cleaning up rendered CPI jobs... Finished (00:00:00)

Succeeded

VirtualBoxを見るとDirector用のVMが起動しているのとStemcell用のテンプレートファイルがいることがわかる。

image

また、NatNetworkという名前のNATネットワーク(10.0.2.0/24)とvboxnet0という名前のホストオンリーネットワークができていることがわかる。

image

image

image

image

Credentials情報はcred.ymlに自動生成されている。

Directorインスタンスは192.168.50.6であり、この環境のエイリアスを作成する。これは以前のbosh target相当。

bosh alias-env vbox -e 192.168.50.6 --ca-cert <(bosh int ./creds.yml --path /director_ssl/ca)

これ以降はbosh -e vbox [command]を実行すれば良い。

ログインというか認証情報の環境変数設定をする。

export BOSH_CLIENT=admin
export BOSH_CLIENT_SECRET=`bosh int ./creds.yml --path /admin_password`

BOSH Lite用のStemcellをDirectorに登録する。

bosh -e vbox upload-stemcell https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent

これもダウンロード速度が遅いととても時間がかかる。進捗が出なくて不安になる場合、次のように実行するとどのくらいダウンロードされているか分かって良い。

wget --content-disposition https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent
bosh -e vbox upload-stemcell ./bosh-stemcell-3312.15-warden-boshlite-ubuntu-trusty-go_agent.tgz

Cloud Config (BOSH Liteの環境情報)を設定。

bosh -e vbox update-cloud-config ~/workspace/bosh-deployment/warden/cloud-config.yml

ここまででBOSH Liteのセットアップ完了。

vagrant upより手間は多いが、細かい設定が可能なのと、他のIaaSでBOSHをセットアップするのとあまり変わらないので練習にも良いと思う。

サンプルBOSH Releaseのデプロイ

BOSH Directorが起動したので、サンプルアプリをデプロイする。

bosh -e vbox upload-release https://github.com/making-dev/hello-legacy-tomcat-boshrelease/releases/download/0.2/hello-legacy-tomcat-0.2.tgz
bosh -e vbox -n -d hello-legacy-tomcat deploy <(wget -O- https://raw.githubusercontent.com/making-dev/hello-legacy-tomcat-boshrelease/master/manifest/simple.yml)

しばらくするとデプロイ完了する。

ログは

Using environment '192.168.50.6' as client 'admin'

Using deployment 'hello-legacy-tomcat'

Task 4

15:48:23 | Downloading remote release: Downloading remote release (00:15:55)
16:04:18 | Verifying remote release: Verifying remote release (00:00:00)
16:04:18 | Extracting release: Extracting release (00:00:01)
16:04:19 | Verifying manifest: Verifying manifest (00:00:00)
16:04:19 | Resolving package dependencies: Resolving package dependencies (00:00:00)
16:04:19 | Creating new packages: hello-legacy-tomcat/9a3f61ae2d9ac62bed2d6ac8daf7c5e9d87b75a4 (00:00:00)
16:04:19 | Creating new packages: java/36c812837a1e61d7393c84a0a2c5270eb1ae51af (00:00:01)
16:04:20 | Creating new packages: tomcat/3055087aee0f5ee08ea6412b921c6d92f19797b5 (00:00:00)
16:04:20 | Creating new jobs: hello-legacy-tomcat/49a453ed9c90a68b4102a2adf616311fac196278 (00:00:00)
16:04:20 | Release has been created: hello-legacy-tomcat/0.2 (00:00:00)

Started  Sun Jan 22 15:48:23 UTC 2017
Finished Sun Jan 22 16:04:20 UTC 2017
Duration 00:15:57

Task 4 done


+ azs:
+ - name: z1
+ - name: z2
+ - name: z3

+ vm_types:
+ - name: default

+ compilation:
+   az: z1
+   network: default
+   reuse_compilation_vms: true
+   vm_type: default
+   workers: 5

+ networks:
+ - name: default
+   subnets:
+   - azs:
+     - z1
+     - z2
+     - z3
+     gateway: 10.244.0.1
+     range: 10.244.0.0/24
+     static:
+     - 10.244.0.34
+   type: manual

+ disk_types:
+ - disk_size: 1024
+   name: default

+ stemcells:
+ - alias: ubuntu
+   os: ubuntu-trusty
+   version: '3312.15'

+ releases:
+ - name: hello-legacy-tomcat
+   sha1: a401254722d4b032f0170b1d17ca3546fd611524
+   url: https://github.com/making-dev/hello-legacy-tomcat-boshrelease/releases/download/0.2/hello-legacy-tomcat-0.2.tgz
+   version: '0.2'

+ update:
+   canaries: 2
+   canary_watch_time: 5000-60000
+   max_in_flight: 1
+   update_watch_time: 5000-60000

+ instance_groups:
+ - azs:
+   - z1
+   instances: 1
+   jobs:
+   - name: hello-legacy-tomcat
+     properties: {}
+     release: hello-legacy-tomcat
+   name: hello-legacy-tomcat
+   networks:
+   - name: default
+   persistent_disk_type: default
+   stemcell: ubuntu
+   vm_type: default


+ name: hello-legacy-tomcat

Continue? [yN]: y

Task 5

16:05:31 | Preparing deployment: Preparing deployment (00:00:00)
16:05:31 | Preparing package compilation: Finding packages to compile (00:00:00)
16:05:31 | Compiling packages: java/36c812837a1e61d7393c84a0a2c5270eb1ae51af
16:05:31 | Compiling packages: tomcat/3055087aee0f5ee08ea6412b921c6d92f19797b5 (00:00:29)
16:06:00 | Compiling packages: hello-legacy-tomcat/9a3f61ae2d9ac62bed2d6ac8daf7c5e9d87b75a4 (00:00:01)
16:06:18 | Compiling packages: java/36c812837a1e61d7393c84a0a2c5270eb1ae51af (00:00:47)
16:06:19 | Creating missing vms: hello-legacy-tomcat/6099534d-509b-4542-a529-1c399d789078 (0) (00:00:23)
16:06:42 | Updating instance hello-legacy-tomcat: hello-legacy-tomcat/6099534d-509b-4542-a529-1c399d789078 (0) (canary) (00:00:32)

Started  Sun Jan 22 16:05:31 UTC 2017
Finished Sun Jan 22 16:07:14 UTC 2017
Duration 00:01:43

Task 5 done

Succeeded

インスタンスの確認。

$ bosh -e vbox vms
Using environment '192.168.50.6' as client 'admin'

Task 7. Done

Deployment 'hello-legacy-tomcat'

Instance                                                  Process State  AZ  IPs         VM CID                                VM Type  
hello-legacy-tomcat/6099534d-509b-4542-a529-1c399d789078  running        z1  10.244.0.2  44356df0-0484-458c-6c19-f3d92bc86c21  default  

1 vms

Succeeded

10.244.0.2で起動していることがわかる。

サンプルアプリにアクセスする。10.244.0.0/16にアクセスするためにルーティングテーブルと追加する。

Macの場合

sudo route add -net 10.244.0.0/16 192.168.50.6

Linuxの場合

sudo route add -net 10.244.0.0/16 gw 192.168.50.6

Windowsの場合

route add 10.244.0.0/16 192.168.50.6

hello-legacy-tomcatにアクセス

$ curl 10.244.0.2:8080/env
PATH: /bin:/usr/bin:/sbin:/usr/sbin:/var/vcap/packages/java/bin
MONIT_SERVICE: hello-legacy-tomcat
MONIT_PROCESS_PID: 0
JAVA_HOME: /var/vcap/packages/java
MONIT_PROCESS_CHILDREN: 0
XFILESEARCHPATH: /usr/dt/app-defaults/%L/Dt
MONIT_EVENT: Started
MONIT_DESCRIPTION: Started
NLSPATH: /usr/dt/lib/nls/msg/%L/%N.cat
MONIT_PROCESS_CPU_PERCENT: 0
MONIT_HOST: localhost
PWD: /etc/sv/monit
MONIT_PROCESS_MEMORY: 0
SHLVL: 0
MONIT_DATE: Sun, 22 Jan 2017 16:07:04 +0000

できた。

このブログのBOSH系の記事、更新しないと・・・

このエントリーをはてなブックマークに追加