PCF OpsManagerでVM Extensionsを作成し、AWSのSpotInstanceを使用する

以前にBOSHでVM Extensionsを使ってSpot Instanceを利用する方法を紹介しましたが、 Pivotal Cloud Foundry 2.1からはOps Manager経由でもVM Extensionsを作成・適用できるようになったので使い方を紹介します。

ドキュメントはこちら。

本記事では次のコマンドを使用します。

また、OpsManager用のIAMのRoleに以下を追加する必要があります。

ec2:RequestSpotInstances
ec2:DescribeSpotInstanceRequests
ec2:CancelSpotInstanceRequests
iam:CreateServiceLinkedRole

VM Extensionの作成

./create-vm-extensions.sh

#!/bin/bash
set -eu

# Change me
SPOT_INSTANCES=""
SPOT_INSTANCES="$SPOT_INSTANCES m4-large,0.0320"
SPOT_INSTANCES="$SPOT_INSTANCES t2-micro,0.0050"
SPOT_INSTANCES="$SPOT_INSTANCES t2-small,0.0095"
SPOT_INSTANCES="$SPOT_INSTANCES t2-medium,0.0190"
SPOT_INSTANCES="$SPOT_INSTANCES r4-xlarge,0.0655"

for spot in ${SPOT_INSTANCES};do
  TYPE=`echo $spot | awk -F ',' '{print $1}'`
  PRICE=`echo $spot | awk -F ',' '{print $2}'`

  om --target "https://${OPSMAN_DOMAIN_OR_IP_ADDRESS}" \
   --skip-ssl-validation \
   --username "${OPS_MGR_USR}" \
   --password "${OPS_MGR_PWD}" \
   create-vm-extension \
   --name spot-instance-${TYPE} \
   --cloud-properties '{"spot_bid_price":"'$PRICE'", "spot_ondemand_fallback": true}'
done

使い方

export OPSMAN_DOMAIN_OR_IP_ADDRESS=aaa.bbb.ccc.ddd
export OPS_MGR_USR=admin
export OPS_MGR_PWD=password

./apply-vm-extensions.sh

VM Extensionの適用

apply-vm-extensions.sh

#!/bin/bash
set -eu

JOB_GUIDS=`om --target https://$OPSMAN_DOMAIN_OR_IP_ADDRESS \
   --skip-ssl-validation \
   --username "$OPS_MGR_USR" \
   --password "$OPS_MGR_PWD" \
   curl \
   --silent \
   --path /api/v0/staged/products/${PRODUCT_GUID}/jobs | \
   bosh int - --path /jobs | grep guid | sed 's|- guid: ||g'`

# Change me
vm_type_consul_server=t2.micro
vm_type_nats=t2.micro
vm_type_nfs_server=t2.medium
vm_type_mysql_proxy=t2.micro
vm_type_mysql=m4.large
vm_type_backup=t2.micro
vm_type_diego_database=t2.micro
vm_type_uaa=t2.medium
vm_type_cloud_controller=t2.medium
vm_type_ha_proxy=t2.micro
vm_type_router=t2.micro
vm_type_service=t2.micro
vm_type_mysql_monitor=t2.micro
vm_type_clock_global=t2.medium
vm_type_cloud_controller_worker=t2.micro
vm_type_diego_brain=t2.small
vm_type_diego_cell=r4.xlarge
vm_type_loggregator_trafficcontroller=t2.micro
vm_type_syslog_adapter=t2.micro
vm_type_syslog_scheduler=t2.micro
vm_type_doppler=t2.micro
vm_type_tcp_router=t2.micro
vm_type_credhub=m4.large

for JOB_GUID in ${JOB_GUIDS};do
    KEY=`echo ${JOB_GUID}| awk -F '-' '{print $1}'`
    VM_TYPE=$(eval echo \$vm_type_${KEY})
    VM_EXTENSION=`echo "spot-instance-${VM_TYPE}" | sed 's/\./-/g'`
    echo "$KEY=$VM_EXTENSION"

    om --target https://$OPSMAN_DOMAIN_OR_IP_ADDRESS \
       --skip-ssl-validation \
       --username "$OPS_MGR_USR" \
       --password "$OPS_MGR_PWD" \
       curl \
       --silent \
       --request GET \
       --path /api/v0/staged/products/${PRODUCT_GUID}/jobs/${JOB_GUID}/resource_config > resource_config-${JOB_GUID}.json

    cat <<EOF > resource_config-${JOB_GUID}-ops.yml
- type: replace
  path: /additional_vm_extensions?
  value:
  - ${VM_EXTENSION}
EOF
    bosh int resource_config-${JOB_GUID}.json -o resource_config-${JOB_GUID}-ops.yml | yq r - -j > resource_config-${JOB_GUID}-new.json

    om --target https://$OPSMAN_DOMAIN_OR_IP_ADDRESS \
       --skip-ssl-validation \
       --username "$OPS_MGR_USR" \
       --password "$OPS_MGR_PWD" \
       curl \
       --silent \
       --request PUT \
       --path /api/v0/staged/products/${PRODUCT_GUID}/jobs/${JOB_GUID}/resource_config \
       --data "$(cat resource_config-${JOB_GUID}-new.json)"
done

rm -f resource_config-*.yml  resource_config-*.json

使い方

export PRODUCT_GUID=cf-xxxxxxxxxxx
./apply-vm-extensions.sh

その後 Apply Changes

om --target "https://${OPSMAN_DOMAIN_OR_IP_ADDRESS}" \
   --skip-ssl-validation \
   --username "${OPS_MGR_USR}" \
   --password "${OPS_MGR_PWD}" \
   apply-changes \
   --ignore-warnings

でOK。これでVMコストが約30%〜20%まで減らせます。開発環境にはこれで良さげ。

IK.AM

PCF OpsManagerでVM Extensionsを作成し、AWSのSpotInstanceを使用する

VM Extensionの作成

VM Extensionの適用