case-kの備忘録

日々の備忘録です。データ分析とか基盤系に興味あります。

Cloud FormationでImage Builderを使ってAMI を自動生成する

Cloud Formationを使ったAWS Image Builderの使い方です。元々RedashのAMIを使っていましたが、環境変数のファイルを変えて再度コンテナをビルドした際AMI側で行ってるビルド処理と重なってしまうせいか、メッセージキューのceleryが動かなくなってしまう事象が確認できたので、RedashのAMIをCloud Formationで自動生成するようにしてみました。インスタンスが停止した際のオートヒーリング出会ったり運用が少しだけ楽になったりました。
www.case-k.jp

  ## AWS Image Builder  --------------------------------------------------------------------------------
  EC2ImageBuilderForRedash:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
                - ssm.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/EC2InstanceProfileForImageBuilder'
        - 'arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore'
        - 'arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess'
  RedashAmiImage:
    Type: AWS::ImageBuilder::Image
    Properties:
      ImageRecipeArn: !Ref Recipe
      InfrastructureConfigurationArn: !Ref InfrastructureConfiguration
      ImageTestsConfiguration:
        ImageTestsEnabled: true
        TimeoutMinutes: 60
  InfrastructureConfiguration:
    Type: AWS::ImageBuilder::InfrastructureConfiguration
    Properties:
      InstanceProfileName: !Ref InstanceProfile
      InstanceTypes: []
      Name: redash-ami-infrastructure-configuration
      SecurityGroupIds: []
      TerminateInstanceOnFailure: True
  InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      InstanceProfileName: ImageBuilderInstanceProfile
      Roles:
        - !Ref EC2ImageBuilderForRedash
  Component:
    Type: AWS::ImageBuilder::Component
    # https://dev.classmethod.jp/articles/ec2-image-builder-now-includes-support-for-aws-cloudformation/
    # https://docs.aws.amazon.com/imagebuilder/latest/userguide/image-builder-action-modules.html#image-builder-action-modules-s3download
    Properties:
      Data: |
        name: InstallApache
        description: InstallApache
        schemaVersion: 1.0
        phases:
          - name: build
            steps:
              - name: UpdateOS
                action: UpdateOS
              - name: RedashDir
                action: ExecuteBash
                inputs:
                  commands:
                    - mkdir /opt/redash
              - name: Download
                action: S3Download
                inputs:
                  - source: s3://redash-infra/*
                    destination: /opt/redash/
              - name: docker-install
                action: ExecuteBash
                inputs:
                  commands:
                    - sudo apt-get update
                    - sudo apt-get install -y \
                      apt-transport-https \
                      ca-certificates \
                      curl \
                      gnupg-agent \
                      software-properties-common
                    - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
                    - sudo apt-key fingerprint 0EBFCD88
                    - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
                    - sudo apt-get update
                    - sudo apt-get install -y docker-ce docker-ce-cli containerd.io
              - name: docker-compose-install
                action: ExecuteBash
                inputs:
                  commands:
                    - sudo apt-get update
                    - sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
                    - sudo chmod +x /usr/local/bin/docker-compose
                    - sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
              - name: aws-cli-install
                action: ExecuteBash
                inputs:
                  commands:
                    - curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
                    - sudo rm /var/lib/dpkg/lock*
                    - sudo dpkg --configure -a
                    - sudo apt update
                    - sudo apt install python -y
                    - sudo python get-pip.py
                    - sudo pip install awscli
                    - sudo apt install jq -y
      Name: redash-ami-component
      Platform: Linux
      # update version when fix the component
      Version: 1.0.0
  Recipe:
    Type: AWS::ImageBuilder::ImageRecipe
    Properties:
      Components:
        - ComponentArn: !Ref Component
      Name: redash-ami-recipe
      # parentImage only accept aws managed image or custom ami installed ssm. so can not use redash ami
      ParentImage: arn:aws:imagebuilder:ap-northeast-1:aws:image/ubuntu-server-18-lts-x86/2020.9.23
      Version: 1.0.0

まとめ
運用は少しだけ楽になりましたが、ビルド〜テストまで30分ほどかかるので頻繁に変更が必要な処理はインスタンスのユーザデータで行った方が良いかもしれないなあと思いました。