[EKS] Terraform으로 EKS Cluster 생성

실습용으로 EKS Cluster를 생성하여 이것 저것 해보고 싶지만 생성할 때마다 Web Console로 만들면 반복 작업인데 반해 시간이 다소 소요되고 계속 만들어놓으면 비용이 발생하기 때문에 Terraform으로 필요할 때마다 Apply 하여 리소스를 생성하고 필요가 없어지면 Destroy를 하여 리소스를 삭제하면 어떨까 하여 Terraform으로 EKS Cluster를 생성하게 되었다.

 

아래 작성한 Terraform Code 외에 추가로 고려해야 할 사항들에 대해서 정리

 

1. EKS의 Worker Node는 Launch Template으로 생성하는 방법과 Managed Worker Node로 생성하는 방법 2가지가 있다.

 

만약 자사 보안상 Worker Node 내부에 여러가지 보안 솔루션이 설치되어야 하거나 Time 설정, 보안 점검에 대해 조치한 Image를 사용해야 한다면 Launch Template을 활용하여 생성한 Worker Node가 생성되어야 할 것이다.

 

=> 이 부분은 본 Terraform Code에서 고려되지 않고 작성하였으며, 추후 다시 스크립트를 작성할 예정이다.

 

 

만약, 기존에 생성되어 있는 VPC를 참조하여 EKS Cluster를 만들고 싶다면 다음의 방법이 있을 것이다.

 

1) VPC 및 Network만을 위한 별도의 Directory를 생성하여 별도의 VPC 디렉터리 관리

2) VPC 디렉터리에서 backend => s3를 지정한 뒤 data 블럭에서 tfstate를 참조하여 vpc의 id 및 subnet id, .. 등 형상을 참조하여 EKS Cluster 및 Node Group을 생성하기

 

디렉터리 구조

❯ tree
.
├── README.md
├── cluster.tf
├── data.tf
├── ec2.tf
├── efs.tf
├── env
│   └── test.tfvars
├── locals.tf
├── oidc.tf
├── oidc_iam_roles.tf
├── parameter-store.tf
├── policies
│   ├── aws_load_balancer_controller.json
│   ├── cluster_autoscaler_policy.json
│   ├── external_dns_policy.json
│   ├── external_secrets_policy.json
│   └── oidc_assume_role_policy.json
├── provider.tf
├── scripts
│   └── common
│       └── amz2_init.sh
├── security_groups.tf
├── templates
│   └── aws-load-balancer-coltroller
│       ├── aws-lb-ingress-demo.yaml
│       └── aws-load-balancer-controller.yaml
├── variables.tf
├── vpc.tf
└── worker_node.tf

6 directories, 23 files

 

전체적인 Terraform Code는 소개하지 않고 부분적으로만 소개함

 

1. locals.tf

고정적으로 사용하는 Name을 locals 변수를 활용하여 지정해주었다.

locals {
  name_prefix = "${var.tags["Service"]}-${var.tags["Environment"]}-${var.tags["RegionAlias"]}"
  cluster     = "${var.tags["Service"]}-${var.tags["Environment"]}-${var.tags["RegionAlias"]}-cluster"
}

 

2. vpc.tf

VPC 모듈을 활용하여 VPC 및  VPC와 연관된 리소스들을 일괄적으로 생성해주었다.

VPC 및 IGW, Subnet Group, Route Table, Tag 등을 VPC 모듈을 사용하여 생성하였으며, VPC Module은 개인적으로 커스텀하게 만들었다.

 

=> 소스 코드를 그대로 따라하여선 안됨. ../../modules/vpc 디렉터리에 모듈이 있어야만 정상 Apply 가능.

 

VPC Endpoint의 경우 S3와 ECR을 VPC 내부에서 Private하게 접근하기 위한 용도이며, data.~~ 을 통해 service_name을 참조하였다.

// VPC
module "main_vpc" {
  source         = "../terraform/modules/vpc"
  vpc_cidr_block = var.cidr_block
  available_azs  = var.available_azs
  // Subnet Settings
  public_subnet_cidr  = var.public_subnet_cidr
  private_subnet_cidr = var.private_subnet_cidr
  db_subnet_cidr      = var.db_subnet_cidr
  tags = var.tags

  // Tags
  vpc_tags = {
    "kubernetes.io/cluster/${local.name_prefix}-cluster" = "shared"
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb"                    = 1,
    "kubernetes.io/cluster/${local.name_prefix}-cluster" = "shared"
  }

  public_subnet_tags = {
    "kubernetes.io/role/elb"                             = 1,
    "kubernetes.io/cluster/${local.name_prefix}-cluster" = "shared"
  }
  // Options
  define_eip           = true
  enable_dns_hostnames = var.enable_dns_hostnames
  enable_dns_support   = var.enable_dns_support
  enable_nat_gateway   = var.enable_nat_gateway
}

// VPC Endpoint

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = module.main_vpc.vpc_id // Network Configuration
  service_name      = data.aws_vpc_endpoint_service.s3.service_name
  vpc_endpoint_type = "Interface"

  security_group_ids  = [aws_security_group.vpc_endpoint.id]
  subnet_ids          = module.main_vpc.private_subnets_ids // Network Configuration
  private_dns_enabled = false

  tags = {
    Name        = "${local.name_prefix}-s3-endpoint"
    Environment = var.tags.Environment
  }
}

resource "aws_vpc_endpoint" "ecr_dkr" {
  vpc_id            = module.main_vpc.vpc_id // Network Configuration
  service_name      = data.aws_vpc_endpoint_service.ecr_dkr.service_name
  vpc_endpoint_type = "Interface"

  security_group_ids  = [aws_security_group.vpc_endpoint.id]
  subnet_ids          = module.main_vpc.private_subnets_ids // Network Configuration
  private_dns_enabled = false

  tags = {
    Name        = "${local.name_prefix}-ecr-endpoint"
    Environment = var.tags.Environment
  }
}

// VPC FlowLogs

 

3. data.tf

 

IAM Policy (JSON) to Terraform data block

URL : https://flosell.github.io/iam-policy-json-to-terraform/

=> 위 URL에서 IAM Policy를 Terraform data block에서 사용될 수 있도록 변환해주는 것을 도와준다.

 

 

// Caller Identity
data "aws_caller_identity" "current" {}

data "aws_region" "current" {}
data "aws_partition" "current" {}

# data "tls_certificate" "cluster" {
#   count = var.create_cluster ? 1 : 0
#   url = aws_eks_cluster.cluster[0].identity.0.oidc.0.issuer
# }

// AMI
// 1. Amazon Linux 2
data "aws_ami" "amazon2" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-ebs"]
  }
}

// OIDC
data "tls_certificate" "cluster" {
  url = aws_eks_cluster.cluster.identity.0.oidc.0.issuer
}

// Endpoint
// 1. S3
data "aws_vpc_endpoint_service" "s3" {
  service_type = "Interface"
  filter {
    name   = "service-name"
    values = ["*s3"]
  }
}

// 2. ECR
data "aws_vpc_endpoint_service" "ecr_dkr" {
  service_type = "Interface"
  filter {
    name   = "service-name"
    values = ["*ecr.dkr*"]
  }
}

// Worker Node Assume Role
data "aws_iam_policy_document" "workers_role_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["ec2.amazonaws.com"]
    }
  }
}

// OIDC IAM Role Policies

// AWS Load Balancer Controller Policy
data "aws_iam_policy_document" "load_balancer_controller" {
  statement {
    actions = [
      "iam:CreateServiceLinkedRole",
      "ec2:DescribeAccountAttributes",
      "ec2:DescribeAddresses",
      "ec2:DescribeAvailabilityZones",
      "ec2:DescribeInternetGateways",
      "ec2:DescribeVpcs",
      "ec2:DescribeSubnets",
      "ec2:DescribeSecurityGroups",
      "ec2:DescribeInstances",
      "ec2:DescribeNetworkInterfaces",
      "ec2:DescribeTags",
      "ec2:GetCoipPoolUsage",
      "ec2:DescribeCoipPools",
      "elasticloadbalancing:DescribeLoadBalancers",
      "elasticloadbalancing:DescribeLoadBalancerAttributes",
      "elasticloadbalancing:DescribeListeners",
      "elasticloadbalancing:DescribeListenerCertificates",
      "elasticloadbalancing:DescribeSSLPolicies",
      "elasticloadbalancing:DescribeRules",
      "elasticloadbalancing:DescribeTargetGroups",
      "elasticloadbalancing:DescribeTargetGroupAttributes",
      "elasticloadbalancing:DescribeTargetHealth",
      "elasticloadbalancing:DescribeTags"
    ]

    resources = ["*"]
  }

  statement {
    actions = [
      "cognito-idp:DescribeUserPoolClient",
      "acm:ListCertificates",
      "acm:DescribeCertificate",
      "acm:GetCertificate",
      "iam:ListServerCertificates",
      "iam:GetServerCertificate",
      "waf-regional:GetWebACL",
      "waf-regional:GetWebACLForResource",
      "waf-regional:AssociateWebACL",
      "waf-regional:DisassociateWebACL",
      "wafv2:GetWebACL",
      "wafv2:GetWebACLForResource",
      "wafv2:AssociateWebACL",
      "wafv2:DisassociateWebACL",
      "shield:GetSubscriptionState",
      "shield:DescribeProtection",
      "shield:CreateProtection",
      "shield:DeleteProtection"
    ]

    resources = ["*"]
  }

  statement {
    actions = [
      "ec2:AuthorizeSecurityGroupIngress",
      "ec2:RevokeSecurityGroupIngress"
    ]
    resources = ["*"]
  }

  statement {
    actions = [
      "ec2:CreateSecurityGroup"
    ]

    resources = ["*"]
  }

  statement {
    actions = [
      "ec2:CreateTags"
    ]

    resources = ["arn:${data.aws_partition.current.partition}:ec2:*:*:security-group/*"]

    condition {
      test     = "StringEquals"
      variable = "ec2:CreateAction"

      values = [
        "CreateSecurityGroup",
      ]
    }

    condition {
      test     = "Null"
      variable = "aws:RequestTag/elbv2.k8s.aws/cluster"

      values = [
        "false",
      ]
    }
  }

  statement {
    actions = [
      "ec2:CreateTags",
      "ec2:DeleteTags"
    ]

    resources = ["arn:${data.aws_partition.current.partition}:ec2:*:*:security-group/*"]

    condition {
      test     = "Null"
      variable = "aws:RequestTag/elbv2.k8s.aws/cluster"

      values = [
        "true",
      ]
    }

    condition {
      test     = "Null"
      variable = "aws:ResourceTag/elbv2.k8s.aws/cluster"

      values = [
        "false",
      ]
    }
  }

  statement {
    actions = [
      "ec2:AuthorizeSecurityGroupIngress",
      "ec2:RevokeSecurityGroupIngress",
      "ec2:DeleteSecurityGroup"
    ]

    resources = ["*"]

    condition {
      test     = "Null"
      variable = "aws:ResourceTag/elbv2.k8s.aws/cluster"

      values = [
        "false",
      ]
    }
  }

  statement {
    actions = [
      "elasticloadbalancing:CreateLoadBalancer",
      "elasticloadbalancing:CreateTargetGroup"
    ]

    resources = ["*"]

    condition {
      test     = "Null"
      variable = "aws:RequestTag/elbv2.k8s.aws/cluster"

      values = [
        "false",
      ]
    }
  }

  statement {
    actions = [
      "elasticloadbalancing:CreateListener",
      "elasticloadbalancing:DeleteListener",
      "elasticloadbalancing:CreateRule",
      "elasticloadbalancing:DeleteRule"
    ]
    resources = ["*"]
  }

  statement {
    actions = [
      "elasticloadbalancing:AddTags",
      "elasticloadbalancing:RemoveTags"
    ]

    resources = [
      "arn:${data.aws_partition.current.partition}:elasticloadbalancing:*:*:targetgroup/*/*",
      "arn:${data.aws_partition.current.partition}:elasticloadbalancing:*:*:loadbalancer/net/*/*",
      "arn:${data.aws_partition.current.partition}:elasticloadbalancing:*:*:loadbalancer/app/*/*"
    ]

    condition {
      test     = "Null"
      variable = "aws:RequestTag/elbv2.k8s.aws/cluster"

      values = [
        "true",
      ]
    }

    condition {
      test     = "Null"
      variable = "aws:ResourceTag/elbv2.k8s.aws/cluster"

      values = [
        "false",
      ]
    }
  }

  statement {
    actions = [
      "elasticloadbalancing:AddTags",
      "elasticloadbalancing:RemoveTags"
    ]

    resources = [
      "arn:${data.aws_partition.current.partition}:elasticloadbalancing:*:*:listener/net/*/*/*",
      "arn:${data.aws_partition.current.partition}:elasticloadbalancing:*:*:listener/app/*/*/*",
      "arn:${data.aws_partition.current.partition}:elasticloadbalancing:*:*:listener-rule/net/*/*/*",
      "arn:${data.aws_partition.current.partition}:elasticloadbalancing:*:*:listener-rule/app/*/*/*"
    ]
  }

  statement {
    actions = [
      "elasticloadbalancing:ModifyLoadBalancerAttributes",
      "elasticloadbalancing:SetIpAddressType",
      "elasticloadbalancing:SetSecurityGroups",
      "elasticloadbalancing:SetSubnets",
      "elasticloadbalancing:DeleteLoadBalancer",
      "elasticloadbalancing:ModifyTargetGroup",
      "elasticloadbalancing:ModifyTargetGroupAttributes",
      "elasticloadbalancing:DeleteTargetGroup"
    ]

    resources = ["*"]

    condition {
      test     = "Null"
      variable = "aws:ResourceTag/elbv2.k8s.aws/cluster"

      values = [
        "false",
      ]
    }
  }

  statement {
    actions = [
      "elasticloadbalancing:RegisterTargets",
      "elasticloadbalancing:DeregisterTargets"
    ]

    resources = ["arn:${data.aws_partition.current.partition}:elasticloadbalancing:*:*:targetgroup/*/*"]
  }

  statement {
    actions = [
      "elasticloadbalancing:SetWebAcl",
      "elasticloadbalancing:ModifyListener",
      "elasticloadbalancing:AddListenerCertificates",
      "elasticloadbalancing:RemoveListenerCertificates",
      "elasticloadbalancing:ModifyRule"
    ]

    resources = ["*"]
  }
}

// Cluster Autoscaler Policy
data "aws_iam_policy_document" "cluster_autoscaler" {
  statement {
    actions = [
      "autoscaling:DescribeAutoScalingGroups",
      "autoscaling:DescribeAutoScalingInstances",
      "autoscaling:DescribeLaunchConfigurations",
      "autoscaling:DescribeTags",
      "autoscaling:SetDesiredCapacity",
      "autoscaling:TerminateInstanceInAutoScalingGroup",
      "ec2:DescribeLaunchTemplateVersions"
    ]

    resources = ["*"]
  }
}

// External Secret Policy
data "aws_iam_policy_document" "external_secrets" {
  statement {
    actions = [
      "secretsmanager:DescribeSecret",
      "secretsmanager:GetSecretValue",
      "secretsmanager:ListSecrets",
      "secretsmanager:ListSecretVersionIds"
    ]

    resources = ["*"]
  }

  statement {
    actions = [
      "ssm:DescribeParameters",
      "ssm:GetParameter",
      "ssm:GetParameterHistory",
      "ssm:GetParameters",
      "ssm:GetParametersByPath"
    ]

    resources = ["*"]
  }
}

// External DNS Policy
data "aws_iam_policy_document" "external_dns" {
  statement {
    actions = [
      "route53:ChangeResourceRecordSets",
      "route53:ListHostedZones",
      "route53:ListResourceRecordSets"
    ]

    resources = ["*"]
  }
}


// EBS CSI Driver
data "aws_iam_policy_document" "csi_driver" {
  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]

    actions = [
      "ec2:CreateSnapshot",
      "ec2:AttachVolume",
      "ec2:DetachVolume",
      "ec2:ModifyVolume",
      "ec2:DescribeAvailabilityZones",
      "ec2:DescribeInstances",
      "ec2:DescribeSnapshots",
      "ec2:DescribeTags",
      "ec2:DescribeVolumes",
      "ec2:DescribeVolumesModifications",
    ]
  }

  statement {
    sid    = ""
    effect = "Allow"

    resources = [
      "arn:aws:ec2:*:*:volume/*",
      "arn:aws:ec2:*:*:snapshot/*",
    ]

    actions = ["ec2:CreateTags"]

    condition {
      test     = "StringEquals"
      variable = "ec2:CreateAction"

      values = [
        "CreateVolume",
        "CreateSnapshot",
      ]
    }
  }

  statement {
    sid    = ""
    effect = "Allow"

    resources = [
      "arn:aws:ec2:*:*:volume/*",
      "arn:aws:ec2:*:*:snapshot/*",
    ]

    actions = ["ec2:DeleteTags"]
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]
    actions   = ["ec2:CreateVolume"]

    condition {
      test     = "StringLike"
      variable = "aws:RequestTag/ebs.csi.aws.com/cluster"
      values   = ["true"]
    }
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]
    actions   = ["ec2:CreateVolume"]

    condition {
      test     = "StringLike"
      variable = "aws:RequestTag/CSIVolumeName"
      values   = ["*"]
    }
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]
    actions   = ["ec2:DeleteVolume"]

    condition {
      test     = "StringLike"
      variable = "ec2:ResourceTag/CSIVolumeName"
      values   = ["*"]
    }
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]
    actions   = ["ec2:DeleteVolume"]

    condition {
      test     = "StringLike"
      variable = "ec2:ResourceTag/ebs.csi.aws.com/cluster"
      values   = ["true"]
    }
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]
    actions   = ["ec2:DeleteSnapshot"]

    condition {
      test     = "StringLike"
      variable = "ec2:ResourceTag/CSIVolumeSnapshotName"
      values   = ["*"]
    }
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]
    actions   = ["ec2:DeleteSnapshot"]

    condition {
      test     = "StringLike"
      variable = "ec2:ResourceTag/ebs.csi.aws.com/cluster"
      values   = ["true"]
    }
  }
}

// EFS CSI Driver
data "aws_iam_policy_document" "efs_csi_driver" {
  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]

    actions = [
      "elasticfilesystem:DescribeAccessPoints",
      "elasticfilesystem:DescribeFileSystems",
    ]
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]
    actions   = ["elasticfilesystem:CreateAccessPoint"]

    condition {
      test     = "StringLike"
      variable = "aws:RequestTag/efs.csi.aws.com/cluster"
      values   = ["true"]
    }
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]
    actions   = ["elasticfilesystem:DeleteAccessPoint"]

    condition {
      test     = "StringEquals"
      variable = "aws:ResourceTag/efs.csi.aws.com/cluster"
      values   = ["true"]
    }
  }
}

 

 

 

4. Cluster.tf

EKS Cluster는 IAM Role에 Policy들이 부착된 이후 정상적으로 생성될 수 있기 때문에 depends_op 블록을 통해 의존성을 지정해주었다.

resource "aws_eks_cluster" "cluster" {
  name                      = "${local.name_prefix}-cluster"
  role_arn                  = aws_iam_role.cluster.arn
  version                   = var.eks_version
  enabled_cluster_log_types = var.eks_enabled_log_types

  vpc_config {
    subnet_ids              = flatten([module.main_vpc.public_subnets_ids, module.main_vpc.private_subnets_ids])
    security_group_ids      = [aws_security_group.eks_cluster_sg.id, aws_security_group.eks_worker_sg.id]
    endpoint_private_access = "true"
    endpoint_public_access  = "true"
  }

  tags = {
    Name        = "${local.name_prefix}-cluster"
    Environment = var.tags.Environment
    Owner       = var.owner_tag
  }

  depends_on = [
    aws_iam_role_policy_attachment.cluster_AmazonEKSClusterPolicy,
    aws_iam_role_policy_attachment.cluster_AmazonEKSServicePolicy,
    aws_iam_role_policy_attachment.cluster_AmazonEKSVPCResourceController,
    aws_cloudwatch_log_group.cluster
  ]
}


resource "aws_cloudwatch_log_group" "cluster" {
  name              = "/aws/eks/${local.name_prefix}/cluster"
  retention_in_days = 7

  tags = {
    Name        = "${local.name_prefix}-cluster"
    Environment = var.tags.Environment
    Owner       = var.owner_tag
  }
}

resource "aws_iam_role" "cluster" {
  name = "${local.name_prefix}-cluster-role"

  assume_role_policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "eks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
POLICY
}

// Policies attached to EKS Cluster
// 1. EKSClusterPolicy, 2. EKSServicePolicy, 3. EKSVPCResourceController
resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.cluster.name
}

resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSServicePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSServicePolicy"
  role       = aws_iam_role.cluster.name
}

resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSVPCResourceController" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.cluster.name
}

 

5. Worker Node

Worker Node의 경우에도 depends_on 블록을 통해 Policy가 먼저 정의될 수 있도록 의존성을 설정하였으며 

// EKS Worker Node
resource "aws_eks_node_group" "worker" {
  cluster_name    = aws_eks_cluster.cluster.name
  node_group_name = "${local.name_prefix}-worker-node"
  node_role_arn   = aws_iam_role.worker.arn
  subnet_ids      = module.main_vpc.private_subnets_ids // Network Configuration

  // Worker Settings
  instance_types = var.worker_instance_types
  disk_size      = var.worker_disk_size

  scaling_config {
    desired_size = var.worker_size.desired
    min_size     = var.worker_size.min
    max_size     = var.worker_size.max
  }

  remote_access {
    source_security_group_ids = [aws_security_group.common_ssh.id]
    ec2_ssh_key               = var.key_pair
  }

  tags = {
    Name        = "${local.name_prefix}-worker-node"
    Environment = "${var.tags.Environment}"
    Owner       = var.owner_tag
  }

  # Ensure that IAM Role permissions are created before and deleted after EKS Node Group handling.
  # Otherwise, EKS will not be able to properly delete EC2 Instances and Elastic Network Interfaces.
  depends_on = [
    aws_iam_role_policy_attachment.eks-AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.eks-AmazonEKS_CNI_Policy,
    aws_iam_role_policy_attachment.eks-AmazonEC2ContainerRegistryReadOnly,
  ]
}


// IAM Role
resource "aws_iam_role" "worker" {
  name               = "${local.name_prefix}-managed-worker-node"
  assume_role_policy = data.aws_iam_policy_document.workers_role_assume_role_policy.json
  tags = {
    Environment = "${var.tags.Environment}"
  }
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.worker.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.worker.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.worker.name
}

 

6. variables.tf

 

env/test.tfvars 파일에서 먼저 참조하게 되고, 없다면 variables.tf 파일에서 참조하게 된다.

 

tfvars 파일 먼저 참고 (우선순위 높음) => variables.tf 파일 내의 variable 블럭에 default 값이 있는지 참조

=> resource 혹은 module 블럭에서 var.xxxx 부분에 기입하게 된다.

// Global
variable "profile_name" { default = "example" }

variable "cred_file" { default = "/home/user/.aws/credentials" }

variable "region" { default = "ap-northeast-2" }

#################################################################
# VPC Configuration
#################################################################

// CIDR
variable "cidr_block" {
  description = "Amazon Virtual Private Cloud CIDR range."
}

variable "public_subnet_cidr" {
  default     = []
  description = "AWS Public Subnet CIDR Block"
}

variable "private_subnet_cidr" {
  default     = []
  description = "AWS Private Subnet CIDR Block"
}

variable "db_subnet_cidr" {
  default     = []
  description = "AWS DB Subnet CIDR Block"
}


variable "available_azs" {
  description = "AWS Available AZs you would like to deploy."
  type        = list(string)
  default     = ["ap-northeast-2a", "ap-northeast-2c"]
}

// VPC Options
variable "enable_dns_hostnames" {
  default     = true
  description = "Enable DNS Hostname"
  type        = bool
}

variable "enable_dns_support" {
  default     = true
  description = "Enable DNS Support"
  type        = bool
}

variable "enable_nat_gateway" {
  description = "Enable NAT Gateway of Not"
  default     = true
  type        = bool
}

#################################################################
# EC2
#################################################################
variable "default_instance_type" {
  default     = ""
  description = "Default EC2 Instance Type"
}

variable "key_pair" {
  default = ""
}

#################################################################
# Security Groups
#################################################################
variable "bastion_ingress_rules" {
  description = "Bastion ingresses. From, To, Protocol, Cidrs, Desc"
  type        = list(any)
  default     = []
}

// EKS
variable "eks_enabled_log_types" {
  type    = list(string)
  default = []
}

variable "eks_version" {}

// Worker Node
variable "worker_size" {
  description = "Worker Node Size"
  type        = map(string)
  default = {
    "desired" = "1"
    "min"     = "1"
    "max"     = "2"
  }
}

variable "worker_instance_types" {
  description = "Maximum number of worker nodes in private subnet."
  type        = list(string)
}

variable "worker_disk_size" {
  description = "Minimum number of worker nodes in private subnet."
  type        = number
}

// Tagging
variable "tags" {
  type        = map(string)
  description = "Default tags"
  default = {
    "Service"     = "Hello_SVC"
    "Environment" = "test"
    "RegionAlias" = "an2"
  }
}

variable "owner_tag" {
  type        = string
  description = "Owner tag for resource pricing management, e.g gildong.hong"
  default     = "gildong.hong"
}


// EFS File System
variable "efs" {
  default = {}
}

 

tfvars 파일

아래 TFVars 파일은 예시이다.

대충 이런식으로 tfvars 파일을 작성하게 된다.

# VPC Configuration
available_azs = ["ap-northeast-2a", "ap-northeast-2c"]
cidr_block = "10.60.0.0/18"
public_subnet_cidr = ["10.60.0.0/20", "10.60.16.0/20"]
private_subnet_cidr = ["10.60.32.0/22", "10.60.36.0/22"]
db_subnet_cidr = ["10.60.40.0/24", "10.60.41.0/24"]

tags = {
  Service           = "SVC"
  Environment       = "test"
  RegionAlias       = "an2"
}

# Security Groups Rules
bastion_ingress_rules =[
  # From, To, Protocol, Source CIDR, Description
  [22, 22, "tcp", ["0.0.0.0/0"], "SSH Inbound"]
]

# EKS Configurations
# upgrade controlplane first then update eks_version_latest_ami to the same version
eks_version           = "1.18"
// Enable Log Types : ["api", "audit", "authenticator", "controllerManager", "scheduler"]
eks_enabled_log_types = []

// Worker Node Configuration
worker_size = {
  desired = 1
  min = 1
  max = 2
}
key_pair = "Test-key"

worker_instance_types = ["t3.medium"]
worker_disk_size = 30

 

 

terraform validate

 

terraform fmt

 

terraform plan --var-file=./env/test.tfvars

 

terraform apply --var-file=./env/test.tfvars

 

terraform destroy --var-file=./env/test/tfvars

 

...