使用KubeCost进行成本可视化

Kubecost 使用户能够查看按 Kubernetes 资源（包括 Pod、节点、命名空间、标签等）细分的成本。 Kubernetes 平台管理员和财务负责人可以使用 Kubecost 可视化其EKS 费用明细、分配成本。 客户可以根据实际的 AWS 账单为其内部团队和业务部门提供透明、准确的成本数据，并获得定制的成本优化建议。

安装KubeCost

安装Kubecost

helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm upgrade --install kubecost kubecost/cost-analyzer --namespace kubecost --create-namespace

确认pod状态变为running：

kubectl get pod -n kubecost

安装AWS load balancer-controller：

curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.7/docs/install/iam_policy.json

aws iam create-policy \
    --policy-name AWSLoadBalancerControllerIAMPolicy \
    --policy-document file://iam_policy.json

export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
export EKS_CLUSTER_NAME="emr-roadshow"
export AWS_REGION="us-east-1"

eksctl create iamserviceaccount \
  --cluster=$EKS_CLUSTER_NAME \
  --namespace=kube-system \
  --region=$AWS_REGION \
  --name=aws-load-balancer-controller \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --attach-policy-arn=arn:aws:iam::$AWS_ACCOUNT_ID:policy/AWSLoadBalancerControllerIAMPolicy \
  --approve

helm repo add eks https://aws.github.io/eks-charts
helm repo update 
   
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=$EKS_CLUSTER_NAME \
  --set region=$AWS_REGION \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

确认AWS load balancer-controller状态变为正常：

kubectl get deployment -n kube-system aws-load-balancer-controller

将kubecost通过ALB进行暴露：

cat << EOF > alb.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kubecost-alb-ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/scheme: internet-facing
spec:
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: kubecost-cost-analyzer
              port:
                number: 9090
EOF

kubectl apply -f alb.yaml -n kubecost
echo "waiting for Kubecost ALB start ...";sleep 20
export ENDPOINT=$(kubectl get ingress kubecost-alb-ingress -n kubecost --output jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "Kubecost UI DNS name: ${ENDPOINT}"

通过Kubecost的UI监控任务花费

在浏览器中访问Kubecost的URL：

image-20231031112540099

在on demand和spot机器上各执行一个任务

aws emr-containers start-job-run \
  --virtual-cluster-id $VIRTUAL_CLUSTER_ID \
  --name spark-demo-ondemand \
  --execution-role-arn $EMR_ROLE_ARN \
  --release-label emr-6.9.0-latest \
  --job-driver '{
      "sparkSubmitJobDriver": {
        "entryPoint": "s3://'$S3BUCKET'/app_code/job/wordcount.py",  
        "entryPointArguments": ["s3://'$S3BUCKET'/output/wordcount_output_demo1"], 
        "sparkSubmitParameters": "--conf spark.executor.instances=1"
        }}'  \
  --configuration-overrides '{
        "applicationConfiguration": [
            {
                "classification": "spark-defaults",
                "properties": {
                    "spark.kubernetes.node.selector.eks.amazonaws.com/nodegroup": "etl-ondemand"
                }
            }
        ]
  }'

aws emr-containers start-job-run \
  --virtual-cluster-id $VIRTUAL_CLUSTER_ID \
  --name spark-demo-spot \
  --execution-role-arn $EMR_ROLE_ARN \
  --release-label emr-6.9.0-latest \
  --job-driver '{
      "sparkSubmitJobDriver": {
        "entryPoint": "s3://'$S3BUCKET'/app_code/job/wordcount.py",
    	  "entryPointArguments": ["s3://'$S3BUCKET'/output/wordcount_output_demo2"], 
        "sparkSubmitParameters": "--conf spark.executor.instances=1"
        }}'  \
  --configuration-overrides '{
        "applicationConfiguration": [
            {
                "classification": "spark-defaults",
                "properties": {
                    "spark.kubernetes.node.selector.eks.amazonaws.com/nodegroup": "etl-spot"
                }
            }
        ]
  }'

Kubecost time range