Deploy ETL on EKS cluster with Application Load Balancer.

The section explains how to deploy ETL in an EKS cluster using an Application Load Balancer.

1. Create an Amazon EKS cluster and node group to deploy Bold BI and Bold Reports.

   https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html

2. Create an Amazon Elastic File System (EFS) volume to store the shared folders for application usage by following this link.

3. Connect to your Amazon EKS cluster.

   https://aws.amazon.com/premiumsupport/knowledge-center/eks-cluster-connection/

4. Deploy the EFS CSI Driver to your cluster.

   https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html

   kubectl apply -k "github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/?ref=release-1.7"
   

5. To install the AWS load balancer controller, please refer to the following documentation.

   https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/deploy/installation/

6. Create the bold-etl-deployment.yaml file and use the following configuration.

   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: efs-sc
     namespace: bold-etl
   provisioner: efs.csi.aws.com
   ---
   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: bold-etl-fileserver
     namespace: bold-etl
   spec:
     capacity:
       storage: 5Gi
     accessModes:
     - ReadWriteMany
     persistentVolumeReclaimPolicy: Retain
     storageClassName: efs-sc
     csi:
       driver: efs.csi.aws.com
       volumeHandle: <efs_file_system_id>
   ---
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: bold-etl-fileserver-claim
     namespace: bold-etl
   spec:
     accessModes:
     - ReadWriteMany
     storageClassName: efs-sc
     volumeName: bold-etl-fileserver
     resources:
       requests:
         storage: 3Gi
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: bold-etl-deployment
     namespace: bold-etl
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: bold-etl
     template:
       metadata:
         labels:
           app: bold-etl
       spec:
         containers:
         - name: bold-etl-container
           image: gcr.io/boldbi-294612/bold-etl:ETL
           ports:
           - containerPort: 80
           resources:
             requests:
               cpu: 500m
               memory: 2Gi
           readinessProbe:
             httpGet:
               path: /health-check
               port: 80
             initialDelaySeconds: 30
             periodSeconds: 15
             timeoutSeconds: 5
           volumeMounts:
           - mountPath: /application/app_data
             name: bold-etl-volume
         volumes:
         - name: bold-etl-volume
           persistentVolumeClaim:
             claimName: bold-etl-fileserver-claim
             readOnly: false
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: bold-etl-service
     namespace: bold-etl
   spec:
     selector:
       app: bold-etl
     ports:
     - protocol: TCP
       port: 6009
       targetPort: 80
     type: ClusterIP
   ---
   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     namespace: bold-etl
     name: boldetl-ingress
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
       alb.ingress.kubernetes.io/target-type: ip
    #   alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    #   alb.ingress.kubernetes.io/certificate-arn: ""
   spec:
     ingressClassName: alb
     rules:
     - http:
         paths:
         - path: /
           pathType: Prefix
           backend:
             service:
               name: bold-etl-service
               port:
                 number: 6009
   

7. Craft the bold-etl-deployment.yaml file by following the steps below:

   • Update the file system ID to the <efs_file_system_id> location in the file that was previously created. If you wish to use a custom storage class name, please specify it as shown below. The default storage class name is “efs-sc”.
     
   • If you want to enable SSL for the ALB, uncomment the following line and apply the certificate ARN as shown below.
     
     

8. Run the following command to create the namespace for ETL service.

   kubectl create ns bold-etl
   

9. Run the following command to deploy the Bold ETL application.

   kubectl apply -f bold-etl-deployment.yaml
   

10. Once Bold ETL application has been successfully deployed, obtain the ingress IP using the following command.

    kubectl get ingress -n bold-etl
    

    

11. Now, with the ingress IP above, you will be able to access the Bold ETL application.

12. Create Project in Bold ETL using this.