Kubernetes Infrastructure Task

Job/Pod Manager with Role-Based Access Control

Kubernetes PostgreSQL RBAC Infrastructure
📋

Task Overview

Design and implement a production-ready infrastructure for a platform that allows users to manage Kubernetes Jobs and Pods through a web UI with role-based restrictions.

Primary Objectives

  • Implement secure access control to Kubernetes resources
  • Design database schema for users, roles and permissions
  • Create infrastructure as code for Kubernetes deployment
  • Implement security best practices throughout the stack
  • Deploy PostgreSQL as part of the Kubernetes cluster

Key Components

  • PostgreSQL database for authentication and authorization
  • Backend API with Kubernetes client integration
  • Frontend for user interaction
  • Kubernetes RBAC configuration
  • PostgreSQL deployment within Kubernetes

System Architecture

graph TD A[User] --> B[Frontend] B --> C[Backend API] E --> F[Pods] E --> G[Jobs] subgraph Kubernetes Cluster B C D[PostgreSQL Database] E[Kubernetes API] F G H[Service Account] I[RBAC Roles] J[PostgreSQL Pod] end C --> H H --> I I --> E C --> J J --> D

Infrastructure Tasks

1PostgreSQL Deployment on Kubernetes

Deploy PostgreSQL as a StatefulSet within the Kubernetes cluster for high availability and data persistence.


# PostgreSQL ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-config
  namespace: platform
  labels:
    app: postgres
data:
  POSTGRES_DB: "<DB_NAME>"
  POSTGRES_USER: "<DB_USER>"
  POSTGRES_PASSWORD: "<DB_PASSWORD>"
  PGDATA: "/var/lib/postgresql/data/pgdata"
                        

# PostgreSQL Service
apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: platform
spec:
  selector:
    app: postgres
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
  clusterIP: None
                        

# PostgreSQL Secrets
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secrets
  namespace: platform
type: Opaque
data:
  <DB_PASSWORD>: cGFzc3dvcmQ= # base64 encoded <DB_PASSWORD>
  username: YWRtaW4=     # base64 encoded username
  database: am9ibWFuYWdlcg== # base64 encoded database name

2PostgreSQL StatefulSet

Create a StatefulSet for PostgreSQL with persistent storage.


apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: platform
spec:
  serviceName: "postgres"
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: <POSTGRES_IMAGE>
        imagePullPolicy: "IfNotPresent"
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_DB
          valueFrom:
            configMapKeyRef:
              name: postgres-config
              key: POSTGRES_DB
        - name: POSTGRES_USER
          valueFrom:
            configMapKeyRef:
              name: postgres-config
              key: POSTGRES_USER
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: <DB_PASSWORD>
        - name: PGDATA
          valueFrom:
            configMapKeyRef:
              name: postgres-config
              key: PGDATA
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: postgres-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "standard"
      resources:
        requests:
          storage: 10Gi

3Kubernetes RBAC Configuration

Create ServiceAccount, Role, and RoleBinding for the backend application.


apiVersion: v1
kind: ServiceAccount
metadata:
  name: job-pod-manager-sa
  namespace: platform
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: job-pod-operator
  namespace: work
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list", "delete"]
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get", "list", "create", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: job-pod-manager-binding
  namespace: work
subjects:
- kind: ServiceAccount
  name: job-pod-manager-sa
  namespace: platform
roleRef:
  kind: Role
  name: job-pod-operator
  apiGroup: rbac.authorization.k8s.io

4Backend Service Deployment

Deploy the backend API with proper service account and database connectivity.


apiVersion: apps/v1
kind: Deployment
metadata:
  name: job-pod-manager-backend
  namespace: platform
spec:
  replicas: 2
  selector:
    matchLabels:
      app: job-pod-manager-backend
  template:
    metadata:
      labels:
        app: job-pod-manager-backend
    spec:
      serviceAccountName: job-pod-manager-sa
      containers:
      - name: api
        image: <BACKEND_IMAGE>
        env:
        - name: DATABASE_URL
          value: "postgresql://<DB_USER>:<DB_PASSWORD>@postgres.platform.svc.cluster.local:5432/<DB_NAME>"
        - name: JWT_SECRET
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: jwtSecret
        ports:
        - containerPort: 8080
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: job-pod-manager-backend
  namespace: platform
spec:
  selector:
    app: job-pod-manager-backend
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080

5Frontend Deployment

Deploy the frontend with connection to the backend API.


apiVersion: apps/v1
kind: Deployment
metadata:
  name: job-pod-manager-frontend
  namespace: platform
spec:
  replicas: 2
  selector:
    matchLabels:
      app: job-pod-manager-frontend
  template:
    metadata:
      labels:
        app: job-pod-manager-frontend
    spec:
      containers:
      - name: web
        image: <FRONTEND_IMAGE>
        env:
        - name: FRONTEND_APP_API_BASE_URL
          value: "http://job-pod-manager-backend.platform.svc.cluster.local"
        ports:
        - containerPort: 80
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 30
          periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: job-pod-manager-frontend
  namespace: platform
spec:
  selector:
    app: job-pod-manager-frontend
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

6Ingress Configuration

Set up Ingress to expose the frontend and backend services.


apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: job-pod-manager-ingress
  namespace: platform
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: <DOMAIN_NAME>
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: job-pod-manager-frontend
            port:
              number: 80
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: job-pod-manager-backend
            port:
              number: 80

Security Checklist

Troubleshooting Guide

  1. Check if PostgreSQL pod is running: kubectl get pods -n platform | grep postgres
  2. Verify the service exists: kubectl get svc -n platform postgres
  3. Test database connectivity: kubectl exec -it [backend-pod] -n platform -- nc -zv postgres.platform.svc.cluster.local 5432
  4. Check database logs: kubectl logs [postgres-pod] -n platform

  1. Check if the ServiceAccount exists: kubectl get serviceaccount -n platform
  2. Verify RoleBinding is correct: kubectl describe rolebinding job-pod-manager-binding -n work
  3. Test API access with: kubectl auth can-i create pods --as=system:serviceaccount:platform:job-pod-manager-sa -n work

  1. Check user roles in the database
  2. Verify permission records for the user's roles
  3. Ensure the namespace in the request matches the permission constraints