6280652d fat security guard

When your security tools cost more than the thing they protect


Lets say you have a micro-services app. Its got a bunch of containers that you’ve orchestrated out with Kubernetes. Deployments, Pods, Daemonsets all over the place. Autoscaling. You are happy. Now it comes time to implement that pesky ‘security’ step. You are a bit nervous, there’s no internal firewall, all the services listen on port 80, no encryption. All the passwords are hard-coded and in the global environment. No one would guess your l33t mysql password right? So you google ‘how is secur networx’. And you click I’m feeling lucky.

Good thing for you google was watching your previous searches and had the microphone on, so it not only corrected your txt-speak spelling but also selected Istio for you.

But suddenly you need to triple the capacity of your cluster. Lets take a look. Here’s kubectl top from my cluster. The lines in red are associated with the securing + auditing. See that last column? Seems we are using 8144MiB for monitoring the thing that is using 2259MiB. And don’t get me started on the CPU.

I said it before, the cloud doesn’t scale down.

Let’s look. istio-system + logging + monitoring == nearly all the resources!

NAMESPACE    NAME                 CPU  MEMORY
default      ingress-nginx-ingre  4m   146Mi
default      ingress-nginx-ingre  0m   3Mi
istio-system istio-citadel-84fb7  0m   12Mi
istio-system istio-egressgateway  2m   35Mi
istio-system istio-galley-655c4f  13m  39Mi
istio-system istio-ingressgatewa  3m   37Mi
istio-system istio-pilot-6cd69dc  8m   84Mi
istio-system istio-policy-77f684  89m  419Mi
istio-system istio-policy-77f684  97m  521Mi
istio-system istio-policy-77f684  99m  492Mi
istio-system istio-policy-77f684  62m  345Mi
istio-system istio-policy-77f684  63m  351Mi
istio-system istio-sidecar-injec  13m  27Mi
istio-system istio-statsd-prom-b  34m  23Mi
istio-system istio-telemetry-77f  76m  440Mi
istio-system istio-telemetry-77f  105m 559Mi
istio-system istio-telemetry-77f  109m 525Mi
istio-system istio-telemetry-77f  106m 574Mi
istio-system istio-telemetry-77f  79m  437Mi
istio-system prometheus-84bd4b97  51m  689Mi
kube-system  cert-cert-manager-6  2m   22Mi
kube-system  heapster-6c4947855f  0m   41Mi
kube-system  kube-dns-v20-5fd69f  18m  27Mi
kube-system  kube-dns-v20-5fd69f  18m  28Mi
kube-system  kube-proxy-5rhch     3m   36Mi
kube-system  kube-proxy-dxk9f     3m   42Mi
kube-system  kube-svc-redirect-d  11m  156Mi
kube-system  kube-svc-redirect-z  5m   110Mi
kube-system  kubernetes-dashboar  0m   15Mi
kube-system  metrics-server-64f6  0m   26Mi
kube-system  tiller-deploy-895d5  0m   45Mi
kube-system  tunnelfront-7794f9f  21m  16Mi
logging      elasticsearch-867b4  567m 1420Mi
logging      fluent-bit-56d6z     21m  11Mi
logging      fluent-bit-8cbnl     17m  12Mi
logging      logging-fluentd-69f  1m   59Mi
logging      logging-kibana-7684  1m   152Mi
logging      sysctl-conf-92l84    0m   0Mi
logging      sysctl-conf-hb2vn    0m   0Mi
monitoring   alertmanager-monito  1m   15Mi
monitoring   monitoring-exporter  3m   37Mi
monitoring   monitoring-exporter  1m   14Mi
monitoring   monitoring-exporter  1m   10Mi
monitoring   monitoring-grafana-  0m   35Mi
monitoring   monitoring-promethe  2m   30Mi
monitoring   prometheus-monitori  7m   176Mi
socks        carts-6994d7d589-6j  5m   340Mi
socks        carts-db-7dd64bfd7b  5m   96Mi
socks        catalogue-849865789  4m   47Mi
socks        catalogue-db-6d6667  3m   236Mi
socks        front-end-855684fd8  4m   118Mi
socks        orders-7d9cf5cb46-d  5m   350Mi
socks        orders-db-6db4678bf  5m   93Mi
socks        payment-6cdc5b656-8  4m   48Mi
socks        queue-master-7b99db  5m   301Mi
socks        rabbitmq-7c5fbf778d  7m   127Mi
socks        session-db-fdd649d6  3m   52Mi
socks        shipping-5b9ffdbdfb  5m   321Mi
socks        user-84ccd5fd57-2vp  4m   47Mi
socks        user-db-7dcc9649dc-  4m   83Mi