How do I get rid of dead containers from cluster?

Due to inevitable issues the first time a container it started (misconfig), it restarts quite a few times before I can set replicas to 0. This creates a lot of dead containers (grey square in the cluster ui) which do not go away after a fixed image/config is pushed. Even after cluster reboot they still stick around.

Even the provided load balancer service creates these zombie containers.