May
29
2022

Deploying Containers to VMs and Instance Groups in GCP

deploying-containers-to-vms-in-gcp

In my last article I’ve outlined how Kubernetes is overused in some scenarios where it does not suit. I’ve even given some good alternatives. One of the obvious alternatives is deploying containers in Virtual Machines or Managed Instance Groups (MIGs). Let’s explore how we can achieve it with minimal effort.

You can access the commands used in the article via this repository.

Deploying Containers to Virtual Machines

Deploying container images to the VM is pretty straight forward. You just need to be ready with your image and run single command, and rest of the things taken care by GCP for you.

Create VM with container image

Following is the command you can use to create Virtual Machine with container image.

gcloud compute instances create-with-container busybox-vm \
--container-image docker.io/busybox:1.27 \
--container-restart-policy on-failure \
--container-privileged

Update VM Containers

You can also update the commands for the container running inside VM.

gcloud compute instances update-container busybox-amv \
--container-command "/bin/ash" --container-arg="-c" \
--container-arg="ls -l"

STDIN and TTY in VM Containers

You can allocate a buffer for STDIN in the container runtime to keep the STDIN stream open in a container. If this is not set, reads from STDIN in the container always result in EOF. Also, you can debug by establishing an interactive shell by enabling pseudo-TTY.

gcloud compute instances update-container busybox-vm \
--container-tty \
--container-stdin

Supply Environment Variables in VM Containers

You can provide environment variables for your container.

gcloud compute instances create-with-container busybox-vm \
--container-image docker.io/busybox:1.27 \
--container-env HOME=/home,MODE=test,OWNER=admin

Or, you can also provide environment variables using env variable file.

gcloud compute instances create-with-container busybox-vm \
--container-image docker.io/busybox:1.27 \
--container-env-file ./env.txt

The file contents look like the following:

# This is an env.text file content
HOME=/home
MODE=test
OWNER=admin

You can remove the environment variables:

gcloud compute instances update-container busybox-vm \
--remove-container-env MODE,OWNER

Mounting Volumes in VM Containers

Mouning a host directory as a data volume:

gcloud compute instances create-with-container busybox-vm \
--container-image docker.io/busybox:1.27 \
--container-mount-host-path mount-path=/logs,host-path=/tmp,mode=rw

Removing volume mounts

gcloud compute instances update-container busybox-vm \
--remove-container-mounts /logs

Mounting tempfs file system as a data volume

gcloud compute instances create-with-container busybox-vm \
--container-image docker.io/busybox:1.27 \
--container-mount-tmpfs mount-path=/cache

Removing tempfs volume

gcloud compute instances update-container busybox-vm \
--remove-container-mounts /cache

Mounting Persistent Disks

Creating and mounting persistent disks

gcloud compute instances create-with-container busybox-vm \
--disk name=my-data-disk \
--create-disk name=my-scratch-disk,auto-delete=yes,image=ubuntu-1710-artful-v20180315,image-project=ubuntu-os-cloud \
--container-image docker.io/busybox:1.27 \
--container-mount-disk mount-path="/disks/data-disk",name=my-data-disk,mode=ro \
--container-mount-disk mount-path="/disks/scratch-disk",name=my-scratch-disk

Removing/Updating mounts

gcloud compute instances update-container busybox-vm \
--container-mount-disk mount-path="/disks/data-disk",name=my-data-disk,mode=rw \
--remove-container-mounts "/disks/scratch-disk"

Allow traffic to VM Containers

VMs with containers use the host network mode, where a container shares the host’s network stack and all interfaces from the host are available to the container.

Container ports have a one-to-one mapping to the host VM ports. For example, a container port 80 maps to the host VM port 80. Compute Engine does not support the port publishing (-p) flag, and you do not have to specify it for the mapping to work.

To publish a container’s ports, configure firewall rules to enable access to the host VM instance’s ports. The corresponding ports of the container are accessible automatically, according to the firewall rules.

Step 1: Create VM with NGINX container

gcloud compute instances create-with-container nginx-vm \
--container-image gcr.io/cloud-marketplace/google/nginx1:1.15 \
--tags http-server

The container shares the host VM’s network stack, and the container’s port 80 is published to the host VM’s port 80. The http-server tag is used as a target tag for the firewall rule

Step 2: Create a firewall rule to enable connections to port 80 of the VM instance

gcloud compute firewall-rules create allow-http \
--allow tcp:80 \
--target-tags http-server

The container automatically starts receiving traffic on port 80.

Debug VM Containers

You can also ssh VM containers and debug the issue you are facing in your containers.

gcloud beta compute ssh nginx-vm \
--zone=us-central1-a \
--container={CONTAINER-ID OR CONTAINER-NAME}

 

Deploying Containers on an MIG (Managed Instance Group)

There are two major steps involved in creating MIGs with container images.

Step 1: Create an instance template

gcloud compute instance-templates create-with-container nginx-instance-template \
--container-image=gcr.io/cloud-marketplace/google/nginx:1.20 \
--container-restart-policy on-failure \
--container-privileged

Step 2: Create an MIG with the Instance Template

gcloud compute instance-groups managed create nginx-group \
--base-instance-name nginx-vm \
--size 2 \
--template nginx-instance-template

SSH your container image

gcloud beta compute ssh nginx-vm-{####} \
--container nginx-instance-template

Create Autoscaling and Multi-zone MIG

Create MIG in multi-zone

gcloud compute instance-groups managed create nginx-group \
--base-instance-name nginx-vm \
--size 8 \
--template nginx-instance-template \
--zones us-east1-b,us-east1-c

Set autoscaling

gcloud compute instance-groups managed set-autoscaling nginx-group \
--max-num-replicas 20 \
--target-cpu-utilization 0.60 \
--cool-down-period 90

Challenge

I left with the challenge here. There is a way to create stateful MIGs with persistent disks. You need to create instance template with container image mounting persistent disks and then create stateful MIG (with persistent disk) with the instance template. You can put your answer as a comment.

Conclusion

So, ideally you have options to Kubernetes when it’s absolutely not required like I mentioned in my last article.