Automatic failover using Interlock + CI/CD(s)
In the last few weeks in the middle of the lockdown i was bored and i (re)started playing around Go since i was a bit rusty.
My main goal was to automagically failover all my static sites from Github & other public hosting sites back and forth in order to maintain an active DR plan when one of my providers is going offline because of connectivity or any other outage/maintenance.
So before going into the blog post here is a quick overview:
You can check it also on the readme there: https://github.com/fnzv/interlock
When i started writing the code this time i tried to force myself to avoid the bad and ugly "exec shell" in order to be able to create a more thin Golang docker image (260MiB)
So to avoid making you read all the README.md i could sum up all the code in this sentence:
"Interlock is a DNS failover and management tool based on Cloudflare APIs"
Maybe i reinvented the wheel because i could have used (paid) the built-in health check of CF or any other DNS provider but i wanted to learn and research underneath in order to get my hands "dirty" and mess around Go again
This piece of code is supposed to run also inside your k8s cluster in order to gather metrics versus your ingresses and monitor internal latency between your cluster nodes, all the metrics are gathered and sent directly to the configured InfluxDB host
Here you can see an example visualization made in a few minutes to monitor some of my public static sites latency
This data could be used to generate post-hook events in order to run external bash scripts via https://github.com/adnanh/webhook (kudos to the dev) + grafana webhooks from Alerts
Cool but how can i use it right now?
Well this is up to you depending on the use case, to run a interlock generic example with a "famous" Public CI/CD you can follow these instructions:
- Clone the Git repo or import directly from your Gitlab account
- Setup your ENV variables (masked) under Settings -> CI/CD -> Variables (Expand)
Keep the dry run env variables in order to avoid any changes to your CF account
Telegram and InfluxDB variables can be kept empty
Just in case keep your repo privacy settings to private
- Change your configuration under interlock.conf
Set your origins
Commit those changes with the commit message "build" in order to prepare your Docker image
- Go under CI/CD -> Pipelines and try to run the already configured one by pressing "Run pipeline"
If everything is set up properly you will see all green
- Schedule your pipeline in order to run multiple times using crontab syntax (if you don't speak crontabese you can go https://crontab.guru/), for example */15 * * * *
To run only one stage of the pipeline you must set the ENV variable CRON to TRUE
Leaving you with this CI/CD meme here, if you have any question or suggestions you can contact me or open a pull request
Links and Resources:
CTRL+R(reverse-i-search)`add': git add . && git commit -m "Initial Commit" && git push origin master