On operating on live environments
Operating on live environments should not be a fire-and-forget task. Care and attention has to be put into every release to production (even to staging) environments as releasing to any of these environments requires operating on a living system that has to keep functioning during this process. These environments will, hopefully, be in a better shape at the end of the process. To put it in perspective, doctors operate on living systems too; the risks and the scale are different, but the principles are the same.
Having thoroughly defined and fully automated development processes makes operations like releasing to production feel mundane. The idea would be for everyone in a team, from the most junior to the most senior member, to feel comfortable putting new code in production.
Automation is here to help us handle common tasks, from the simple ones like continues integration to the scariest ones like releasing to production. By automating processes we are removing the need of manual repetition of these common tasks as well as reducing the probability of human errors.
However, machines are not smarter than humans, they are just better at following orders, executing steps, and remembering things. It doesn’t matter how many checks and automation the release process has, my mind is never fully at peace when automated operations are being executed. I have come to accept this and, that it is better this way. The reason is that there can always be an unhandled, not automated, case that can break an environment and the automated processes can only protect from things have been added to it already.
We have come a long way from the earlier days when people would be afraid of touching production environments, maybe because of how manual the process was, the lack of proper monitoring or because there was not enough care put into building healthy development workflows. They used to say: If it ain’t broken, don’t fix it, some people still do.
Not being afraid of operating on live environments is a great win. This allows us to do it more often and more safely. Let’s not forget that these are still living environments. It doesn’t matter how many times the environment is operated, thought and care should always be put into every action.