Week 10/22/2017 - 10/29/2017 Repository migration & Deployment process improvement
In the past week I've been working on mainly one thing: migrating our existing repositories from bitbucket to github.
Git is the first source control tool I used and has become dominant over the past few years. To be honest I'm surprised to see my project uses mercurial.
The task of migration is fairly simple: fast-export will do the heavy lifting for you. Here's an article on how to do it. The reason why it took me a lot of time is: my new project has baked a lot of secrets / keys inside source code and now it's going to be more public than only accessible by my group.
After I started digging into these secrets, I've discovered more issues:
- configuration management and secret management are done with the same configuration file
- configuration and source code is not a perfect 1:1 map. With a lot of copy-paste, many sections of configuration are not even being used in code
- deployment is a mess. some repos have dedicated (not 100% accurate) jenkins jobs for deploying, others rely on "experience" to deploy(0 documentation & 0 automation)
- branching strategy is weird: hard to perform rollback if needed
- deployment of my current project exposes write access to source code
Ideally what I wanted:
- a reliable service for configuration management. e.g. consul
- a reliable secret management service. e.g. vault
- prune the configurations to reflect what is actually needed by the application
- only fetch configuration / secrets when needed and do not persist a copy locally
The actual plan:
- migrate the repo with history to private github repos first. learning how to use mercurial is taking a lot of my time and I'm doing that just to avoid using mercurial in the future, seems contradicting.
- update existing deployment jobs & create new deployment jobs to have 100% accurate service deployment coverage.
- delete the configuration file and fetch configuration file from a persisted data store. the requester needs provisioning to access the data store, and in order not to expose more secrets / keys, I'm using AWS IAM instance profile to get around this.
- test the changes on dev, followed by production. If everything looks good, expose the git repo to public audience.
What I'm planned but didn't do: building config & secret management service stack and migrate our application to use that. Too much work for what I wanted to achieve initially :P
Git is the first source control tool I used and has become dominant over the past few years. To be honest I'm surprised to see my project uses mercurial.
The task of migration is fairly simple: fast-export will do the heavy lifting for you. Here's an article on how to do it. The reason why it took me a lot of time is: my new project has baked a lot of secrets / keys inside source code and now it's going to be more public than only accessible by my group.
After I started digging into these secrets, I've discovered more issues:
- configuration management and secret management are done with the same configuration file
- configuration and source code is not a perfect 1:1 map. With a lot of copy-paste, many sections of configuration are not even being used in code
- deployment is a mess. some repos have dedicated (not 100% accurate) jenkins jobs for deploying, others rely on "experience" to deploy(0 documentation & 0 automation)
- branching strategy is weird: hard to perform rollback if needed
- deployment of my current project exposes write access to source code
Ideally what I wanted:
- a reliable service for configuration management. e.g. consul
- a reliable secret management service. e.g. vault
- prune the configurations to reflect what is actually needed by the application
- only fetch configuration / secrets when needed and do not persist a copy locally
The actual plan:
- migrate the repo with history to private github repos first. learning how to use mercurial is taking a lot of my time and I'm doing that just to avoid using mercurial in the future, seems contradicting.
- update existing deployment jobs & create new deployment jobs to have 100% accurate service deployment coverage.
- delete the configuration file and fetch configuration file from a persisted data store. the requester needs provisioning to access the data store, and in order not to expose more secrets / keys, I'm using AWS IAM instance profile to get around this.
- test the changes on dev, followed by production. If everything looks good, expose the git repo to public audience.
What I'm planned but didn't do: building config & secret management service stack and migrate our application to use that. Too much work for what I wanted to achieve initially :P
Comments
Post a Comment