Tool selection guide for secret protection in Git.

It is common to find application credentials in the source code of brand new projects. The developers are eager to get the project off the ground, fake data is used everywhere and everything is still running on the development machine. Even the database server runs locally.

At some point the project must get onto a server. It is still development only mode and data is still mostly fake or unimportant but the application starts to depend on outside services and API keys are now real. These keys can be compromised and misused to the detriment of the company.

This is the time many developers become conscious of the security risk and starts looking for methods to securely deal with application credentials. When they are in an environment with other server software there may be a ready made solution to deal with the application secrets. Those who are not need the quickest and easiest solution to secure the credentials without slowing down development. The obvious question arises: How do I store the application secrets securely in Git?

A broadly accepted opinion about application security is to “never store plaintext secrets with the code”. This is often phrased as “never store secrets in the code” or “never store secrets in Git” which are then provided as unhelpful responses when someone has a question on how to store application credentials securely in Git. While the last 2 phrases are pretty close to the first, they are ignorant of the security principle behind it.

That principle is the one of separation between access to the code and access to the plaintext secrets. There are definitely better options for secrets management than Git but for some cases it can provide the necessary separation.

One reason that makes this choice attractive is that it only requires one extra tool that interacts with Git. There are no new servers or services to set up and maintain.

When considering a secrets management tool or service there are a few points that should be considered in determining the suitability of each option. The relevance of each point was discussed in a previous post.

If after evaluating these considerations and it is still an option to store your secrets in Git, then there are some more issues to consider which are directly relevant to Git-based secrets management solutions.

Storage repo

The secrets can be stored in the same repo as the code or in a different one.

Storing them in the same repo has the following advantages:

• The secret and code histories are tracked together. There is never a question of which version of the secrets repo matches which version of the code repo.
• There is less Git admin

Using different repos has these advantages:

• There is more separation between code and secrets.
• It is easier to prevent environments from using the wrong set of secrets.

Encryption scope

There are options to encrypt specific files in a repo or the whole repo.

If the whole repo is encrypted but the code and secrets are stored in the same one then there still is no separation between access to the code and access to the secrets. Encrypting the whole repo also limits Git’s capability to operate with hosting providers.

When individual files are encrypted it allows storage of the secrets and code in the same repo while still allowing developers to work on the code without access to the secrets.

Secret versioning

Some of the tools implement access control in a way that loses access to older secret versions when the access permissions change. This may become a security hole when access permissions are not updated because access to the older versions is needed.

Secret rotation

When a single repo is used it requires an application redeployment when secrets are rotated.

A dual repo setup allows secret rotation to happen with hot-reloading the secrets or an application restart.

Access control

Access to the secrets can be granted with shared passwords (symmetric key encryption) or with GPG keys (public key encryption).

Shared passwords are easier for the team members because it does not bring GPG into their workflow but it is riskier because the password must now be distributed through a secure channel. There is also no individual access control. Anybody with the password has access to the secrets. When the password gets compromised it must be changed and redistributed to everyone.

GPG keys have some overhead for each developer because they now need to manage their keys. It is however a much more secure solution because only the public keys should be communicated between members and that does not require a secure channel. Access is also granted individually. When a key gets compromised, only one person’s key need to be revoked.

Deterministic encryption

Deterministic encryption means the same input text always results in the same encrypted text. Encryption tools normally avoid this because it is easier to break than non-deterministic encryption.

Some of the tools here implement deterministic encryption because it needs less complicated logic to determine when the secrets file should be re-encrypted. It also requires that the secret file is always encrypted with the same symmetric key. If the encryption key is changed all previous versions of the secret file becomes inaccessible.

Revoking access to the secrets are normally done when a developer leaves the team or when the encryption key has been compromised. The tool author can deal with this need in one of two ways:

1. Do not implement key rotation, keep access to all old secret versions.
2. Implement key rotation, lose access to pre-rotation secret versions.

With option 1 it is not possible to revoke someone’s access to the secrets, with option 2 it is not possible to roll back the secrets past the time of rotation.

Transparent operation

Some tools operate transparently, that is, they do the encryption and decryption on-the-fly during the normal Git commit/update flow without any human intervention. This is accomplished by hooking into Git’s clean/smudge filters. The encryption/decryption is done with every commit and update so there is a performance hit that may become significant when a large number of files are encrypted.

Tool proficiency

All the possible tools add additional commands to Git or have their own list of commands. Many of them also require using GPG. That is a lot of extra commands to learn and few, if any, of them have GUI integrations. There will be an initial impact on developer productivity.

Deployment process

Using any of these tools will have an effect on your deployment process. There will be management overhead to ensure all the necessary people and CI machines have the necessary access to deploy the application. The deployment process itself will need to change to accommodate the new secret setup.

Tool options

Six tools were evaluated and their features tabled below. Note that this review is not an endorsement of any tool’s security claims.

• git-crypt
• git-secret
• git-remote-gcrypt
• blackbox
• transcrypt
• keybase