- Published on
Dotenv-vault: The New Way to Manage .env
- Authors
- Name
- Hoh Shen Yien
Table of Contents
Sharing .env
values manually through copy-pasting seems to still be the most common approach in the software industry. However, I thought we've always been boasting about our technological prowess, but why are we stuck with this old-school tradition?
In this article, I will be sharing a somewhat opinionated tool, dotenv-vault that modernizes the way we handle .env
files!
As a bonus, I'll also show you how it can simplify your docker deployment process.
What's Wrong with Manual Sharing?
While sharing .env through chats doesn't necessarily lead to disaster, but it does come with some unpleasant effects.
PS: I will not consider security here. (not that it's insignificant, but I doubt that multimillion project owner still doing it)
Messy
Here are some scenarios
- You asked someone for a secret, they sent you through DM's or Telegram.
- Someone updated a feature to include a new secret, they sent them to a group
- Your team leader thought the secret might better live in the documentation
PS: Only some - During onboarding, the secrets were sent to you through an email
- etc.
So where can you find secret A? Telegram? Slack? Direct Message? Group Message? Which Group?
The secrets are scattered everywhere!
Of course, it's not the end of the day, you can always ask another developer for the secrets.
Just that we developers don't like to ask people sometimes.
Error-prone
What if you accidentally deleted a secret from staging/production while editing the file? Or what if you edited the wrong variable? What if you deleted a character while scrolling down?
Maybe it wasn't noticeable right away as .env are cached, but the production might go down during the next deployment!
As long as edits happen in terminals, it's prone to errors, especially when juniors or interns who aren't familiar with bash get access to it. (I believe the readers are not one of these)
PS: From one of my experiences, we update the production/staging .env using ssh, and that's my assumption here.
Again, these two reasons aren't huge enough issues, but they make us less happy at times. Also, it's always a good reason to progress forward, isn't it?
In the worst-case scenario, it may even lead to huge mistakes, such as deleting important env secrets while updating production .env
, and breaking the production.
Env Vault
Introducing .env
secrets manager, dotenv-vault.
In short, it's like Git, but for .env
files.
It's pretty intuitive for most programmers out there, with operations involving only pull
, push
, and build
most of the time.
Benefits
Clearly, when it comes to benefits, it must solve the aforementioned issues. Let's see how dotenv-vault
does it
Clean
Even with just a quick peek at the screenshot above, you can already see how the secrets will be kept centralized.
The best part about this is that if you're building with microservices or having multiple projects, then these secrets can be accessed by all of them, without repetition! Say bye to copy pasting values across multiple projects.
Every update will be made directly to the vault, so everyone can access all updated values easily. Beyond that, it also offers different environments to separate the secrets of our environments, so it does cater to most of our needs.
Safety
This is not about security, but corresponds to error prevention.
Remember how I commented on the insecurity of editing file from the terminal? What if you can only edit one variable at one time? While carelessness can still lead to mistakes, the possibility of errors will be minimized now.
Simplicity
How many commands do we need to remember? 2, for most cases.
npx dotenv-vault pull [development/staging/production] # to pull in new changes
npx dotenv-vault push # to push changes from local
Security
I'm no security expert here, but according to the docs, they do look pretty secure! All decrypted values exist only in memory and are flushed after using it.
Others
Unfortunately, the paid version is required to access more features of dotenv-vault. Nonetheless, $5 per month per user seems rather affordable for companies out there.
You probably don't need dotenv-vault for your personal projects anyways, who are you sending your secrets to?
Here are some cool features I find on paid version:
- Version history, similar to Git, allows you to rollback and examine changes at the individual level
- Slack notification, who wouldn't want to be notified when someone changes the secrets?
- User Access Controls, it's probably best to just let the DevOps guys manage these keys. Or perhaps the seniors.
Of course, these are some nice additions, dotenv-vault is totally amazing in its free version anyways.
What about Config Server?
If you come from a Spring Boot background, you might find this surprisingly similar to config server! Nevertheless, config servers aren't the convention for JavaScript & Python projects, so yeah.
There are more differences, but I'd stop here since it's not the purpose of this article. You can read more about it here.
Project Setup
To demo the usage of dotenv-vault
, I'll be using a Next.js project. More specifically, I'll be using my next-power-starter template, to get started, simply run
yarn create next-app dotenv-hello-world -e https://github.com/HohShenYien/next-power-starter
Not self-promotion, just simply because it's an empty project on my computer that I can use directly.
Usage
Sign Up
Well, I guess you know how to do that. Here's the link.Create a new vault
The vault is required to link the project with dotenv-vault server. This step is required for new projects.npx dotenv-vault new
After pressing
y
, your browser will launch into a page like thisAfter clicking next, you will notice that a new file,
.env.vault
is added to your project!The value there will point to the remote project on dotenv. (doesn't it sound like Git?)
ThisDOTENV_VAULT
is not a secret, it's just like the github project link, nothing special about it. Therefore it's okay to be shown.Login from terminal
The next step will require us to prove that we're a member of the project before we can access the secrets.npx dotenv-vault login
You'll either be logged in already like below, or you'll need to login in dotenv.
Going back to the project, you should see a new file,
.env.me
. This file is like the auth token that proves that you can access the project. Therefore, if you share the file, then others can also access the secrets without having to login again.PS: This file is like the top secret, don't share it!
Add some secrets
Next, let's head back to your dashboard. You should see your project being listed there.Clicking into the project, you should see something like
Since we have already setup, just click the click here.
Next, we can add a secret, say NEXT_PUBLIC_APP_NAME and MY_NAME.
PS: The prefix NEXT_PUBLIC... means that it can be used in the client, while other environmental variables can only be accessed in the server. Read more about it in the docs.
It should look like this in the end.
Note that the environment is
development
Pull from local
Now that we've set it up indotenv-vault
, we can now download it to our local project. Let's runnpx dotenv-vault pull development # depends on the environment you want to pull
and voila, the image is now in your project!
Push from local
Maybe you'd prefer to update the.env
and publish it ondotenv
? Let's do itI've added a new variable,
MY_EMAIL
and modifiedMY_NAME
to my full name. Let's push itnpx dotenv-vault push development # depends on the environment you want to push to
Checking it on the
dotenv-vault
, you should see the new keys appearing there
Checking the environmental variable in project
Let's use the variables from the Next.js project. Open up the page, app/page.tsx
Yes, this project uses the app directory, you can read more about it here.
// src/app/page.tsx
...
export default function Home() {
// I can do this because this is a server component
const myName = process.env.MY_NAME;
...
<p className="text-slate-600">
The template which balances flexibility and simplicity
</p>
<p className="font-sm text-slate-600">By {myName}</p>
...
Spin up the development server,
yarn dev
You can now find the variable shown up (The By Hoh Shen Yien)
Bonus: Deployment with Docker
Finally, let's look at how we can deploy the project with docker.
Production Environment
Let's first push our .env
to production,
.env.production
Let's create a new.env.production
file which will be pushed to productioncp .env .env.production
Before we push, we will also need to edit the first line, which indicates the environment
# production@v1 NEXT_PUBLIC_APP_NAME="dotenv-hello-world" MY_NAME="Hoh Shen Yien" MY_EMAIL="hohshenyien@gmail.com"
Pushing to dotenv
npx dotenv-vault push production
You should see the secrets appearing on remote dotenv
, in the production environment.
Setting up Docker
You can install the Docker here if you haven't already.
Let's add a Dockerfile, here's the starting file from Nextjs
Copy the entire content, and paste it into /Dockerfile
.
Finally, also update /next.config.js
to use the "standalone" output type
/** @type {import('next').NextConfig} */
const nextConfig = {
output: 'standalone',
}
module.exports = nextConfig
Setting up Next.js
To deploy on Docker, we'll need to add a new dependency,
yarn add dotenv-vault-core
You'll need this line before you can access the environmental variables.
It will download the .env to the server during execution
require('dotenv-vault-core').config()
Since there's technically no index.js
file that will be called on every page (if just page, then it can lie in app/layout.tsx
, but they won't be available in API routes) & API.
Hence, I'll create a env.ts
file that contains all environmental variables
// src/utils/configs/env.ts
require('dotenv-vault-core').config()
export const MY_NAME = process.env.MY_NAME
And to access these variables, just import them! Simple and straightforward.
Retrieving Production Key
Finally, we will need to pass the production vault key for dotenv
to download the .env
file. To retrieve the key, you can run
npx dotenv-vault keys production
You'll get the production key in the next line. Copy it, we'll use it to run the docker image.
PS: the key should look like dotenv://key_..../.env.vault?environment=production
Building Docker Image
Finally, just build the docker image as usual
docker build -t docker-next .
and to test it locally on localhost:8080 (Because 3000 is occupied by the running Next.js project earlier)
docker run -e DOTENV_KEY="your_key_here" --rm -it -p 8080:3000 --init docker-next
And that's it. You have a perfectly working Docker image.
You can check out their documentation for more details.
Conclusion
Migrating to dotenv-vault
is relatively simple, and its benefits are quite significant.
I used it once and truly enjoyed it, so perhaps you should try it too!
Disclaimer: I am not sure how will dotenv
and Docker work in a serverless environment (Vercel), maybe it'll retrieve it for every different server?