I started a new open-source software project called Parameters. It’s written in modern C++. You can use Parameters to automatic replace variables in configuration file templates from environment variables. In this blog post I describe how to use Parameters to create a .env file for Symfony 4.
I am using Symfony 4 now for awhile. It’s a really cool webframework. I like it.
Since version 3 (or so), Symfony is using the Dotenv Component to manage parameters for database connection, mail server access, etc. The kind of parameter which you do not want in your Git history.
We put our environment variables into GitLab’s CI variables.
(This is how the variables look like for my blog. Top secret.)
On deployment these variables, from GitLab CI, will be replaced in a .env-template file. Let’s call this template file .env.dist.
Normally .env can look like this:
# .env
DB_USER=root
DB_PASS=pass
Here is an example how the .env.dist template file could look like:
# .env.dist
DB_USER=@SYMF_DB_USER@
DB_PASS=@SYMF_DB_PASS@
The variable names within @ are directly coming from the GitLab CI/CD variables.
The Problem
Sometimes you do not just have one single value for the same variable. In most the projects we have multiple environments. For example, Testing, Staging and Production. By using GitLab you cannot distinguish variables based on environments in your CI/CD variables. Of course you can put it into your .gitlab-ci.yml. But here lies the problem: you do not want sensitive data in .gitlab-ci.yml. This would mean that it’s in Git, forever.
So we need another way to use the same variable for multiple environments. For this purpose I created the Parameters C++ project. To get around this problem we just create multiple variables for one and the same template variable in GitLab CI/CD. Then using the Parameters program to replace the variable based on the environment (Testing, Staging, Production, etc).
Let’s take @SYMF_DB_PASS@ from the example above. We set one default password in GitLab CI/CD variables. For the purpose of this blog post I’ll use bash syntax to describe which GitLab CI/CD variables are set:
# GitLab CI/CD Variables
SYMF_DB_PASS=hello_world
For this example we assume that only the Testing environment has a different password. Staging and Production are the same. So set the default password to the once you use on Staging and Production:
# GitLab CI/CD Variables
SYMF_DB_USER=user1
SYMF_DB_PASS=production_password
But we want to use the same @SYMF_DB_USER@ string regardless which environment we are deploying. And we neither want to create a separete .env.dist template file for each environment. So, let’s create another variable for only the Testing environment:
# GitLab CI/CD Variables
SYMF_DB_USER=user1
SYMF_DB_PASS=production_password
SYMF_DB_PASS_TESTING=testing_password
We want @SYMF_DB_PASS@ to be replaced on Staging and Production environment and only on Testing to be replaced by the value of the SYMF_DB_PASS_TESTING environment variable.
And this is exactly what the Parameters program does. It reads the needed environment variables and replaces the variables in the template file based on a given environment.
Based on our .env.dist file from above, at the end the .env file for Testing should look like:
# .env Testing
DB_USER=user1
DB_PASS=testing_password
For Staging and Production:
# .env Staging/Production
DB_USER=user1
DB_PASS=production_password
Ok, this was a very common use case. With this solution we can cover the most simplest project configuration. Let’s move to the next level.
Distinguish variables based on the environment and instance
In some rare cases a project setup should not only be distinguished by the environment but also by an instance (or entity).
For example, you have two Shop instances. ShopA and ShopB. Both are based on the exact same source code. And both should be available on all environments (Testing, Staging, Production, etc). In the simplest case these both shops only differ in their database credentials. Regardless how many variables differ, the approach is always the same.
For the purpose of this blog post we assume that both shops are stored in different databases for each environment, Production and Testing.
First, let’s set our template file .env.dist:
# .env.dist
DB_USER=@SYMF_DB_USER@
DB_PASS=@SYMF_DB_PASS@
DB_NAME=@SYMF_DB_NAME@
Now set our GitLab CI/CD variables:
# GitLab CI/CD Variables
SYMF_DB_USER=user1
SYMF_DB_PASS=password1
SYMF_DB_NAME=default_name
SYMF_DB_NAME_SHOPA=my_shop_a_production
SYMF_DB_NAME_SHOPB=my_shop_b_production
SYMF_DB_NAME_TESTING_SHOPA=my_shop_a_testing
SYMF_DB_NAME_TESTING_SHOPB=my_shop_b_testing
In this example ShopA has my_shop_a_testing in the testing environment. Since there is no extra variable set for the Production environment, ShopA would use the my_shop_a_production database in Production.
The .env output for each shop and environment will be:
# .env ShopA Production
DB_USER=user1
DB_PASS=password1
DB_NAME=my_shop_a_production
# .env ShopB Production
DB_USER=user1
DB_PASS=password1
DB_NAME=my_shop_b_production
# .env ShopA Testing
DB_USER=user1
DB_PASS=password1
DB_NAME=my_shop_a_testing
# .env ShopB Testing
DB_USER=user1
DB_PASS=password1
DB_NAME=my_shop_b_testing
Assuming we would also have a ShopC, using the same variables and the same template file from above:
# .env ShopC Production/Testing
DB_USER=user1
DB_PASS=password1
DB_NAME=default_name
Fallback example
As you can see in the previous example, ShopC got the default name for both environments. This is because we didn’t set a specific variable for ShopC.
Here is another example for fallback variables. (Or sometimes also called default variables.)
# GitLab CI/CD Variables
SYMF_DB_NAME=name1
SYMF_DB_NAME_TESTING=name2
SYMF_DB_NAME_TESTING_SHOPA=name3
In this example the fallback for ShopB in Testing would be SYMF_DB_NAME_TESTING. Like for every other shop instance except ShopA. If SYMF_DB_NAME_TESTING is also not set,
# GitLab CI/CD Variables
SYMF_DB_NAME=name1
SYMF_DB_NAME_TESTING_SHOPA=my_shop_a_testing
the fallback would be SYMF_DB_NAME for ShopB for every environment.
These are just a few examples. There are more combinations. It’s also a good idea to always set a default value. But even better is to set exact those variables which you need.
Conclusion
The goal is to have as less as possible files and everything set as GitLab CI/CD variables. This solution avoids sensitive data to be commited to Git. Sensitive data should never be commited to Git. Only as environment variables.
Of course this program is not limited to be used for Symfony. You can use it for whatever configuration files you need to replace variables.
How to install Parameters
See the Parameters GitHub README file for more examples and how to build it from source, install it via apt-get under Debian, or via Homebrew under macOS.
