I have been using Symfony 4 now for awhile. It’s a really cool webframework. I like it.
Since version 3 (or so), Symfony is using the Dotenv Component to manage parameters for database connection, mail server access, etc. The kind of parameter which you do not want in your Git history.
We put our environment variables into GitLab’s CI variables.
(This is how the variables look like for my blog. Top secret.)
On deployment these variables, from GitLab CI, will be replaced in a
.env-template file. Let’s call this template file
.env can look like this:
# .env DB_USER=root DB_PASS=pass
Here is an example how the
.env.dist template file could look like:
# .env.dist DB_USER=@SYMF_DB_USER@ DB_PASS=@SYMF_DB_PASS@
The variable names within
@ are directly coming from the GitLab CI/CD variables.
Sometimes you do not just have one single value for the same variable. In most the projects we have multiple environments. For example, Testing, Staging and Production. By using GitLab you cannot distinguish variables based on environments in your CI/CD variables. Of course you can put it into your
.gitlab-ci.yml. But here lies the problem: you do not want sensitive data in
.gitlab-ci.yml. This would mean that it’s in Git, forever.
So we need another way to use the same variable for multiple environments. For this purpose I created the Parameters C++ project. To get around this problem we just create multiple variables for one and the same template variable in GitLab CI/CD. Then using the Parameters program to replace the variable based on the environment (Testing, Staging, Production, etc).
@SYMF_DB_PASS@ from the example above. We set one default password in GitLab CI/CD variables. For the purpose of this blog post I’ll use bash syntax to describe which GitLab CI/CD variables are set:
# GitLab CI/CD Variables SYMF_DB_PASS=hello_world
For this example we assume that only the Testing environment has a different password. Staging and Production are the same. So set the default password to the once you use on Staging and Production:
# GitLab CI/CD Variables SYMF_DB_USER=user1 SYMF_DB_PASS=production_password
But we want to use the same
@SYMF_DB_USER@ string regardless which environment we are deploying. And we neither want to create a separete
.env.dist template file for each environment. So, let’s create another variable for only the Testing environment:
# GitLab CI/CD Variables SYMF_DB_USER=user1 SYMF_DB_PASS=production_password SYMF_DB_PASS_TESTING=testing_password
@SYMF_DB_PASS@ to be replaced on Staging and Production environment and only on Testing to be replaced by the value of the
SYMF_DB_PASS_TESTING environment variable.
And this is exactly what the Parameters program does. It reads the needed environment variables and replaces the variables in the template file based on a given environment.
Based on our
.env.dist file from above, at the end the
.env file for Testing should look like:
# .env Testing DB_USER=user1 DB_PASS=testing_password
For Staging and Production:
# .env Staging/Production DB_USER=user1 DB_PASS=production_password
Ok, this was a very common use case. With this solution we can cover the most simplest project configuration. Let’s move to the next level.
Distinguish variables based on the environment and instance
In some rare cases a project setup should not only be distinguished by the environment but also by an instance (or entity).
For example, you have two Shop instances. ShopA and ShopB. Both are based on the exact same source code. And both should be available on all environments (Testing, Staging, Production, etc). In the simplest case these both shops only differ in their database credentials. Regardless how many variables differ, the approach is always the same.
For the purpose of this blog post we assume that both shops are stored in different databases for each environment, Production and Testing.
First, let’s set our template file
# .env.dist DB_USER=@SYMF_DB_USER@ DB_PASS=@SYMF_DB_PASS@ DB_NAME=@SYMF_DB_NAME@
Now set our GitLab CI/CD variables:
# GitLab CI/CD Variables SYMF_DB_USER=user1 SYMF_DB_PASS=password1 SYMF_DB_NAME=default_name SYMF_DB_NAME_SHOPA=my_shop_a_production SYMF_DB_NAME_SHOPB=my_shop_b_production SYMF_DB_NAME_TESTING_SHOPA=my_shop_a_testing SYMF_DB_NAME_TESTING_SHOPB=my_shop_b_testing
In this example ShopA has
my_shop_a_testing in the testing environment. Since there is no extra variable set for the Production environment, ShopA would use the
my_shop_a_production database in Production.
.env output for each shop and environment will be:
# .env ShopA Production DB_USER=user1 DB_PASS=password1 DB_NAME=my_shop_a_production
# .env ShopB Production DB_USER=user1 DB_PASS=password1 DB_NAME=my_shop_b_production
# .env ShopA Testing DB_USER=user1 DB_PASS=password1 DB_NAME=my_shop_a_testing
# .env ShopB Testing DB_USER=user1 DB_PASS=password1 DB_NAME=my_shop_b_testing
Assuming we would also have a ShopC, using the same variables and the same template file from above:
# .env ShopC Production/Testing DB_USER=user1 DB_PASS=password1 DB_NAME=default_name
As you can see in the previous example, ShopC got the default name for both environments. This is because we didn’t set a specific variable for ShopC.
Here is another example for fallback variables. (Or sometimes also called default variables.)
# GitLab CI/CD Variables SYMF_DB_NAME=name1 SYMF_DB_NAME_TESTING=name2 SYMF_DB_NAME_TESTING_SHOPA=name3
In this example the fallback for ShopB in Testing would be
SYMF_DB_NAME_TESTING. Like for every other shop instance except ShopA. If
SYMF_DB_NAME_TESTING is also not set,
# GitLab CI/CD Variables SYMF_DB_NAME=name1 SYMF_DB_NAME_TESTING_SHOPA=my_shop_a_testing
the fallback would be
SYMF_DB_NAME for ShopB for every environment.
These are just a few examples. There are more combinations. It’s also a good idea to always set a default value. But even better is to set exact those variables which you need.
The goal is to have as less as possible files and everything set as GitLab CI/CD variables. This solution avoids sensitive data to be commited to Git. Sensitive data should never be commited to Git. Only as environment variables.
Of course this program is not limited to be used for Symfony. You can use it for whatever configuration files you need to replace variables.
How to install Parameters
See the Parameters GitHub README file for more examples and how to build it from source, install it via
apt-get under Debian, or via Homebrew under macOS.