filters: |env lets you use environment variables in any template

CHANGELOG.md

@@ -1,6 +1,9 @@
+## 0.3.7 (2019-04-23)
+* The new `{{ VAR_NAME |env }}` filter lets you use environment variables in every template.
+
 ## 0.3.6 (2019-03-21)
 * Fixed support for Python 2.6
-* Dropped Python 2.6 from unit-tests
+* Dropped Python 2.6 from unit-tests~~~~
 * Fixed a warning issued by PyYAML. 
   See [issue #33](https://github.com/kolypto/j2cli/issues/33)
 

Makefile

@@ -9,20 +9,18 @@ clean:
 	@pip install -e .  # have to reinstall because we are using self
 README.md: $(shell find j2cli/) $(wildcard misc/_doc/**)
 	@python misc/_doc/README.py | python j2cli/__init__.py -f json -o $@ misc/_doc/README.md.j2
-README.rst: README.md
-	@pandoc -f markdown -t rst -o README.rst README.md
 
 
 .PHONY: build publish-test publish
-build: README.rst
+build: README.md
 	@./setup.py build sdist bdist_wheel
-publish-test: README.rst
+publish-test: README.md
 	@twine upload --repository pypitest dist/*
-publish: README.rst
+publish: README.md
 	@twine upload dist/*
 
 
-.PHONY: test test-tox test-docker test-docker-2.6
+.PHONY: test test-tox
 test:
 	@nosetests
 test-tox:

README.md

@@ -251,3 +251,21 @@ And then uses `format` to format it, where the default format is '{addr}:{port}'
 
 More info here: [Docker Links](https://docs.docker.com/userguide/dockerlinks/)
 
+### `env(varname, default=None)`
+Use an environment variable's value inside your template.
+
+This filter is available even when your data source is something other that the environment.
+
+Example:
+
+```jinja2
+User: {{ user_login }}
+Pass: {{ USER_PASSWORD|env }}
+```
+
+You can provide the default value:
+
+```jinja2
+Pass: {{ USER_PASSWORD|env("-none-") }}
+```
+

README.rst

@@ -1,302 +0,0 @@
-`Build Status <https://travis-ci.org/kolypto/j2cli>`__
-`Pythons <.travis.yml>`__
-
-j2cli - Jinja2 Command-Line Tool
-================================
-
-``j2cli`` is a command-line tool for templating in shell-scripts,
-leveraging the `Jinja2 <http://jinja.pocoo.org/docs/>`__ library.
-
-Features:
-
--  Jinja2 templating
--  INI, YAML, JSON data sources supported
--  Allows the use of environment variables in templates! Hello
-   `Docker <http://www.docker.com/>`__ :)
-
-Inspired by
-`mattrobenolt/jinja2-cli <https://github.com/mattrobenolt/jinja2-cli>`__
-
-Installation
-------------
-
-::
-
-   pip install j2cli
-
-To enable the YAML support with `pyyaml <http://pyyaml.org/>`__:
-
-::
-
-   pip install j2cli[yaml]
-
-Tutorial
---------
-
-Suppose, you want to have an nginx configuration file template,
-``nginx.j2``:
-
-.. code:: jinja2
-
-   server {
-     listen 80;
-     server_name {{ nginx.hostname }};
-
-     root {{ nginx.webroot }};
-     index index.htm;
-   }
-
-And you have a JSON file with the data, ``nginx.json``:
-
-.. code:: json
-
-   {
-       "nginx":{
-           "hostname": "localhost",
-           "webroot": "/var/www/project"
-       }
-   }
-
-This is how you render it into a working configuration file:
-
-.. code:: bash
-
-   $ j2 -f json nginx.j2 nginx.json > nginx.conf
-
-The output is saved to ``nginx.conf``:
-
-::
-
-   server {
-     listen 80;
-     server_name localhost;
-
-     root /var/www/project;
-     index index.htm;
-   }
-
-Alternatively, you can use the ``-o nginx.conf`` option.
-
-Tutorial with environment variables
------------------------------------
-
-Suppose, you have a very simple template, ``person.xml``:
-
-.. code:: jinja2
-
-   <data><name>{{ name }}</name><age>{{ age }}</age></data>
-
-What is the easiest way to use j2 here? Use environment variables in
-your bash script:
-
-.. code:: bash
-
-   $ export name=Andrew
-   $ export age=31
-   $ j2 /tmp/person.xml
-   <data><name>Andrew</name><age>31</age></data>
-
-Usage
------
-
-Compile a template using INI-file data source:
-
-::
-
-   $ j2 config.j2 data.ini
-
-Compile using JSON data source:
-
-::
-
-   $ j2 config.j2 data.json
-
-Compile using YAML data source (requires PyYAML):
-
-::
-
-   $ j2 config.j2 data.yaml
-
-Compile using JSON data on stdin:
-
-::
-
-   $ curl http://example.com/service.json | j2 --format=json config.j2
-
-Compile using environment variables (hello Docker!):
-
-::
-
-   $ j2 config.j2
-
-Or even read environment variables from a file:
-
-::
-
-   $ j2 --format=env config.j2 data.env
-
-Reference
-=========
-
-``j2`` accepts the following arguments:
-
--  ``template``: Jinja2 template file to render
--  ``data``: (optional) path to the data used for rendering. The default
-   is ``-``: use stdin
-
-Options:
-
--  ``--format, -f``: format for the data file. The default is ``?``:
-   guess from file extension.
--  ``--import-env VAR, -e EVAR``: import all environment variables into
-   the template as ``VAR``. To import environment variables into the
-   global scope, give it an empty string: ``--import-env=``. (This will
-   overwrite any existing variables!)
--  ``-o outfile``: Write rendered template to a file
--  ``--undefined``: Allow undefined variables to be used in templates
-   (no error will be raised)
-
--  ``--filters filters.py``: Load custom Jinja2 filters and tests from a
-   Python file. Will load all top-level functions and register them as
-   filters. This option can be used multiple times to import several
-   files.
--  ``--tests tests.py``: Load custom Jinja2 filters and tests from a
-   Python file.
-
-There is some special behavior with environment variables:
-
--  When ``data`` is not provided (data is ``-``), ``--format`` defaults
-   to ``env`` and thus reads environment variables
--  When ``--format=env``, it can read a special “environment variables”
-   file made like this: ``env > /tmp/file.env``
-
-Formats
--------
-
-env
-~~~
-
-Data input from environment variables.
-
-Render directly from the current environment variable values:
-
-::
-
-   $ j2 config.j2
-
-Or alternatively, read the values from a file:
-
-::
-
-   NGINX_HOSTNAME=localhost
-   NGINX_WEBROOT=/var/www/project
-   NGINX_LOGS=/var/log/nginx/
-
-And render with:
-
-::
-
-   $ j2 config.j2 data.env
-   $ env | j2 --format=env config.j2.
-
-This is especially useful with Docker to link containers together.
-
-ini
-~~~
-
-INI data input format.
-
-data.ini:
-
-::
-
-   [nginx]
-   hostname=localhost
-   webroot=/var/www/project
-   logs=/var/log/nginx/
-
-Usage:
-
-::
-
-   $ j2 config.j2 data.ini
-   $ cat data.ini | j2 --format=ini config.j2
-
-json
-~~~~
-
-JSON data input format
-
-data.json:
-
-::
-
-   {
-       "nginx":{
-           "hostname": "localhost",
-           "webroot": "/var/www/project",
-           "logs": "/var/log/nginx/"
-       }
-   }
-
-Usage:
-
-::
-
-   $ j2 config.j2 data.json
-   $ cat data.json | j2 --format=ini config.j2
-
-yaml
-~~~~
-
-YAML data input format.
-
-data.yaml:
-
-::
-
-   nginx:
-     hostname: localhost
-     webroot: /var/www/project
-     logs: /var/log/nginx
-
-Usage:
-
-::
-
-   $ j2 config.j2 data.yml
-   $ cat data.yml | j2 --format=yaml config.j2
-
-Extras
-======
-
-Filters
--------
-
-``docker_link(value, format='{addr}:{port}')``
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Given a Docker Link environment variable value, format it into something
-else.
-
-This first parses a Docker Link value like this:
-
-::
-
-   DB_PORT=tcp://172.17.0.5:5432
-
-Into a dict:
-
-.. code:: python
-
-   {
-     'proto': 'tcp',
-     'addr': '172.17.0.5',
-     'port': '5432'
-   }
-
-And then uses ``format`` to format it, where the default format is
-‘{addr}:{port}’.
-
-More info here: `Docker
-Links <https://docs.docker.com/userguide/dockerlinks/>`__

j2cli/cli.py

@@ -145,7 +145,10 @@ def render_command(cwd, environ, stdin, argv):
     renderer = Jinja2TemplateRenderer(cwd, args.undefined)
 
     # Filters, Tests
-    renderer.register_filters({'docker_link': filters.docker_link})
+    renderer.register_filters({
+        'docker_link': filters.docker_link,
+        'env': filters.env,
+    })
     for fname in args.filters:
         renderer.import_filters(fname)
     for fname in args.tests:

j2cli/extras/filters.py

@@ -1,4 +1,5 @@
 """ Custom Jinja2 filters """
+import os
 
 from jinja2 import is_undefined
 import re
@@ -41,3 +42,29 @@ def docker_link(value, format='{addr}:{port}'):
 
     # Format
     return format.format(**d)
+
+
+def env(varname, default=None):
+    """ Use an environment variable's value inside your template.
+
+        This filter is available even when your data source is something other that the environment.
+
+        Example:
+
+        ```jinja2
+        User: {{ user_login }}
+        Pass: {{ USER_PASSWORD|env }}
+        ```
+
+        You can provide the default value:
+
+        ```jinja2
+        Pass: {{ USER_PASSWORD|env("-none-") }}
+        ```
+    """
+    if default is not None:
+        # With the default, there's never an error
+        return os.getenv(varname, default)
+    else:
+        # Raise KeyError when not provided
+        return os.environ[varname]

misc/_doc/README.md.j2

@@ -156,7 +156,7 @@ Extras
 
 ## Filters
 
-{% for filter in extras.filters|sort() %}
+{% for name, filter in extras.filters|dictsort() %}
 ### `{{ filter.qsignature }}`
 {{ filter.doc }}
 {% endfor %}

misc/_doc/README.py

@@ -15,10 +15,12 @@ README = {
         for name, f in j2cli.context.FORMATS.items()
     },
     'extras': {
-        'filters': [doc(v)
+        'filters': {k: doc(v)
                     for k, v in getmembers(j2cli.extras.filters)
-                    if inspect.isfunction(v) and inspect.getmodule(v) is j2cli.extras.filters]
+                    if inspect.isfunction(v) and inspect.getmodule(v) is j2cli.extras.filters}
     }
 }
 
+assert 'yaml' in README['formats'], 'Looks like the YAML library is not installed!'
+
 print(json.dumps(README))

setup.py

@@ -26,7 +26,7 @@ if sys.version_info[:2] == (2, 6) or True:
 
 setup(
     name='j2cli',
-    version='0.3.6-1',
+    version='0.3.7',
     author='Mark Vartanyan',
     author_email='kolypto@gmail.com',
 

tests/render-test.py

@@ -4,6 +4,7 @@ from __future__ import unicode_literals
 
 import unittest
 import os, sys, io, os.path, tempfile
+from copy import copy
 from contextlib import contextmanager
 from jinja2.exceptions import UndefinedError
 
@@ -23,6 +24,15 @@ def mktemp(contents):
         os.unlink(path)
 
 
+@contextmanager
+def mock_environ(new_env):
+    old_env = copy(os.environ)
+    os.environ.update(new_env)
+    yield
+    os.environ.clear()
+    os.environ.update(old_env)
+
+
 class RenderTest(unittest.TestCase):
     def setUp(self):
         os.chdir(
@@ -31,7 +41,8 @@ class RenderTest(unittest.TestCase):
 
     def _testme(self, argv, expected_output, stdin=None, env=None):
         """ Helper test shortcut """
-        result = render_command(os.getcwd(), env or {}, stdin, argv)
+        with mock_environ(env or {}):
+            result = render_command(os.getcwd(), env or {}, stdin, argv)
         if isinstance(result, bytes):
             result = result.decode('utf-8')
         self.assertEqual(result, expected_output)
@@ -124,6 +135,23 @@ class RenderTest(unittest.TestCase):
             # Python 3: environment variables are unicode strings
             self._testme(['resources/name.j2'], u'Hello Jürgen!\n', env=dict(name=u'Jürgen'))
 
+    def test_filters__env(self):
+        with mktemp('user_login: kolypto') as yml_file:
+            with mktemp('{{ user_login }}:{{ "USER_PASS"|env }}') as template:
+                # Test: template with an env variable
+                self._testme(['--format=yaml', template, yml_file], 'kolypto:qwerty123', env=dict(USER_PASS='qwerty123'))
+
+                # environment cleaned up
+                assert 'USER_PASS' not in os.environ
+
+                # Test: KeyError
+                with self.assertRaises(KeyError):
+                    self._testme(['--format=yaml', template, yml_file], 'kolypto:qwerty123', env=dict())
+
+            # Test: default
+            with mktemp('{{ user_login }}:{{ "USER_PASS"|env("-none-") }}') as template:
+                self._testme(['--format=yaml', template, yml_file], 'kolypto:-none-', env=dict())
+
 
     def test_custom_filters(self):
         with mktemp('{{ a|parentheses }}') as template: