Merge pull request #279 from myii/ci/standardise-structure

chore: use `semantic-release` cross-formula standard structure

Author: myii

docs/TOFS_pattern.rst

@@ -39,7 +39,7 @@ Using SaltStack is a simple and effective way to implement configuration managem
 
 To avoid this situation we can use the `pillar mechanism <http://docs.saltstack.com/en/latest/topics/pillar/>`_, which is designed to provide controlled access to data from the minions based on some selection rules. As pillar data could be easily integrated in the `Jinja <http://docs.saltstack.com/en/latest/topics/tutorials/pillar.html>`_ templates, it is a good mechanism to store values to be used in the final rendering of state files and templates.
 
-There are a variety of approaches on the usage of pillar and templates as seen in the `saltstack-formulas <https://github.com/saltstack-formulas>`_' repositories. `Some <https://github.com/saltstack-formulas/nginx-formula/pull/18>`_ `developments <https://github.com/saltstack-formulas/php-formula/pull/14>`_ stress the initial purpose of pillar data into a storage for most of the possible variables for a determined system configuration. This, in my opinion, is shifting too much load from the original template files approach. Adding up some `non-trivial Jinja <https://github.com/spsoit/nginx-formula/blob/81de880fe0276dd9488ffa15bc78944c0fc2b919/nginx/ng/files/nginx.conf>`_ code as essential part of composing the state file definitely makes SaltStack state files (hence formulas) more difficult to read. The extreme of this approach is that we could end up with a new render mechanism, implemented in Jinja, storing everything needed in pillar data to compose configurations. Additionally, we are establishing a strong dependency with the Jinja renderer.
+There are a variety of approaches on the usage of pillar and templates as seen in the `saltstack-formulas <https://github.com/saltstack-formulas>`_' repositories. `Some <https://github.com/saltstack-formulas/nginx-formula/pull/18>`_ `developments <https://github.com/saltstack-formulas/php-formula/pull/14>`_ stress the initial purpose of pillar data into a storage for most of the possible variables for a determined system configuration. This, in my opinion, is shifting too much load from the original template files approach. Adding up some `non-trivial Jinja <https://github.com/saltstack-formulas/nginx-formula/blob/f74254c07e188bd448eaf1c5f9c802d78c4c005e/nginx/files/default/nginx.conf>`_ code as essential part of composing the state file definitely makes SaltStack state files (hence formulas) more difficult to read. The extreme of this approach is that we could end up with a new render mechanism, implemented in Jinja, storing everything needed in pillar data to compose configurations. Additionally, we are establishing a strong dependency with the Jinja renderer.
 
 In opposition to the *put the code in file_roots and the data in pillars* approach, there is the *pillar as a store for a set of key-values* approach. A full-blown configuration file abstracted in pillar and jinja is complicated to develop, understand and maintain. I think a better and simpler approach is to keep a configuration file templated using just a basic (non-extensive but extensible) set of pillar values.
 
@@ -325,6 +325,7 @@ We can simplify the ``conf.sls`` with the new ``files_switch`` macro to use in t
 
 
 * This uses ``config.get``, searching for ``ntp:tofs:source_files:Configure NTP`` to determine the list of template files to use.
+* If this returns a result, the default of ``['/etc/ntp.conf.jinja']`` will be appended to it.
 * If this does not yield any results, the default of ``['/etc/ntp.conf.jinja']`` will be used.
 
 In ``libtofs.jinja``, we define this new macro ``files_switch``.
@@ -426,18 +427,20 @@ The list of ``source_files`` can be given:
      tofs:
        source_files:
          Configure NTP:
-           - '/etc/ntp.conf.jinja'
            - '/etc/ntp.conf_alt.jinja'
 
 Resulting in:
 
 .. code-block:: sls
 
          - source:
-           - salt://ntp/files/theminion/etc/ntp.conf.jinja
            - salt://ntp/files/theminion/etc/ntp.conf_alt.jinja
-           - salt://ntp/files/Debian/etc/ntp.conf.jinja
+           - salt://ntp/files/theminion/etc/ntp.conf.jinja
            - salt://ntp/files/Debian/etc/ntp.conf_alt.jinja
-           - salt://ntp/files/default/etc/ntp.conf.jinja
+           - salt://ntp/files/Debian/etc/ntp.conf.jinja
            - salt://ntp/files/default/etc/ntp.conf_alt.jinja
+           - salt://ntp/files/default/etc/ntp.conf.jinja
+
+Note: This does *not* override the default value.
+Rather, the value from the pillar/config is prepended to the default.
 

postgres/libtofs.jinja

@@ -10,7 +10,7 @@
 
     Params:
       * source_files: ordered list of files to look for
-      * lookup: key under '<tplroot>:tofs:source_files' to override
+      * lookup: key under '<tplroot>:tofs:source_files' to prepend to the
         list of source files
       * default_files_switch: if there's no config (e.g. pillar)
         '<tplroot>:tofs:files_switch' this is the ordered list of grains to
@@ -55,14 +55,13 @@
       tplroot ~ ':tofs:files_switch',
       default_files_switch
   ) %}
-  {#- Lookup source_files (v2), files (v1), or fallback to source_files parameter #}
+  {#- Lookup source_files (v2), files (v1), or fallback to an empty list #}
   {%- set src_files = salt['config.get'](
       tplroot ~ ':tofs:source_files:' ~ lookup,
-      salt['config.get'](
-          tplroot ~ ':tofs:files:' ~ lookup,
-          source_files
-      )
+      salt['config.get'](tplroot ~ ':tofs:files:' ~ lookup, [])
   ) %}
+  {#- Append the default source_files #}
+  {%- set src_files = src_files + source_files %}
   {#- Only add to [''] when supporting older TOFS implementations #}
   {%- set path_prefix_exts = [''] %}
   {%- if v1_path_prefix != '' %}
@@ -83,18 +82,25 @@
     {%- for fs in fsl %}
       {%- for src_file in src_files %}
         {%- if fs %}
-          {%- set fs_dir = salt['config.get'](fs, fs) %}
+          {%- set fs_dirs = salt['config.get'](fs, fs) %}
         {%- else %}
-          {%- set fs_dir = salt['config.get'](tplroot ~ ':tofs:dirs:default', 'default') %}
+          {%- set fs_dirs = salt['config.get'](tplroot ~ ':tofs:dirs:default', 'default') %}
         {%- endif %}
-        {%- set url = [
-            '- salt:/',
-            path_prefix_inc_ext.strip('/'),
-            files_dir.strip('/'),
-            fs_dir.strip('/'),
-            src_file.strip('/'),
-        ] | select | join('/') %}
+        {#- Force the `config.get` lookup result as a list where necessary #}
+        {#- since we need to also handle grains that are lists #}
+        {%- if fs_dirs is string %}
+          {%- set fs_dirs = [fs_dirs] %}
+        {%- endif %}
+        {%- for fs_dir in fs_dirs %}
+          {%- set url = [
+              '- salt:/',
+              path_prefix_inc_ext.strip('/'),
+              files_dir.strip('/'),
+              fs_dir.strip('/'),
+              src_file.strip('/'),
+          ] | select | join('/') %}
 {{ url | indent(indent_width, true) }}
+        {%- endfor %}
       {%- endfor %}
     {%- endfor %}
   {%- endfor %}