.. title: Ansible testing with Molecule
.. date: 2019-01-11
.. slug: ansible-testing-with-molecule
.. updated: 2019-06-21
.. status: published
.. tags: configuration management, ansible, molecule,
.. category: configuration management
.. authors: Elia el Lazkani
.. description: A fast way to create a testable ansible role using molecule.
.. type: text


When I first started using `ansible <https://www.ansible.com/>`_, I did not know about `molecule <https://molecule.readthedocs.io/en/latest/>`_. It was a bit daunting to start a *role* from scratch and trying to develop it without having the ability to test it. Then a co-worker of mine told me about molecule and everything changed.

.. TEASER_END

I do not have any of the tools I need installed on this machine, so I will go through, step by step, how I set up ansible and molecule on any new machine I come across for writing ansible roles.

Requirements
============

What we are trying to achieve in this post, is a working ansible role that can be tested inside a docker container. To be able to achieve that, we need to install docker on the system. Follow the instructions on `installing docker <https://docs.docker.com/install/>`_ found on the docker website.

Good Practices
==============

First thing's first. Let's start by making sure that we have python installed properly on the system.

.. code:: text

   $ python --version
   Python 3.7.1

Because in this case I have *python3* installed, I can create a *virtualenv* easier without the use of external tools.

.. code:: text

   # Create the directory to work with
   $ mkdir -p sandbox/test-roles
   # Navigate to the directory
   $ cd sandbox/test-roles/
   # Create the virtualenv
   ~/sandbox/test-roles $ python -m venv .ansible-venv
   # Activate the virtualenv
   ~/sandbox/test-roles $ source .ansible-venv/bin/activate
   # Check that your virtualenv activated properly
   (.ansible-venv) ~/sandbox/test-roles $ which python
   /home/elijah/sandbox/test-roles/.ansible-venv/bin/python

At this point, we can install the required dependencies.

.. code:: text

   $ pip install ansible molecule docker
   Collecting ansible
     Downloading https://files.pythonhosted.org/packages/56/fb/b661ae256c5e4a5c42859860f59f9a1a0b82fbc481306b30e3c5159d519d/ansible-2.7.5.tar.gz (11.8MB)
       100% |████████████████████████████████| 11.8MB 3.8MB/s
   Collecting molecule
     Downloading https://files.pythonhosted.org/packages/84/97/e5764079cb7942d0fa68b832cb9948274abb42b72d9b7fe4a214e7943786/molecule-2.19.0-py3-none-any.whl (180kB)
       100% |████████████████████████████████| 184kB 2.2MB/s

   ...

   Successfully built ansible ansible-lint anyconfig cerberus psutil click-completion tabulate tree-format pathspec future pycparser arrow
   Installing collected packages: MarkupSafe, jinja2, PyYAML, six, pycparser, cffi, pynacl, idna, asn1crypto, cryptography, bcrypt, paramiko, ansible, pbr, git-url-parse, monotonic, fasteners, click, colorama, sh, python-gilt, ansible-lint, pathspec, yamllint, anyconfig, cerberus, psutil, more-itertools, py, attrs, pluggy, atomicwrites, pytest, testinfra, ptyprocess, pexpect, click-completion, tabulate, future, chardet, binaryornot, poyo, urllib3, certifi, requests, python-dateutil, arrow, jinja2-time, whichcraft, cookiecutter, tree-format, molecule, docker-pycreds, websocket-client, docker
   Successfully installed MarkupSafe-1.1.0 PyYAML-3.13 ansible-2.7.5 ansible-lint-3.4.23 anyconfig-0.9.7 arrow-0.13.0 asn1crypto-0.24.0 atomicwrites-1.2.1 attrs-18.2.0 bcrypt-3.1.5 binaryornot-0.4.4 cerberus-1.2 certifi-2018.11.29 cffi-1.11.5 chardet-3.0.4 click-6.7 click-completion-0.3.1 colorama-0.3.9 cookiecutter-1.6.0 cryptography-2.4.2 docker-3.7.0 docker-pycreds-0.4.0 fasteners-0.14.1 future-0.17.1 git-url-parse-1.1.0 idna-2.8 jinja2-2.10 jinja2-time-0.2.0 molecule-2.19.0 monotonic-1.5 more-itertools-5.0.0 paramiko-2.4.2 pathspec-0.5.9 pbr-4.1.0 pexpect-4.6.0 pluggy-0.8.1 poyo-0.4.2 psutil-5.4.6 ptyprocess-0.6.0 py-1.7.0 pycparser-2.19 pynacl-1.3.0 pytest-4.1.0 python-dateutil-2.7.5 python-gilt-1.2.1 requests-2.21.0 sh-1.12.14 six-1.11.0 tabulate-0.8.2 testinfra-1.16.0 tree-format-0.1.2 urllib3-1.24.1 websocket-client-0.54.0 whichcraft-0.5.2 yamllint-1.11.1

Creating your first ansible role
================================

Once all the steps above are complete, we can start by creating our first ansible role.

.. code:: text

   $ molecule init role -r example-role
   --> Initializing new role example-role...
   Initialized role in /home/elijah/sandbox/test-roles/example-role successfully.

   $ tree example-role/
   example-role/
   ├── defaults
   │   └── main.yml
   ├── handlers
   │   └── main.yml
   ├── meta
   │   └── main.yml
   ├── molecule
   │   └── default
   │       ├── Dockerfile.j2
   │       ├── INSTALL.rst
   │       ├── molecule.yml
   │       ├── playbook.yml
   │       └── tests
   │           ├── __pycache__
   │           │   └── test_default.cpython-37.pyc
   │           └── test_default.py
   ├── README.md
   ├── tasks
   │   └── main.yml
   └── vars
       └── main.yml

   9 directories, 12 files

You can find what each directory is for and how ansible works by visiting docs.ansible.com.

``meta/main.yml``
-----------------

The meta file needs to modified and filled with information about the role. This is not a required file to modify if you are keeping this for yourself, for example. But it is a good idea to have as much information as possible if this is going to be released. In my case, I don't need any fanciness as this is just sample code.

.. code:: yaml

   ---
   galaxy_info:
     author: Elia el Lazkani
     description: This is an example ansible role to showcase molecule at work
     license: license (BDS-2)
     min_ansible_version: 2.7
     galaxy_tags: []
   dependencies: []

``tasks/main.yml``
------------------

This is where the magic is set in motion. Tasks are the smallest entities in a role that do small and idempotent actions. Let's write a few simple tasks to create a user and install a service.

.. code:: yaml

   ---
   # Create the user example
   - name: Create 'example' user
     user:
       name: example
       comment: Example user
       shell: /bin/bash
       state: present
       create_home: yes
       home: /home/example

   # Install nginx
   - name: Install nginx
     apt:
       name: nginx
       state: present
       update_cache: yes
     notify: Restart nginx

``handlers/main.yml``
---------------------

If you noticed, we are notifying a handler to be called after installing *nginx*. All handlers notified will run after all the tasks complete and each handler will only run once. This is a good way to make sure that you don't restart *nginx* multiple times if you call the handler more than once.

.. code:: yaml

   ---
   # Handler to restart nginx
   - name: Restart nginx
     service:
       name: nginx
       state: restarted

``molecule/default/molecule.yml``
---------------------------------

It's time to configure molecule to do what we need. We need to start an ubuntu docker container, so we need to specify that in the molecule YAML file. All we need to do is change the image line to specify that we want an ``ubuntu:bionic`` image.

.. code:: yaml

   ---
   dependency:
     name: galaxy
   driver:
     name: docker
   lint:
     name: yamllint
   platforms:
     - name: instance
       image: ubuntu:bionic
   provisioner:
     name: ansible
     lint:
       name: ansible-lint
   scenario:
     name: default
   verifier:
     name: testinfra
     lint:
       name: flake8

``molecule/default/playbook.yml``
---------------------------------

This is the playbook that molecule will run. Make sure that you have all the steps that you need here. I will keep this as is.

.. code:: yaml

   ---
   - name: Converge
     hosts: all
     roles:
       - role: example-role

First Role Pass
===============

This is time to test our role and see what's going on.

.. code:: text

   (.ansible-role) ~/sandbox/test-roles/example-role/ $ molecule converge
   --> Validating schema /home/elijah/sandbox/test-roles/example-role/molecule/default/molecule.yml.
   Validation completed successfully.
   --> Test matrix

   └── default
       ├── dependency
       ├── create
       ├── prepare
       └── converge

   --> Scenario: 'default'
   --> Action: 'dependency'
   Skipping, missing the requirements file.
   --> Scenario: 'default'
   --> Action: 'create'

       PLAY [Create] ******************************************************************

       TASK [Log into a Docker registry] **********************************************
       skipping: [localhost] => (item=None)

       TASK [Create Dockerfiles from image names] *************************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Discover local Docker images] ********************************************
       ok: [localhost] => (item=None)
       ok: [localhost]

       TASK [Build an Ansible compatible image] ***************************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Create docker network(s)] ************************************************

       TASK [Create molecule instance(s)] *********************************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Wait for instance(s) creation to complete] *******************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       PLAY RECAP *********************************************************************
       localhost                  : ok=5    changed=4    unreachable=0    failed=0


   --> Scenario: 'default'
   --> Action: 'prepare'
   Skipping, prepare playbook not configured.
   --> Scenario: 'default'
   --> Action: 'converge'

       PLAY [Converge] ****************************************************************

       TASK [Gathering Facts] *********************************************************
       ok: [instance]

       TASK [example-role : Create 'example' user] ************************************
       changed: [instance]

       TASK [example-role : Install nginx] ********************************************
       changed: [instance]

       RUNNING HANDLER [example-role : Restart nginx] *********************************
       changed: [instance]

       PLAY RECAP *********************************************************************
       instance                   : ok=4    changed=3    unreachable=0    failed=0

It looks like the **converge** step succeeded.

Writing Tests
=============

It is always a good practice to write unittests when you're writing code. Ansible roles should not be an exception. Molecule offers a way to run tests, which you can think of as unittest, to make sure that what the role gives you is what you were expecting. This helps future development of the role and keeps you from falling in previously solved traps.

``molecule/default/tests/test_default.py``
------------------------------------------

Molecule leverages the `testinfra <https://testinfra.readthedocs.io/en/latest/>`_ project to run its tests. You can use other tools if you so wish, and there are many. In this example we will be using *testinfra*.

.. code:: python

   import os

   import testinfra.utils.ansible_runner

   testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
       os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')


   def test_hosts_file(host):
       f = host.file('/etc/hosts')

       assert f.exists
       assert f.user == 'root'
       assert f.group == 'root'


   def test_user_created(host):
       user = host.user("example")
       assert user.name == "example"
       assert user.home == "/home/example"


   def test_user_home_exists(host):
       user_home = host.file("/home/example")
       assert user_home.exists
       assert user_home.is_directory


   def test_nginx_is_installed(host):
       nginx = host.package("nginx")
       assert nginx.is_installed


   def test_nginx_running_and_enabled(host):
       nginx = host.service("nginx")
       assert nginx.is_running



.. warning::

   Uncomment ``truthy: disable`` in ``.yamllint`` found at the base of the role.

.. code:: text

   (.ansible_venv) ~/sandbox/test-roles/example-role $ molecule test
   --> Validating schema /home/elijah/sandbox/test-roles/example-role/molecule/default/molecule.yml.
   Validation completed successfully.
   --> Test matrix

   └── default
       ├── lint
       ├── destroy
       ├── dependency
       ├── syntax
       ├── create
       ├── prepare
       ├── converge
       ├── idempotence
       ├── side_effect
       ├── verify
       └── destroy

   --> Scenario: 'default'
   --> Action: 'lint'
   --> Executing Yamllint on files found in /home/elijah/sandbox/test-roles/example-role/...
   Lint completed successfully.
   --> Executing Flake8 on files found in /home/elijah/sandbox/test-roles/example-role/molecule/default/tests/...
   /home/elijah/.virtualenvs/world/lib/python3.7/site-packages/pycodestyle.py:113: FutureWarning: Possible nested set at position 1
     EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
   Lint completed successfully.
   --> Executing Ansible Lint on /home/elijah/sandbox/test-roles/example-role/molecule/default/playbook.yml...
   Lint completed successfully.
   --> Scenario: 'default'
   --> Action: 'destroy'

       PLAY [Destroy] *****************************************************************

       TASK [Destroy molecule instance(s)] ********************************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Wait for instance(s) deletion to complete] *******************************
       ok: [localhost] => (item=None)
       ok: [localhost]

       TASK [Delete docker network(s)] ************************************************

       PLAY RECAP *********************************************************************
       localhost                  : ok=2    changed=1    unreachable=0    failed=0


   --> Scenario: 'default'
   --> Action: 'dependency'
   Skipping, missing the requirements file.
   --> Scenario: 'default'
   --> Action: 'syntax'

       playbook: /home/elijah/sandbox/test-roles/example-role/molecule/default/playbook.yml

   --> Scenario: 'default'
   --> Action: 'create'

       PLAY [Create] ******************************************************************

       TASK [Log into a Docker registry] **********************************************
       skipping: [localhost] => (item=None)

       TASK [Create Dockerfiles from image names] *************************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Discover local Docker images] ********************************************
       ok: [localhost] => (item=None)
       ok: [localhost]

       TASK [Build an Ansible compatible image] ***************************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Create docker network(s)] ************************************************

       TASK [Create molecule instance(s)] *********************************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Wait for instance(s) creation to complete] *******************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       PLAY RECAP *********************************************************************
       localhost                  : ok=5    changed=4    unreachable=0    failed=0


   --> Scenario: 'default'
   --> Action: 'prepare'
   Skipping, prepare playbook not configured.
   --> Scenario: 'default'
   --> Action: 'converge'

       PLAY [Converge] ****************************************************************

       TASK [Gathering Facts] *********************************************************
       ok: [instance]

       TASK [example-role : Create 'example' user] ************************************
       changed: [instance]

       TASK [example-role : Install nginx] ********************************************
       changed: [instance]

       RUNNING HANDLER [example-role : Restart nginx] *********************************
       changed: [instance]

       PLAY RECAP *********************************************************************
       instance                   : ok=4    changed=3    unreachable=0    failed=0


   --> Scenario: 'default'
   --> Action: 'idempotence'
   Idempotence completed successfully.
   --> Scenario: 'default'
   --> Action: 'side_effect'
   Skipping, side effect playbook not configured.
   --> Scenario: 'default'
   --> Action: 'verify'
   --> Executing Testinfra tests found in /home/elijah/sandbox/test-roles/example-role/molecule/default/tests/...
       ============================= test session starts ==============================
       platform linux -- Python 3.7.1, pytest-4.1.0, py-1.7.0, pluggy-0.8.1
       rootdir: /home/elijah/sandbox/test-roles/example-role/molecule/default, inifile:
       plugins: testinfra-1.16.0
   collected 5 items

       tests/test_default.py .....                                              [100%]

       =============================== warnings summary ===============================

   ...

       ==================== 5 passed, 7 warnings in 27.37 seconds =====================
   Verifier completed successfully.
   --> Scenario: 'default'
   --> Action: 'destroy'

       PLAY [Destroy] *****************************************************************

       TASK [Destroy molecule instance(s)] ********************************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Wait for instance(s) deletion to complete] *******************************
       changed: [localhost] => (item=None)
       changed: [localhost]

       TASK [Delete docker network(s)] ************************************************

       PLAY RECAP *********************************************************************
       localhost                  : ok=2    changed=2    unreachable=0    failed=0

I have a few warning messages (that's likely because I am using python 3.7 and some of the libraries still don't fully support the new standards released with it) but all my tests passed

Conclusion
==========

Molecule is a great tool to test ansible roles quickly and while developing them. It also comes bundled with a bunch of other features from different projects that will test all aspects of your ansible code. I suggest you start using it when writing new ansible roles.
