2019-06-23 23:29:02 +00:00
.. title: Ansible testing with Molecule
.. date: 2019-01-11
.. slug: ansible-testing-with-molecule
.. updated: 2019-06-21
.. status: published
.. tags: configuration management, ansible, molecule,
.. category: configuration management
2019-08-31 09:40:11 +00:00
.. authors: Elia El Lazkani
2019-06-23 23:29:02 +00:00
.. description: A fast way to create a testable ansible role using molecule.
.. type: text
When I first started using `ansible <https://www.ansible.com/> `_ , I did not know about `molecule <https://molecule.readthedocs.io/en/latest/> `_ . It was a bit daunting to start a *role* from scratch and trying to develop it without having the ability to test it. Then a co-worker of mine told me about molecule and everything changed.
.. TEASER_END
I do not have any of the tools I need installed on this machine, so I will go through, step by step, how I set up ansible and molecule on any new machine I come across for writing ansible roles.
Requirements
============
What we are trying to achieve in this post, is a working ansible role that can be tested inside a docker container. To be able to achieve that, we need to install docker on the system. Follow the instructions on `installing docker <https://docs.docker.com/install/> `_ found on the docker website.
Good Practices
==============
First thing's first. Let's start by making sure that we have python installed properly on the system.
.. code :: text
$ python --version
Python 3.7.1
Because in this case I have *python3* installed, I can create a *virtualenv* easier without the use of external tools.
.. code :: text
# Create the directory to work with
$ mkdir -p sandbox/test-roles
# Navigate to the directory
$ cd sandbox/test-roles/
# Create the virtualenv
2019-08-31 09:40:11 +00:00
~/sandbox/test-roles $ python -m venv .ansible-venv
2019-06-23 23:29:02 +00:00
# Activate the virtualenv
~/sandbox/test-roles $ source .ansible-venv/bin/activate
# Check that your virtualenv activated properly
(.ansible-venv) ~/sandbox/test-roles $ which python
/home/elijah/sandbox/test-roles/.ansible-venv/bin/python
At this point, we can install the required dependencies.
.. code :: text
$ pip install ansible molecule docker
Collecting ansible
Downloading https://files.pythonhosted.org/packages/56/fb/b661ae256c5e4a5c42859860f59f9a1a0b82fbc481306b30e3c5159d519d/ansible-2.7.5.tar.gz (11.8MB)
100% |████████████████████████████████| 11.8MB 3.8MB/s
Collecting molecule
Downloading https://files.pythonhosted.org/packages/84/97/e5764079cb7942d0fa68b832cb9948274abb42b72d9b7fe4a214e7943786/molecule-2.19.0-py3-none-any.whl (180kB)
100% |████████████████████████████████| 184kB 2.2MB/s
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
...
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
Successfully built ansible ansible-lint anyconfig cerberus psutil click-completion tabulate tree-format pathspec future pycparser arrow
Installing collected packages: MarkupSafe, jinja2, PyYAML, six, pycparser, cffi, pynacl, idna, asn1crypto, cryptography, bcrypt, paramiko, ansible, pbr, git-url-parse, monotonic, fasteners, click, colorama, sh, python-gilt, ansible-lint, pathspec, yamllint, anyconfig, cerberus, psutil, more-itertools, py, attrs, pluggy, atomicwrites, pytest, testinfra, ptyprocess, pexpect, click-completion, tabulate, future, chardet, binaryornot, poyo, urllib3, certifi, requests, python-dateutil, arrow, jinja2-time, whichcraft, cookiecutter, tree-format, molecule, docker-pycreds, websocket-client, docker
2019-08-31 09:40:11 +00:00
Successfully installed MarkupSafe-1.1.0 PyYAML-3.13 ansible-2.7.5 ansible-lint-3.4.23 anyconfig-0.9.7 arrow-0.13.0 asn1crypto-0.24.0 atomicwrites-1.2.1 attrs-18.2.0 bcrypt-3.1.5 binaryornot-0.4.4 cerberus-1.2 certifi-2018.11.29 cffi-1.11.5 chardet-3.0.4 click-6.7 click-completion-0.3.1 colorama-0.3.9 cookiecutter-1.6.0 cryptography-2.4.2 docker-3.7.0 docker-pycreds-0.4.0 fasteners-0.14.1 future-0.17.1 git-url-parse-1.1.0 idna-2.8 jinja2-2.10 jinja2-time-0.2.0 molecule-2.19.0 monotonic-1.5 more-itertools-5.0.0 paramiko-2.4.2 pathspec-0.5.9 pbr-4.1.0 pexpect-4.6.0 pluggy-0.8.1 poyo-0.4.2 psutil-5.4.6 ptyprocess-0.6.0 py-1.7.0 pycparser-2.19 pynacl-1.3.0 pytest-4.1.0 python-dateutil-2.7.5 python-gilt-1.2.1 requests-2.21.0 sh-1.12.14 six-1.11.0 tabulate-0.8.2 testinfra-1.16.0 tree-format-0.1.2 urllib3-1.24.1 websocket-client-0.54.0 whichcraft-0.5.2 yamllint-1.11.1
2019-06-23 23:29:02 +00:00
Creating your first ansible role
================================
Once all the steps above are complete, we can start by creating our first ansible role.
.. code :: text
$ molecule init role -r example-role
--> Initializing new role example-role...
Initialized role in /home/elijah/sandbox/test-roles/example-role successfully.
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
$ tree example-role/
example-role/
├── defaults
│ └── main.yml
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── molecule
│ └── default
│ ├── Dockerfile.j2
│ ├── INSTALL.rst
│ ├── molecule.yml
│ ├── playbook.yml
│ └── tests
│ ├── __pycache__
│ │ └── test_default.cpython-37.pyc
│ └── test_default.py
├── README.md
├── tasks
│ └── main.yml
└── vars
└── main.yml
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
9 directories, 12 files
You can find what each directory is for and how ansible works by visiting docs.ansible.com.
`` meta/main.yml ``
-----------------
The meta file needs to modified and filled with information about the role. This is not a required file to modify if you are keeping this for yourself, for example. But it is a good idea to have as much information as possible if this is going to be released. In my case, I don't need any fanciness as this is just sample code.
.. code :: yaml
---
galaxy_info:
2019-08-31 09:40:11 +00:00
author: Elia El Lazkani
2019-06-23 23:29:02 +00:00
description: This is an example ansible role to showcase molecule at work
license: license (BDS-2)
min_ansible_version: 2.7
galaxy_tags: []
dependencies: []
`` tasks/main.yml ``
------------------
This is where the magic is set in motion. Tasks are the smallest entities in a role that do small and idempotent actions. Let's write a few simple tasks to create a user and install a service.
.. code :: yaml
---
# Create the user example
- name: Create 'example' user
user:
name: example
comment: Example user
shell: /bin/bash
state: present
create_home: yes
home: /home/example
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
# Install nginx
- name: Install nginx
apt:
name: nginx
state: present
update_cache: yes
notify: Restart nginx
`` handlers/main.yml ``
---------------------
If you noticed, we are notifying a handler to be called after installing *nginx* . All handlers notified will run after all the tasks complete and each handler will only run once. This is a good way to make sure that you don't restart *nginx* multiple times if you call the handler more than once.
.. code :: yaml
---
# Handler to restart nginx
- name: Restart nginx
service:
name: nginx
state: restarted
`` molecule/default/molecule.yml ``
---------------------------------
It's time to configure molecule to do what we need. We need to start an ubuntu docker container, so we need to specify that in the molecule YAML file. All we need to do is change the image line to specify that we want an `` ubuntu:bionic `` image.
.. code :: yaml
---
dependency:
name: galaxy
driver:
name: docker
lint:
name: yamllint
platforms:
- name: instance
image: ubuntu:bionic
provisioner:
name: ansible
lint:
name: ansible-lint
scenario:
name: default
verifier:
name: testinfra
lint:
name: flake8
`` molecule/default/playbook.yml ``
---------------------------------
This is the playbook that molecule will run. Make sure that you have all the steps that you need here. I will keep this as is.
.. code :: yaml
---
- name: Converge
hosts: all
roles:
- role: example-role
First Role Pass
===============
This is time to test our role and see what's going on.
2019-08-31 09:40:11 +00:00
.. code :: text
2019-06-23 23:29:02 +00:00
(.ansible-role) ~/sandbox/test-roles/example-role/ $ molecule converge
--> Validating schema /home/elijah/sandbox/test-roles/example-role/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
└── default
├── dependency
├── create
├── prepare
└── converge
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
--> Scenario: 'default'
--> Action: 'dependency'
Skipping, missing the requirements file.
--> Scenario: 'default'
--> Action: 'create'
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY [Create] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Log into a Docker registry] ***** ***** ***** ***** ***** ***** ***** ***** ***** *
2019-08-31 09:40:11 +00:00
skipping: [localhost] => (item=None)
2019-06-23 23:29:02 +00:00
TASK [Create Dockerfiles from image names] ***** ***** ***** ***** ***** ***** ***** **
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Discover local Docker images] ***** ***** ***** ***** ***** ***** ***** ***** *** *
ok: [localhost] => (item=None)
ok: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Build an Ansible compatible image] ***** ***** ***** ***** ***** ***** ***** *** *
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Create docker network(s)] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Create molecule instance(s)] ***** ***** ***** ***** ***** ***** ***** ***** *****
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Wait for instance(s) creation to complete] ***** ***** ***** ***** ***** ***** *
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY RECAP ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *** *
localhost : ok=5 changed=4 unreachable=0 failed=0
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
--> Scenario: 'default'
--> Action: 'prepare'
Skipping, prepare playbook not configured.
--> Scenario: 'default'
--> Action: 'converge'
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY [Converge] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *** *
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Gathering Facts] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** **
ok: [instance]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [example-role : Create 'example' user] ***** ***** ***** ***** ***** ***** ***** *
changed: [instance]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [example-role : Install nginx] ***** ***** ***** ***** ***** ***** ***** ***** *** *
changed: [instance]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
RUNNING HANDLER [example-role : Restart nginx] ***** ***** ***** ***** ***** ***** ***
changed: [instance]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY RECAP ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *** *
instance : ok=4 changed=3 unreachable=0 failed=0
It looks like the **converge** step succeeded.
Writing Tests
=============
It is always a good practice to write unittests when you're writing code. Ansible roles should not be an exception. Molecule offers a way to run tests, which you can think of as unittest, to make sure that what the role gives you is what you were expecting. This helps future development of the role and keeps you from falling in previously solved traps.
`` molecule/default/tests/test_default.py ``
------------------------------------------
Molecule leverages the `testinfra <https://testinfra.readthedocs.io/en/latest/> `_ project to run its tests. You can use other tools if you so wish, and there are many. In this example we will be using *testinfra* .
.. code :: python
import os
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
import testinfra.utils.ansible_runner
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
def test_hosts_file(host):
f = host.file('/etc/hosts')
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
assert f.exists
assert f.user == 'root'
assert f.group == 'root'
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
def test_user_created(host):
user = host.user("example")
assert user.name == "example"
assert user.home == "/home/example"
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
def test_user_home_exists(host):
user_home = host.file("/home/example")
assert user_home.exists
assert user_home.is_directory
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
def test_nginx_is_installed(host):
nginx = host.package("nginx")
assert nginx.is_installed
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
def test_nginx_running_and_enabled(host):
nginx = host.service("nginx")
assert nginx.is_running
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
.. warning ::
Uncomment `` truthy: disable `` in `` .yamllint `` found at the base of the role.
.. code :: text
(.ansible_venv) ~/sandbox/test-roles/example-role $ molecule test
--> Validating schema /home/elijah/sandbox/test-roles/example-role/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
└── default
├── lint
├── destroy
├── dependency
├── syntax
├── create
├── prepare
├── converge
├── idempotence
├── side_effect
├── verify
└── destroy
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /home/elijah/sandbox/test-roles/example-role/...
Lint completed successfully.
--> Executing Flake8 on files found in /home/elijah/sandbox/test-roles/example-role/molecule/default/tests/...
2019-08-31 09:40:11 +00:00
/home/elijah/.virtualenvs/world/lib/python3.7/site-packages/pycodestyle.py:113: FutureWarning: Possible nested set at position 1
2019-06-23 23:29:02 +00:00
EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
Lint completed successfully.
--> Executing Ansible Lint on /home/elijah/sandbox/test-roles/example-role/molecule/default/playbook.yml...
Lint completed successfully.
--> Scenario: 'default'
--> Action: 'destroy'
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY [Destroy] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *****
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Destroy molecule instance(s)] ***** ***** ***** ***** ***** ***** ***** ***** *** *
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Wait for instance(s) deletion to complete] ***** ***** ***** ***** ***** ***** *
ok: [localhost] => (item=None)
ok: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Delete docker network(s)] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY RECAP ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *** *
localhost : ok=2 changed=1 unreachable=0 failed=0
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
--> Scenario: 'default'
--> Action: 'dependency'
Skipping, missing the requirements file.
--> Scenario: 'default'
--> Action: 'syntax'
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
playbook: /home/elijah/sandbox/test-roles/example-role/molecule/default/playbook.yml
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
--> Scenario: 'default'
--> Action: 'create'
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY [Create] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Log into a Docker registry] ***** ***** ***** ***** ***** ***** ***** ***** ***** *
2019-08-31 09:40:11 +00:00
skipping: [localhost] => (item=None)
2019-06-23 23:29:02 +00:00
TASK [Create Dockerfiles from image names] ***** ***** ***** ***** ***** ***** ***** **
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Discover local Docker images] ***** ***** ***** ***** ***** ***** ***** ***** *** *
ok: [localhost] => (item=None)
ok: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Build an Ansible compatible image] ***** ***** ***** ***** ***** ***** ***** *** *
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Create docker network(s)] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Create molecule instance(s)] ***** ***** ***** ***** ***** ***** ***** ***** *****
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Wait for instance(s) creation to complete] ***** ***** ***** ***** ***** ***** *
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY RECAP ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *** *
localhost : ok=5 changed=4 unreachable=0 failed=0
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
--> Scenario: 'default'
--> Action: 'prepare'
Skipping, prepare playbook not configured.
--> Scenario: 'default'
--> Action: 'converge'
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY [Converge] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *** *
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Gathering Facts] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** **
ok: [instance]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [example-role : Create 'example' user] ***** ***** ***** ***** ***** ***** ***** *
changed: [instance]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [example-role : Install nginx] ***** ***** ***** ***** ***** ***** ***** ***** *** *
2019-08-31 09:40:11 +00:00
changed: [instance]
RUNNING HANDLER [example-role : Restart nginx] ***** ***** ***** ***** ***** ***** ***
changed: [instance]
PLAY RECAP ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *** *
instance : ok=4 changed=3 unreachable=0 failed=0
--> Scenario: 'default'
--> Action: 'idempotence'
Idempotence completed successfully.
--> Scenario: 'default'
--> Action: 'side_effect'
Skipping, side effect playbook not configured.
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /home/elijah/sandbox/test-roles/example-role/molecule/default/tests/...
============================= test session starts ==============================
platform linux -- Python 3.7.1, pytest-4.1.0, py-1.7.0, pluggy-0.8.1
rootdir: /home/elijah/sandbox/test-roles/example-role/molecule/default, inifile:
plugins: testinfra-1.16.0
collected 5 items
2019-06-23 23:29:02 +00:00
tests/test_default.py ..... [100%]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
=============================== warnings summary ===============================
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
...
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
==================== 5 passed, 7 warnings in 27.37 seconds =====================
Verifier completed successfully.
--> Scenario: 'default'
--> Action: 'destroy'
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY [Destroy] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *****
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Destroy molecule instance(s)] ***** ***** ***** ***** ***** ***** ***** ***** *** *
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Wait for instance(s) deletion to complete] ***** ***** ***** ***** ***** ***** *
changed: [localhost] => (item=None)
changed: [localhost]
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
TASK [Delete docker network(s)] ***** ***** ***** ***** ***** ***** ***** ***** ***** ***
2019-08-31 09:40:11 +00:00
2019-06-23 23:29:02 +00:00
PLAY RECAP ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *** *
localhost : ok=2 changed=2 unreachable=0 failed=0
I have a few warning messages (that's likely because I am using python 3.7 and some of the libraries still don't fully support the new standards released with it) but all my tests passed
Conclusion
==========
Molecule is a great tool to test ansible roles quickly and while developing them. It also comes bundled with a bunch of other features from different projects that will test all aspects of your ansible code. I suggest you start using it when writing new ansible roles.
