Yet another look at Symfony2 functional testing

Functional testing Symfony2 controllers is not that hard once you get the idea. The real problem comes to light when you need to optimize those brilliant functional tests you just wrote because they are slow as hell.

So let's start at the beginning.

Create the tests

If you are using the included CRUD generator you will end with a ready-to-run functional test for each one of the generated controllers, neatly saved in your bundle's /Tests/Controller folder. Just follow the instructions and you are all set.
Hey, but there is a problem: the generated functional tests won't run, mainly because they are commented out. You need to uncomment the code and make the required modifications: change each occurrence of yourEntity[fieldName] with yourNamespace_yourBundle_yourEntityType[fieldName].
Example:

• Let's say that your entity is called Country and your bundle is called Ns\CountryBundle, and it has a property name.
• You will need to change country[name] by ns_countrybundle_country[name].

Just in case you are curious enough, the string ns_countrybundle_country comes from class Ns\CountryBundle\Form\CountryType, returned by method getName().

Go ahead and execute the test. Come back here when you have at least one functional test running.

What is tested?

The generated functional tests perform the following actions:

1. Open the base route parameterized when creating the crud. Let's say is /country.
2. Open the new view. Submit the form with data filled in to create a new entry in the database and check that it is in fact created.
3. Edit the new entry and change some field. Submit the changes and check that the entry is in fact changed.
4. Delete the new entry and check that it is in fact deleted.

A fairly comprehensive test, huh? And the better part is that, for the most part, it is auto generated.

Use a separate database

There is a problem, though: by default, the functional tests use the same database you configured and created for the dev environment. If the tests work perfectly you won't notice it, because it deletes the database entry it creates. But it could happen that the test failed, and suddenly you will end with an unexpected entry hanging in your database.
Not a problem, I hear you screaming: just define a new database for testing purposes. OK, let's do it. But, instead of using a server-based database (like MySql, the one you are most likely to be using for your development database) we will use an SQLite database to try to make the tests run as fast as possible.
Add the following to your config_tests.yml:

#/app/config/config_test.yml
...
doctrine:
    dbal:
        driver: pdo_sqlite
        path: %kernel.root_dir%/../data/%kernel.environment%/database.sqlite
    orm:
        auto_generate_proxy_classes: true
        auto_mapping: true

At first look you (yes, you, the SQLite savvy) may think the best approach should be using an in-memory database instead of incurring in the costs of hitting the disk. But it won't work because, after each functional test performs a submit, all the connections to that database are closed, causing the database to be removed from memory (as explained here).

Why is this not good? Because we intend to re-create the database and load the tests fixtures just once before the whole tests suite is run, instead of doing it before each test. You can find a number of tutorials and examples on how to use an sqlite database to load the fixtures before each functional tests, so I won't be explaining it once more. But that approach has a big problem: It is really sloooooooow!!!!!

Add database initialization and fixtures 

Create the following WebDoctrineTestCase somewhere inside your project's structure (for the sake of good code organization I created a new bundle just to hold all my common test-related classes). Do not forget adjusting the namespace accordingly:

<?php

namespace MyNamespace\TestBundle\Lib\Test;

use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Doctrine\DBAL\Driver\PDOSqlite\Driver as SqliteDriver;

abstract class WebDoctrineTestCase extends WebTestCase
{

    protected static $entityManager;
    protected static $client;
    protected static $application;
    
    protected static $isFirstTest = true;

    /**
     * Prepare each test
     */
    public function setUp()
    {
        parent::setUp();

        static::$client = static::createClient();

        if (!$this->useCachedDatabase()) {
            $this->databaseInit();
            $this->loadFixtures();  
        }
    }

    /**
     * Initialize database
     */
    protected function databaseInit()
    {
        static::$entityManager = static::$kernel
            ->getContainer()
            ->get('doctrine.orm.entity_manager');

        static::$application = new \Symfony\Bundle\FrameworkBundle\Console\Application(static::$kernel);
        
        static::$application->setAutoExit(false);
        $this->runConsole("doctrine:schema:drop", array("--force" => true));
        $this->runConsole("doctrine:schema:create");
        $this->runConsole("cache:warmup");
    }

    /**
     * Load tests fixtures
     */
    protected function loadFixtures()
    {
        $this->runConsole("doctrine:fixtures:load");
    }
    
    /**
     * Use cached database for testing or return false if not
     */
    protected function useCachedDatabase()
    {
        $container = static::$kernel->getContainer();
        $registry = $container->get('doctrine');
        $om = $registry->getEntityManager();
        $connection = $om->getConnection();
        
        if ($connection->getDriver() instanceOf SqliteDriver) {
            $params = $connection->getParams();
            $name = isset($params['path']) ? $params['path'] : $params['dbname'];
            $filename = pathinfo($name, PATHINFO_BASENAME);
            $backup = $container->getParameter('kernel.cache_dir') . '/'.$filename;

            // The first time we won't use the cached version
            if (self::$isFirstTest) {
                self::$isFirstTest = false;
                return false;
            }
            
            self::$isFirstTest = false;

            // Regenerate not-existing database
            if (!file_exists($name)) {
                @unlink($backup);
                return false;
            }

            $om->flush();
            $om->clear();
            
            // Copy backup to database
            if (!file_exists($backup)) {
                copy($name, $backup);
            }

            copy($backup, $name);
            return true;
        }
        
        return false;
    }

    /**
     * Executes a console command
     *
     * @param type $command
     * @param array $options
     * @return type integer
     */
    protected function runConsole($command, Array $options = array())
    {
        $options["--env"] = "test";
        $options["--quiet"] = null;
        $options["--no-interaction"] = null;
        $options = array_merge($options, array('command' => $command));
        return static::$application->run(new \Symfony\Component\Console\Input\ArrayInput($options));
    }
}

What this class does is:

1. Look  if the SQLite test database is already cached.
2. If not cached (when executing the first test in the suite), create it and populate with the data fixtures.
3. But, if it is already cached, just copy the database cached version (a single file, as you may know) replacing the real database file used by tests.

The cache file is created inside your /app/cache and is automatically recreated before running the first test in the suite.

Of course, for that to work you have to change the functional tests class to extend WebDoctrineTestCase instead of WebTestCase:

<?php

namespace MyNamespace\MyBundle\Tests\Controller;

use MyNamespace\TestBundle\Lib\Test\WebDoctrineTestCase;

class MyEntityControllerTest extends WebDoctrineTestCase
{
    public function testCompleteScenario()
    {
        ....
        ....
    }
}

Disclaimer: The solution outlined here was inspired by LiipFunctionalTestBundle, but I simplified it because it may be overkill for simple projects and difficult to understand for those still learning (like myself). Anyway, once you understand the principles may be you could switch to that great bundle.

That's all. I've seen at least a 50% decrease in overall tests run time, but of course it will depend on how many functional tests you have.

EDITED (2012-09-03): Added '--no-interaction' option when running console commands.