Standalone PHP code coverage without PHPUnit

When developing a PHP application you can use PHPUnit in order to execute unit tests and gather code coverage statistics. Sometimes, though, it would be useful to gather code coverage statistics from the real world usage of the application. Although that induces a time penalty, it could be used in a debug deployment of the application for the sake of gathering the required statistics.

Let’s go step by step to perform the above mentioned procedure:

  1. Locate php.ini and uncomment the following line in order to enable the XDebug extension.

    ...
    [XDebug]
    ...
    zend_extension_ts=<path to php_xdebug.dll>
    ...
    
  2. Restart your web server.

  3. Next you have to prepend a PHP script fragment in every PHP script of your application. I have accomplished this task by adding the following line into the .htaccess file. That way every PHP script that resides under the directory of the .htaccess file will get automatically prepended. Notice that the .htaccess file applies recursively to directories.

    php_value auto_prepend_file <absolute path to prepend.php>
    
  4. The contents of the prepended script (prepend.php) are the following:

    <?php
    
    function __xdebug_stop()
    {
      $data = xdebug_get_code_coverage();
      xdebug_stop_code_coverage();
    
      $dir = <absolute path to generated code coverage files>;
      $file = sprintf('%s%s.code_coverage', $dir, uniqid());
      file_put_contents($file,serialize($data));
    }
    
    if (function_exists('xdebug_start_code_coverage'))
    {
      xdebug_start_code_coverage(XDEBUG_CC_UNUSED | XDEBUG_CC_DEAD_CODE);
      register_shutdown_function('__xdebug_stop');
    }
    
  5. We can now gather code coverage statistics by just visiting the web application from our browser. We need now a utility to transform the serialized code coverage files into a nice HTML or XML report. For that matters, we have the following script (report.php):

    <?php
    
    require_once('PHPUnit/Framework/TestResult.php');
    require_once('PHPUnit/Util/Report.php');
    require_once('PHPUnit/Runner/Version.php');
    require_once('PHPUnit/Util/Log/CodeCoverage/XML/Clover.php');
    
    class MyTestResult extends PHPUnit_Framework_TestResult
    {
      public function topTestSuite()
      {
        return new PHPUnit_Framework_TestSuite();
      }
      
      public function getCodeCoverageInformation($filterTests = TRUE)
      {
        foreach (CoverageFiles::getNames() as $filename)
        {
          print $filename."\n";
          $codeCoverage = unserialize(file_get_contents($filename));
          $this->appendCodeCoverageInformation(new MyTestCase, $codeCoverage);
        }
        
        $ignoreLines = file(CoverageFiles::getIgnoreFile());
        foreach ($ignoreLines as $ignoreLine)
        {
          $ignoreLine = rtrim($ignoreLine);
          $pos = strrpos($ignoreLine, ':');
          $filename = substr($ignoreLine, 0, $pos);
          $lineno = substr($ignoreLine, $pos+1);
          $ignore[$filename][] = $lineno;
        }
        
        foreach ($ignore as $file=>$lines)
        {
          foreach ($lines as $lineno)
          {
            foreach ($this->codeCoverageInformation as &$codeCovInfo)
            {
              $x = $codeCovInfo['executable'][$file][$lineno];
              if ($codeCovInfo['executable'][$file][$lineno] == 1)
              {
                print 'Warning: Covered by tests but ignored: '.$file.':'.$lineno."\n";
              }
              else
              {
                unset($codeCovInfo['executable'][$file][$lineno]);
              }
            }
          }
        }
        
        return parent::getCodeCoverageInformation($filterTests);
      }
    }
    
    class MyTestCase extends PHPUnit_Framework_TestCase
    {
    }
    
    class CoverageFiles
    {
      public static function getNames()
      {
        return glob(<absolute path to *.code_coverage>);
      }
      public static function getIgnoreFile()
      {
        return <path to ignore file>;
      }
    }
    
    $htmlOutputDir = <path to directory for the HTML report>;
    $xmlOutput = <path to the XML report>;
    
    $res = new MyTestResult;
    PHPUnit_Util_Report::render($res, $htmlOutputDir);
    $xmlClover = new PHPUnit_Util_Log_CodeCoverage_XML_Clover($xmlOutput);
    $xmlClover->process($res);
    
    foreach (CoverageFiles::getNames() as $filename)
    {
      unlink($filename);
    }
    
  6. Sometimes, we know that a certain piece of code will not be executed, so we would like to have a mechanism to ignore specific lines of code. This purpose is served by the ignore file. Every line of this file specifies a line of code that should be ignored if it is not executed (because we expect that it will not be executed). But if the ignored line is actually executed, the report.php will issue a warning, which may indicate that there is a problem in the ignore file, e.g. some lines of code got shifted out of position.

    ...
    <absolute path to prepend.php>:6
    <absolute path to prepend.php>:8
    <absolute path to prepend.php>:9
    <absolute path to prepend.php>:10
    <absolute path to prepend.php>:11
    ...
    
  7. The final step is to generate a nice HTML report that summarises the not executed lines of code in our application. For this reason, we have the following Python script (filter.py):

    from xml.dom.minidom import parse
    
    htmlOutputDir = <path to directory for the HTML report>
    xmlInputFile = <path to the XML report>
    htmlOutputFile = <path to the generated HTML summary report>
    projectPathPrefix = <absolute path to the web application root directory>
    reportFilesPrefixLength = len(<common prefix of the paths of the covered files>)
    
    def getLink(filename, lineno):
      fileReport = filename.replace('\\','_')
      href = '%s/%s.html#%d' % (htmlOutputDir, fileReport[reportFilesPrefixLength:], lineno)
      text = filename + ':' + str(lineno);
      return '<a href="%s">%s</a><br>' % (href, text)
    
    dom = parse(xmlInputFile)
    htmlout = open(htmlOutputFile,'w')
    
    htmlout.write('<pre>')
    htmlout.write('<h3>Lines not covered by tests</h3>')
    
    files = dom.getElementsByTagName('coverage')[0] \
               .getElementsByTagName('project')[0].getElementsByTagName('file')
    
    for file in files:
      name = file.getAttribute('name')
      if (not name.startswith(projectPathPrefix)):
        continue
      #name = name[reportFilesPrefixLength:]
      lines = file.getElementsByTagName('line')
      for line in lines:
        count = int(line.getAttribute('count'))
        if count > 0:
          continue
        lineno = int(line.getAttribute('num'))
        pair = (name,lineno)
        htmlout.write(getLink(name, lineno))
    

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: