buy Windows 8 Product Key Chrome Hearts Earring Curly Full Lace human Wigs Chrome Hearts Sale windows 8 product key ccie lab workbook

Automating the PHP deployment process with Phing, dbdeploy and FTP

Posted by buildmeister on January 5th, 2009, last updated on July 28th, 2009
Filed under:  database  php  deployment  build automation  build scripting  web  build  tools 
There are 8 comments on this article.
Bookmark and Share

Introduction

PHP is a scripting language, primarily used for rapidly creating dynamic web applications. Deployment of PHP applications can simply be a matter of copying completed PHP source and database scripts to a production web server. Although such an approach might be fine for small applications, it does not scale for larger applications that are created by multiple developers and need to be deployed to multiple environments for development, testing and production (customer facing) usage. A "well defined" build and deployment process (supported by a number of key tools or scripts) can often help reduce errors and increase the quality of delivered applications. In comparison with other languages, the PHP community has been somewhat slow in implementing build, deployment, code inspection and testing tools, however there are now a sufficient number available that have reached maturity. This article therefore looks specifically at some of these and describes how Phing - a PHP build tool - can be used with dbdeploy - a database change management tool - and FTP to implement a secure, controlled and automated PHP deployment process.

Getting Started with Phing

You could in theory use any number of tools for building and deploying PHP applications, including popular ones such Apache Ant, GNU make or SCons. However it is usually best to use of a tool that understands the language domain that applications are being written in. In the case of PHP one such tool is Phing which is written in PHP and has a large number of tasks for integrating with related PHP tools (including PHPUnit, PHPDocumentor, PHP CodeSniffer and many others). It is also worth mentioning that Phing's build scripting format and model is based on Apache Ant, so if you have experience of Ant will have an advantage in learning and using Phing.

If you have a working PEAR Package Manager installation, then installing Phing can be carried out by navigating to your PHP home directory and executing the following two commands:

>pear channel-discover pear.phing.info
>pear install phing/phing

The input to Phing is a textual build script by default called build.xml (although any filename can be used). Each build script describes a set of targets, which are the core parts of the composite build process, e.g. parse files for errors, run unit tests, create a PEAR package and so on.

Each of the targets can make reference to any of Phing's built-in tasks to carry out a predefined operation, e.g. copy a file. Parameters can be specified for the task, defining its exact invocation. There are a large number of built-in tasks (see the Phing manual for details), however, you can quite easily extend Phing by creating new tasks yourselves.

An example of a simple Phing build script to deploy PHP files via FTP to a specific web server is illustrated below:

<?xml version="1.0" encoding="UTF-8"?>
    <project name="buildmeister.com" default="help" basedir=".">

    <property name="dir.public"    value="public"/>
    
    <!-- define a fileset for PHP Web sources -->
    <fileset dir="${dir.public}" id="php.sources">
        <include name="**"/>
        <exclude name="images/**"/>
        <exclude name="**/.svn/**"/>
    </fileset>

    <!-- define a fileset for PHP Web images -->
    <fileset dir="${dir.public}/images" id="php.images">
        <include name="**"/>
        <exclude name="**/.svn/**"/>
    </fileset>

    <!-- get environment -->
    <target name="get-env" description="get the environment for an action">
       <input propertyname="environment" 
           validargs="dev,test,prod">Enter environment name </input>
       <property file="${environment}.properties"/>
    </target>

    <!-- deploy the applications files to the specific environment -->
    <target name="deploy" depends="get-env"
        description="update the application files in a specific environment">
        <ftpdeploy host="${ftp.server}" port="${ftp.port}" 
            username="${ftp.user}" password="${ftp.pass}" 
            dir="." mode="ascii" clearfirst="true">
            <fileset refid="php.sources"/>
        </ftpdeploy>
        <ftpdeploy host="${ftp.server}" port="${ftp.port}" 
            username="${ftp.user}" password="${ftp.pass}" 
            dir="images" mode="binary" clearfirst="true">
            <fileset refid="php.images"/>
        </ftpdeploy>
    </target>
    
</project>

In this build script there are two targets: get-env and dist. Sometimes a target will have a dependency on another target. Dependencies are specified via a depends attribute in the target. You can see this with the deploy target (where the target requires the get-env target to be executed first to prompt for the environment to deploy to). Phing will automatically execute any dependencies.

On line 4 you will see the script makes reference to a property value. In Phing, properties are equivalent to most programming language's concept of variable constants; that is to say they are immutable and once properties have been assigned a value they cannot be changed. Here they are being used to define a directory name. You make references to properties by using the ${property_name} symbols (see line 7 for an example). An interesting feature of properties is that they can be overridden from the command line. For example, if you wanted to override the dir.public property, you could use the following:

>phing -f mybuild.xml deploy -Ddir.public=mypublic

On lines 6-17 two Fileset types are defined. The Fileset type is one of a number of types in Phing and can be used to refer to a related list of files and directories. Types can either be specified as an argument to a task or defined upfront and referred to later by an "id" as in this example. This approach allows you to reuse fileset definitions throughout tour build script. The two Fileset types in this script refer to the set of source and image files that are to be deployed. They are separated out because we want to specify different transfer modes (ascii or binary) when we deploy the files via FTP.

The get-env target on line 20 prompts the user for the environment to deploy to. Based on the input it reads a property file that contains all of the settings relevant to the environment, i.e. FTP server and database server credentials.  An example of the properties file for production (called prod.properties) is illustrated below:

ftp.server=ftp.myserver.com
ftp.port=21
ftp.user=ftpadmin
ftp.pass=password
ftp.remotedir=
db.server=localhost
db.user=root
db.pass=root
db.name=buildmeister
site.basedir=http://myserver/buildmeister.com/public

If this build script was used to start a deployment to the production environment then the output result would look similar to the following:

>phing deploy
Buildfile: C:\Temp\buildmeister.com\build.xml
 [property] Loading C:\Temp\buildmeister.com\default.properties

buildmeister.com > init:

buildmeister.com > get-env:

Enter environment name (dev,test,prod) prod
 [property] Loading C:\Temp\buildmeister.com\prod.properties

buildmeister.com > deploy:

[ftpdeploy] Clearing directory .
[ftpdeploy] Clearing directory images

BUILD FINISHED

Configuring for different environments

So far we have deployed files to an environment but we have not carried out any configuration of the source files for the environment. Unless you are very lucky you will often have different database names, accounts and web root directories in each of the environments. Many developers create a single file to contain constant definitions (typically called constants.php) and include it in each of their PHP source files, i.e. include_once('include/constants.php'). One way of configuring for deployment is to use the information contained in the Phing environment.properties file to replace placeholder strings with environment specific settings. In order to configure these constants however, it would be better to have a file called template_constants.php which includes the placeholder strings to be replaced. This file could then be used as the basis for the real file. As an example the template_constants.php file might look something like the following:

// database constants for MySQL
define("DB_SERVER", "@db_server@");
define("DB_USER", "@db_user@");
define("DB_PASS", "@db_pass@");
define("DB_NAME", "@db_name@");
// site constants
define("SITE_BASEDIR", "@site_basedir@");

This file could then be copied to the real file (constants.php) at build time and the placeholder values replaced with the environment specific settings using the following addition to the Phing build script.

    <!-- configure application for specific environment -->
    <target name="config" depends="get-env"
        description="configure application for a specific environment">
        <input propertyname="db.pass">Enter database password for ${environment}:</input>
        <property file="${environment}.properties"/>
        <copy overwrite="true"
             file="${dir.public}/include/template_constants.php"
             tofile="${dir.public}/include/constants.php">
             <filterchain> 
                 <replacetokens begintoken="@" endtoken="@">
                     <token key="db_server" value="${db.server}" />
                     <token key="db_user" value="${db.user}" />
                     <token key="db_pass" value="${db.pass}" />
                     <token key="db_name" value="${db.name}" />
                     <token key="site_basedir" value="${site.basedir}" />
                 </replacetokens>
            </filterchain>
         </copy>
    </target>

    <!-- deploy the applications files to the specific environment -->
    <target name="deploy" depends="config" ...

Note that this example also prompts for the database password for the environment - this is for security reasons as it might not be a good idea to hold this information in a publicly visible file.

An alternative deployment target

Although the FTP deployment task currently defined works, in practice it can be somewhat slow for large code bases since it needs to transfer all of the files each time. An alternative is to use a specific FTP synchronization tool such as CyberKikp FTPSync. Although this is a Windows only tool it can perform incremental updates and cache passwords. This tool uses "INI" files to identify the source and destination directories to be synchronized. You would create an "INI" file for each environment. An example of one for production would look like the following:

[Source]
Type=F
Dir=./public
ExcludeDir=_*;.svn;.settings;upload;db
ExcludeFile=_*;*.log;*.exe
Case=Sensitive

[Destination]
Server=ftp.myserver.com
User=ftp_admin@myserver.com
Pass=password
Dir=/
Type=M
TimeOut=60
Passive=No

The password in this file will be replaced with an encrypted password after the tool is first run. To make use of FTPSync you would then change the deploy target to look similar to the following:

    <property name="dir.bin"       value="bin"/>
    <property name="dir.sync"      value="sync"/>

    <!-- deploy the application files to the specific environment --> 
    <target name="deploy" depends="config"
        description="update the application files in a specific environment">
       <exec command="${project.basedir}\${dir.bin}\FTPSync.exe ${environment} /FTPSYNCDATA:${project.basedir}\${dir.sync} /INCREMENTAL" />
    </target>

In this example, the FTPSync.exe file has been placed in the bin directory and the prod.ini INI file in the sync directory. The sync directory will also be used to hold log files on execution.

Database deployment

The vast majority of PHP applications are database driven (usually the popular MySQL database). Although your application's database schema will not change as regularly as your PHP source files, it is still desirable to have a process in place for deploying database changes. One approach is to simply to create an SQL script each time you need to update your database schema and then run this script on each database environment. However, what if you want to create a database from scratch, roll back a number of changes or migrate database changes to different environments at different times. Writing the update scripts to cater for all of these circumstances would be very time consuming not to mention difficult. A recommended alternative approach is to use a database change management tool such as dbdeploy or liquibase. Such tools allow you to deploy "changesets" to a database environment and manage the different change levels for you so that you can successfully deploy or rollback incremental changes. Since Phing already has a DbDeploy task, we will look at how we can make use of this tool as part of our deployment process.

First you need to create a "changelog" table in each of your databases by executing the following SQL:

CREATE TABLE changelog (
  change_number BIGINT NOT NULL,
  delta_set VARCHAR(10) NOT NULL,
  start_dt TIMESTAMP NOT NULL,
  complete_dt TIMESTAMP NULL,
  applied_by VARCHAR(100) NOT NULL,
  description VARCHAR(500) NOT NULL
);

ALTER TABLE changelog ADD CONSTRAINT Pkchangelog PRIMARY KEY (change_number, delta_set);

This changelog is used by dbdeploy to keep track of the incremental changes deployed to each database. This SQL is for MySQL only, if you are using another database you can download dbdeploy and use the relevant createSchemaVersionTable script contained within the download to create the changelog table.

Next you should create a directory structure to hold all of your dbdeploy "delta" scripts. An example directory structure is illustrated in the diagram below, with the scripts being contained in the db/delta directory:

[dbdeploy Directory Structure]

Each of the scripts you create should start with a number indicating the order of deployment and a descriptive name. The content of each of the scripts should then be created in the following format:

...SQL creation statements...

--//@UNDO   
  
...SQL rollback statements...
  
--//  

As an example, the 1-create-users.sql delta script contains the following content:

ALTER TABLE users (
  username varchar(32) NOT NULL,
  `password` varchar(32) default NULL,
  firstname varchar(100) NOT NULL,
  lastname varchar(100) NOT NULL,
  userid varchar(42) NOT NULL,
  userlevel tinyint(1) unsigned NOT NULL,
  `email` varchar(50) default NULL,
  `timestamp` int(11) unsigned NOT NULL,
  notify tinyint(1) NOT NULL default '0',
  verifystring varchar(16) NOT NULL,
  active tinyint(1) unsigned NOT NULL default '0',
  PRIMARY KEY  (username)
);

--//@UNDO   
  
DROP TABLE IF EXISTS users;
  
--//  

You must use the "--//" format to delimit deploy and undo actions but other than that the script is pure SQL. This particular example creates a new "users" table.

If at a later stage you wanted to refactor this table by changing one of its columns, then you would write a new delta script (say 15-alter-email-size.sql) similar to the following:

ALTER TABLE users CHANGE `email` `email` VARCHAR(100) 
  NOT NULL DEFAULT 'user@yourdomain.com';

--//@UNDO   
  
ALTER TABLE users CHANGE `email` `email` VARCHAR(50) NULL;
  
--// 

Each script can contain any number of SQL statements, but it is good practice to keep the number small so you can better understand the scripts intention.

Once you have a number of these delta scripts in place and have created the changelog table, then each update to a database environment can be carried out by implementing a Phing target similar to the following:

    <property name="dir.build"     value="build"/>
    <property name="dir.db"        value="db"/>
    <property name="mysql.exe"     value="C:\\mysql\\mysql5.1.30\\bin\\mysql.exe"/>

    <!-- update the relevant database -->  
    <target name="db.update" depends="get-env" 
        description="update database of a specific environment">     
        <input propertyname="db.pass">Enter database password for ${environment}:</input>
        <property file="${environment}.properties"/>

        <!-- load the dbdeploy task -->  
        <taskdef name="dbdeploy" classname="phing.tasks.ext.dbdeploy.DbDeployTask"/>  

        <!-- generate the deployment scripts -->  
        <dbdeploy url="mysql:host=${db.server};dbname=${db.name}"  
            userid="${db.user}" password="${db.pass}"  
            dir="${project.basedir}/${dir.db}/deltas"
            outputfile="${dir.build}/all-deltas.sql"  
            undooutputfile="${dir.build}/undo-all-deltas.sql" />  
  
        <!-- execute the SQL -->  
        <exec command="${mysqlexe} -h${db.server} -u${db.user} -p${db.pass} ${db.name} &lt; ${dir.build}\all-deltas.sql" checkreturn="true" />  
    </target> 

This example adds a new target called db.update. Executing this target will as before prompt you for the environment to deploy to and then call dbdeploy to create a single update script (all-deltas.sql) as well as an undo script (undo-all-deltas.sql); both of which will be placed in the build directory. In this example it will also automatically execute the all-deltas.sql script to deploy the changes using the mysql.exe command line client.

The only issue with dbdeploy is that you will need to be able to connect to each of your databases directly from your client machine. If your database is hosted by a third party (i.e. an Internet Service Provider), then you will probably not be able to execute dbdeploy (or mysql.exe) directly on it, in which case you will have to unfortunately make sure you transfer and deploy the contents of the deployment scripts yourself.

Summary

In this article we have looked at how to implement a simple, yet effective and configurable deployment process for PHP applications. I would encourage you to see how these practices can be implemented in your own environment. Since deployment can only deploy what has already been created, I would also recommend that you look at how you can improve your development and build process using Phing and some its "quality" tasks such as PHPUnit, XDebug, and so on. You will find more information about how to do this using the references below and from other articles on this site.

As always, if you believe things can be done better, or have any ideas or suggestions based on this article please feel free to comment below.

References

 

Bookmark and Share

Comments

Posted by Pedro

Thanks a lot for this helpful article! This is exactly what I was looking for. Pedro

Posted on February 8th, 2011
Posted by Karsten

Nice article. I'd like to know one thing though: Why is the PK (change_number, delta_set) and not just change_number?

Posted on April 27th, 2010
Posted by Buildmeister

Phing is a "build control" tool it doesn't really have much deployment capabilities itself. Capistrano would be a better bet (see here for an example) but you have to learn rails to get the most out of it. Scripting your own process in Phing to call tools like scp/sftp etc could also be an option.

Posted on July 4th, 2009
Posted by Abhishek

Is phing or capistrano efficient  tool for deploying php applications on 100 web servers? what should be best approach to deploy php application on huge number of servers.?

Posted on June 26th, 2009
Posted by Hannes Dorn

Thanks for the article! I will have a look at Phing.

Instead of ftpsync I am currently using sitecopy, which is available on Windows, Mac and Linux. If SSH is available, I am using rsync.

Posted on April 6th, 2009
Posted by Daniel

Excellent post! For anyone interested in working with DBDeploy and rsync on linux, I posted a while back on the buildscript setup I had at my previous job.

http://www.toosweettobesour.com/2008/07/23/net-perspective-cross-post-time-is-money-save-time-with-automation-and-phing/

Posted on April 3rd, 2009
Posted by Sean

Terrific article!  I've been searching for a detailed walk through of php deployment using Phing... great work and thanks!

Posted on April 2nd, 2009
Posted by Jim O'Halloran

Hi,

Great article.  I've been learning about Phing lately and this post makes me want to dive in and get it up and running!  Thanks!

Jim.

Posted on March 25th, 2009

Back to Top

Submit a new comment

All fields in bold are required.

Fatal error: Class 'FCKeditor' not found in /homepages/44/d346279547/htdocs/buildmeister/pages/articles/view.php on line 325