Background

Building a large Drupal website can be a daunting and complex process. There are many engineering, project and other risks associated with it. More often than not, budget and deadline overrun occurs.

Moreover, large projects often result in a prolonged Q/A, testing and troubleshooting period which once again can pose additional risks. 

FFW was recently presented with the following challenge by a client:

• 1200 budget hours
• 2 month of development time
• 4 developers
• 1 hard launch date

To add another challenge to the mix, there was minimal Q/A engineer availability for the project.

In order to meet the client’s budget and timeline requirements above, we needed to rethink a new development workflow that largely incorporated quality into the project in order to meet both the deadline and budget requirements. 

Typical Development Workflow

Propeople selected GIT as the revisioning software to manage the development process/workflow for the project. GIT is a popular, open source distributed versioning system used by many organizations of all sizes. For more information on GIT, you can visit this site.

Organizations that employ a repository management system such as GIT, typically one of the following workflow models is used.

1. Using the Master Branch

This workflow is typically employed by smaller organizations with a small number of developers working on the project. In this workflow, all of the code changes are committed into the master branch by all of the developers. It results in a linear development workflow:

This workflow works well at keeping everything simple, allowing developers to avoid branching and merging. It also allows for the creation of some tags to designate stable release or state of the code repository.

However, there are many drawbacks to this workflow. One of the most overlooked facts is that most likely, John and Joe will not check each other’s commits. Once a change is committed to the master branch, the change is incorporated into the project until someone notices any poorly written code or commits. 

2. Development -> Stage -> Production

One of the most widely adopted workflows, this three-tiered workflow fully leverages GIT’s ability to create and merge branches. It also gives web developers the ability to mirror the branches to a different “working environment” so that they can have:

• Development website
• Staging website
• Production website

3. Feature Branch

We ultimately chose to adopt a “feature” branch workflow for this project. This model takes advantage of the git branching/merging model. It is also chosen because we did not necessarily need a “staging” or “production” environment during the agile development phase.

This workflow allows developers to independently work on different "features" on the project. A branch is created for each "feature". Once they are finished, they will be merged into the master branch. Completed "feature" branches can then be deprecated or removed. 

This worked well for us because it provided a checkpoint, when every feature is merged into the master branch for a Q/A and review of that feature. 

Introduction of Github and the Magical Pull Request

One of the biggest reasons that we adopted the feature branch approach is Github. Github.com is one of the longest-running and most popular hosted git management platforms. It offers many developer friendly tools. One of the most useful and popular tools and functions is the Pull Request.

For the project, we set a few very simple rules for all developers that maximized the “feature branch” model using pull requests.

1. No one can commit DIRECTLY on master branch.

2. Developers should initiate pull requests when a feature/task is finished.

3. Each pull request should be reviewed by by another team member other than the PR initiator.

4. The Pull request initiator (owner) CANNOT  merge their own pull request.

Through the pull request and peer review process, we were able to build quality control into every commit that ended up on the master branch and eventually into the production code.

Adding Continuous Integration (CI)

In order to aid the pull review process, we also added automated tools that were triggered when the pull request is initiated and any new commits to the pull request is pushed subsequently.
 

1. Pull Request Build

A temporary development snapshot of the website is built for every pull request. This website will include all the new commits from the feature branch as well as any database changes invoked through hook_update_N.

2. PHP Code Sniffer

PHP Code sniffer automatically checks for coding standards during every pull request. This promotes a better commenting and code style for all of the developers. It also checks code for common php mistakes and security issues.

3. CSS/SCSS Lint

Additional CSS/SCSS style checking libraries also scan and check for css/scss complexity and coding style.

4. JSHint

JShint was also installed as part of the CI workflow to check for JavaScript code quality.

Testing JavaScript can be a challenge, but with an adopted-for-CI-workflow, JShint checking quality for JavaScript code becomes easier to do.

Below is an example of the screen when a new PR is created:

Looking ahead

We are looking ahead to adding automated regression testing to the pull requests. This will allow developers to easily see page changes caused by their commits.

A more integrated issue and time tracking system would help consolidate all of the project efforts into Github. We used a separate issue tracking and hour tracking system for the project.

Conclusion

Largely due to the new development workflow, we were able to meet the the hard deadline set by client. Although the project had a 8% budget overrun, this number is a great success, given the 27%* industry average. We were also able to reduce our post launch Q/A cycle by 66.67%, from 30 days to 10 days.

A big thanks to the masterminds behind the new development workflow:

Yuriy Gerasimov

Andriy Podanenko

 

* http://hbr.org/2011/09/why-your-it-project-may-be-riskier-than-you-think/