SpiderLogic.com

Organizing Large Software Projects

Brennan Stehling

March 26, 2004

Overview

Large software projects can become unwieldy quickly. New dependencies and requirements creep into the system without any concern for the added workload it may require or how it could affect deadlines. IT projects with ever changing technologies and increasingly complex business needs make it difficult to maintain consistent and reliable builds.

If you find yourself in the enviable position of joining a project from the start, not after the scope creep has infested your infinite workload, you will have the time to prepare for change. Your best option is to stay flexible and separate the major pieces of your project into manageable components. Perhaps your massive project could be broken up into the core infrastructure, database persistence, business rules, and the user interface. That is a good first step to encapsulate changes within a component.

Division of Labor

With the project broken up into various components the development team can specialize in one or more components as their skills fit. Perhaps developers with extensive experience with the company and/or industry will have an advantage in handling the business rules component. Other developers may be skilled UI designers and will be best suited to handling the interface. As a result, the code in each component will be of higher quality including the Javadoc which will provide informed detail about every aspect of the component.

Breaking up the team, even a small team, will speed up development efforts. Team meetings could be limited to just the component teams most of the time so the discussion is focused on the development issues and less time reviewing complex business requirements or the nuances of the UI framework. Since the meetings will include fewer people it will be easier to schedule and should be much shorter. Issues discussed in the meetings will be more focused and the developers will be capable of providing the right solution the first time. But ultimately, a technical architect and project manager should watch over each component-oriented team to ensure each component passes code review guidelines and fulfills overall project goals.

Design Contracts

Part of breaking up a project into many components will be integrating them with a stable interface. The implementation of the code will continually change as the software is edited, but the API should remain consistent between components. Class names and locations, method signatures and constants will need to be set in stone, at least between major versions. If a significant change is necessary, versioning should be used to manage the change. With sufficient planning a system designer will be able to produce a stable interface to protect against change. In light of these constraints it is clearly going to be easier to introduce new classes, methods and constants than it will be to rename or remove them.

Automation

With design contracts in place and the team specializing in various components the next step will be to automate the more time-consuming and tedious tasks, such as builds and unit testing. Today the de facto standard for automated builds in Java is Ant. (see Tips for Ant) With Ant you may choose to run routine builds automatically with Cruise Control which will happily run your unit tests and report on the result.

Unit Testing

With the design contracts in place it will be possible to enforce them with unit tests. The JUnit framework provides a means to simplify unit testing. Tests can be created with regular Java classes which use the JUnit framework to verify the agreed upon behavior. These tests will also confirm the required functionality for the project and send up flags whenever a test fails during a Cruise Control build.

Revision Control

An important part of any project will be managing revisions. The most common revision control system (RCS) is the Concurrent Versioning System (CVS) which is integrated with nearly every development tool available. I highly recommend SmartCVS which provides a powerful, yet intuitive, interface for working with a CVS repository outside of the IDE. Some CVS work can be difficult in an IDE like Eclipse, but is much simpler with SmartCVS. For example, it is very easy to create branches and tags within SmartCVS. And merging between branches is also a breeze in a way that Eclipse does not provide. The newly released IDEA 4.0 has integrated the SmartCVS engine. The ability to carry out the frequent housekeeping tasks for revision control makes the right tool invaluable.

A lesser known and unfortunately unused CVS feature is the notification system. With tools like SmartCVS or Eclipse it is possible to monitor changes on a file. Eclipse can be set to send an edit notification to the CVS server each time a local file is changed. It will also place a temporary watch on the file to alert you whenever someone else edits the same file. The watch does not act as a lock. Instead it acts as an early warning system to prevent complications and conflicts. A developer can also place a permanent watch on a file to get all change notifications on a file regardless of what files are edited locally. A lead architect may choose to watch critical infrastucture throughout a project to keep an eye on important files. Typically the CVS server is configured to send emails for the notices, but any means can be used, such as messages to an IM client or adding entries to an issues database. The behavior can be configured with the CVS adminstrative files. Typical on a Unix or Linux system the CVS server will use the mail utility to send mail using the username of the committer. But in Windows the committer name may not match up with a username or email address perfectly. A custom script will need to be created.

Typical Notify Command In Unix/Linux

ALL mail %s -s "CVS notification"

With the codebase carefully managed with CVS and tested routinely with JUnit through Cruise Control scheduled Ant builds, it will be a possible to discover issues early and often. It will not be necessary to wait till the last minute or have each developer stop working once a week to attempt a manual integration build. With properly maintained design contracts and unit tests the quality control processes will be an easier task as the project is wrapped up for release.