Mark Gibson; PgMP, PMP, IBM certified Executive Project Manager

Implementation Methodologies: Good or Bad?

Are implementation methodologies helpful to project success or a hindrance?  It depends on how the project manager executes the methodology, and adapts to the unique situations on each project, and how well the project conforms to the assumptions within the methodology.  I am a firm believer in the value of an implementation methodology having used them previously and developed several myself.  The methodologies are a good starting point for developing a project plan providing the project manager with:

1. Templates for all project deliverables that the consulting/contractor staff are familiar with thus eliminating the startup costs associated with training the team and agreeing internally on the format and content of deliverables.
2. A development methodology the staff is both trained in and is comfortable executing.
3. Checklists of tasks to consider within the context of the project.
4. Typical staffing profiles and associated task durations and dependencies.

These are just a few of the benefits of a methodology and all have a positive impact on project success.  These methodologies are also based on fundamental assumptions that many times get missed by project managers including:

1. The customer is familiar with and agreeable to the development model proposed.
2. The customer can support the customer staffing model with roles such as a  business analyst, testing leads, etc.
3. The customer is agreeable with and supportive of the set of deliverables defined.
4. The statement of work or project charter is compatible with the methodology.
5. Individual projects should adjust the tasks, durations, and staffing to fit the unique project needs.

The specific assumptions associated with each model are unique to each model so identifying the implicit assumptions inherent with any methodology becomes a key component in determining how to tailor the methodology to each unique project.  In my experience, few methodologies formally document their assumptions creating a situation whereby PMs must derive the assumptions.  The tailoring concept is also a key as we all know that each project has its own idiosyncrasies that we have to deal with.  When methodologies are treated as a “cookbook” and applied in the exact same manner for each customer, then they can be a detriment to the project.

By understanding the implicit assumptions behind a methodology and tailoring the methodology when 1 to all of these assumptions are not valid, the project manager can effectively leverage the strengths and mitigate any weaknesses of a methodology.

No Test Team: Ideas for System Testing

I recently started a new project in Canada which is an enterprise implementation of a commercial order capture and order management product.  My initial discussions with the PMO went well until we began discussions around the testing strategy where I typically discuss the different testing phases, the number and availability of testing environment, etc.  When we came to the composition and contact information for the system test team, the customer's face went blank...

I discovered this customer did not have a test team only a business analyst.  Needless to say, I was a little taken a back but went to work on how to test an enterprise level order capture and order management system without a formal test team. I also learned there would be no formal test plans either but there would be a user acceptance test used by the business to make the production deployment decision.

My first thought was to hire a testing firm to come in and execute a traditional testing phase, but I do not have the budget for it so needed to be more creative.  

As I am currently in the requirements definition phase, I plan to use this time with the technical and business community to document not only the requirements but also the use cases in a fashion typical of the Rational Unified Process (RUP).  Not only will this enable me to confirm that I have requirements in all areas of the business, but will also provide me an informal set of test scenarios.  My technical team will use these scenarios to guide the integration testing and perform an integration test of greater than typical depth as it will include many fringe cases identified early and typically left to a more formal test phase.  Using the development team to execute on all the test cases also will flush out aspects of the system configuration that may have been overlooked.

After meeting with the business analyst, I have good confidence that she is well versed in all aspects of the business.  The issue becomes finding time for a single person to effectively test a system of this size. What I came up with is three fold strategy 1) apply the agile development approach to a commercial software implementation and provide incremental releases of functionality at the soonest possible time and build the system over time, 2) enhance unit and integration testing to include all the use cases collected during the current requirements phase, 3) engage my off shore performance testing group.

As is customary with agile custom development, the superset of functionality is divided into small testable releases.  These releases build on each other and the team get into a rhythm whereby a group is designing, a second developing the next release and a third testing what has been delivered.  The entire solution is build one releases at a time with the development and testing time periods overlapping.  The net result is a longer test cycle. The testable release concept is also applicable to commercial software as subsets of functionality are configured vice coded.

I typically start with the initial activities in the business process under development.  In this case, the authentication of users with Active Directory and all fringe cases such as invalid password, password resets, and password changes.  As this is an order capture system, the next step will be capturing orders but not sending them to the order management system, no accounting entries, or inventory transactions. This additional functionality is added in subsequent releases in the order of the business process.  The final steps in the business process and therefore releases are reverse logistics.

This approach will extend the test cycle from weeks to months and allow the BA the time to focus on smaller sets of functionality as they roll out.  The additional time comes from overlapping the development and testing cycles.  A formal system test phase can also be added where the group does exist.  The performance testing group is off shore so I can get them cheaply to cover the potential issues around performance and effectively isolate the sub systems where the performance is below expectations.
Given the integration testing is enhanced and included the end to end scenarios, I will engage the performance testing team in an incremental fashion as well thus providing the technical team a longer lead time to address any identified performance issues.

The risks in this approach are:
    1) The interfaces which the customer is building may not be available in a time frame to support the extended testing cycle
    2) The fringe test cases not identified during requirements definition may be missed until the UAT
    3) The stabilization period after the production date may be extensive and have a greater than tolerable error issue rate given the less than ideal testing
    4) I am relying on the testing expertise of a single individual and my development team where neither group are testers by profession

Below is what I plan to execute and will write a follow up post in the summer of 2010 on how it went.

Requirements Documentation- Let me Count the Ways

As I finish preparations to begin a new project, I have spent some time discussing several methods of documenting requirements.  More specifically in a requirements collection phase, at what point and with what audience do we put pen to paper with the deliverable being a requirements document.

The discussions in which I have been engaged in the last week have defined four basic scenarios that can occur, each with their own advantages and disadvantages and degrees of passion about the method.

     Conduct all meetings first and document later

In this method, the project team conducts the days or week worth of meetings, meeting minutes are kept and published.  After all meetings are conducted, a subset of the team begins the writing.  The writing is typically accomplished by a business analyst but any team member(s) can also accomplish. 

The advantages of this method are: 1) decreased time defining requirements so less business user time, 2) complete focus on requirements without need to discuss verbiage, 3) during writing, rest of team can accomplish other startup tasks. 

The disadvantages are: 1)  longer approval cycle as requirements wording is left to the end,    2) results of discussions may be lost.  This is a greater risk the longer the requirements meetings occur,  3) writers must spend time initially preparing and then reviewing meeting minutes to write final document.  Initial sets of meeting minutes are non-value add as they are duplicative to the final requirements document.

     Conduct meetings and document weekly

In this method, the meetings are conducted for a number of days during week which is typically either 3 or 4 days.  The remaining 1 to 2 days are used to document the results of the week.  A subset of the team is typically used for the actual document preparation and is well suited for a business analyst. Typically, the emerging document is reviewed weekly and prior to beginning the new week of material.

The advantages of this method are: 1) all parties can focus on requirements definition without overhead of documentation, 2) writers have dedicated time to document at the end of each week, 3) less delay between definition and documentation so time lapse issues are less

The disadvantages are: 1) some time lapse issues and verbiage issues remain, 2) time to define requirements is extended as time is spent dedicated to documentation, 3) time lapse issues created additional uncertainty on the approval cycle length and thus the associated schedule risks, 4) end user subgroups may be required to attend sessions split across disjointed weeks.

      Conduct meetings and document daily

With this method, the writers keep up with the daily definition of requirements and document said requirements during the evening hours.  This enables up to a 5 day workweek to document which may be useful when limited time is available either all together or with a particular end user group.  A business analyst remains well suited to the writing tasks but technicians with good functional skills can also accomplish easily. Typically, the prior day's language is reviewed prior to beginning the day with new material.

The advantages are: 1) lag time between definition and language agreement is substantially smaller eliminating the loss of agreement over the lag time, 2) the team is better able to monitor progress as the definition and language are agreed upon in a more incremental fashion, 3) individual subgroups of users are needed for a confined time as they are not required to come back at a later time to reach agreement on language.

The disadvantages are: 1) writers are required to put in additional hours at night and will loose effectiveness due to fatigue, 2) the days can be disjointed as language is reviewed at the beginning of each day causing greater stress on the end users, 3) the time lag of even a day can result in loss of agreement on the material from the previous day.

     Conduct meeting and build document in real time

In this final method, the requirements document is populated in real time as the requirements are defined.  All parties agree review the language on either a PC projector and/or via a web-based conferencing solution such as NetMeeting or Webex.  As individuals articulate the requirement, the writer types the new requirement into the final deliverable.  If the need for diagrams is identified, the writer can shift to a presentation graphics or business process flow application such as PowerPoint or Visio.  In some cases, PMs may decide to have a short document review after the sessions are complete, but great care should be taken to ensure topics are not reopened.

The advantages of this method are: 1) progress is incremental and easily tracked against the available schedule, 2) once all material is covered, the document is ready for signature and all parties have agreed on the language, 3) no time lag is created between requirement identification and documentation, 4) each end user subgroup is needed for only a single period of time vice coming back for a document review.

The disadvantages are: 1) writers have great pressure of them to either type in real time while interpreting the requirement or creating a diagram in real time, 2) end user may feel additional pressure since requirements are documented in real time vice having some number of days to review after documentation is complete.

In summary, I prefer the build as you go method as it provides a better gauge of progress, keeps all project team members focused on the goal of completing the deliverable, and typically has a smaller approval tail then the other methods.  The key assumption here is that parties are held accountable to the verbiage written as they go along and wholesale revisions are not made at the end.  This risk is further mitigated by publishing the draft on a daily basis.

Program Manager Role in License Sale

In course of selling commercial software, there will inevitably come a time when a potential sale begins to slowdown or even  stall.  This can occur for many reasons including:

1. Uncertainty of internal risks of large project
2. Uncertainty of load of project on internal staff
3. Uncertainty of how project tasks are inter-linked
4. Uncertainty if vendor is capable of successfully completing the project
5. Uncertainty if services staff understands the business and can model the business within the tool

For a IT executive, these items can slow or even prevent a software sale.  A Program Manager can help by briefing the executive staff on these items or any other concerns expressed by the prospect.  The software sales team can leverage the Program Manager in this capacity to close the sale as the Program Manager can build confidence within the prospect that is difficult for the sales team to achieve.

Given this opportunity in my past, the following item have helped me.

• Brief an implementation/development methodology including key decision points and risky aspects of the implementation

Whether the project is a commercial software implementation, custom development project or any combination thereof, the PM can begin with the overall approach to the project.  Start with a template or shell project plan and define the major phases including the major objectives and activities in each.  Keep the conversation at a high level and focus on the flow of the activities and their relationship to the project objectives.  The staffing needs, key decision points, and risks will naturally fall out of the discussion.

Identify the activities where deliverables are produced and where intense business involvement may be required.  The initial requirements gathering sessions are a typical example.  Describe how these activities have been conducted in the past, highlight and stress the interaction between the vendor staff and the technical and business staff of the prospect.  This reiterates the consultative nature of the activities and that the prospect will drive the end product.

• Focus on the non-technical aspects of the project including communication, training, testing, risk management, and change management

In my experience, many of the concerns come from the non-technical areas of the project and include: How will we manage change?  Will we come in on time and within budget? How will we ensure the prospect staff is trained and able to operate the solution once built?  How will the solution be tested?  How will data be converted from the legacy systems?

Walk the audience through the project plan template and highlight these areas in a separate discussion from above.  This will demonstrate the completeness of the template, sensitivity to the long term operation of the solution, and and also enable the audience to focus on individual areas of concerns outside the context of the technical aspects of the plan.

• Present multiple options for business related issues without committing

Always present multiple options for consideration as the options will be discussed and that decisions are made during the requirements and design phases of the project.  Attempting to make them in this timeframe is premature until all topics are thoroughly discussed.  Having multiple options to consider demonstrates an overall understanding of the industry and that the project may uncover additional business improvements not previously identified by the prospect.

• Present business examples directly related to the prospects business and how they were solved previously

In preparation for the meeting, work with the sales team to understand and identify multiple business examples from the prospect's current environment and that are relevant to the project scope.  The audience will relate much better to examples from their business rather than more general examples. If there are no known examples from their business, keep all the examples within their line of business as a retailer will likely have difficulty relating to a business example from a financial services company.

When presenting the business examples, focus on the solution to the problem and ideally the conditions under which the decision was made.  This helps the audience identify the problem solving and consultative skills of the Program Manager and how the teams will interact during the implementation.

• Explain the rationale behind any preferences

During the project plan template discussions, the Program Manager may have desired implementation options, techniques, or tools.  When identifying any constraints such as this, explain the rationale behind these preferences and how this preference has prevented issues in the past.  This level of transparency highlights the thought process behind the decisions and helps the audience understand how decisions will be made on the project.

• Questions from the executives typically highlight areas of concern

During the presentation, questions will inevitably be asked.  These questions tend to identify the areas where the audience is concerned and provide insight into the areas to focus the presentation.  The Program Manager should spend any time necessary to address these questions.  The main presentation is of secondary importance to addressing these questions.

Have a second services individual in the room with you as well to read body language.  This person can also help steer the conversation to address additional areas of concern, provide additional business examples, or additional options to consider.

In Summary, the Program Managers can be very helpful in closing business when the prospect has implementation concerns or questions.  By introducing to the prospect the individual who will be with them throughout the process, the sales team and PM can build the necessary confidence to close the sale.