10 Formal Processes

Most of what you do as the business analyst and requirements documenter will be unstructured time and informal processes, but there will be some scheduled time and formal processes, too. Below, many of those are listed.

10.1. Non-Agile

To a great extent, “non-Agile” means Waterfall, but it can also mean formal processes before or outside of Agile processes. Some of these processes are outlined below.

10.1.1. Request for Proposal – RFP

RFP stands for Request For Proposal. It generally means that some large company or a governmental agency wants a project done. They will issue a Request for Proposals, outlining what the project has to be or do, and it’s up to the bidding team to produce a proposal to answer the request.

10.1.2. Proposal

The proposal in response to a RFP is not just a quote. Often the RFP is fairly vague as to exactly what has to be done, instead just laying out the need that needs to be handled. For instance, recently NASA announced that they want to return to the moon in a new spacecraft, and issued an RFP to that effect. They didn’t give more detailed requirements than that, and it was up to the people writing the proposal to determine the exact requirements and how it would be done. When you get an RFP, you need to respond with a proposal (if you’re going to respond at all), and this will usually take some amount of requirements engineering to determine what will be in the proposal, and then what that proposal will be priced at.

10.1.3. Bidders’ Conference

Some time after the RFP and before the proposal is due, there will often be a bidders’ conference. In the bidders’ conference, you and the other prospective bidders are allowed to ask questions, in front of all the other bidders. So, there’s a strategy involved. You want to get your questions answered, but at the same time, you don’t want to tip your hand to the other potential bidders.

10.1.4. Request for Quote – RFQ

RFQ means Request For Quote. The company or agency will issue a RFQ when the requirements are already well-known (usually because they issued the RFQ with a complete set of requirements), and they just need a numeric bid. If the requirements are not well known, or are ambiguous, be careful.

10.1.5. Quote

Of course, the response to a RFQ is the quote. Often, that’s just a numeric bid, but often other information is required.

10.1.6. Change Request

In a time-and-expenses contract, you’ll get informal change requests all the time. You just handle those without a second thought.

But, in a fixed-price contract, you’ll often be constrained to a formal process, because the contract will specify that. Normally, the process starts with the customer issuing a formal change request. That change request may be in the form of changed requirements, but it may be in some more vague terms, and will need to be translated into proposed requirements changes. You would reply, showing them those proposed requirements changes, and with a proposed price change, if any. If there is more work to do, or work that would need to be redone, there will generally be an increased price. But, if the work to be changed is only future work, and there will not be added costs, then you may or may not want to charge them a fee. If the change request will somehow actually save you money, then you would just send them a note that their change request has been accepted as a change order (see below).

10.1.7. Change Order

Once the customer has accepted your bid on their request, they’ll issue a change order. That means to go ahead and make the change.

A change order will cascade into requirements tracking, in that some requirements will become moot, some requirements will change and some requirements will be added. In very formally controlled projects, there will be a requirements traceability matrix. In this matrix, one coordinate is the list of all the requirements, usually by ID and a short description, and the other coordinate is a list of controlling documents. A requirement present and unchanged since the onset of the project would only trace back to the initial document, but a requirement that was modified by two change orders would show that three documents control it.

This brings up a slightly broader subject: Who logs, tracks, and reports on the progress through the requirements. Sometimes it’s the project manager (PM), sometimes it’s the lead business analyst (BA), and on some Scrum projects, it’s the Scrum Master. On smaller projects, those two or three positions will be the same person.

10.1.8. Engineering Change Proposal – ECP

Sometimes the producing end of a fixed price contract wants to make changes to the plan. When this happens, because you found a better or cheaper way, or some component becomes unavailable and needs to be replaced, you’ll develop a set of changed requirements, and issue an Engineering Change Proposal (ECP) to the customer, which they’ll either accept or reject. If they reject it, you just keep making whatever you were making. If they accept it, then the new requirements lock in.

10.1.9. Progress Reports

Often, progress reports or intermediate deliverables are required. Even if they’re not required, it’s generally a good idea to send them to the customer. In your progress reports, state what happened in the last period, and what is planned for the next period, and any problems you’re up against. There are some pluses and minuses to this: You don’t want to give them too much information, but at the same time, you don’t want to give them too little. Customers don’t like surprises, so use the progress reports to prevent surprises. Unfortunately, it’s impossible to give more specific advice here.

10.2. Agile

Where non-Agile projects tend to have documentation and notices, Agile projects tend to have events, often called “ceremonies”. Many of those are detailed below.

10.2.1. Sprint Planning

As mentioned above, a sprint is the unit of time on an Agile project between versions, or at least the period of time on which assignments are handed out. This is typically a week to a month.

In each sprint, the team will plan on accomplishing a certain amount of work. What work will be done during the sprint is generally worked out with the project manager, the scrum master, and a business analyst. Sometimes this sprint planning is done with the whole team, and sometimes not. Generally, during the sprint planning session, points, sizes, or estimated hours are assigned to each task to be done during the sprint. Tasks can be sized any time after the requirements have been documented, so a task may be sized but not included in the sprint. Assignments of tasks to people is then done, either in a Scrum session, or through a Kanban board.

10.2.2. Daily Stand-up

All Scrum-based projects, and many other projects, have a Daily Stand-up meeting. It’s called “Stand-up”, because the intent of the meeting is to be a quick recap of yesterday’s events and the plans for today, with the idea that if everyone stands for the whole meeting, that the meeting will tend to take less time than it would if people were seated.

10.2.3. Retrospective

When the sprint finishes, there is a Retrospective meeting. In the Retrospective, there is generally a discussion of what went well, what didn’t, and how things could be improved for the next sprint, as a means of process improvement.

10.3. Other Forms of Tracking and Paperwork

There are sometimes additional types of work done in tracking a project. We won’t claim that each of these is always useful to a project, nor will we claim that each of these is always a waste of time. Each of the following have their purposes, advantages, and costs. Although there are a lot of management-related charts, graphs, and lists done, this section will only deal with those pertaining directly to requirements engineering.

10.3.1. The RACI Matrix

RACI stands for Responsible, Accountable, Consulted, and Informed. The idea here is that you (or someone in management) will draw up a spreadsheet. One dimension will list these four classes of stakeholders, and the other dimension will list each of the project phases and components. So in concept, the matrix is really three dimensional, but two of the dimensions, phase and component, are generally listed from top to bottom. Within each cell is a list of the people or positions who fulfill the attribute for the phase and component. You’re “doing it right” if you have one or more people listed in each cell.

The advantage to doing this is that you know that you’ve covered your bases of who needs to be talked to. The disadvantage is that if this is all obvious, there’s no point in doing this.

10.3.2. PERT Charts

PERT stands for Program Evaluation Review Technique, but almost nobody knows that or cares. What they know is that a PERT Chart will show, visually and clearly, what relies on what. It’s an excellent way to do schedule planning no matter what the management style, and there are a lot of programs that help you to product this chart. The idea is simple. You have a number of rectangles (or rounded rectangles) that represent tasks. Very often, a task is the implementation of a requirement, function, or user story. But, they can also represent tasks, such as flushing out the requirements for a user story, theme, or epic. The box will be labeled as to it’s title or requirement ID, in the center. Once people and/or equipment (collectively, “resources”) are assigned to the task, that assignment appears under the title. The left side of the rectangle is called the “input”, but not in the programming sense. Anything that has to happen before the task can be started is the tasks “inputs”, “preconditions”, or “blockers”. So, the leftmost rectangle will be the project kickoff, and it has no inputs. The right side of the rectangle is called the “output”. The outputs, or “reliances” are connected to any task that requires this task to be done before they can start. So, the rightmost box will be “Done”, and has no outputs. The boxes are connected to each other in such a way as to indicate what relies on what. By assigning time to complete, man-hours, level of effort, or “points” (the kind you can add up) to each box, you can get an estimate of the work remaining by adding up the remaining boxes. The path from left to right that has the highest score (hours, points, dollars, or whatever) is the “critical path”, in that any delay on that path will delay the entire project’s completion. Often color is used to denote Blocked, Started and on pace, Started and running late, and Completed, often with some sort of bolding to denote the critical path.

In any nontrivial project, tracking what has to be done next, and knowing the critical path is very important. The upside of this far outweighs the downside virtually every time, unless no component or task relies on another.

Image by “Rm w a vu”

10.3.3. The Keeping of Statistics

A lot of projects keep a lot of statistics and benchmarks. These may include how many requirements each person gathers over any given time, how much of their work is reworked, how many ambiguities are found in their work, how complex areas are, the average size of each story or function, and the like. There are also an even larger number of statistics that can be kept on the programmers and QA people. Having gathered these statistics, you can find various indices, such as the requirements quality index, the productivity index, and others.

On the plus side, you can find out about how the organization is really working, find out everyone’s strengths and weaknesses, make better predictions about the next project, and improve the process. On the plus-and-minus side, you can get rid of whoever is not producing. On the drawback side, you can find yourself spending an inordinate amount of time gathering and calculating these values, and you can purposefully or inadvertently make people feel so much pressure that they’ll burn out and/or leave.

10.4. Decommissioning of Obsolete Requirements

In an Agile process, things change all the time. That will mean that some information is refined to a better depth of understanding, some things will flat-out change, including from a No to a Yes or a Yes to a No, some requirements will be added, and some requirements will be deleted. When a requirement changes in a Waterfall project, it’s much easier to notice, because there’s a formal process for change management. But either way, you’ll be left with a different set of requirements than before.

• An added requirement is the easiest to deal with. You just add it to the backlog in an Agile project, or in a Waterfall project, add it to the task list (in the specification document, in the requirements traceability matrix, and in the PERT Chart). Eventually, someone will be assigned to it. Be sure to make any documentation changes using some sort of change tracking, so that change bars appear in the document at least, and to note the change in the version history log.

A modified requirement is a bit harder to deal with, as there are a few possibilities:

• If a requirement is modified before work has begun on it, then only documentation changes to the backlog or the requirements document need be made.
• If a requirement is modified once development has started, and before QA is started, then in addition to the document changes above, the developer must be notified immediately. Scheduling adjustments will also most likely have to be made.
• If a requirement is modified once QA has begun work on the affected item(s), then in addition to the document changes above, both the developer and QA must be notified immediately, so that QA can stop their work, and so that the developer can either begin the rework or at least schedule it.
• If a requirement is modified after QA has signed off, then all of the above holds, and one more more components may have to be recalled, and the release schedule may be affected.

The deletion of a requirement might involve more work than you might expect:

• When a requirement is deleted (often called an obsolete software requirement or OSR), the first step is documentation. Never just remove the items or requirements from the backlog or the documentation. Edit the documentation, usually using strikethrough text (“like this“) or a shaded background (“like this“) to indicate that the requirement is known but no longer valid. The reason for this is to prevent the deleted requirement from being “rediscovered”, which is a strong risk. Typically someone will come up with the rejected idea too, and add it, or find that the deleted requirement is “missing” and add it back. If the requirement is still in the document, showing as deleted, then you won’t get into the trouble of accidentally reinstated requirements.
• As above, if the item is in development or in QA, it must be stopped for rework.

If a deleted requirement has already cleared QA, then it must be categorized:

• If the deleted-but-completed requirement is still useful, but was only deleted because the cost was no longer deemed worthwhile, then the work-product of that requirement falls into the “you bought it; you own it” category. The feature is no longer required, but it’s still a feature. Remove it from the requirements set, but leave it in the user documentation and in the product.
• If the deleted-but-completed requirement is not useful, but is of no direct harm, then schedule the code behind it for removal, but at the absolute last priority. The reason to schedule the code for removal is that in the product’s life cycle, added code complexity means added costs on maintenance, if for no other reason that people have to consciously ignore it, But, if dead code ends up being maintained, that’s an unneeded cost. It’s cheaper to remove the code at the end of primary development than to maintain it later.
• If the deleted-but-completed requirement uses any significant amount of system resources, such as screen-space, power, memory, disk, or CPU time, then the story is the same, but the priority on removing it goes up commensurate with its load.

10.5. How Things Can Go Wrong

Skipping a formal process that’s contractually required can become a compliance, contractual, and money issue. Skipping any regular periodic step from time to time can cause confusion in the staff. Skipping the PERT chart can lead to (a) resources (mostly people) going idle, and/or (b) not understanding the critical path. Both of which can cost time and money. Keeping too many statistics is a waste of time, and keeping too little can mean that you don’t know something you need to know, so know what it is you need to know, and plan for that. Keeping obsolete requirements active in the system will usually mean too much maintenance cost.

10.6. Discussion Questions

1. Have you been on any projects where any of these techniques have helped?
2. Have you been on any projects where any of these techniques has been a hindrance?
3. Of these techniques, which seems most useful (if any)?