Every once in a while we get close to releasing a feature and then, at the last minute, sometimes even after we’ve announced its coming, we pull the plug on the release. Why?
The extended version and variations of the ÒWhy?Ó questions include ÒHow can you get so close to being done and then find a problem so critical, you delay giving it to customers?Ó and ÒWhy does it take so long to build (X)?Ó
To answer those questions I need to take a bit of this article to outline the process of building a feature:
-
Research: What is the problem we are solving? What are all the ways that someone might use this and will it solve all the use cases?
-
Design and Testing: Once we have the requirements from the research phase, we need a design. Usability testing the design with customers tells us whether our proposed solution both solves all the key problems and ensures it’s easy to use.
-
Coding: Developers write the code!
-
Code Review: Another developer reviews the code for stability and functionality
-
Quality Assurance: The quality assurance specialist then reviews it and puts it through an extensive battery of tests
-
Usability Review: The user experience team does a final review to make sure it matches the design specs.
-
Internal Testing and Beta Testing *: we make the feature available for staff and a limited number of customers to try it out.
*ÓBeta testingÓ is an abbreviation for what can be a lot more complicated; pre-alpha, alpha, closed beta and open beta testing, labs and slow releases, etc. …but let’s keep it simple for the sake of brevity.
When we start talking about a feature getting close to release, we’re usually at least as far as the QA phase of the project.
For those of you who have been around the Unbounce community forums for a while, you might have seen some threads that are are over a year old talking about features that are coming soon. You might also have noticed that we do that a lot less. By talking about features we are planning to build in too early (e.g. the research phase) we are often a long way from it actually being shippable. To better set the expectations, we now only talk about features when they are much further along.
Ok, I’ve digressed a bit. Back to the questions!
Sometimes we get deep into the development cycle, we’re almost done and we’ve started talking about a feature. If all goes well, (and it usually does) we ship it. However, there are times where we’re in the final innings and we uncover a bug. We have to stop the release and send it back to coding to fix it. Then the process begins again, following every step in the process. This is not uncommon. Software is complicated and we take pride in releasing good, well functioning software. This is actually the least expensive type of problem we can face.
A more expensive mistake is a design flaw. One where the feature does solve the problem, but it’s hard to use, unintuitive or kludgy. These will make it through the coding process because they meet the design spec but we might pull the release and redevelop if our internal and beta testers give it the thumbs down.
The worst case scenario is if we make a mistake at the research phase. Those are the most expensive and time consuming types of mistakes and the ones that keep me up at night. A mistake at this phase means we didn’t fully understand what we should be building. The developers, designers, and QA testers won’t pick up on it - it meets the specs. This means having to go back to the drawing board and reassess every decision.
Part of the test plan for adding one new button to the page builder. Shit’s complicated yo.
Smooth scrolling (Not yet built, we’re still early in the research and design phase)
So what does this mean? Testing, testing and more testing helps us release better quality software - but because of the inherent complexity, sometimes we find issues at the last minute and have to pause and regroup.
I hope this has been helpful in some capacity; if you have any questions about our Q/A process, or want to chime in with how your company handles Q/A, be sure to fire them off in the comments below.