How to Write an Effective Bug Report
March 4, 2015
I’ve worked for a few start-up software companies in my day, and nothing drives me more bat-shit crazy than a vague, poorly-worded bug report on the software I’m writing. I mention the start-ups specifically because the QA stage of their development process is usually less-defined than more seasoned shops. It’s typically performed by one or more of the designers or project managers who are familiar with how the app should behave. When bugs pop up, they jot a few hastily-conceived words into the issue tracking system and push them over the wire for the developer to make sense of — e.g. I tried to open an image in the app and it didn’t work.
After seeing some particularly poorly-written bug reports lately, I felt compelled to write this piece. Here are a few things you can do to help preserve the sanity of your software engineers — and save valuable time and money in getting your software to market.
Write Meaningful Titles
Try to get right to the heart of the matter in just a few simple words. Is it a crash? Is it a layout issue? Is it an unexpected behavior? Where and when do you see the issue taking place? Here are a few examples of acceptable titles for your bug reports.
- App crashes on submit when sending an activity to Facebook
- User avatars are out of proportion on the discussion thread detail screen
- Unexpected redirect after updating a contact’s details
Keep it short, but make sure it captures the essence of the issue.
Be Specific With Environmental Variables
Whether you think it’s important or not, list all the details of your test environment when logging an issue. This list might include (but shouldn’t be limited to) the following:
- Device model
- Operating system version
- App build & version numbers
- List of connected bluetooth devices
- Network connection details (cellular vs wifi, name of network, existance of a proxy or firewall)
- User credentials used for testing (for apps that limit access via authentication)
Do this everytime. For every issue. Even if you’re using the same testing environment for all the bugs you’re submitting, issues are closed at different intervals and things change over time. Don’t assume the environment will be the same, just include the information in each report you write.
List the Steps to Reproduce the Behavior
When we describe a process, we have a natural tendancy to gloss over the details — to assume everyone is familiar, to a degree, with the situation. While this is nice for getting to the point of a story you might be telling your co-worker at the water cooler, it’s not so nice for the developer trying to understand why her software is breaking. For example, you might be working on reporting a bug that appears while trying to log in. You might start to write that the app crashes on login and may even be savvy enough to include the credentials you’re using when experiencing the problem. But, can you tell me if you used the enter key on your keyboard to submit the login form, or did you click the login button? Was the “Keep me logged in” checkbox checked or not? Did the crash seem to happen at the precise moment the login button was tapped, or was there a pause between releasing the button and the actual point of the crash? These are all important details that the developer will need. If she can’t reproduce the issue, she has no way of telling what the cause may be or even be able to test if her actions have actually fixed it.
Expected Behavior - vs - Actual Behavior
Start out by giving a detailed description of the bug you’re experiencing. This should be a lengthy version of what was stated in the issue title. Avoid vague references and confusing pronouns. Don’t telescope your language to save time — e.g. Tried open image. Didn’t work. Closed and opened. Worked second time. (Closed and opened what? The app or the screen showing the image?). Be verbose, your developer will thank you for it. If written descriptions are not your strongsuit (and even if they are), include some marked up screen shots for visual representation. There are some great apps available for marking up images for exactly this purpose — Bonfire and Skitch to name a few.
Also, while it’s great to get a clear picture of what went wrong, it’s also just as nice to know how you expected the app to behave. I’ve received bug reports on screen layout, for example, that just said, “Tighten up the spacing around the logo.” Tell me, how much should I tighten up that spacing? I’d rather not go back and forth with the designer one pixel at a time until satisfied. Tell me the exact spacing you expect in your layout and I’ll match it. Over and done in one round and everyone’s happy.
How About a F’rinstance?
So, now you should know the basics of bug reporting. Let’s pull that all together in a sample. The following is a fake bug report for a piece of personal finance software. It’s not the definitive end-all-be-all report format, but illustrates the point of how you should organize your information. Formatting in your organization may be different. In fact, consult your software engineers to see how they would prefer to see bug reports. Afterall, they’re the ones who will have to read them the most. At the end of the day, the important thing to remember is to include as much detail as possible in a format that is easy to read.
Monthly financial report expense pie chart is missing the segment for the largest expense for the month
- iPad Air 2 running iOS 8.1.3
- Application version 0.1.2 build #23
- Software keyboard with no peripheral devices attached
- Office wifi
- Use test user credentials email@example.com || testuserpassword1
Steps to reproduce
- Tap the ‘Log Expenses’ button on the main screen
- Generate at least one expense for each of the expense categories for the current month e.g. $800.00 for rent, $150.00 for food, $100.00 for entertainment, $125.00 for commute
- Submit the expense report by tapping the save button in the upper right corner of the expense log screen
- Tap the ‘View Current Month’ button on the main screen
- Observe the pie chart only contains 3 of the 4 segments (food, entertainment, and commute — rent is not displayed)
The app should display a segment in the pie chart for each expense category logged for the month.
The app is consistently missing the pie chart segment for the largest expsense logged in the given month. In the reproduction steps above, that was the rent category. Changing the values so any of the other categories comes in larger will result in that category missing from the pie chart. See the screen shot below.