Writing bug reports should be easy, the hard part should be reading them.
Earlier this year, I read a really wonderful post by Peter Steinberger on writing good bug reports. This post reached me at a really personal level, for a bunch of reasons. First and foremost, it is discussing a topic that’s near and dear to my heart. It may seem like bug reports are a funny topic to care so much about, but I do. I’ve been writing software for a long time, and it really matters to me when someone using that software has a problem. The post also focuses on reporting bugs to Apple. I’ve been fortunate enough to see radar from the inside, so I feel a connection to this kind of report in particular. And, finally, as if all that wasn’t enough, it lead with a quote from a friend and former coworker (Hi Tanya!).
I want to say that I think Peter’s recommendations are excellent. He has a great take here on how to increase the probability of getting the desired outcome from a report — a fix. However, I cannot help but feel sad that Peter had to do this. I sincerely wish that his post wasn’t necessary. The outside world should not have to rely (partially) on social engineering to get attention to their issues. So, I though it could be helpful, especially since I was once on the other side of these reports, to offer a complimentary take.
I’d like to start with a story — back to my time at Apple.
For a number of years, a regular part of my day was to triage and investigate iOS battery life radars. These came from all the sources you could imagine — external developers to internal Apple folks. That included Apple Store employees. One day, I received a novel of a bug report from a Store Genius. This person was attempting to help a customer, who was experiencing poor battery life after updating to a newer version of iOS. First, they followed all procedures to the letter, including hardware and software configurations, the customer’s own account of the issue, and well as observations they’d made. But, boy oh boy, they went further than that.
They also grabbed a number of difference devices, installed the new and old OSes on them, and ran experiments. They attempted to measure the battery level changes over a period of time, while emulating the setup the customer had. They included a spreadsheet of their results, along with some requests for additional instrumentation that would help them produce better test results. The tests were run over a number of days, possibly even a week if I remember correctly.
I was just blown away by the effort this person had put in, ostensibly just to help one customer. I was proud to see that kind of dedication, and also felt for them. I’m sure this customer was frustrated, and I can only imagine that at least some of this testing was done on personal time. But, I also couldn’t help but feel sorry for this poor Genius, bending over backwards here.
Now, this was a while ago, so I cannot remember all the details. But, I do recall that had I narrowed down the issue to maybe 2–3 bugs almost right away. And, I was able to do that just via the customer account + configuration. The whole time I’m reading this report, I’m getting more and more amazed at how much unnecessary info is there. All that work, testing, and writing. All I really needed to know was something like “customer has both Exchange 2007 and 2010 accounts, and is running iOS 5”.
Of course, this Genius couldn’t have known that. But, had he just submitted the bug with the minimum info, I could have saved him a bunch of effort and addressed that customer’s issue way faster.
This account was just one of the countless interactions I had via Radar. Of course, there were less positive experiences that shaped my thoughts as well. But, I think this story speaks to many of the approaches I now use when dealing with bug reports. I’ve used many different systems. I currently use no tracking system at all, and I highly recommend it. It took me a while, but I now look at bug tracking systems kinda like I look at UML diagrams. <ducks>
Here are the things I now do/think about when dealing with a bug report. Some of these are contentious. Some conflict with each other. This are just my guidelines, but they have served me very well. If you have a different approach that works for you, I’d love to hear about it!
My Guidelines
Consider every report as a gift
Feedback is incredibly valuable, in all cases. Here, it is an opportunity to make your product better AND impress a customer. Tell me there is something your time is better spent doing than that.
Try to respond to every report
I happen to like it, but I know that not everyone enjoys interacting with customers. Find a way to respond in some fashion, if you can. I have a theory that it’s bad for business to ignore customers.
Do everything you can to encourage reports
Your software is riddled with bugs. You don’t need a fully-reduced, reproducible example to know that the stupid table re-ordering bug is happening again. It’s your job to debug, not your customers. Raising the bar too high will just prevent you from getting reports.
Ask for clarification, even if you’re sure you know what’s up
You’ll rarely get all the info you need in a report. Instead, just ask for more info. You have all the domain knowledge anyways, so you know best how to narrow things down. Worst-case, you get no answer. This can also be a good technique for helping customer’s realize that it really is a feature and not a bug :)
Of course, your customers may not always be willing to help. Consider taking some time to brainstorm on better tooling and/or diagnostics. A bug report template is the lowest-common denominator — you can do better.
Save your responses
Over time, you’ll find you have to respond in similar ways over and over. Save your responses, so you can refer to them and reuse them if needed. Also makes for great knowledge-base fodder.
I’ve never been great at this, and I always regret it.
Don’t laugh off low-priority bugs
Sometimes, I find it hard to even take reports seriously. Obscure use-cases, esoteric configurations — these are the kinds of things that tempt you to just hit delete. But, you have to keep in mind that to this customer, this problem was important enough to take the time to write this all up. Give it the respect it deserves, even if that is just a “I’m afraid that’s not in the plans right now”.
Never ignore a rude report
Believe me, I know. This is the hardest thing to deal with. I’ve lost sleep, many times, over rude feedback around my work.
My general approach is to get especially technical and thorough when someone is being belligerent. I find it can be quite disarming, while also helping to make sure I spend the time to really know what I’m taking about. It tends to be times like this when I’m most likely to make mistakes and say something wrong/dumb. Go slow, and get it right.
Value The Feedback
Dealing with bug reports can be really draining. Especially when people are rude or mean. But, it’s also an opportunity to learn about your work. Your software has all kinds of bugs, everywhere, and customers are seeing them. Only a tiny fraction are going to go through the trouble of reporting them. Find a way to make sure they feel great when they do that. Do something, because your product is your responsibility and no one else’s.
There are people out there willing to work for free to make your stuff better. All you have to do is listen.
Hacker Noon is how hackers start their afternoons. We’re a part of the @AMIfamily. We are now accepting submissions and happy to discuss advertising &sponsorship opportunities.
To learn more, read our about page, like/message us on Facebook, or simply, tweet/DM @HackerNoon.
If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!