Client: Noibu
Timeline: April - May 2022
Roles: Lead Product Designer, Research, Visual Design, Testing
Prototype: Source Maps
Within Noibu, every error we flag has a details page that contains data and information regarding said error. The details page has a section dedicated to the developer, including a stacktrace made up of a minified version of the original code.
A stacktrace is a timeline of steps leading up to and from where the error occurred. The downside is that the code is in an unreadable minified format. A source map is a guide that helps third-party services–like Noibu–navigate through the labyrinth of transformed and minified code to the original source code. For this first version, a late decision was made that the source maps would have to be manually uploaded until we had the resources to work on an API.
We provide the developer with an unreadable minified version of their code. Only using Noibu and this minified code makes it nearly impossible to determine what caused the error. To begin their investigation, the developer must leave Noibu, enter their codebase, and search for where the error occurred using different attributes from the minified code.
Our data shows that customers who use stacktrace thirty-five times or more have a 94% chance of remaining a customer. If they fall below a threshold of only utilizing the stacktrace nine times or less, that percentage plummets to 20%.
How do we make life easier for that 94% group and then incentivize the 20% group to start using the stack trace? All while fulfilling their needs.
During the kickoff meeting, it was still up in the air whether our Engineers would be building an API or we would ask the developer to upload their source maps. We are at a crossroads. If they went the API route, there wouldn't be much design needed, maybe a light facelift to the UI (assumption). On the other hand, if they went the manual upload route, it would be the exact opposite.
We spoke to a few of our Engineers a-sync to get an idea of how much work it would take to build an API. From those discussions, we felt comfortable assuming that the external developers would be manually uploading their source maps. We lacked the resources to build the API. A few days after these conversations, there was a decision to have developers upload their source maps manually.
I had many questions since I was not a developer. To kick things off, I booked calls with our Engineers to try and get some technical answers. Does every frame in the stacktrace require a file uploaded to it? Suppose a developer uploads a source map within an errors details page. Does it only un-minify that code locally or across the entire domain? What happens when they push a release, does that impact what we are showing? Can we track when they push a release? How can we tell them their source map is outdated?
I received some answers, but needed to speak to some external developers. I talked to twelve developers and asked them all the same open-ended questions.
Our main findings were:
The API would provide the most significant value, which we all assumed.
They all had different release schedules. Six developers adhered to a strict bi-weekly code push. Three would push code only a few times a month, and the other three whenever necessary. However, I did notice a pattern for our customers that outsource their dev work; those seemed to lean towards a stricter release schedule.
It would not be an annoyance for developers to upload their source maps.
Developers would most likely forget to upload new source maps after pushing code.
One developer informed me that some companies have public source maps.
I took this information into my design.
For the design, I knew that:
Source maps had to be designed so that when the API was ready, we could seamlessly transition the user into a fully automated experience.
One uploaded source map could unminify code across the entire domain.
I needed to figure out a way to indicate to the developer that their source code might be outdated. There was no way of automatically advising at this time. I leaned into the data and decided that a warning would appear two weeks after an uploaded source map.
Visual indicators for minified, unminified, public source maps, and error states were all needed.
Keep the current meat and potatoes of the stack trace design the same, but add value and a slight facelift.
The real highlight for me on this feature was the ability to build something specifically for our developer persona, whom Noibu has unfortunately overlooked for some time. Most of Noibu is catered towards product owners, providing them with the errors, the data surrounding it, and how much revenue they will lose if it isn't fixed. They get to show their superiors how much money they saved their employer.
On the other hand, we have a developer who may have written that code, which is now costing the company a large sum of money. Then we provide them with limited information on how to fix the error. We do a poor job of setting them up for success.
The feedback we have received from developers on source maps has been overwhelmingly positive. The one complaint we often hear is that our warning indicator is sometimes wrong, which we anticipated.
I feel comfortable calling this version of source maps a success. A fully automated version wasn't feasible then, but we still managed to provide the developer with value and fulfill needs.