Chris Nandor, Principal Software Engineer
At Marchex, technology is our core. It’s foundational to everything we do, from driving phone calls to advertisers to understanding, deeply, why an ad campaign does – or doesn’t – work.
One of the most important and dangerous questions I can ask myself when digging into an issue is “What went wrong?” It’s a loaded question, and the answer can lead you down some ridiculously wasteful paths. But asking this and other hard questions is necessary to fix problems and ensure we don’t continually make the same mistakes.
We have more than 130 engineers working at Marchex to keep our technologies running at optimal levels. But what, exactly, does maintaining and developing our systems entail?
A lot of it is front-end detective work. For instance, if there is a misconfiguration, was it due to human error? Can we come up with processes or code to help solve this? If it was a bug, how did it happen? Should we have had additional acceptance criteria or tests? Did it work previously, but we broke it because we didn’t have tests?
Engineers want to dig, we want to explore how and why things work. We think of problems as opportunities for discovery. And I’d like to say, “don’t be afraid to dig deep,” but you should be afraid, because digging also means bracing for some harsh realities. Engineers have to decide when it’s appropriate to start dismantling things. Don’t go there unless the problem is systemic and requires it. The investigation itself – let alone what it may uncover – may make people defensive and angry, sidetrack more practical work and generally upend much of what you’re trying to accomplish. The image of using a sledgehammer to drive a thumbtack into a wall comes to mind.
That doesn’t mean you shouldn’t push forward. Why? Because there’s always a possibility you might be able to solve an important technical problem – and perhaps, other underlying issues – without as much effort as you thought.
Being agile doesn’t mean sidestepping the thorns. It just means asking the right questions and being prepared to handle the answers.
If you need to get out the sledgehammer and start wrecking stuff, do it.
Chris has been working at Marchex since 2010. Previously he worked at Geeknet as a Senior Programmer.