‘Test, Learn and Test Again’: How 3 Software Engineers Overcome Technical Hurdles

Software engineers from CarGurus, Vicarious Surgical and more know that the most complex technical challenges often present the greatest growth opportunities.

Written by Olivia Arnold
Published on Jan. 31, 2023
‘Test, Learn and Test Again’: How 3 Software Engineers Overcome Technical Hurdles
Brand Studio Logo

Jeff Forsyth recently faced a unique challenge at his job. He needed to find a way for doctors to control three robotic arms — all at the same time — during surgeries.

Two robotic arms would have been “trivial,” Forsyth said. After all, his embedded software engineering team at Vicarious Surgical is used to mapping two robotic arms to a surgeon’s two arms. To make it work with three robotic arms, however, Forsyth and his team needed to get creative. 

“For our embedded software engineers, this meant rewriting the mapping of our hand controller boards to handle the additional button inputs in a short timeframe,” Forsyth said. “The end result is very slick, and it was well received by our surgeon advisors.”

Forsyth is one of three software engineers at Boston-based companies who regularly tackle and solve complex technical issues. By implementing an array of strategies, including adopting more accessible digital tools, collaborating closely with their teams and testing products repeatedly, these featured employees — from Vicarious Surgical, CarGurus and PathAI — have overcome their hurdles and gained impressive technical and communication skills along the way. 

 

The CarGurus team
CarGurus
Andrew Carey
Senior Software Engineer • CarGurus

Cambridge-based e-commerce company CarGurus offers a digital platform for selling, financing and purchasing cars. 

 

What’s the biggest technical challenge that you’ve faced recently in your work? 

CarGurus aims to help people confidently and conveniently buy and sell any vehicle anywhere, with the best selection and price. 

One way that we facilitate the buying part is by creating and deploying digital ad campaigns that alert car shoppers to available nearby vehicles matching their search criteria after they’ve left our site. My first project at CarGurus was to help us migrate this advertising program to the Google Display and Video 360 online ad platform. With no benchmarks in place, we had to do a lot of testing, learning and testing again to define what success would look like and the best way to achieve it.

With no benchmarks in place, we had to do a lot of testing, learning and testing again to define what success would look like.”

 

How did you overcome this challenge in the end? What were some specific tools or technologies that you used?

When I was interviewing and onboarding with CarGurus, people raved about our collaborative, data-driven culture. Weeks later, I experienced it firsthand as I developed a testing protocol for the migration project and shared my work with teams across engineering, product, advertising, operations and marketing.

Our process involved identifying dozens of factors that could influence outcomes — such as ad frequency, time of day and device type — and combining them into A/B test pairs. Comparing these tests using Google’s internal tooling enabled me to find the optimal combinations for reaching car shoppers in an efficient way.

 

How did this challenge help you strengthen a specific skill, technical or otherwise?

This migration project exposed me to extensive data analysis work, which I hadn’t done much of in my past roles as a backend engineer. This is now foundational to my ability to partner across the organization. 

I learned that while having a lot of data can be incredibly helpful, drawing firm conclusions from it is quite tricky, especially if you’re comparing data across different time frames and regions. I think the best tests are the simple ones, especially when the results need to be digestible and concise to drive decision making. That said, you can’t get too attached to findings in the digital ad space because even the strongest takeaways will evolve as consumers’ interests and behaviors evolve.

 

 

The office at Vicarious Surgical
Vicarious Surgical
Jeff Forsyth
Senior Manager of Embedded Software • Vicarious Surgical

Vicarious Surgical is a Waltham-based robotics company that aims to improve the efficiency of surgical procedures.  

 

How do you lead your team in overcoming technical challenges in your work?

As an embedded software engineer at Vicarious Surgical, I am contributing to the development of a next-generation surgical robotics system. 

One of the ways that I help my team is by taking on software development tasks in our software sprints. By continuing to stay hands-on in our continuous integration/continuous deployment build environment, I am able to better relate to the challenges that my team faces. 

By being involved in the day-to-day development tasks, I can also provide the best advice for my team members when they look to me for guidance. We are able to work closely together in collaboration, which often leads to creative solutions to some of our biggest technological challenges.

Working as a manager who still codes can be demanding. We tend to be pulled into many meetings. I organize my schedule in a way that leaves time for me to do development work in the middle of the day, which helps me avoid frequent context switching. 

I also schedule blocks of focused time for development work. Not only does working this way help me be a better manager, it increases the amount of code we are able to produce as a team.

By continuing to stay hands-on in our CI/CD build environment, I am able to better relate to the challenges that my team faces.”

 

How does this way of working help you strengthen your skills and those of your team?

Being a hands-on manager allows me to see the code progressing firsthand and gives me opportunities to challenge its state as I go. Even though I am often a reviewer on code changes, doing actual development work myself means that I am working with different domains, classes and files at a deeper level of focus.

 

 

The PathAI team.
PathAI
Chris Burke
Staff Software Engineer • PathAI

PathAI applies machine learning and AI to improve diagnoses and disease treatment. 

 

What’s the biggest technical challenge that you’ve faced recently in your work? 

At PathAI, we have a number of distinct products with limited cross-product communication, and each product is focused on specific aspects of the market. 

Technical drivers point to a world of more integration and data sharing, so we recently began reviewing what we could do to evolve in support of those needs. We didn’t want to embark on a major rewrite or drop-everything refactor to shared services, so we looked at ways to improve incrementally. 

Around the same time, data science and machine learning teams highlighted that they were facing challenges in discovering interesting data sets and improving existing models based on real-world data from our products. These challenges resurfaced the need for a catalog of data; in turn, that provided an opportunity to begin laying the groundwork for easier cross-product communication and integration for the long term, while solving a near-term problem that will provide immediate benefits. 

The main difficulty faced here was trying to set us up for future success while minimizing impact to the current deliverables of a number of teams.

 

How did you overcome this challenge in the end? What were some specific tools or technologies that you used?

While each product at PathAI has its own data model, the contents of those models are very similar. We don't want to introduce the dreaded core enterprise data model, so instead we are working to define a JSON Schema-based data model for communicating across traditional boundaries. 

Products will then be able to opt-in to translating to this format to make their data available for downstream use. The first consumer of this data will be the data catalog, which begins the process of allowing data scientists and machine learning engineers to find relevant data for their day-to-day tasks more effectively. 

For some of the large-scale backfill of existing data, we are able to leverage extract-transform-load pipelines with Airflow. Moving forward, we plan to support event-driven communication. With this approach, we are able to make progress on the near-term goal of making data easier to find and use via the data catalog, while also proving out use of the new data model for interchange between products and potential future services.

It was important to build consensus while designing the solution, since it has the potential to impact a number of different teams in the future.”

 

How did this challenge help you strengthen a specific skill, technical or otherwise?

This challenge helped grow my ability to communicate and coordinate with various stakeholders throughout the company — from engineers and product managers to data scientists and machine learning engineers. It was important to build consensus while designing the solution, since it has the potential to impact a number of different teams in the future. 

On the purely technical front, this was the first time that I used Airflow, so it was fun to dig into that space and learn about what it offers.

 

Responses have been edited for length and clarity. Images via listed companies and Shutterstock.

Hiring Now
Simply Business
Fintech • Information Technology • Insurance • Software