back
esr_

~/esr_work/stories/testing

~/work/stories

Tester’s Paradise

If your e-mail program freezes, it might be annoying. But if the software in your car freezes, it can be dangerous, even deadly. That’s why an ever-growing team at ESR Labs does nothing but make sure everything works as intended.

/
/

Garages have often been the birthplace of something special, something unexpected, something extraordinary. Many computer companies, from Apple to Hewlett-Packard, were started in garages. As were trailblazing bands from The Who to Nirvana. Walt Disney started out in a garage, and the electric pacemaker was invented in one. The testing garage on the ground floor of the ESR Labs building in Munich is also an extraordinary place where the future is being built. But in this space, new things are checked and verified to make sure they really are ready for the world.

The central area of the garage has room for two cars. If they’re unreleased prototypes – the “Erlkönige” (“elven kings”) hunted by journalists and car enthusiasts – the testing team can close the curtains for perfect confidentiality. “It’s much better to do tests on the cars in here rather than in a public parking garage where it can be freezing cold in the winter, and you have to be careful about who’s getting a look at the cars,” says Ihsane, one of the roughly dozen people who work in “Tester’s Paradise,” as they call their realm.

“We’re growing, and there’s a good team spirit everywhere at ESR Labs, but our team is especially close.” Ihsane, Software Engineer in Test

Chris, a trained electrician, was nicknamed “Dünndraht” (“thin wire”) in his former job, because the tinier the device and more finicky the task, the more he enjoys it.

“When the quality assurance team told me something in my code wasn’t working – it hurt! So I can understand they sometimes hate us a little.” Sergio, Ruby Framework Architect.

Poking the code

Testing the code programmers have written, and that makes modern cars run, is essential. Of course, the developers themselves already run the first tests on their everyday computers. But the software ultimately won’t run on an ordinary computer or smartphone, but inside a car. So in addition to this “off-target” testing, “on-target” testing is even more critical – but also more demanding. Every day a new version of the code makes its way from the upper floors down into the testing garage, where it’s loaded – or “flashed” – onto the respective control unit. “We not only carefully check if the code we flashed onto the different controllers does what it’s supposed to do,” says Ihsane. “We also poke around and manipulate the input or certain conditions to find out how the code reacts if things don’t go according to plan.”

Only for very few tests is it necessary to have the actual car in the testing garage: ESR Labs is developing software for far too many companies and models to make that feasible. The dozens of testing racks that simulate the car in most cases are the work of Chris, a trained electrician nicknamed “Dünndraht” (“thin wire”) in his former job because the tinier the device and more finicky the task, the more he enjoys it. Chris builds the testing racks in a little laboratory (“I just call it my workshop”) then rolls them to the other end of the garage where they reside safely inside a huge glass cube. Once Chris has installed the racks, the testers can use them from anywhere there’s an internet connection. That came in handy during the pandemic, but saves time and hassle no matter what’s going on.

Tinkering tools: Building the testing benches requires a special kind of craftsmanship as well as a steady hand and a good eye for details.

A relay card being repaired in the lab. Small changes and repairs of certain peripherals can be done in-house.

These two test racks will soon be filled with about a dozen test benches then moved into the air-conditioned test rack room.

This test bench setup seen from above only looks chaotic – every wire and every connector is exactly where it needs to be.

Tester’s Paradise: There is also room for up to two cars in the garage. ESR often works with prototypes so this is a question of security and confidentiality.

The test-rack room from the inside: What used to be a confusing jungle of cables is being replaced by a cleaner, less cluttered setup.

To find bugs and improve software quality, the data from the automated test runs gets post-processed and visualized in dashboards.

United by an
obsession for bugs

Compared to other teams in the company that might get rearranged when projects change, it feels like the testing team has been together forever. “We’re growing, and there’s a good team spirit everywhere at ESR Labs, but our team is especially close,” says Isahne, who joined the company after growing up in Morocco. Others in the team come from Mexico, India, China, or Belarus. Good software testers are hard to find – and ESR Labs looks all over the globe to find the best of them.

Sergio was working in Bogotá, the capital of Colombia, when ESR Labs reached out to him. Several remote interviews and two in-person interviews later, he calls Tester’s Paradise his home. “I’m going to be brutally honest: Nobody likes to be told what they did isn’t working,” he says when asked about relations between programmers and testers. Sergio knows both sides: “When I was a developer at my previous employer and the quality assurance team told me something in my code wasn’t working – it hurt! It also puts pressure on you because you need to find a solution. So I can understand they sometimes hate us a little.” But in the end everyone in the testing team agrees it’s about professionalism and finding bugs as early as possible.

New day,
new surprises

Two concepts are essential for the scrupulous testing at ESR Labs: scalability and continuous integration. Scalability is vital because more and more code has to be tested in a growing number of different environments as quickly as possible. Automation of the sometimes tens of thousands of tests a new line of code must undergo is therefore crucial. On the other hand, continuous integration means that it isn’t enough to analyze a new line of code by itself: It has to be merged with the already existing software that someone completely different might have written – possibly years ago. “Sometimes if you write code, you break something you weren’t even aware existed,” as Sergio describes the problem. So the whole thing has to be tested in context. And when new code comes along the next day, the entire game starts over.

And when it does, Tester’s Paradise in the garage on the ground floor becomes a special place again. Maybe the testers aren’t inventing the future right there and then. But they’re ensuring that it works every single time.

We use cookies to enable website functionality, understand the performance of our site and serve relevant content to you. More information: Privacy Policy and Cookie Policy.