Google Search Algorithm Is A Lot Of Trial and Error Coding?

A post from a former Google search engineer, Kevin Lacker, from 2006 recounts a conversation this engineer had with the former head of Google search Amit Singhal. It discusses how sometimes these engineers just tried trial and error to see what works, even if it didn’t make logical sense.

Kevin Lacker worked for Google as a software engineer building search algorithms from January 2005 through November 2009, so for almost five years working under Amit. When Amit left the company he posted on Hacker News shortly after that “This is certainly a loss for Google.”

But he also added that he was so good at helping engineers keep things simple and not overthink things. He said “One conversation we had about when it made sense to take the square root of things, which I still recall which whiteboard it was on, in my noob-cubicle in building 43, was really the time when I realized, aha I get how to design search algorithms now.”

He added not only was using square roots a good method for smoothing but “more so that a it’s trial and error thing.” “When I started out after dropping out of grad school I was working on search algorithms and a bunch of my coworkers had a background in informational retrieval and I did not, and I was kind of thinking about it in the wrong way. I was thinking about it like it was a math puzzle and if I just thought really hard it would all make sense,” he said.

Kevin said “one day I was stumped in how to make this signal useful and Amit suggested, hey why don’t you take the square root. I was like, why a square root, it doesn’t make much sense, nothing is getting squared ever, and he went up to the whiteboard and just drew a square-root-ish-wiggle-arc-shape and said look you just want something that looks like this. Square root, log, whatever.

I was like, oh… am I allowed to write code that doesn’t make any sense? I thought I wasn’t supposed to do that. And he was just like, well, just don’t worry about it, you are overthinking it, you can take all the square roots you want, multiply by 2 if it helps, add 5, whatever, just make things work and we can make it make sense later.”

It was “at that point I realized that real world software engineering was much different than research had been, and also that this was going to be way more fun,” he said.