Inside ad fraud: What it takes to dismantle a US$5.8 billion enterprise

Tamer Hassan is edging on paranoid. Sometimes he conducts weekly wipes of all his devices. But he also knows far more than most of us about what cybercriminals are capable of, and the almost unconquerable battle ahead in keeping a lid on ad fraud.

Hassan is one of the co-founders of White Ops, the cybersecurity and ad verification firm responsible for uncovering two of the largest online ad fraud operations in history: Methbot in 2016, and 3ve (pronounced Eve) in 2018. Hassan was the chief technology officer at the firm when he spotted a particularly resistant type of botnet in 2017; the ghost of Methbot that would rapidly escalate into 3ve. He led the 3ve project from start to finish and is credited with being the lynchpin of the global consortium to take it down. For his work, Hassan was promoted to CEO by the White Ops board in April.

Ad fraud is a low-risk, high-profit, recurring-revenue crime. It is tipped to become the second most lucrative form of organised crime (behind drug trafficking) within the next decade, according to the World Federation of Advertisers. White Ops has projected US$5.8 billion of advertising dollars will be lost to fraud in 2019. Other organisations set the number as high as US$42 billion.

As the price tag of ad fraud has skyrocketed, law enforcement has taken note. After the historical involvement of the FBI in the take-down of Methbot in 2016, government cyber teams and US attorneys are now having to learn how programmatic advertising works, so they can step in and take action earlier, according to Hassan. The fact that law enforcement are upskilling in adtech shows the severity of the problem, but advertisers still aren't taking it seriously enough, he believes.

But there is hope. For the first time, more fraud will be stopped this year than will succeed, as a result of DSPs and SSPs filtering fraudulent bid requests, clawbacks, and other preventative measures. White Ops estimates that loses would have been a monumental $14 billion annually without such measures.

In a comprehensive interview with Campaign Asia-Pacific, Hassan goes behind-the-scenes on how a sophisticated fraud operation is run and what it takes to shut it down, while also shining a light on the causes and solutions.

What happens behind-the-scenes

In their infancy, ad fraud schemes would be operated by a device farm in a data centre. Hundreds of devices are ordered to repeat actions such as clicks, registrations, installs and engagement to create the illusion of legitimate activity. Device farms are still common practice, but they are easy to detect and therefore have a short life span.

Malware-based botnets, which infect devices used by real humans, are, by their nature, much better at masquerading as humans. They have a real device ID, a real history. Three-quarters of the bot activity that White Ops tracks comes from this method.

“The game is always to look like a million humans,” Hassan explains. “Once you can look like a million humans you can do a lot of things on the internet—ad fraud being one of the biggest.”

Both Methbot and 3ve falsified audiences as well as inventory to siphon away maximum advertising dollars. 3ve generated 3 billion daily bid requests and siphoned away more than $29 million in fraudulent ad revenue before it was dismantled last year. The full cost of Methbot is not known, but it is estimated to have generated a high of 400 million impressions a day on falsified websites at an average CPM of $13, equating to $5.2 million in daily revenue.

3ve obtained control over 1.7 million unique IPs by leveraging computers infected with Boaxxe/Miuref and Kovter malware, as well as Border Gateway Protocol-hijacked IP addresses. The malware leveraged by 3ve had been around for almost a decade and was used to facilitate everything from click fraud to ransomware. It spread through email attachments and drive-by downloads. Over time the malware got more and more sophisticated at evading detection. Before performing ad fraud, it would check for geolocation, security software, VPNs or proxies typically used in security sandboxing.

“The anti forensics built in were so advanced, it was hard to even reverse-engineer, because it knew it was being reverse engineered,” Hassan explains.

So sophisticated, in fact, that the seeds of the next version of that malware are already starting to sprout, just months after it was taken down by an international law enforcement operation.

Methbot wasn't malware-based but a data centre-based botnet—the first time White Ops had seen this kind of activity. Over a number of years the developers obtained or leased almost 1 million real IP addresses and used them to generate fraudulent ad calls that appeared to come from legitimate residential internet providers in the US, such as Verizon and Comcast. Ad slots were placed on more than 6,000 spoofed premium domains. Methbot similarly employed advanced techniques to evade detection, such as fake clicks, mouse movements, and social network login information to make it appear as if a user was logged in when an impression occurred. In its prime, Methbot was sending 3 to 5 billion bid requests per day.

It’s not just the technology behind fraud that has evolved; cybercrime has also become more organised. Hassan explains that 10 years ago malware was often run by a single person who would be responsible for everything from writing the code to laundering the money. But as the crime has become more lucrative, operating it has become a business. There are now whole teams running different parts of the operation, with a few people at the top pulling the strings.

“It's almost like drug cartels where you have suppliers and distributors, and it's more of an ecosystem, which means it's a bit more robust,” explains Hassan. It’s why when Methbot went down, 3ve filled its place. Many of the same people were behind both schemes.

“These are professionals, this is their full-time job, and they're making a lot more money than we are," he says. "They stand to lose millions of dollars from their annual salaries if they're not winning. So you have to count on that, from an adaptation perspective. How do you play against an adversary like that?”

There are still kids in their basements looking to make a quick buck on the dark web, but that is not what keeps Hassan up at night.

Where is organised criminal cybercrime coming from?

Methbot and 3ve were concentrated from Russian actors, but White Ops also sees a lot of non-human activity originating in China.

However, due to the secrecy of the market, Hassan says it is difficult to differentiate how much of the bot activity is cybercrime and how much is “just part of day-to-day business in China”. It’s also difficult to put a price tag on ad fraud there. GroupM has estimated fraud in China is worth US$18.7 billion, representing 83.4% of total ad fraud globally, but some industry execs have questioned this figure in conversations with Campaign.

If you can look like a million humans, what can you do?

Botnets can be weaponised to do a variety of things, some more nefarious than others, with money the main motivator. Bots can click on ads, listen to music to increase royalties, generate fake social-media followings, inflate app store downloads, and create fake reviews. They can commit distributed denial-of-service (DDoS) attacks, whereby a server or site is flooded with traffic to slow or temporarily shut it down. But the crime that has drawn perhaps the most attention in recent years is election manipulation.

The level to which votes or elections have been—and are still being—influenced by bots has not yet come to light, Hassan believes. While there is an ongoing investigation into Russian interference in the 2016 US presidential election, other cases have been uncovered. In 2017 millions of fake comments were posted on a portal run by the Federal Communications Commission to influence a vote on net neutrality.

“That was uncovered and it was a relatively simple bot, which was a bit alarming," Hassan says. "If it was anything remotely more sophisticated, we may have never seen it. So you have to wonder how many of those we have not actually seen.”

More recent innovations in fraud have been centred around controlling the mobile phone. Malware on phones can generate ghost clicks and revenue, drain the battery, and in some cases have full root access to a user’s phone—in theory allowing them to scrape files or turn on a microphone.

The complex adtech supply chain has facilitated fraud

While ad fraud as a concept has been around for 30 years, the more recent branching out of the advertising supply chain—from a linear chain to one which includes thousands of intermediaries—has provided fraudsters with a plethora of dark corners to hide in.

Since advertisers often don’t know exactly which sites they are buying space on, it is relatively easy for a fraudster to masquerade as a legitimate partner in this ecosystem.

A common practice that exploits this is called domain spoofing. It involves fraudsters passing off low quality inventory as a high-quality or premium site. The same practice is employed with apps, called app name spoofing. For more on prevalent types of fraud in Asia, see here.

No wonder, then, that fraud attempts currently amount to 20% to 35% of all ad impressions throughout the year, according to White Ops’ latest Bot Baseline report. This is why trends like supply path optimisation—which helps DSPs streamline and remove some of the opacity of the supply chain—are growing.

It’s also behind the Internet Advertising Bureau (IAB)’s ads.txt tool, introduced in 2017 to help ad buyers avoid illegitimate sellers. Ads.txt is a text file listing vendors a publisher has authorised to sell its inventory.

The tool was an “important step” in shrinking the number of opportunities for fraudsters, but it “certainly doesn't solve the majority of the problem”, suggests Hassan.

“It almost certainly has been gamed to a certain extent, and it can’t weed out professional cybercrime too much," he says. "But it can weed out the other players—the teenagers in their basements, the smaller groups." Ads.cert, which will use cryptographically stamped bid signatures to determine whether an impression belongs to the correct website, will add a more comprehensive line of defense, Hassan notes, but is still a few years away.

While ads.txt filtered out the majority of 3ve traffic, 20% was still noted as 'authorised'

Hassan likens initiatives like ads.txt to “building a wall around your castle that you’re trying to defend, and limiting the number of entry points so you can focus your defense efforts”. But the problem with this approach is the ad industry’s defence will never be as strong as fraudsters’ offense.

“It’s a game of cat and mouse,” Hassan says.

So what are the solutions?

The only long-term way to suppress fraud and cybercrime is to address the economics of it, Hassan suggests.

“The moment you raise the cost of doing fraud to the level of the profit or above, you start to disincentivise it. That’s how we build our detection engine—it’s offensive in nature, it interrogates, it's much more costly for an adversary to get around it.” he says. “It's the only way to win.”

Leveraging the industry’s collective power and raising the criminal consequences of fraud also acts as a deterrent. Methbot and 3ve were the first ever large-scale FBI investigations into ad fraud, and a lot of resource was put into finding the individuals responsible.

“Most of the time you stop at active prevention and blocking of the fraud, to actually go the last mile to uncover and attribute to an individual or company is a high resource activity,” Hassan says. It’s why ad fraud is one of the lowest-risk crimes around.

Scary stuff, right? Knowing what he does about the ever-evolving ad fraud threat, Hassan has resigned himself to the inevitability of being targeted by fraud, at some point.

“I think it's a misconception that we can defend the perimeter perfectly. I just assume that my emails, my devices are compromised,” he says. “Sometimes I reset my whole device weekly or monthly, but that’s probably more on the paranoid end.”