What’s Behind Your Web Traffic — Is it Human or Bots?
By Edward Roberts, Director of Strategy, Application Security at Imperva
As a marketer, how would you react if you learned that the money you’re spending on digital advertising every year isn’t generating any engagement from humans whatsoever, but actually attracting automated bots? And what if you learned that 40.8% of the traffic on your website or campaign landing pages last year didn’t originate from an actual human customer?
Unfortunately, for many businesses, this is their unsettling reality. In 2020, human traffic on websites fell by 5.7% while automated “bad bot” traffic grew to record levels. This trend is not only a concern for IT administrators and security professionals but it’s also costing marketers money every year.
Marketing Technology News: Vericast Survey: How Deals Play a Role in Return to Pre-Pandemic Activities
Bots — software applications that run automated tasks over the internet – are a nemesis for marketers. While bots have different purposes, none of the traffic is coming from legitimate human customers, which is ultimately what marketers are spending ad dollars to attract.
By better understanding how bots work and how to manage the problem, marketers can get more accurate data insights about their actual, legitimate customers. This in turn will give them the ability to optimize demand generation campaigns and better understand the customer journey.
The Many Faces of Bots
For years, bots have powered the basic underpinnings of modern internet functionality – including search engine crawlers, web analytics reporting scrapers, and automated customer support chats. These are all examples of “good” bots that help augment business operations.
However, there’s another breed of bots that are harder to detect and stop, yet closely resemble human behavior. These sophisticated bad bots are programmed to conduct price scraping, content scraping, account creation, account takeover, fraud, denial of service, scalping, and denial of inventory. The concern for marketers is that some of these bots help your competitors scrap pricing and product info so they can remain a step ahead. At its worst, this bot activity can be a threat to your business’ data security strategy and intellectual property.
Who’s Actually Engaging with Your Content?
Whether it’s a good or bad bot, automated activity represents something other than the desired human traffic you originally built your website to attract.
Today, bots pollute Google Analytics reporting data and display ad clicks. This creates a potential reporting challenge for marketers who need web traffic data and ad engagement data to inform business decisions. When bots skew web analytics, it makes some pages appear more popular than they actually are and has the potential to adversely influence the entire marketing strategy. Many marketers are making campaign decisions based on what they assume is human traffic, but is actually originating from bot behavior.
Bots have also been identified as the culprit behind driving up page views and engagement on social media influencers content and posts. Some platforms, like Twitch, have proactively addressed the challenge by removing fraudulent accounts. For marketers, this trend of influence driven by bot activity calls into question what is really engaging and what is actually garnering human attention.
When a site is polluted with any kind of bot traffic, it also slows web performance and makes it harder for legitimate users to access the information or services they need. Being able to intelligently separate traffic generated by legitimate human users, good bots and bad bots is essential for making informed business and marketing decisions.
Human or Bot: How Can You Tell?
Bots can appear as human users, with an IP address, header data and other identifiable attributes. This makes it difficult to sort through human and bot traffic. While blocking individual IP addresses or entire IP ranges might seem like an effective approach, this is time-consuming and labor intensive. It’s also not a solution to the growing bot problem. Bots can cycle through hundreds or thousands of IP addresses at a time, allowing them to associate themselves with another IP after getting blocked.
The best solution is to establish challenges with graduated levels of difficulty, making it harder for automated bots to evade defenses. Because certain behaviors are impossible for a bot to do, but simple for a human to complete, it manages the problem while creating the least amount of friction for a legitimate user.
Marketing Technology News: MarTech Interview With George Donovan, Chief Revenue Officer at Allego
There are also steps marketers can take to proactively mitigate the level of bot activity on their website:
- Consider how and when you announce new products. If you communicate the precise day and time for a limited product release to the public, it enables bot operators to schedule bot activity for that exact moment and makes it harder for your legitimate customers to access your site. The ticketing, sneaker and gaming industries all rely on this approach to generate buzz and demand, but it results in high levels of bot behavior on the day of launch.
- Evaluate what functionality you put on your website. Businesses make decisions about their website based on consumer behavior and user experience. However, sometimes those decisions can backfire and become lucrative targets for automated bots. Comment fields, appointment scheduling and gift card balance audits are all examples of functionality that can be ravaged by bots. Even listing your pricing and proprietary content on your website is subject to web scraping.
Today, data-driven marketing is the goal of every organization. Unfortunately, bot activity could be skewing up to 40% of web traffic and tarnishing the quality of your data. In turn, this will impact campaign performance and reduce ROI.
The bot problem is a growing one and organizations need to learn how to manage it. Unfortunately, stopping it altogether is unlikely. For marketers, the need to do this has never been greater as the quality of your data is paramount and campaign success is vital for future business success.