©1996- Aspire Systems
This article is part of an ongoing series on launching a flash or a hype sale using Sales Force B2C Commerce. Read the first and second blog to know how to launch a successful flash sale.
In 2012, Nike released their sneaker Air Jordan Doernbecher 9 on Twitter. Shoppers could reserve the shoe by being first to direct message (DM) the company. This prompted the creation of bots to scour Twitter's API and DM Nike after any tweets with terms like "reserve now" or "Doernbecher". You could send hundreds of DMs in a tenth of a second. Real customers did not stand a chance.
This is what 'bad bots' are capable of during flash or hype sales. For something as critical as a flash sale, where a large amount of traffic is driven to a storefront for a limited product range, bad bots are inevitable. So, it is wise to gear up against malicious users who eye profits by buying your stock to resell through bad bots. Since the risk of a bot attack increases with great discounts and limited inventory on offer, before you "Get set, go!" on your Flash Sale, you need to do something about bad bot behavior.
Let us unpack bad bot behavior and know how it can impact the success of your flash sale. In this article, you will also read about successful bot mitigation strategies that let you stay ahead in the game and attract customer loyalty and new customers.
Why are 'bad' bots bad?
While good internet bots help you get ranked higher on search engine results, evasive bad bots can bring your website to a stand-still. This moderate and advanced group of bad bots made up an approximate 39% of all internet traffic in 2021. They are hard to detect as they elude standard security defenses and use the latest techniques to evade detection. This breed of bots can cycle through random IPs, enter through anonymous proxies, change identities, and above all, mimic human behavior. And for those retailers eyeing the metaverse or the gaming industry, you need to gear up against increased data scraping and credential stuffing attempts.
Bad bots cost global businesses millions in direct and indirect losses. Any business with an online presence is exposed to bots. Internet bots systematically browse the web for indexing. Web indexing is an automated task that makes sure that your website, for instance your Flash Sale and relevant product pages, appear on the search engine results pages (SERPs). While these "good" bots can help you, "bad" bots are used by malicious users and criminal hackers, who want to make gains from your website, or even steal sensitive information such as passwords and financial information of customers, cause slowdowns, relay spam, launch cyberattacks and much more, causing you huge loss of revenue as well as reputation.
What You Can Do to outmaneuver Bad Bots
Innovation in sneaker bot technology has made them a hard nut to crack. Getting a sneaker bot is as easy as booking a ticket to automate your life to sneaker (or any product) heaven. These bots take care of searching for the right product, down to its specific size and take care of payments and shipping. They are fast evolving, hard to catch and switching tactics and attack vectors. A thriving developer community supports everything about bot software: from acquisition to attack.
Your flash sale is a ripe target for automated attacks by bad bot operators through vectors. By using simple tools or Python scripts, bots can use script routing to scrape sites for data, attack popular controllers, and direct traffic at its origin in an attempt to bypass protection at the CDN layer.
Bot Mitigation Strategies in the SFCC bag
Even though enterprises know that bots are a problem, there is a lack of awareness about what techniques and technologies are effective against bots, where these bots are being used, and who uses them. Of most concern is that over two-thirds of businesses think that web application firewalls (WAFs) and distributed denial of service (DDoS) protection will keep them secure against bot attacks. These tools are valuable and recommended, but they are not effective against sophisticated bots — leaving businesses vulnerable to attacks that may be the difference between profit and loss. These strategies protect against data theft or compromise of customer information, your business information such as inventory or server data.
A multi-tiered approach to bot mitigation
Salesforce B2C Commerce has an embedded Content Delivery Network (CDN) that gets enabled by default when creating proxy zones. It also provides you the freedom to choose the CDN from another provider and integrates seamlessly. The eCDN Web Application Firewall (WAF) is a layered approach to security and an important component of a multi-tiered approach to bot mitigation.
The eCDN provides the following bot protection strategies:
Allowlist of traffic sources: The CDN gets a list of safe IP addresses to allow during your Flash Sale, and those identified as harmful are blocked. Allowlist IP ranges on the eCDN so that blocking mechanisms are bypassed for known sources of legitimate traffic. When used with a firewall rule, this approach blocks any traffic that isn’t from allowlisted IP ranges.
'Under Attack' mode: When you use the built-in eCDN functionality, you can even consider increasing the threat level or enabling ‘Under Attack’ mode during a flash sale. The only impact on your customer when you are ‘Under Attack’ mode is every unique user needs to solve a CAPTCHA before they are allowed to see the storefront.
Firewall rules: This approach is an effective way to control storefront access and minimize malicious traffic during a flash sale. When you set up firewall rules on the eCDN, you are basically setting up minimum criteria that a browser or a network needs to meet. These rules help log, block, or challenge suspect traffic. Firewall rules are based on a variety of conditions and are built on filters and regular expressions. Salesforce Support can help you create firewall rules that help manage suspect traffic, such as traffic from specific countries, user agents, or paths.
There's more to blocking bot traffic
In addition to the eCDN layer, Salesforce B2C Commerce offers more options for blocking bot traffic effectively. They stop bots that try to bypass the eCDN or CDN stack. The bots target sensitive endpoints. A combination of the following strategies helps us protect your storefronts from malicious access, while still allowing legitimate traffic.
Selective origin shielding (SOS): To restrict external sources from bypassing the eCDN or directly accessing the Commerce Cloud POD origin, we implemented selective origin shielding (SOS) on all PODs. SOS prevents bots from sources, such as third-party integrations or partners, from accessing the origin outside the eCDN, and increases site security and availability by allowing traffic from only a known list of essential IPs. Other traffic is not responded to, and the requests time out.
Rewrite rules: Rewrite rules are a URL rewrite framework that can block and redirect traffic at the origin. This framework checks for certain paths and identifies mistakes that bots make when trying to spoof regular calls. This feature is available on the backend servers as part of the Commerce Cloud infrastructure and protects sensitive paths like the Cart and Checkout pipelines.
Secret headers: A secret header is an extra HTTP header that gets typically attached at the CDN level. This extra header is not visible to shoppers and bot operators. If an incoming request has to pass through the eCDN, the header should be present and validated at the Commerce Cloud origin. This validation is done by rewrite rules that check for the header and block malicious traffic. All requests that fail the header check return a 403-response code.
Here’s what we recommend for you to mitigate bad bots, even the complex ones
Allowlist legitimate traffic sources: You can use the Salesforce eCDN to allowlist legitimate IP ranges for incoming traffic. But you can also use the CDN Zones API to allowlist IP ranges.
Leverage WAF protection on eCDN: The eCDN Web Application Firewall (WAF) can help mitigate many bot attacks. The WAF rule engine powered by intelligence and heuristics.
Set up firewall rules: Consider working with your Salesforce Support representative to set up firewall rules to block and challenge suspect traffic. For example, you can have rules that block traffic from certain countries, IPs, and autonomous system numbers (ASNs).
Analyze traffic patterns and call formats: Traffic patterns on a storefront constantly change, and bot traffic can begin subtly with a heightened number of calls against common endpoints, such as Product-Variation or product detail pages (PDP). Check for and know the typical traffic patterns for your site, especially during sale events. This information helps you to identify patterns that are consistent (or inconsistent) with your implementation or line of business. Then, you can work with your Salesforce Support representative to mitigate bot activity.
Evaluate bot management solutions: You can employ specialized third-party solutions that integrate with Salesforce B2C Commerce to tailor a protection package against shopping bots. Key features of most bot management tools include a constantly updated blocklist, and advanced machine learning that scores and rates requests prior to making block or allow decisions. Just like shopping bots, bot management solutions continually evolve to update their strategies. Some options include PerimeterX, DataDome, Shape, Akamai BotMan, Cloudflare Bot Management, etc. If you are using one of these solutions, Salesforce B2C Commerce can advise and evaluate traffic patterns.
Create a virtual waiting room: Especially during the first few minutes of a flash sale, bots send requests from many IPs. To protect your storefront from overwhelm by bot traffic for popular products in limited stock, you can create a virtual waiting room that limit the number of shoppers allowed to purchase products at a given time. When the maximum number of shoppers is reached on designated pages, new shoppers receive a waiting page that doesn't access backend systems. When paired with presale load testing and traffic forecasting, waiting rooms help control the storefront load while maintaining a positive shopper experience. You can even create a customized waiting room experience that enhances your brand. Virtual waiting rooms lets you analyze incoming requests for a traffic signature and understand how human shoppers (and bots) are interacting with your storefront. This knowledge can provide insights that help you tailor your bot mitigation strategies. Consult with your Salesforce Support representative to identify the levels of traffic bursts your storefront can handle, while maintaining optimal performance. You can determine the waiting room size, how many shoppers to allow into a waiting room, and which pages should have a waiting room.
Review the settings in robots.txt: To control the crawl rate and site paths for non-malicious bots (for example, search spiders, known marketing crawlers, and feed fetchers), review and update the settings in your robots.txt file. This approach helps us be proactive with managing "good" bot activity and avoids scenarios of heightened activity during a flash sale.
Conclusion
The availability of more digital services, new online functionality, and the development of expansive API ecosystems has unfortunately caused an array of new endpoints. When you choose Salesforce Commerce Cloud as your platform, you can timely identify and mitigate bot attacks.