It’s in the Data: How COVID-19 is Affecting the Digital Landscape

I’m sure almost everyone reading this has been affected by the emergence of the novel coronavirus disease (COVID-19), in addition to noticing some serious disruptive economic changes across most industries. Our data research department here at Oxylabs has confirmed these movements, especially in the e-commerce, human resources (HR), travel, accommodation and cybersecurity segments. 

According to data compiled by our research department, some industries have had increased request activity while others have had a decrease in requests. 

The data presented here is a sample of the types of requests processed by our residential and data center proxies, in addition to requests using our Real-Time Crawler solution.

Recommendations by worldwide health organizations have prompted governments to implement policies requiring citizens to stay at home in an effort to contain the virus. This has led to a predictable increase in some business activity on the internet, while other areas have seen a decrease, like the following: 

As expected, anyone that intended to take business or recreational trips was forced to put their plans on hold because flights to most destinations were suspended in most parts of the world.

This put a serious dent in the travel industry, as confirmed by a drop of 24% in data requests in March when compared to February. Most travel at this time largely fell into the “essential” category - pointing to a specific customer segment that may be of interest to some companies. 

As with the travel segment mentioned above, booking activity for hotels and other forms of travel accommodation dropped worldwide. Accordingly, the number of related requests across all our data resource infrastructure went down by 21% for March versus February. It should be noted though that this was already following a downward trend since February was down 7% compared to January. 

While some research metrics may not be useful during this time (like cruises), other types may still be of interest, particularly the types of bookings that were made by the “essential travel” segment referenced in the previous section. 

Decreased worldwide economic activity, according to some analysts, will likely result in a sharp slowdown in the new motor vehicles sector. Since the demand for personal transportation is likely to remain unchanged, this may result in a market shift to used vehicles or an increase in the vehicle repair industry. 

Data scraping practices can be used in this case to obtain information on the makes, models and years being sold most often, along with customer reviews that give useful information about vehicle quality. 

Since tough financial times often force people to repair older cars rather than buying new ones, parts dealers can use web scraping to monitor competitor information that includes obtaining data on their catalog, sales activity and any relevant customer reviews. 

It should be noted that the preceding information (including that provided in the links) is largely speculative at this point and it may be too early to draw conclusions. We will be monitoring this vertical over the next few months for any changes in how market stakeholders will address the challenges the industry will likely face. 

Millions of people have lost their jobs as a result of the panic. While the general recruiting and HR segments were expected to decrease, our data research department actually reports a 5% increase in requests which may be due to increased searches for remote jobs. 

A cyberattack is an attempt to gain unauthorized access, destroy, expose, change, disable, and/or steal data from an information system. Since internet activity is increasing at this time, the cybersecurity industry is seeing increased threats of all types, from unauthorized access of websites to email phishing schemes. 

Our data confirms this - cybersecurity requests were up 18% for March (compared to February), outperforming all sectors mentioned in this report. 

The E-Commerce sector has seen high volatility in recent weeks, moving in different directions depending on the industry. 

While online retailing for some sectors, like event ticketing,  big-ticket items (like appliances), hotels and flights has taken a hit, other sectors like grocery, pharmaceuticals, personal care and electronics have had an upswing. While these increases vary significantly, data collection and analysis in all these areas provides useful information for pricing strategies and customer behaviour.

Brand protection measures, such as counterfeiting and copyright infringement are other important factors being considered by our clients. As a result, they are harnessing the power of proxies to crawl e-commerce websites, auction sites, and other relevant marketplaces to detect fraudulent activity, in addition to obtaining the information required to prosecute anyone attempting to misuse intellectual property.

Some people have opportunities to work at home, while others have more spare time than ever. E-Learning opportunities are everywhere on the internet, and interesting data is being pulled to discover how people are upgrading their skills to deal with job losses.

The first step to efficient data mining is data extraction, and tools like this are built specifically to extract data from search engines and e-commerce websites. Another option is to use data center and residential proxies in combination with an in-house team of developers.

Here are two specific case studies (names changed) that illustrate solutions for handling the challenges currently being faced in their respective industries: 

Tech-Deals.com is a popular online store that sells laptops, gadgets and office accessories. 
The company offers a huge array of brands and competes mostly on price and customer service. With COVID-19 confining many people to their home offices, the demand for certain equipment and tech gadgets has increased. 

Current challenge:

The industry is more competitive and dynamic than ever, and price sensitivity is off the charts now that customers have ample time to research prices. Since Tech-Deals.com competes on price, they need to track competitor prices constantly in order to stay competitive.

Their in-house data team was doing a great job prior to COVID-19, but now the company is finding itself in a position where costs are climbing and sales are decreasing. This is pointing to a need for them to get an edge over the competition with higher-quality data. 

Solution: 

Tech-Deals leveraged the power of this customizable online scraping tool that was able to address their diverse business requirements, give them the most relevant up-to-the-minute data, and save on costs. 
The increased sophistication of this data collection technique gave them more relevant data on the most popular brands, prices, best-selling categories & customer reviews. 

After analyzing the data the company was able to quickly optimize their pricing and advertising strategy, allowing them to stay competitive and on top of the industry.  

SecureCom Ltd. serves a wide range of industries by identifying, predicting and preventing cyber threats using security analytics and sophisticated threat management technology. 

Current challenge:

Since the COVID-19 outbreak, the company has observed an increased risk of cybersecurity threats, particularly related to an extensive list of newly registered COVID-19-related domains.
These domains were connected to a surge of highly advanced COVID-19-themed phishing emails that were impersonating trusted health organizations and government bodies. 

Solution:

Proxy solutions that scan for malicious links and attachments prior to delivery are a solution to this issue. Companies can use a large pool of dynamic IP addresses to continuously monitor incoming emails for threats. 

The behavior of consumers is drastically changing - even as you read this. In order to stay competitive, businesses must adapt as quickly as possible.

Companies can use web scraping to track relevant metrics across a wide variety of markets and industries so they can adjust their strategies and stay competitive. Interest in big data has been growing exponentially in recent years, and it now seems to be more important than ever.