2022 Big Data Trends: Retail and eCommerce become one of the hottest sectors for AI/ML

Nowadays, we can see that AI/ML is visible everywhere, including advertising, healthcare, education, finance, automotive, public transport, manufacturing, security, and government sectors. Each company on this planet is likely asking itself the same question: “What can AI do for my business?” In many cases, they find very promising answers. However, based on the data/ML-related projects that we implement at GetInData, our internal market research, and the content of the conferences that we organize, I have noticed that eCommerce and especially in retail have become one of the hottest sectors for AI/ML.

In this post, I will discuss what factors favor the development of this Big Data trend, explain the specifics of AI/ML in the retail and eCommerce sector, discuss how “black swan” events and the introduction of regulations affect it and try to predict how this trend will behave in the future. Can we expect this Big Data trend to grow bigger?

AI/ML is visible everywhere, but especially in retail and eCommerce

The growth of data and AI in the retail and especially eCommerce sectors was fueled by COVID-19, or more precisely, by the way governments and people responded to COVID-19. The government distributed a lot of money directly to consumers in the form of stimulus checks, the interest rates were slashed to record-low levels (near to zero in many countries) so that people could get loans cheaper, and they also rushed to buy new stuff such as electronics, sporting goods, health and personal care goods. They bought new bigger homes, or renovated their existing homes to design home offices or e-schooling rooms for kids, which increased the sale of furniture, electronics, building materials, etc. Due to the lockdown, online shopping became the main option available. It has become ubiquitous on a scale we have never faced before. So many retailers accelerated the growth of their digital channels and improved their eCommerce websites, chatbots, and mobile apps. Because frankly, it has become a matter of survival in the market. The fundamentals in 2020 and 2021 were great for data and AI in retail and eCommerce. This year is more challenging, due to the record-high inflation worldwide and the rising costs of living. Therefore it will likely decrease the profits of companies such as Target, Amazon and Walmart but I will come back to this later in the article. 

Many different types of data - clickstream, text, images, voice and even videos. 

Instead, let’s now take a look at the retail industry from the data and AI perspective. Why is the retail (including eCommerce) industry so hot? First of all, in this sector, you can analyze many different types of quite useful data. For example, in finance, you focus on time series data, in media companies you focus on clickstream, but in retail you have almost everything - clickstream, IoT stream, text, images, voice and even videos. 

This is because their interactions with customers happen in many different places, online with the website or mobile app, in physical stores with real cashiers or smart hardware, and often using text in the chat window or voice when you speak with bots. Btw, according to a recent Ubisend report, 1 out of 5 consumers is willing to purchase goods from a chatbot.  Also, more and more people shop using visual search. According to data that I found on the internet, 62% of millennials prefer visual search tools to any other search technology because visual information is more useful to them than text information. When you read about visual search, you can also consider the metaverse or multiverse. For example, today we have smart mirrors that recommend clothes for you and you can see how you look with them in your room, but tomorrow much more will be possible in the metaverse, for example, you will be able to see how you look with your clothes in various countries (e.g. on vacation in Mexico) or places (e.g. going to a fancy restaurant with your partner). This will also generate a lot of complex real-time data. Of course, the different types of data require defining and implementing a good omnichannel strategy and customer journey so that you can later leverage this data for many different business use cases. 

Many different interesting and useful use-cases

This brings me to the second reason. There are many interesting and useful use-cases for analytics in the retail industry. There are many different units such as logistics, delivery, customer support, eCommerce, and use-cases such as dynamic product pricing, product recommendation, analyzing public customer reviews, and reporting adverse events where you can apply data, analytics, and domain knowledge. You will also find cross-unit use-cases such as forecasting. Having many different data sources and many different units at a large company have also introduced complexity. Such complexity might also require the adoption of concepts like Data Mesh or similar, to make sure that this data is properly collected, accessed and analyzed.  

As I have already mentioned, we have more and more opportunities to look at use cases in the area of retail and eCommerce. I made these predictions among others, by looking at the development of projects that we run at GetInData. If projects in this sector interest you, check out our open positions. 

Black Swan, MLOps, real-time and automation

In many cases, the data must be analyzed very quickly by retailers - ideally using some automated tools and techniques. For example, successful retailers should have their MLOps platform working well so that they quickly iterate on AI/ML models. This is especially important today when we have so many “black swan” events e.g. COVID-19, wars, high inflation, or possibly stagflation or even recession in the future. These events can suddenly make their current AI/ML models not work well and need to be reimplemented or re-trained because customers have changed their shopping habits dramatically. In other words, successful retailers have now become “truly tech” companies that simply happen to sell some products such as clothes, cosmetics, or LEGO bricks.

Most companies don’t build real-time analytics solutions, because it requires more engineering effort (this is what GetInData can make simpler with our experience - write to us, and we will get back to you). However, everyone agrees that real-time data is much better than latent data. As Benn Stancil notably wrote in his article “We all have dashboards that update a little too slowly, or marketing emails we wish we could send a little sooner”. However, retailers and eCommerce companies benefit from real-time analysis even more, e.g. displaying personalized search results after the few first clicks to accelerate sales, giving customer support the real-time context of who contacts them at that present moment via chatbot, discovering issues with some products on the same day, etc.

Responsible AI

Maybe you already know that LEGO has recently announced that it is entering a long-term partnership with Epic Games to shape the future of the metaverse to make it safe and fun for children and families. The CEO of The LEGO Group said: “Kids enjoy playing in digital and physical worlds and move seamlessly between the two.” This will likely generate a lot of complex data such as voice and video that will need to be processed in real-time with many different AI/ML use-cases on top of it to make sure that it's fun and safe.

Of course, the AI that is safe will need to meet many regulations that are being introduced, that are sometimes cohesively called: Responsible AI. The parts of Responsible AI can include Secure AI, Ethical AI, Explainable AI, or Interpretable Machine Learning. This is also the trend that I see that companies and data scientists are talking about more often. This is also a topic that governments are discussing as well. For example, the European Union is working on the so-called Artificial Intelligence Act that will introduce many strict compliance obligations for AI systems, especially for so-called high-risk AI systems. I will not talk much about these regulations because in this article you can read our analysis of the changes it will bring: EU Artificial Intelligence Act - where are we now? These regulations will likely introduce some barriers to entry for smaller companies that will have to put more resources to comply with them, especially if they don’t have their own legal teams. IMHO, it will be much easier for companies such as Facebook, Amazon, and Google to comply with these regulations because they already have their own legal teams and resources required. Please don’t get me wrong, I know how powerful AI and ML are, and they shouldn't be abused, therefore there should be well-defined regulations to protect people and their personal data. Yet on the other hand, they should not introduce too much bureaucracy, paperwork, and legal efforts so that only big players can comply with them and get even bigger.

2022 can define winners

I would like to mention that this year, 2022, will be more challenging for retail and eCommerce companies, because of the current macro-fundamentals such as rising inflation, rising supply chain issues and the rising cost of living which may lead to an official recession this year. Walmart and Target have recently published their results for Q1 and Q2 2022. Not only were the results for Q1 and Q2  2022 disappointing from a financial perspective, but also the guidance for the next quarters. Furthermore, in late July Shopify announced that it is letting go of ten percent of its workforce (about 1000 employees) in the face of slowing sales growth for four consecutive quarters. This is because consumers are returning to their old shopping habits, decreasing their reliance on e-commerce and/or simply shopping less due to the lower purchasing power of their money (during inflationary times, a rise in personal spending doesn't have to reflect a strong consumer, but inflation, as consumers can buy fewer items because they have to pay more for them).

However, the AI/ML perspective can be different. For example, in its memo, Shopify wrote that most of the impacted roles are in recruiting, support and sales. Data and AI engineering teams didn’t seem to be affected. Something similar happened in May 2022 when Karna (a large Swedish fintech company from the BNPL sector), announced massive layoffs (around 600 employees) and published the list of affected employees, so that they could be offered jobs by other companies and recruiters. In this list, you don’t find data, AI, or cloud engineers because the company wanted them to stay. These two examples show that data, AI and cloud are very important assets for many companies and they still want to invest in them, even when they also need to navigate through turbulent times.

In 2020 and 2021, the demand from consumers was large and people were shopping a lot online because of the lockdown or remote working, so companies invested a lot in AI/ML to accelerate their digital channels and eCommerce platforms. However, in 2022 I predict they will adopt many AI solutions that will help them to reduce costs, automate processes, discover better pricing and inventory strategies, optimize marketing efforts and so on. It might turn out that some companies will see a reduction in the sales of their products and run into trouble, but on the other hand, many companies will grow very fast, because they will sell the right stuff for the right price and they will invent many new ideas on how data and AI can be used that others will adopt later. So it is my belief is that the retail and eCommerce sectors may not grow in real terms of sales, revenue, and consumer spending (adjusted by inflation), but I assume that the adoption of data and AI in these sectors will grow significantly, because there are many new business use-cases there, many business opportunities and still many ways to reduce costs and increase sales.

This blog post summarises one of the 5 trends that I talked about in the Radio DaTa Podcast episode '5 current trends in the data and AI landscape (H1 2022):. You can listen to its full version on Spotify, Google Podcasts, Apple Podcasts, or YouTube and discover in which direction Big Data is changing. These trends include Modern Data Platforms (SQL, hiring, open-source, data engineering pipelines), Public Cloud (data residency, multi-cloud & cloud-agnostic approach) Data quality, Data Auditing and Data Access (data cataloging, data discovery and data mesh).

Don’t forget to subscribe to our Radio DaTa Podcast to get notifications about future episodes. In the podcast, we host experienced experts from companies such as Shopify, Datadog, Reckit, and Zalando, who share insights into projects (successes and challenges) in the fields of AI, ML, Cloud, analytics and Big Data.

Interested in ML and MLOps solutions? How to improve ML processes and scale project deliverability? Watch our MLOps demo and sign up for a free consultation.