With effective solutions, Cars2Click reached an ingestion of ~5 million ads per day in under 30 minutes. In partnership with Zitec, they reduced the processing time, thus improving their data management.
In today’s digital landscape, collecting, ingesting and processing massive amounts of data in real time are essential ingredients for successful organizations. Changing customer preferences and emerging business models heavily based on digital products and interactions have altered a multitude of industries irreversibly. The automotive market is no exception in this regard.
Understanding that offering customer-centric, scalable and data-driven solutions is key to success. So, Cars2Click, in partnership with Zitec’s DevOps and Cloud consultancy team, embarked on their digital transformation journey. More precisely, a journey to build and scale a unique global platform: Cars2View.
Market context - automotive industry
The automotive sector is a central contributor to the European Union (EU) economy. It generates over 7% of the EU Gross Domestic Product (GDP). This means that in 2020, the automotive market totaled around EUR 936 billion.
Moreover, this sector opens opportunities for the EU trade suppliers who want to move up in the value chain. More than 5.6 million vehicles were exported per year globally. This generated a surplus of EUR 74 billion.
Cars2Click and the car trading industry
Cars2Click started as a car trading company focused on exporting cars from Sweden to other countries in Europe. Today, the company is a digital car trader with a presence in 16 countries and continuously expanding to new markets.
The company has developed an AI-driven platform to help automotive dealers advance their business decisions based on detailed market data. More precisely, millions of online car ads from across Europe indexed and processed on a daily basis. Cars2Click’s partners are able to use this data in order to create the best strategy to assess their car inventory and maximize selling opportunities.
Building a modern, data-driven software experience
To improve the efficiency of the development processes and reduce time-to-market, Cars2Click realized they needed to modernize their software architecture.
Learn about modernizing your platform infrastructure
As an initial step in this partnership, Cars2Click worked with Zitec’s DevOps, Security and Cloud architecture consultancy teams. The aim was to define a robust and scalable architecture for the Azure-based applications they were looking to build.
✅ Large scale ingestion of car ads using Azure Functions
By using a serverless architecture based on chained Azure Functions and Azure Storage Queues, Cars2View managed to improve the daily ingestion of millions of car ads, resulting in a setup that is both performant and highly cost-efficient to run and operate. This improvement has allowed the company to provide more insights in a faster manner to the customers. In this setup, Functions allowed Cars2Click’s development team to fully focus on writing code and not on the server management hassle.
After implementing this tech stack, Cars2View managed to reach an ingestion of ~5 million ads per day in under 30 minutes.
✅ Highly performant ETL Pipeline stack
Processing, storing and querying large amounts of data in near-real time is essential in order to provide timely insights to partners. To accomplish this, Zitec’s team focused on optimizing the data storage and data management processes. The solution came naturally for the latter: the use of the technical stack Azure Tables, Data Factory, Azure Synapse, and Functions. The value of adding Azure Synapse Analytics as part of the tech stack is the fact that we could take advantage of its massive parallel processing engine. For the data storage optimization, we used MongoDB provisioned on Azure Premium Storage capable VMs.
This data-driven solution not only used Azure's full potential, but Cars2View was able to process a 10 times higher amount of data. The platform also allowed for data exploration, big data analytics, and machine learning tasks performed from a unified environment. Due to this solution, the processing volume increased from ~5 million to ~50 million events daily.
“The Azure Functions-based solution we implemented based on the architecture designed by Zitec allows us to scale to over 10 times the current volumes without compromising on data accuracy or performance.”
✅ Increased performance with microservices and AKS migration
Zitec’s DevOps team helped Cars2Click adopt AKS, Azure Kubernetes Service, as the main application deployment environment. When processing the high amount of daily workloads, on AKS, Cars2View’s infrastructure was able to scale much faster. We also split the monolithic application into microservices, which helps in optimizing the resources allocation. This way we reduced the processing time from 2 hours to under 10 minutes for a 10x higher volume of data.
Future plans
“All our requirements and needs for our cloud infrastructure were delivered on time by Zitec.”
For the future, the company plans to scale 10x more the volume of data that will be ingested and processed. The already implemented setup by Zitec has been tested for this load and is fully ready to support it.In today’s fast-changing business environment, using modern technologies to improve user experience, optimize the software delivery process and make sense of the data that surrounds your organization can turn into a significant competitive advantage. If you need a partner to help you adopt state-of-the-art technologies that generate tangible results, let’s talk!
Share via:
