Privalia - Case study
E-commerce growth predictions estimate that the market size will likely double and even triple between 2014 and 2021. This translates to a constant need to improve user experience and performances. Privalia as a company is particularly aware of this situation.
Founded in 2006, Privalia holds short brand oriented flash sales on several markets like Brazil, Mexico, Spain and Italy. Privalia belongs to the vente-privee Group, creator and worldwide leader of flash sales concept.
As architecture leader, Philippe is also the manager of the performance team at Privalia. Team that is also directed by Alex Garcia.
This team regroups a lot of skill sets and relies on a mix of the best tools available for APM, monitoring, etc.
A big challenge¶
The biggest challenge with flash sales is the fast ramp up of users and high activity concentrated on short periods of time. Some sales being larger than other, with sometimes the entire stock sold over 5 minutes. On top of that mobile users receive push notifications for new sales. Which means they also rush to the application.
In such a context, performance is critical to achieving good sales but that’s not all. Events like the black friday might not be that big in Europe yet. But it sure is in Brazil and Mexico where sales are up by 45% compared to a regular day. In comparison, Spain will “only” have 34% more traffic. But if you factor this in you get a very difficult mix of constraints.
In response to this, Privalia had to put in place a very efficient testing process. The first step is to get accurate previsions from the business team for major events. This is then used to run the most realistic load test possible. This load test is compared to the previous one to assess the impact of each new version. To do this, every week the performance team tests the new release in pre-production. And every month they run a nightly test in production. The goal is to confirm the pre-production results in a more realistic environment.
The need for speed¶
To achieve this Privalia needed a fast paced tool.
And as Philippe puts it:
The scenario creation process is shorter.
Debug and comparison features make it even faster to check scenarios against each new release. A reduced maintenance is the key towards running more tests. Even more so since Philippe's team maintains a mix of desktop and mobile scenarios.
Another important topic is test automation. Since a test is run on each release in pre-production, they should be part of the continuous integration process. To achieve this, the load should also come from a local source.
This way, tests could be triggered through OctoPerf's API and integrated in Jenkins for build to build comparison. A continuous process like this means the performance team can focus on analysis.
Realistic tests¶
The next step is to run a realistic test in production. Pre-production testing is only relevant to a certain point. Going further requires a test under real conditions. Here again Philippe confirms OctoPerf was a great help:
A test takes a few minutes to configure which means that updating the user ramp up or test location is very quick. Then results provide a lot of insight on response times for each location or each page of the application.
Another functionality that Philippe liked a lot is exporting to Jmeter or importing from JMeter:
Results¶
As a result of the collaboration between Privalia and OctoPerf, no more rollback had to be done. Early tests have definitely helped identify large issues upfront. Also, hardware growth could stay under control even though traffic on the application increased by 18%. OctoPerf helps run more regular tests. Thanks to this Privalia can focus on their expertise on software and hardware optimization.
Since OctoPerf has been implemented in 2016, no incident happened.
The application is breaking new records of concurrent users. And large flash sales are scheduled without any stress for the architecture team.
Other key optimizations include a 32% reduction of load time. This means user satisfaction is on the rise, which is why the number of pages per session increased by 12%.
Key takeaways¶
Philippe is very satisfied with the new testing process:
- Over a one year period the user base also increased by 18%.
- Privalia is preparing its largest flash sales to date.
- Philippe and his team are planning on running even more tests next year: