Web performance optimization for an online retail chain
We eliminated 30% of technical debt, doubled the indexing speed, and cut the number of HTTP 404 errors tenfold for a leading European retailer.
Context
Our customer is a leading European supplier of office equipment and stationery products. The company owns a large chain of online stores spanning 90K+ products and serving thousands of corporate clients via multiple logistic centers.
The online supermarket was built on top of SAP Commerce (formerly SAP Hybris) that allowed for effective management of catalogs, navigation, marketing campaigns, content, etc.
The platform communicated with external systems for analytics collection, email marketing, payment processing, and geolocation information sourcing via REST API. The store’s internal systems (ERP, CRM, etc.) were connected to the online supermarket via the SAP PI integration bus using SOAP protocols. HotFolder was used for those systems that couldn’t be directly connected via SAP PI.
Most of the system’s problems resulted from the incorrect work of Apache Solr, which caused multiple 404 errors, incorrect display of faceted search results, slow indexing, delayed price database updates, errors in autocomplete and synonym searches, as well as recurring failures in payment processing.
In addition to technical difficulties, the customer had to deal with other issues:
- Unsynchronized teams. The customer’s team wasn’t synchronized with the geographically distributed teams of the contractors who were developing the system for the company.
- Lack of workflow visibility. It was difficult to track workflows of different teams collaborating on the same projects, which made it impossible to plan accordingly.
- A poorly organized structure. Any employee from the development, sales, marketing or other departments could create and assign arbitrary tasks without consulting a business analyst first.
- Out-of-sync delivery schedules. Multiple teams were working on separate tasks and projects, and product features were released out of sync.
- No knowledge transfer. After the customer’s SAP Commerce technical lead left the project, the product updating and version delivery processes were stalled.
- No documentation system. There was no unified system for maintaining project documentation with a shared access for all the teams.
The customer was searching for a technological partner to help set up collaboration principles, implement best practices, solve technical issues, and optimize the web performance of their online store. They chose Itransition due to our proven expertise in SAP Commerce and software maintenance and support.
Solution
After a thorough analysis of the customer’s solution, we uncovered and solved the following technical issues:
Faulty indexation
We identified numerous code errors that caused incorrect data indexing and irrelevant search results hampering the user experience. To solve this problem, we reworked and optimized the Apache Solr indexing process that resulted in 10x fewer 404 errors. We also fixed problems with the indexing speed, doubling it.
Faceted search errors
The faceted search with enabled filters was working incorrectly. As a result, users were unable to find certain products. After our team fixed the errors, users could find necessary products much quicker.
Price update cache failures
The store’s platform used data from the customer’s other systems in order to update and calculate prices. As outdated values were deleted incorrectly, prices were recalculated with errors, which led to system crashes and errors when displaying products on the website. To fix the problem, the customer had to manually clear the cache or restart individual system nodes.
We decided to rewrite the outdated data removal mechanism and update caching. Due to these changes, the number of errors associated with incorrect price caching decreased to zero.
Errors in autocomplete and synonym search
This feature used by the marketing department became highly inefficient. We rewrote and optimized the mechanism, allowing the website users to find specific products much faster.
Checkout failures
There were multiple errors at certain steps of the automatic checkout that could result in hours spent on customer service assistance. We found out that the reason for failures was the ineffective functionality implementation that caused asynchronous order processing on different nodes of the cluster.
Process
Itransition assisted the customer in implementing best development practices and optimizing processes to correspond to the corporate goals and their project management environment.
We set up transparent processes between different teams by syncing their workflows in Atlassian Jira. Following our recommendation, the customer added a separate phase for business analysis, task creation and assignment. We also developed a plan for organizing communication and collaboration within each microteam as well as across all teams.
During regular meetings, we shared the project vision, analyzed feedback from key decision makers, discussed interdependencies and bottlenecks to sync the release schedule. We also developed some functionality in accordance with the WSJF (Weighted Shortest Job First) principles to prioritize tasks efficiently.
Itransition’s team helped customize the repository management module to meet the customer’s needs, while also applying the continuous integration (CI) system for this module. Since the customer had already been using Jenkins, we helped them fix the problem associated with storing sensitive configuration data by implementing a separate storage space and integrating it with the CI system as well.
Additionally, we helped implement such best development practices as:
-
A standardized coding style
We developed and implemented a set of rules and conventions for the source code in accordance with the customer’s needs. We also participated in automating the process of Checkstyle verification, which cut the code verification time and increased code readability.
-
Code branching
We proposed a branching process configured specifically for several geographically distributed teams working on multiple parallel projects, in order to make collaboration more transparent and coherent.
-
Cross-reviews
We helped form new task management principles that took into account the interests of all team members and supported various project management models. We helped the customer establish processes that allowed teams to plan release dates and assign tasks, thus improving the results of our mutual efforts.
Results
Throughout more than 14 months of collaboration, Itransition’s team were the customer’s trusted technological partner. We minimized manual operations, which increased system stability, performance, and productivity. We also implemented best development and web performance optimization practices that brought the following results:
- 30% of the technical debt was eliminated
- The Apache Solr indexing became twice as fast
- HTTP 404 errors were reduced tenfold
Later on, the company hired us for another project that involved online supermarket optimization for low-frequency search queries.
