SAP Commerce-based feeds creation and management system

Solution

Our team developed an SAP-based engine that generates unique feeds for products and groups of products, regions of sales, and marketplaces so that clients could see accurate item information on different advertising platforms.

Preliminary price calculation

Analyzing the customer's system, we noticed that marketplaces banned users for incorrect feed files (for example, out-of-date information, including product prices). To prevent irrelevant prices bans, provide up-to-date product feed data, and increase feed generation speed, we integrated a preliminary price calculation module into the customer’s ecommerce system. Тhe module plans the price changes based on price configurations (purchase price, base price, planned discounts, etc.) for a certain period, region of sale, etc. Then, the system sends prices aggregated for a certain period, region, company, or category to the feed generation system.

Feed generation

Our SAP specialists improved the customer’s feed generation system, adding the ability to define the fields included in or excluded from the feed, set up a UTM code, and configure promo text. The system now also allows for flexible generation of product feed files to be uploaded to marketplaces. The system creates a feed file based on:

• product properties (category, brand name, SKU number, description, price, images, discounts, delivery terms)
• sales region
• selected marketplaces (Google Merchant Center, Facebook, Instagram, Criteo, RTB House)

Then, the system saves the feed data as an XML file in the database and generates links to feed files. This way, the feed data in marketplaces refreshes automatically when the marketplace is set up. Feeds are generated at different times to distribute the system load. Since some feeds include business-sensitive information, we’ve added the ability to set up, change and delete the password to access the feed.

We also added the ability to classify feeds in the admin panel. While uploading the product list in XML format, admins can massively include or exclude products, organizing them by trademarks, brands, product groups, markets, or negative keywords.

The customer can also step up their promotional product sales by generating the feed at the product level, adding a promotions type, gifts with purchases, delivery terms and cost, etc.

The system pulls the relevant data from the preliminary price calculation module and imports prices that are valid for a certain period, region, customer, or group of products in the feed files. This proved to be beneficial, as many clients base their choices on relevant and up-to-the-minute data.

We enabled the customer to configure the feed files that meet technical requirements, like file size or specific fields availability, for all the selected marketplaces to run effective shopping campaigns on each marketplace. For marketplaces that cannot process feeds larger than 100 MB, the system splits a large feed file into several smaller ones.

Reports creation

We added a product report generation system to monitor and eliminate errors when generating feeds. The system generates two types of reports, one for tracking errors related to incorrect data entry and another for errors stemming from marketplace tech requirements. Two reports are created at the end of feed generation.

To analyze marketing campaigns, the report includes the number of products included in or excluded from the feed with the reason specified.

We also created templates for reports containing technical information, such as the size of the feed file in MB, date of feed generation, feed generation time, last time the balance was updated, the time of price, discounts last update status, etc. Here's how a feed technical summary looked like:

Number of products included in the feed	77,275
File size	197Mb (100Mb, 97Mb)
Date and time of feed generation	11/12/2021 20:15
Feed generation time	3 minutes
Last time product balance status was updated	8 minutes
Last time price status was updated	10 minutes
Last time gifts status was updated	274 minutes

Analyzing feed quality changes

We also developed a system for analyzing changes in the quality of feeds by various metrics (like feed generation speed, last time of prices, and balance updates). If the metrics deviate from the setting value, the admin is sent a percentage deviation and a short description of the problem. Thanks to this functionality, the admin is timely informed about the reasons for specific errors and doesn’t need to constantly monitor the feed generation system, because it is automated.

Process

To increase transparency, we divided the project into several phases. We started with the preliminary price calculation system development, creating the solution’s core features and then enhancing the feeds generation module, scaling regions of sales for up to 30, adding new fields, creating a reporting feature, and integrating modules with each other. 

Then, we gradually scaled the solution’s functionality and finalized the enhancement of the SAP feed generation module. Now, the system can generate 250+ feeds for all customer’s products and 50+ regions of sales in Google Merchant Center, Facebook, Instagram, Criteo, RTB House, and other marketplaces in line with their technical requirements in less than five minutes.