Is your Shopware ERP integration syncing slowly?

We decided to write a new blog post on this topic to help anyone currently struggling with the performance of their ERP integration via the API. Let’s say this up front: the API is not slow in itself. With the right configuration, it can be very powerful.

But let’s start at the beginning, without getting too technical. What is the advantage of importing and reading data via the Shopware API? Thanks to Shopware’s API-first approach, we can transfer information such as customer data, master data, product data or orders to Shopware and create, update or delete it there.

The same applies to deleting and extending data. We can handle all of this with a single connection and the corresponding API requests. To make tools such as an ERP or PIM communicate as cleanly as possible with a shop system, these API interfaces are the standard today. Unlike a CSV import or a direct database injection, data is not simply written. The API reacts to an incorrect entry, missing mandatory fields, a string exceeding its length limit (for example a product number/SKU that is too long) or other errors. This makes it possible to read an error message in the shop and report the same error back to the ERP or PIM.

This protects the shop from incorrect input and prevents malfunctions. So why does this API process in Shopware take so long with some ERPs, middleware or PIMs? Depending on how complex the data is, the API goes through several steps. The ERP, for example, opens a connection and checks whether an item already exists. In our example the shop answers “no”, so the ERP sends the command to create the product in the shop. The shop responds, generates a “create” command, creates the item and sends the UUID (a kind of randomly generated number that identifies the item in the database) to the ERP. If we are lucky, the ERP stores it internally.

Now the ERP sets off again and wants to assign the manufacturer to the product, for example. So the ERP checks whether the manufacturer already exists and has its UUID reported back as well. Now the ERP can tell the shop to assign the manufacturer to the product using that UUID. The shop plays this game of ping-pong for a large number of fields. Property groups and properties, manufacturers and images in particular tend to slow the process down.

So how can this process be sped up when creating a product?

Put simply, the first step of our optimisation is that the ERP must learn to cache the UUIDs. Then, in our earlier example, the item would be created and the second update would write the manufacturer and property assignments directly instead of requesting them again.

It sounds trivial, but if we scale it up, optimising data flows like this can reduce the time from around 15 seconds per product to under one second. That means roughly 60 products per minute instead of around 4. For larger data volumes, it is advisable to generate a delta and only update data that actually needs an update, if your ERP can’t make that distinction itself.

There are solutions for this, such as Yedi or Lobster, with which we have created more than 900,000 products per day in the past. We expect that, depending on the approach, more than 1.5 million full updates can be achieved.

Alternatively, you can set up a dedicated push for the most important processes that only updates stock and price, and then add new products overnight. If that isn’t possible, a partial update can also be implemented as a plugin that automatically processes a CSV file.

Tip: we don’t recommend direct database injection, because

• 1. you have no control over incorrect entries,
• 2. it can bring the shop down,
• 3. the product still has to go through the indexer to appear in the storefront and in the search. If you need technical support to improve the performance of your ERP integration with Shopware 6, we will be happy to advise you!