One of the most common tasks I have to deal with in Magento are integrations. Almost any system I worked with needs to push order or product data to an ERP or marketing software, or get stock/product info/tracking numbers from an external source. Every integration is unique in a few ways…
- the communication channel. REST, SOAP, sending files via SFTP are common variations
- the data format. JSON, XML, CSV to name a few
- the actual data mapping. We either pass fields as-is or combine/transform them.
However, any good integration has a lot in common:
- a log must exist so we can refer back to when and how each piece of information was synchronized
- errors must be logged, with an alert system so we are aware of any failures
- the system should be able to queue data in case the integration is down. We cannot lose info especially when dealing with financial records.
- the system must be able to retry failed records
- the actual field mapping must be easy to change, ideally without changing code
I have been looking for good solution to build integrations for a while. On the lower end, there is the do it yourself custom shell script. Easy to build, but usually missing key elements, like retries or flexible data mapping. On the higher end, we have full ETL solutions. They tend to be expensive and add new software components to the mix.
Almost accidentally I stumbled upon Xtento’s Import/Export suite – https://www.xtento.com/magento-2-extensions.html . They flew under my radar as I had bad experiences in the past with such extensions and for a while, concluded that the best import/export is the one you built on top of Magento’s default.
Let’s go over the steps involved to export orders from Magento via Xtento. First, one starts with defining destinations. A destination is the place where your orders will go. You have a few options:
- on the local server in a specified folder
- on a remote server via a SFTP or FTP connection
- to an email
- to a HTTP server (i.e. REST)
- to a web service. Use this for XML-RPC or SOAP integrations
- via a custom class, where you can define the communication with your exotic system
So the above options should cover any possible target. One nice thing is that you can have multiple destinations, so you could place the orders on a SFTP but also mail a copy of the files to you for later review.
After defining the destinations, the next step is to define the export profile. Most options are obvious so I will go only over the important ones:
- you can choose the entity to export, i.e. orders. Usually each exportable entity corresponds to a specific extension that you need to buy from Xtento.
- you can define filters. For example, you can decide that you only want to export “processing” orders, keeping the “pending” ones in queue until they are reviewed
- You can define the export mapping as an XSLT template. This was the feature that I was sold on. XSLT templates allow you to be as expressive as you need to. You can use all the standard fields, apply transformations, use all the fields in the order/related tables (your custom ones too). All this with a nice autocomplete interface and a dead-simple way to test the template with an existing order. Once you get the hang of it, you almost never need to refer to the docs/examples, it’s that easy. If you do need help though, https://support.xtento.com/wiki/Magento_2_Extensions:Magento_Order_Export_Module#XSL_Template has you covered.
- You can define when your export runs. Do you want to export orders on schedule? How often? Do you export all orders in the last N hours or only what’s new? If the export process is time consuming, you have a CLI command to run it outside the main Magento cron.
- You have a flexible manual export option in case you need to replay any of the missed exports, or simply test the process.
Everything comes with logging. You have filterable grids where you can see the exported entries and errors. You also have the option to locally store the export files for later review.
If you need to import entities, Xtento has you covered too, The process is very similar in that you still have sources where you can pull from, profiles you can define, same logging capabilities, a way to map the data. In addition to imports, you have an actions section where you can define what happens when an entry is imported. For example, when you import a tracking number, you can have Xtento ship and bill the order automatically.
I should mention that currently Xtento does not offer a product import solution. You can import stocks, but not product data. I’d love to see that on their offer sometimes.
What I really like about the extensions is that they are developer-friendly. Almost everything in their system has a fallback to a custom class. You have a very exotic export destination? You can define a class to implement the communication logic. Need to map your data in a way the XSLT template does not support? You can define a class and a method just for that. Finally, having logs for all operations make it easy to identify random issues. It scales ok too, I have been exporting importing 100k records per day with no performance issues.
Here are a few usecases where I have used Xtento successfully, usually without writing any line of code:
- product exports to marketplaces/product aggregators. I still prefer specific extensions for big marketplaces, like Amazon or Google Shopping, but will use Xtento for the ones that have poor extensions or none at all.
- pushing order data to ERPs and getting back tracking info, billing and shipping automatically. That’s a huge one, before Xtento I used to spend a lot of time on these type of implementations.
- pushing order and product data to marketing software, like mailing list managers.
- importing stocks from an external source (usually ERP).
Xtento might be lacking all the bells and whistles of an ETL solution, but in an ecosystem where not everyone has a fortune to spend on an integration, their extension suite is more than decent to get things done.