ETL-Suite in Odoo
The ETL suite from braintec integrates the standardized workflow – Extract, Transform, Load – for managing data processes directly in Odoo. The suite enables seamless migration, integration, and synchronization of data from external sources or other Odoo instances.
The ETL workflow of braintec for Odoo
The ETL suite from braintec automates the data exchange between third-party systems and the Odoo business software. While the classic ETL workflow processes data in three phases, braintec's ETL suite separates mapping and validation into their own phases to optimize modularity, clarity, and maintenance.
1
Extract
Extracting raw data from any sources – either sequentially or in parallel using flexible extractor components.
2
Transform
Normalization and cleansing of different data formats using configurable rules and automatic type conversion.
Mapping: Intelligent assignment of data to Odoo target fields or user-defined structures from third parties, including validation and supplementation of default values.
3
Load
Import of data via loader components directly into the Odoo file structure or proprietary target systems.
ETL-Suite: The Swiss Army Knife for your data management in Odoo
Third-party integrations
Seamless integration of Odoo with specialized or proprietary enterprise ERP systems such as SAP, Abacus, Microsoft Navision, as well as CRM and Product Information Management (PIM) systems like Salesforce, HubSpot, or Akeneo.
Data Warehousing and BI
Export of structured and normalized Odoo data to data warehouses or data lakes for advanced reporting and business intelligence.
Master data management
Automated processing of large data sets from catalogs, inventories, or customer data with guaranteed data record integrity thanks to configurable validation rules.
e-Commerce Integration
Reading and processing data with complex and nested data structures from online shop systems via webhooks or HTTP-REST APIs in Odoo.
ERP data migration
Structured transfer of extensive and complex datasets from legacy systems or older Odoo versions for a clean start in the new Odoo environment.
Omnichannel and Retail Sync
Central data hub for synchronizing data from POS cash register systems, e-commerce platforms, or planning and project apps with Odoo.
Functions of the Odoo ETL Suite by braintec
Data processing
Hierarchical Parent-Child Imports
Processing of complex data sets such as sales orders with associated items like lines in a single pass.
Dry-Run-Modus
Simulation and preview of the exact results from the extraction, transformation, and mapping phases, without actual changes in the target system.
Parallel processing and intelligent grouping
Division of processing phases into individual packages as well as parallel processing in multiple queue jobs. The intelligent grouping processes related data records in the same job and prevents conflicts during simultaneous updates.
Error control and traceability
Validation errors are stored directly with the affected record, with each record documented by a detailed processing log.
Upsert functionality
Existing records are recognized and updated instead of duplicated.
Planned Pipeline Execution (Cron)
The integrated cron job runs every five minutes and automatically processes all pipelines with the "Auto-Fetch" function enabled. In combination with SFTP/HTTP/Cloud sources, this enables fully automated, scheduled imports.
Repetition in case of erroneous records
Failed records are reset to "Draft," and new sub-orders – only for these records – are created, which is very helpful for resolving partial errors.
Multi-Object Export with Join
Exports multiple pipeline objects into a single file (CSV/XLS/XLSX). The merging can be done either by row index or by a key value.
Data Connectors
- Tables + Calculations: Bidirectional for delimiter-based text files CSV, TSV, SSV, PSV via configurable delimiters as well as Excel files (xls and xlsx).
- Structured data: Bidirectional for JSON and XML user data.
- Direct SQL connection for user-defined data queries or for uploading to database tables.
- API interfaces: HTTP/REST APIs (with OAuth2) and SOAP Web Services (with WS-Security).
- Odoo-to-Odoo: Direct extraction from Odoo models and transfer to Odoo target instances.
- Secure file transfer: Enhanced SFTP connection from braintec with connection pooling.
- Multi-Backend Cloud Storage: Supports S3, Azure Blob, FTP, local file systems, WebDAV, and any fsspec-compatible backend.
* The ETL suite offers numerous interfaces for read (extraction) and write (loading) operations for many data formats and is continuously being updated.
Data transformation
Predefined Rules
Extensive library with integrated text formatting (removing whitespace, adjusting case, etc.), type conversions (floating point, decimal) for mathematical calculations and date conversion, value mapping, etc.
Integrated validations
For strict data integrity before the loading process. The validations include:
Required field (not empty) | Is Integer (whole number) | Is Float/Number |
Is Positive | Is Not Negative | Minimum/Maximum value |
Minimum/Maximum length | Is date and time indication | Is email |
Regex-Match | "In List" | "Not in list" |
In addition, custom validation rules can be defined using user-defined Python.
Odoo-specific search
Querying the relational Odoo database for linked record IDs or values through Odoo domains, XML IDs, or exact field matches.
Customized Python scripts and extensibility
For complex logic, custom Python scripts can be created for both data transformation and mapping validation.
AI and Big Data
While standard Python libraries such as re and datetime, dateutil, calendar, random, and time are directly available, external libraries like Pandas for extensive data processing or LangChain for AI-driven data processing can be directly integrated into one's own ETL pipeline.
Book your Odoo demo
Whether it's a one-time migration or daily mass import: Our Odoo experts will show you how to quickly and easily transfer large amounts of data into your Odoo environment.
Arrange your free consultation here
Contact
FAQ ETL-Suite
Here we answer the frequently asked questions regarding the ETL suite from braintec for Odoo.
By separating the field mapping from the transformation, the process becomes more modular and easier to maintain. Validations can be specifically carried out before loading, which ensures data quality in Odoo.
The ETL suite utilizes parallel processing and the Odoo Queue Job System. Data packets are divided into chunks and processed simultaneously. "Smart Grouping" prevents conflicts during simultaneous write access to related records.
Yes, the framework has a comprehensive "Dry Run Mode." This simulates the phases of Extract, Transform, and Map completely, without making changes to the target system, allowing results to be checked in advance.
Yes, the suite supports hierarchical parent-child imports. This allows complex datasets (e.g., headers and their corresponding lines) to be processed consistently in a single pass.
