Continuing our journey through the paradoxes of productivity, we find ourselves grappling with the elusive problem of data duplication in enterprise management systems—a malady affecting thousands of businesses worldwide, much like a persistent cold that keeps coming back.
Amid numerous discussions seeking solutions, the team launched a brilliant idea: What if we created a General Merger? With this tool, we could resolve issues across all areas. The idea of killing multiple birds with one stone, of course, ignited enthusiasm in the development team, and users got excited like children with a bag of candy.
The challenges of Data Merging: A slippery slope
However, there were some things to define before entering the battle. Merging basic data was feasible in an initial version with reasonable effort, but how do you integrate relational fields, such as the list of responsible parties or the followers of a task? Any solution on this path would require lengthy design and development time. So, we made a practical decision: the solution would always take these data from the object selected as the primary one. Thus, we embarked on the adventure with the simplest variant, driven by the desire to find a solution that could see the light as soon as possible.
When we launched it, it felt like a pioneer race in the West. Eager to unload the workload, we applied the solution to every problem that arose, seeking to lighten the volume of work. With the solution, we could create merge actions for any object in Odoo: opportunities, tasks, labels, and support tickets. For each of them, we merge the fields according to the nature of the information required.
With the mixing actions created by the administrators, users in sales, projects, support, and other modules could easily organize the information they were working with. We consolidated duplicate tasks, unified lost opportunities, and standardized labels and categories throughout the system, saving the souls of administrative mediocrity. But, as is often the case in software, the General Consolidator had an exposed flank we hadn’t imagined.
Soon, problems began to bloom like cracks in an ancient wall, appearing without warning and threatening the stability of work. It all started with the salespeople, eager to organize their work, who began unifying multiple opportunities at the same time, often without adequately reviewing what they were combining. “Everything got thrown into one!” they shouted, as if they had dropped a jar of sand on the beach. Imagine the scene: salespeople calling the tech team, asking, “How do I recover them? Can I undo the merge?”
But, as is often the case in the software world, at that moment, there was no solution. We could only explain to users how to locate the messages in the new opportunity to redo the opportunities they had lost. From there, the salespeople would have to take on the work of ancient art restorers: reconstructing a broken vase from ancient Greece, piece by piece.
So we came up with a solution: it wasn't enough to teach them what they were merging; there had to be a limit on what could be combined. First, we used a fixed number for all objects. However, this was not sufficient, as not all merging actions carried the same level of risk. For project standardization managers (generally advanced users), the maximum number of elements to be combined was too small compared to the number of elements they needed to unify. This forced them to merge multiple times to organize the project labeling. However, the sellers merging opportunities were less skilled and had to handle a larger amount of information, which significantly increased the risk in each merge.
In a new approach, we made the limit configurable. Because, let's be honest, some objects are more delicate than others, and some users are more careful than others.
Lessons from the General Merger
La aplicación general para fusionar datos en los módulos de odoo se convirtió en nuestra arma secreta en la lucha por la limpieza de datos. Pero, justo cuando pensábamos que todo estaba bajo control, los
problemas aparecieron de nuevo. Los usuarios clamaron: "¡El sistema está
mezclando los datos incorrectamente!"
As often happens in support teams, after the initial confusion, laughter usually follows. In the effort to resolve all data duplication problems, consolidated actions were configured, producing results that resembled Picasso’s work in its most abstract phase.
In this sense, businesses could learn a valuable lesson from medicine. You can’t combine medical treatments without considering their interactions; similarly, in data merging, it’s crucial to first understand the nature of the problem before proposing a solution. Only then can we avoid complications and ensure effective results.
Those interested in exploring the subtleties of designing strategies for data deduplication in the business realm will find more details in our article: The Art of Data Merging in Business.
The tool, while powerful, is not a panacea; its implementation has shown us that organizing information requires not only technology but also a deep understanding of the processes and the people managing them.
From this experience, we learned that organization is often a game of chance, and that sometimes the real challenge is simply not to lose one’s head in the process. So here we are, unifying objects in Odoo to maintain order and work more efficiently.