Integrations and distributed systems

Getting systems that were not made for each other to talk, and to keep talking when one of them changes. It is the core of my work: almost none of the services I maintain solves the problem alone.

On the other end of the line there is a bit of everything. ERP, CRM, customer service platform, card acquirer, anti-fraud, marketing, marketplace, carrier, credit bureau. None of them speaks the other’s protocol, and a good part of what I write is the midfield between them.

What you learn doing this is not variety of technology, it is variety of ways to fail. A bank responds differently from a marketplace, which responds differently from a manufacturer. Each has its own way of going down, its own request limit and its own interpretation of “I already processed that”. And these are not résumé integrations: they are contracts that have already broken, changed without warning or responded differently from the documentation while I was the one responsible.

What I do

• Design integration contracts between systems with different owners, including what happens when the other side goes down
• Write middleware that translates the domain model between platforms with incompatible vocabulary
• Build ETL for loading and reconciliation between a source and a destination that disagree
• Handle idempotency, reprocessing and arrival order: the order that arrives twice, the callback that arrives before the confirmation
• Publish an internal SDK so that whoever consumes my service does not rewrite the HTTP client
• Decide between a synchronous call, a domain event and a scheduled job by the nature of the coupling

The trade I make without thinking much

A dedicated translator per platform, instead of a generic bus. It is more code and less coupling, and the whole reasoning is in How I work.

Where I applied it

Payments and anti-fraud · Manufacturer reporting · Experience · Customer service

What I have not done yet

My integration pattern is predominantly synchronous over HTTP plus scheduled jobs. I have never built an event-driven architecture with a central broker in production, nor applied event sourcing. It is the gap I would choose to close first.