Scheduled data processing & budget reconciliation
UI-to-Core Flight Pipeline
A nightly pipeline translating customer-facing budgets into concrete flight records for downstream delivery services.
Business problem
The challenge
Recurring UI schedules needed to become monthly core flights without losing identifiers, spend history or billing compatibility.
Technical solution
Our approach
BullMQ scheduled jobs, flight conversion and reconciliation against existing core records.
- 01
Translate the schedule
Converted recurring customer budgets into monthly core flights with proration and advertiser timezone handling.
- 02
Reconcile existing records
Preserved or updated identifiers, archived removed flights and maintained spend history and downstream compatibility.
- 03
Process scheduled and immediate changes
Built nightly future-month generation and queued user-triggered budget/billing updates, with logging and script-based dry runs.
Results & impact
Project outcomes
A nightly pipeline translating customer-facing budgets into concrete flight records for downstream delivery services.
Future-flight generation
Ongoing budgets produce the core records required for the next month.
Spend and flight continuity
Updates account for existing IDs, spend history and removed schedules.
Visual assets
How the workflow fits together
UI-to-Core Flight Pipeline
flowchart TD A["Nightly schedule or budget edit"] --> B["BullMQ job"] B --> C["UI-to-core conversion"] C --> D["Reconcile existing flights"] D --> E["Downstream delivery records"]
Tech stack
Technology and ownership.
Built the conversion and scheduled-processing pipeline connecting UI budgets to core backend services.