All news with #cloud composer tag

Back Market Migrates to Google Data Cloud, Cuts Costs

🔁 Back Market migrated its data and core tech stack from AWS-based Snowflake and Databricks to Google Cloud, consolidating all historical and operational data in BigQuery. The team executed a two-week proof of concept and a live double-run migration that kept production on Databricks while writing to cloned BigQuery tables until outputs matched. They replaced AWS DMS with Datastream, implemented hourly batching to control small-file costs, and completed critical switchover in six months. The move halved data processing times, cut CDC costs by 90%, reduced technical debt, and improved observability, governance, and developer productivity.