DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Containerization for Data Engineering: A Practical Guide with Docker and Docker Compose
j_m47 profile

Containerization for Data Engineering: A Practical Guide with Docker and Docker Compose

#docker #dataengineering #devops #containers
Comments
2 min read
Join OSA CON 2025: Two Days of Open‑Source Analytics and AI (Nov. 4–5)
altinity profile

Join OSA CON 2025: Two Days of Open‑Source Analytics and AI (Nov. 4–5)

#events #opensource #ai #dataengineering
Comments
3 min read
AWS Glue for ETL
godofgeeks profile

AWS Glue for ETL

#aws #dataengineering #serverless
Comments
5 min read
What to use for data preparation in report, query or analysis business?
esproc_spl profile

What to use for data preparation in report, query or analysis business?

#analytics #dataengineering #datascience
5
Comments
10 min read
Optimizing Data Processing on AWS with Data Compaction
datapenguin profile

Optimizing Data Processing on AWS with Data Compaction

#aws #dataengineering #iceberg #lambda
4
Comments
7 min read
Real-Time Earthquake CDC Pipeline
oliver_samuel_028c6f65ad6 profile

Real-Time Earthquake CDC Pipeline

#architecture #database #dataengineering
Comments
5 min read
Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS
datapenguin profile

Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS

#aws #dataengineering #lambda #sqs
3
Comments
6 min read
The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane
markolekic profile

The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane

#python #dataengineering #api #sql
Comments
5 min read
Understanding Kafka Architecture, Schema Registry, ksqlDB, PostgreSQL, Couchbase, and Microservices
jumptotech profile

Understanding Kafka Architecture, Schema Registry, ksqlDB, PostgreSQL, Couchbase, and Microservices

#architecture #dataengineering #microservices #beginners
2
Comments
3 min read
Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025
bolewara profile

Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025

#machinelearning #deeplearning #dataengineering #pca
1
Comments
4 min read
Introduction to the Confluent REST Proxy
jumptotech profile

Introduction to the Confluent REST Proxy

#api #architecture #dataengineering
2
Comments
4 min read
Why We Need Schema Registry in Kafka
jumptotech profile

Why We Need Schema Registry in Kafka

#systemdesign #distributedsystems #dataengineering #architecture
2
Comments
17 min read
An Exploration of the Commercial Iceberg Catalog Ecosystem
alexmercedcoder profile

An Exploration of the Commercial Iceberg Catalog Ecosystem

#dataengineering
Comments
14 min read
🧠 ClickHouse LEFT JOINs: Why join_use_nulls Matters
ranjbaryshahab profile

🧠 ClickHouse LEFT JOINs: Why join_use_nulls Matters

#clickhouse #sql #dataengineering #join
5
Comments
2 min read
Getting Started Building a Data Platform
jorinvo profile

Getting Started Building a Data Platform

#dataengineering #database #sql
Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.