Description
- Azure Data Engineer Description for Experienced Candidates
- Role summary: An Azure Data Engineer designs, builds, and operates scalable data platforms on Microsoft Azure to enable analytics and BI.
- Core responsibility: Own end‑to‑end data pipelines (ingest, ETL/ELT, transform, load) and ensure reliable data delivery for downstream consumers.
- Platform skills: Deep hands‑on experience with Azure Data Factory, Azure Synapse, Azure Databricks, Azure SQL/Managed Instances, and Blob/ADLS storage.
- Big data and processing: Implement batch and streaming solutions using Spark, Delta Lake, and event frameworks to process large volumes with low latency.
- Data modelling: Define robust logical and physical data models, partitioning strategies, and schema evolution patterns for performance and maintainability.
- Security and governance: Enforce RBAC, encryption, data masking, lineage, and compliance controls to meet enterprise policies and regulatory requirements.
- Performance and cost: Tune query performance, optimize storage formats, and apply cost‑management practices across compute and storage.
- Automation and DevOps: Build CI/CD pipelines, infrastructure as code (ARM/Bicep/Terraform), automated testing, and deployment for repeatable releases.
- Observability: Implement monitoring, alerting, and SLA reporting for pipelines, jobs, and data quality metrics.
- Advanced engineering: Lead architecture for multi‑tenant, hybrid, or multi‑cloud data platforms and design for high availability and disaster recovery.
- Leadership and mentoring: For senior levels, drive technical strategy, mentor engineers, conduct design reviews, and liaise with stakeholders to translate business needs into data solutions.
- Career progression: Expect growth from hands‑on pipeline development to solution architecture, platform ownership, and data leadership roles.
