Apache Beam Software Engineer Maestro Technologies, Inc.

About Maestro Technologies, Inc.

Job Description

Job Description: As a Software Engineer with 4 to 6 years of experience, you will be responsible for designing and developing framework-level components using Apache Beam, GCP Dataflow, and Dataproc. Your primary focus will be on building scalable and reusable data processing frameworks for both batch and streaming use cases. You will collaborate closely with architects to implement best practices and ensure high-performance data frameworks. Strong programming skills in Java or Python are essential for this role, along with deep experience in Apache Beam and GCP Dataflow. Your key responsibilities will include designing and developing framework-level components using Apache Beam, GCP Dataflow, and Dataproc. You will be building scalable, reusable libraries and abstractions in Python or Java for distributed data processing. Your role will also involve working closely with architects to implement best practices for designing high-performance data frameworks. Additionally, you will ensure software reliability, maintainability, and testability through strong coding and automation practices. Participation in code reviews, architectural discussions, and performance tuning initiatives will be part of your routine tasks. Moreover, contributing to internal tooling or SDK development for data engineering platforms will be expected. The ideal candidate should have a solid understanding of streaming versus batch processing concepts, familiarity with CI/CD pipelines, GitHub, and test automation. Preferred skills include experience with workflow orchestration tools such as Airflow (Composer), exposure to Pub/Sub and BigQuery, and understanding of observability, logging, and error-handling in distributed data pipelines. Experience in building internal libraries, SDKs, or tools to support data teams will be an added advantage. In this role, you will work with a tech stack that includes Cloud services like GCP (Dataflow, Dataproc, Pub/Sub, Composer), programming languages such as Java and Python, frameworks like Apache Beam, and DevOps tools like GitHub and CI/CD (Cloud Build, Jenkins). Your focus areas will include framework/library development, scalable distributed data processing, and component-based architecture.,