Databricks Developer
Job Description
ARE YOU READY to step up and take your technology expertise to the next level?
There is never a typical day at Accenture, but that's why we love it here! This is an extraordinary chance to begin a rewarding career at Accenture Technology. Immersed in a digitally compassionate and innovation-led environment, here is where you can help top clients shift to the New using leading-edge technologies on the most ground-breaking projects imaginable.
Cloud First - Software Engineering is a high performing team to join!
We focus on client adoption of disruptive technologies, technology architecture and providing specialized skills related to integration, custom software engineering, testing, application modernization, agile and more. We help our clients with the most complex projects including working on open web platforms, DevOps platforms as well as intelligent Computing and Architecture enhancement.
YOUR ROLE: Databricks Developer
What You'll Do
As a Databricks Developer, you will play a critical role in designing and implementing robust data pipelines, stream processing solutions, and data migration solutions using Databricks. Your expertise will be crucial in designing & optimizing operational/analytical data processes and mentoring junior team members. You'll work closely with developers, analysts, product owners, and business stakeholders to ensure that data solutions are scalable, efficient, and aligned with business objectives.
Qualifications
Key Responsibilities
- Design and Implementation: Lead the design and development of ETL pipelines and real-time data processing solutions using Databricks and Apache Kafka. Ensure data integrity, quality, and performance through best practices.
- Architecture and Optimization: Architect scalable data solutions to handle large volumes of data. Optimize existing pipelines and workflows for performance, reliability, and cost-efficiency.
- Cross-Functional Collaboration: Work closely with team members, and product managers to understand data requirements and deliver solutions that meet business needs. Participate in project planning and technical discussions.
- Innovation and Continuous Improvement: Stay updated with the latest advancements in data engineering technologies and methodologies. Evaluate and recommend new tools and technologies to improve data processing and analytics capabilities.
- Documentation and Best Practices: Develop and maintain comprehensive documentation for data pipelines, architectures, and processes. Establish and enforce best practices for data engineering and ETL processes.
Basic Qualifications
- Experience: Minimum of 3 years of professional experience in data engineering with a strong focus on Databricks and Apache Kafka. Proven track record in designing and implementing large-scale ETL pipelines and stream processing solutions.
- Technical Skills: Deep understanding of Databricks, Schema management, SQL, Spark/Python, and Kafka. Proficiency in Delta Tables, DLT, and Postgres, , with experience in other programming languages such as Scala or Java being a plus.
- Data Architecture: Solid experience in designing scalable data architectures and optimizing data workflows. Familiarity with data warehousing concepts and technologies is highly desirable.
- Education: Bachelor's degree in computer science, Engineering, or a related field. Advanced degree or relevant certifications in data engineering or cloud technologies is a plus.
- Language: English is required for this position as this role will regularly interact with stakeholders across Canada, US and other countries across our Global footprint where English is the common language. Due to the significant high volume of interactions with these English-speaking stakeholders, which is inherent to this position, it is not possible to reorganize the company's activities to avoid this requirement.