Big Data Engineer jobs

Spaulding Ridge is an advisory and IT implementation firm. We help global organizations get financial clarity into the complex, daily sales, and operational decisions that impact profitable revenue generations, efficient operational performance, and reliable financial management.

At Spaulding Ridge, we believe all business is personal. Core to our values is our relationships with our clients, our business partners, our team, and the global community. Our employees dedicate their time to helping our clients transform their business, from strategy through implementation and business transformation. 

What You Will Do and Learn

We are seeking a highly experienced and motivated ETL Architect to lead the design, development, and optimization of scalable ETL pipelines and modern data integration solutions. The ideal candidate will possess deep expertise in ETL architecture, data modeling, and hands-on experience with industry-leading tools such as Fivetran, Informatica, Azure Data Factory (ADF), and Snowflake. Strong communication skills and a proven track record in team leadership and cross-functional collaboration are essential for this role. 

Key Responsibilities: 

• Architect, design, and implement robust ETL solutions, ensuring scalability, reliability, and performance. 

• Lead the development of complex ETL pipelines that extract, transform, and load data from multiple sources into data warehouses or data lakes. 

• Ensure efficient integration of systems using tools such as Fivetran, Informatica, ADF, and Snowflake. 

• Set up and configure ETL environments, including choosing appropriate tools, platforms, and technologies. 

• Define and implement standards for ETL environments across development, testing, and production stages. 

• Ensure all required software, libraries, and configurations are in place to support ETL jobs. 

• Establish ETL monitoring processes and alerting mechanisms for operational support. 

• Build and manage data pipelines using tools like Fivetran, Informatica, Azure Data Factory, and Snowflake. 

• Design and implement CI/CD pipelines for automated testing, deployment, and monitoring of ETL processes. 

• Collaborate with DevOps and engineering teams to integrate ETL pipelines with version control (Git), CI/CD tools (Jenkins, GitLab, Azure DevOps), and testing frameworks. 

• Ensure seamless integration and deployment of ETL jobs across environments (e.g., dev, test, prod). 

• Design and implement solutions for managing SCD types 1, 2, and 3 in the data warehouse. 

• Ensure that data integrity and historical data tracking are maintained across changing datasets. 

• Design efficient data models and data warehousing solutions optimized for Snowflake. 

• Ensure data quality, consistency, and reliability through rigorous testing and monitoring. 

• Provide technical leadership and mentorship to junior engineers. 

• Coordinate with project managers and stakeholders on project scope, timelines, and deliverables. 

 

Required Skills & Qualifications: 

• 8–10 years of experience in ETL development and architecture. 

• Extensive experience in designing and architecting ETL pipelines using tools such as Fivetran, Informatica, ADF, and Snowflake. 

• Proven expertise in setting up and managing ETL environments. 

• Strong hands-on experience with CI/CD pipelines for automated deployment, testing, and monitoring of ETL processes. 

• Experience working with APIs, unstructured data, and JSON, Parquet, Flat files etc. 

• Deep understanding of Slowly Changing Dimensions (SCD), including implementation of SCD Type 1, Type 2, and Type 3 in the data warehouse. 

• Expertise in data modeling (e.g., dimensional, normalized models, star/snowflake schemas). 

• Strong proficiency in SQL, with experience working on large-scale datasets and optimizing query performance in Snowflake or other relational databases. 

• Familiarity with cloud data platforms (e.g., Azure, AWS, GCP) and technologies like data lakes, data warehouses, and big data solutions. 

• Ability to lead teams, mentor junior developers, and collaborate effectively with business stakeholders. 

• Experience with data governance and data security practices. 

• Strong problem-solving skills and ability to troubleshoot complex data and performance issues. 

• Experience working with applications such as Oracle ERP / SAP / Microsoft ERP / Salesforce / NetSuite / etc.  

• Good understanding of data within ERP & CRM. Functional expertise in areas like Finance / Procurement / HRMS / etc.   

 Preferred Qualifications: 

• Experience with modern data stack tools (e.g., dbt, Airflow, Matillion). 

• Understanding of data governance, privacy, and compliance frameworks. 

• Experience working in Agile/Scrum environments. 

• Certifications in Snowflake, Azure Data Factory are a plus. 

Spaulding Ridge’s Commitment to an Inclusive Workplace 

When we engage the expertise, insights, and creativity of people from all walks of life, we become a better organization, we deliver superior services to clients, and we transform our communities and world for the better.

At Spaulding Ridge, we believe our team should reflect the rich diversity of society and we take seriously the responsibility to cultivate a workplace where every bandmate feels accepted, respected, and valued for who they are. We do this by creating a culture of trust and belonging, through practices and policies that support inclusion, and through our employee led Employee Resource Groups (ERGs): CRE (Cultural Race and Ethnicity), Women Elevate, PROUD and Mental Wellness Alliance.

The company is committed to offering Equal Employment Opportunity and to providing reasonable accommodation to applicants with physical and/or mental disabilities. If you are interested in applying for employment with Spaulding Ridge and are in need of accommodation or special assistance to navigate our website or to complete your application, please send an e-mail with your request to our VP of Human Resources, Cara Halladay ([email protected]). Requests for reasonable accommodation will be considered on a case-by-case basis.

Qualified applicants will receive consideration for employment without regard to their age, race, religion, national origin, gender, sexual orientation, gender identity, protected veteran status or disability.