Data Engineer, AI Pipelines
Advocate
Software Engineering, Data Science
San Francisco, CA, USA
Posted on Thursday, May 2, 2024
Advocate is a mission-driven technology company revolutionizing the way Americans access critical federal benefits. Our cutting-edge AI platform streamlines the application process, ensuring that every submission is complete, optimized, and tailored to the specific requirements of each federal program. Our innovative technology not only simplifies the process for applicants but also significantly reduces the administrative burden on federal agencies, enabling faster and more efficient eligibility determinations.
The Opportunity
Benefits Advocate is seeking a talented Senior Data Engineer to contribute to the data infrastructure and workflows that underpin our AI-driven platform. In this crucial role, you will have the opportunity to make a significant impact on modernizing public benefit program administration by working closely with AI developers, product managers, and various stakeholders.
As a Senior Data Engineer, you will be responsible for researching and integrating comprehensive historical case data sets and truth sets, creating a solid foundation for AI model training and validation. You will design and implement scalable, resilient data architectures that accommodate AI pipelines/services, ensuring seamless integration and functionality of advanced AI models. Additionally, you will optimize data processing pipelines by crafting, refining, and managing ETL (Extract, Transform, Load) processes for optimal data movement and transformation, prioritizing performance and reliability.
Ensuring data integrity and compliance will be a key aspect of your role. You will implement strict data management practices, including data validation, cleansing, and anonymization, to uphold the highest standards of data quality and compliance with legal and ethical guidelines. Moreover, you will support advanced data analysis by equipping data scientists and AI developers with the infrastructure and tools necessary for deep analytics and the operational deployment of AI, enabling data-driven innovation and decision-making.
As a Senior Data Engineer, you will have the opportunity to stay at the forefront of data engineering by continuously exploring and integrating the latest data engineering tools, technologies, and methodologies to enhance the capabilities of our data platform. This role offers an exciting opportunity to leverage your expertise in data engineering to drive the development of cutting-edge AI solutions that will revolutionize the administration of public benefit programs, ultimately improving the lives of countless individuals and families.
Join Us and Make a Difference
We believe in the power of technology to drive positive change and promote fairness and equality in the distribution of federal assistance. By joining Advocate, you will have the opportunity to be part of a mission-driven team that is making a tangible impact on society. Our work is centered around the belief that every American deserves access to the benefits they have earned through their contributions to our nation.
If you are passionate about leveraging cutting-edge technology to solve complex social challenges and are driven by the desire to make a meaningful difference in people's lives, Advocate is the perfect place for you. Join our talented and dedicated team as we work towards building a more equitable and inclusive society, one application at a time.
Requirements
- Advanced degree (Master's or Ph.D.) in Computer Science, Engineering, or a related field
- Extensive experience (5+ years) in data engineering, particularly in designing and implementing data systems for AI applications
- Proficiency in databases (SQL, NoSQL), big data frameworks (Spark), cloud services (AWS), and experience in developing AI pipelines and services
- Deep understanding of data modeling, data warehousing, and data integration techniques
- Experience with data quality assurance, data governance, and performance optimization
- Analytical problem-solving skills, adept at tackling complex data challenges and devising innovative solutions
- Strong team player with excellent communication abilities, capable of effectively collaborating with both technical and non-technical colleagues
- Experience working in agile development environments and familiarity with version control systems (e.g., Git)
- Familiarity with compliance standards such as HIPAA and SOC2, demonstrating understanding of critical security and privacy considerations within the healthcare and data sectors
- Knowledge of data anonymization techniques and experience working with sensitive data
- Proven track record of successfully delivering complex data engineering projects, preferably in the healthcare or public sector domains
- Proactive, self-motivated, and capable of working independently as well as part of a team
- Strong problem-solving skills, attention to detail, and the ability to thrive in a fast-paced, dynamic environment