Senior Software Engineer - Data Platform Group

Company Description

Job Description

Epsilon is the leader in outcome-based marketing. We enable marketing that’s built on proof, not promises. Through Epsilon PeopleCloud, the marketing platform for personalizing consumer journeys with performance transparency, Epsilon helps marketers anticipate, activate and prove measurable business outcomes. Powered by CORE ID®, the most accurate and stable identity management platform representing 200+ million people, Epsilon’s award-winning data and technology is rooted in privacy by design and underpinned by powerful AI. With more than 50 years of experience in personalization and performance working with the world’s top brands, agencies and publishers, Epsilon is a trusted partner leading CRM, digital media, loyalty and email programs. Positioned at the core of Publicis Groupe, Epsilon is a global company with over 8,000 employees in over 40 offices around the world.

For more information, visit epsilon.com. Follow us on Twitter at @EpsilonMktg.

At Epsilon, we do more than collect and store data. We help some of the world’s biggest brands discover real opportunities inside the data types, delimiters and decimals. At Epsilon, we can analyze anonymized, privacy-safe data at internet scale and handle 300B+ of online interactions a day and have $3.8 trillion in multichannel purchases in our database.

We are looking for a Senior Data Engineer with extensive experience in the big data ecosystem to work in our Data Pipeline team. As part of the Epsilon DMS Data Organization, the Data Pipeline team’s responsibilities include but is not limited to maintaining streaming and batched data pipelines collecting hundreds of billions of data rows daily, near real time aggregation of a variety of data sets (using Spark Structured Streaming) and maintaining system of record raw and aggregate data sets (HDFS and HIVE). Additionally, as part of the Data Pipeline team work closely with Real Time Bidding (RTB), data warehousing, ETL and decision science teams in building, maintaining and optimizing solutions.

The candidate must be proficient in Scala or Java, with the ability to be a key contributor in building and maintaining Spark jobs and AirFlow DAGs. This role will be hands on in code development and data architecture to optimize the platform for growth and maximum efficiency. The person in this role will need to be able to work both independently and as part of a team to meet required specifications of solution delivery.

• You will design and code data pipelines connecting Real Time Bidding platform, HDFS, Elastic, MPP and Postgres databases utilizing Flume, Kafka, Spark, Cassandra/Scylla and other technologies as needed.
• Maintain and extend Spark framework used by DMS Data Organization to support various aspects of the business.
• Build and maintain AirFlow DAGs for job management.
• Work closely with infrastructure teams in capacity planning, hardware procurement and build outs.
• Build and maintain metrics collection to help with identifying production issues, optimizing job performance and alerting on error conditions (Pager Duty).
• Develop test cases to demonstrate new code meets functional requirements.
• Ideal candidate can lead in one or more areas of: design, code development, data modeling, cross team communication, application maintenance and identifying opportunities for improving code quality or performance through refactoring and/or incorporating new technologies.

• Bachelor’s Degree in Computer Science or equivalent degree is required.
• 4+ years of experience developing in Java and/or Scala
• Spark experience a big plus
• Strong experience in SQL, bash and Python
• Experience with Hadoop Stack (HDFS, Hive, YARN, HBase)
• Experience with Kafka and the Kafka producer and consumer APIs (Kafka Connect, K-streams and ksql a plus)
• Experience with Postgres and Cassandra/Scylla a big plus
• Docker and Kubernetes experience a big plus
• ELK stack experience a plus
• Experience with scheduling applications with complex interdependencies
• Good experience in working with geographically and culturally diverse teams
• Excellent written and verbal communication skills
• Excellent analytical and problem-solving skills
• Ability to diagnose and troubleshoot problems quickly

Qualifications

Additional Information

For decades, we’ve been helping marketers from the world’s top brands personalize experiences for millions of people with our cutting-edge technology, solutions and services.

Thanks to the work of every employee, Epsilon products and services have been recognized as industry-leading by the Forrester Wave™. We process more than 500 billion consumer actions each day and hold many patents of proprietary technology, including real-time modeling languages and consumer privacy advancements. And our CORE ID® is the industry’s most accurate, stable and scalable identity solution that helps brands recognize and reach 200M+ U.S. consumers in a privacy-safe way.

Positioned at the core of Publicis Groupe, Epsilon is a global company with more than 8,000 employees around the world. For more information, visitepsilon.com

We know that we have some of the brightest and most talented associates in the world, and we believe in rewarding them accordingly. If you work here, expect competitive pay, comprehensive health coverage, and endless opportunities to advance your career.

Epsilon is an Equal Opportunity Employer. Epsilon’s policy is not to discriminate against any applicant or employee based on actual or perceived race, age, sex or gender (including pregnancy), marital status, national origin, ancestry, citizenship status, mental or physical disability, religion, creed, color, sexual orientation, gender identity or expression (including transgender status), veteran status, genetic information, or any other characteristic protected by applicable federal, state or local law. Epsilon also prohibits harassment of applicants and employees based on any of these protected categories.

Epsilon will provide accommodations to applicants needing accommodations to complete the application process.

#LI-AM1

REF134146K