Senior Data Engineer

Job Description

Cybersecurity is at the beating heart of our culture. Our diligence and expertise is what makes us the undisputed leader in electronic payments. We’ve made it our priority to create a top-tier Security Architecture team, poised to defend us against any potential cyber threats.CyberSecurity Architecture team is looking for a Data Engineer with Hadoop admin experience.Candidate will be a part of the Visa Security Analytics Engineering teamand involved with administering Hadoop-ecosystem and Data Engineering activities. The position will be based at Visa's headquarters in Foster City, California.

The successful candidate will beworking oncutting-edge technologies to perform statistical profiling, inference, classification, clustering and predictive analysis. As a key member of the Data Engineering team, you will be working on large datasets of Security Dataand implement Data mining, Data parsing contributing to data availability buildingmachine learning models to help derive new insights to defend against cyber-attacks.

You will be working withworld-class operation teams including Cyber Defense team and Data Science team to create awesome analytics for security and other business units.You'll need to have excellent communication and cross-group collaboration skills, be able to make forward progress despite ambiguous circumstances, be a self-starter, a quick learner of new technologies and have experience in Data engineering to maintain DataQuality and Integrity.

As a Data Engineer, you will be responsible to establish processes, automations, structures and big data systems based on business and technical requirements to channel multiple requirements, route appropriately and plan proper big data technology using combination of open source and vendor supported big data technologies, databases, and other applicable big data technologies as required.

• Interact and aid to provide data for other investigative teams within Visa on time sensitive, critical investigations.
• Work with cross functional teams to fully understand business requirements and desired business outcomes
• Assist in scoping and designing analytic data assets, implementing modeled attributes and contributing to brainstorming sessions
• Build and maintain a robust data engineering process to develop and implement self-serve data and tools for Visa’s data scientists
• Find opportunities to create, automate and scale repeatable analyses or build self-service tools for business users
• Execute data engineering projects ranging from small to large either individually or as part of a project team
• Experience on working with very large data sets and knowledge of building programs that leverage Massively Parallel Processing (MPP) Data warehouse platforms.
• The engineer will be working on Big Data technologies like Hadoop and tools and security related datasets doingData parsing, Data mining activities and writingparsers.
• Responsible for loading data from several disparate datasets, documentationand debugging applications.

Qualifications

Basic Qualifications:

• 5 years of work experience with a Bachelor’s Degree or an Advanced Degree such as Computer Science.(e.g. Masters, MBA, JD, MD, or PhD).
• 4+ years of software development experience (with a concentration in data centric initiatives), with demonstrated expertise in leveraging standard development best practice methodologies.
• Experience with Data Engineering/ETL/Administration experience in production ETL pipelines, utilizing data engineering techniques that enable statistical modeling solutions to solve business problems.

Preferred Qualifications:

• Extensive experience withbig data technologies (Hadoop,Hive, Druid, etc.) tools for large scale data processing, data transformation and machine learning pipelines.
• Ability to create and manage big data pipeline using Syslog-ng, kafka, flume, Airflow etc.
• Hands-on expertise with Java orPython, Spark and any scripting languages. Go lang and Scala is a plus.
• Experience with highly distributed, scalable, concurrent and low latency systems working with one or more of the following database technologies:MySQL, Postgres SQL,NoSQL DB, Elastic Search.
• Experience with data visualization and business intelligence tools like Power BI, Tableau, Microstrategy, or other programs.
• Experience in developing Rest API.
• Experience with Continuous Integration and Automated Test tools such as Jenkins, Artifactory, Git, Ansible.
• Experience working in an Agile and Test-Driven Development environment.
• Demonstrated intellectual and analytical rigor, strong attention to detail,team oriented, energetic, collaborative, diplomatic, and flexible style.
• Previous exposure to financial services, credit cards or Security analytics is a plus, but not required.

Additional Information

This position requires the incumbent to travel for work “0-5%” of the time

This position will be performed in an office setting. The position will require the incumbent to sit and stand at a desk, communicate in person and by telephone, frequently operate standard office equipment, such as telephones and computers, reach with hands and arms, and bend or lift up to 25 pounds.

Visa will consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.