Staff Data Engineer - Platform as a Service

Job Description

Visa is seeking an Staff Data Engineer in the Data and AI Platform department tobuild and manage Visa’s Open-Source Hadoop in the Data and AI Platform.

This individual will act as a trusted technical expert and strategic contributor to the Data and AI Platform department. The successful candidate should be comfortable navigating challenging projects in big data and Data and AI Platform areas and be comfortable in a fast-paced environment with significant changes, developing/implementing strategies and processes that enable the organization to innovate and grow fast. You will have the opportunity to lead, participate, guide, and mentor other people in the team on architecture and design in a hands-on manner. This position reports to the head of the Large-Scale Shared Platform team in Palo Alto. The ideal candidate will have very strong development skill while exposure Open-Source Hadoop and big data / large-scale software platforms like Hadoop, Presto, Hive and HDFS are required.

Essential Functions

• Provide Platform-as-a-Service offerings that are easy to consume, scalable, secure, and reliable using open source based solutions for Big Data technologies
• Work at the intersection of infrastructure and software engineering, and design and deploy cloud infrastructure build on top of open-source components, including Hadoop, Hive, Spark, HBase, Kafka streaming and other Open Source Big Data technologies.
• Collaborate with various teams to build and maintain the most innovative, reliable, secure, and cost-effective distributed solutions.
• Bring together cross-functional teamsand build automation to deliver programmable and vendor-neutral infrastructure that transforms virtualization to new compute paradigm. Design the Cloud, and Container strategy around PaaS Solutions.
• Facilitate knowledge transfer to the Engineering and Operations team during and after deployment.
• Work with geographically distributed teams on technical challenges and process improvements.
• Responsible for the design and implementation of agile-innovative infrastructure solutions that take advantage of technology advances that allow cost reduction, standardization, and commoditization.
• Drive the adoption of open standard toolsets to reduce complexity and support operational goals for increasing automation across the enterprise.
• Champion the adoption of open infrastructure solutions that are fit for purpose while keeping technology relevant.

Qualifications

Basic Qualifications:

• Experience in building developer productivity tools to improve the overall developer productivity
• Expertise in design and management of complex data structures and data processes
• Expertise in managing and operating distributed big data systems, including but not limited to Hadoop eco system - Hive, HDFS, Spark, Kafka and etc.
• Deep knowledge and hands on experience cloud computing technologies.
• Expertise on design/coding using any one of the programming languages: Python, Golang, Java, C/C. Can write very solid prototypes or core components quickly.
• Experience in service architecture and development experience with high performance and scalability
• Technical background in data with deep understanding of issues in multiple areas such as data acquisition and processing, data management, distributed processing, and high availability is required
• Outstanding verbal, written, presentation, facilitation, and interaction skills, including ability to effectively communicate architectural issues and concepts to multiple organization levels and executive management

Additional Information

Visa has adopted a COVID-19 vaccination policy to safeguard the health and well-being of our employees and visitors. As a condition of employment, all employees based in the U.S. are required to be fully vaccinated for COVID-19, unless a reasonable accommodation is approved or as otherwise required by law.

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Visa will consider for employment qualified applicants with criminal histories in a manner consistent with applicable local law, including the requirements of Article 49 of the San Francisco Police Code.