JobDescription : The Director of Data Engineering will lead the engineering activities that will form the new Platts' data platform, constructing innovative systems that will unite the collection, parsing, enriching, linking and delivering mechanisms that form the large portfolio of energy and pricing data for Platts. The Director of Data Engineering will lead the development and execution of highly complex and large-scale data structures and pipelines that link and enrich data, generating insights whose value are exposed via APIs, analytical models and artificial intelligence, thus increasing the value of our data sets for internal teams and customers of Platts' applications. In short, the director of Data Engineering will be responsible for delivering on a multi-year data strategy that will catapult Platts as one of the leading data companies in the market.
The ideal candidate is an innovative, current practitioner with a strong data engineering background, who will lead multiple data engineering teams globally and will work closely with operations, technology, product management, analysts and other teams located in our offices in London, NY, Denver, Dallas and India as well as work with the S&P Global Chief Data Officer in shaping and refining the corporate standards to ensure they meet the needs of Platts' data users and customers.
As Director of Data Engineering you will be responsible for the following facets of data engineering:
Data Strategy & Architecture: o Data Models (Relational, Time Series, Document & Graph Models) o Knowledge/Semantic Models (Taxonomies, Ontologies, Reference & Master Data, Knowledge Graphs) o Operational Models (Multi-tenant, Cloud, Batch & Streaming models) o Storage Models (NoSQL, Relational, Graph, Time Series, in-memory)
Data Collection & Ingestion o Data Wrangling o Cleanup & Deduplication o Data Scraping
Data Processing & Automation o Data Pipelines (ETL & ELT Models) o Event driven Data Processing o Change Data Capture o Batch & Stream operations o Data Quality Automation (using rules engines and statistical methods)
Data Access & Delivery o APIs o Analytics & Data Science o Visualizations o Data Exports o Streams / Alerts / Events o Channel Partner distributions
Data Linking & Enrichment o Data Tagging (Automated & Manual) o Machine learning based data linking
Infrastructure & Operations o Data Scalability o Availability o Performance o Observability (Monitoring, Logging, Alerting) o Telemetry o Security
Data Governance o Standards o Risk Management o Access Policies o Data Quality o Data Lineage & Auditing o Corporate Governance o Continuous improvement
Additional Responsibilities: Development Management
Partner with product management and co-lead an agile transformation
Partner with customer-facing peers to create and deliver on multi-year roadmaps
Manage and build teams off-shore
Lead the transformation from multiple applications/tools into a few platforms
Migration legacy applications onto the platform
Identify and consolidate duplicate product functionality, infrastructure environments and teams
Automate manual tasks
Provide constant coaching to the agile teams in the program
Provide mentoring and guidance to team members
Have effective 1:1's with your direct reports
Ensure managers and their team members work at capacity to ensure deadline are meet
Ensure agile teams estimate development effort by breaking down components into work items
Review their team's Scrum and ScrumBan boards weekly and ensure the teams operate efficiently and meet sprint goals
Continually work on process improvement
Work closely with application support to ensure escalated items are addressed accordingly
Ensure production operations of 99.99% uptime
Partner with infrastructure to efficiently manage the AWS environments. Implement elastic infrastructure to control costs.
Lead technology audit compliance based on regulators and SOC2
Degree in computer science or related quantitative field of study. Masters degree is preferred
10+ years of experience in software industry with emphasis on Data Engineering
Experience in software development with a major modern language (e.g. Java, Scala, Python, etc.)
Strong understanding of data structures, algorithms and data design.
Expertise in data engineering topics (APIs, automation, big data, open source packages, and secure systems architectures)
Experience in AWS data storage, processing and analytical services.
Experience with stream-processing systems, such as Kafka, Spark Streaming, Kinesis.
Experience with data lake/warehouse technologies (e.g. Athena, Redshift, EMR) and distributed NoSQL databases (e.g. DynamoDB, MarkLogic).
Experience with semantic web technologies (Ontologies, Taxonomies, Knowledge Graphs) and Graph theory in general.
Familiarity with data science techniques or machine learning.
Good knowledge and experience in Agile development
Proven technical abilities, but must be capable of communicating complex analyses effectively
Capable to lead multiple work streams simultaneously in a fast-paced environment and partner with multiple business stakeholders
Most of all, it requires an individual who is comfortable striking a balance between cutting-edge and pragmatic solutions.
Internal Number: 5791126
About S&P Global
eFinancialCareers is a career site specializing in financial services.