| Position Type: | Full-time |
| Location: | London, ON |
| Date Posted: | Nov 23, 2024 |
Job Description
Data Engineer
The Data Engineer will build and operationalize data necessary for enterprise data and analytics initiatives. This role will build, manage and optimize data architecture and pipelines in accordance with industry standard practices and using industry standard tools, and will move these data pipelines effectively into production for data and analytics use cases across the organization. The incumbent will ensure compliance with data governance and data security requirements while creating, improving and operationalizing these integrated and reusable data pipelines and analytical models that will support and promote self-service analytics. Will occasionally provide support to end users on the use of data visualization solutions. Responsible for maintaining and making recommendations to enhance technology as the needs of our client’s change.
Apply for this job
The Data Engineer will build and operationalize data necessary for enterprise data and analytics initiatives. This role will build, manage and optimize data architecture and pipelines in accordance with industry standard practices and using industry standard tools, and will move these data pipelines effectively into production for data and analytics use cases across the organization. The incumbent will ensure compliance with data governance and data security requirements while creating, improving and operationalizing these integrated and reusable data pipelines and analytical models that will support and promote self-service analytics. Will occasionally provide support to end users on the use of data visualization solutions. Responsible for maintaining and making recommendations to enhance technology as the needs of our client’s change.
Data & Analytics
-
- Design, build and maintain data pipelines (i.e. data flow from data sources to integration to consumption for specific use cases).
- Develop data models (conceptual, logical and/or physical) as required.
- Manage metadata using data preparation, integration and AI-enabled tools and techniques.
- Drive automation in data integration. Recommend and lead the implementation techniques and architectures to automate the most-common, repeatable and tedious data preparation and integration tasks partially or completely to minimize manual and error-prone processes.
- In collaboration with functional leaders, identify the company’s inefficiencies, inaccuracies, missed opportunities, other issues and report on the recommended improvements to the executive team.
- Provide technical leadership for Azure platform by directing and prioritizing the work of others on the team related to Athena.
- Act as a subject matter expert for the Azure platform.
Stakeholder Engagement
-
- Assist cross-functional groups, Business analysts, and other stakeholders to gather, define and refine their data requirements for various initiatives.
- Propose appropriate (and innovative) data ingestion, preparation, integration and operationalization techniques in optimally addressing stakeholder requirements.
- Assess ways to further leverage existing systems and recommend methods for streamlining data flow and data integration.
- While identifying and evaluating new projects or requests for enhancements, work with the functional leader to determine the project cost, timeline, goals and feasibility and then support the project through completion.
Data Compliance & Governance
-
- Work with corporate leadership to manage and promote compliance with the corporate data strategy
- Collaborate with data stewards or appropriate stakeholders to develop policies to ensure the responsible use of data by users and consumers.
- Lead the development and maintenance of documentation and the data catalog for Alimentiv data assets.
- Promote the available data and analytics capabilities and expertise to business unit leaders and educate them in leveraging these capabilities in achieving their business goals.
- Ensure data or software solutions continue to meet customer needs by capturing changes in requirements and making corresponding updates to existing software or data analytics.
Qualifications
- Microsoft SQL Server:
Azure Data Platform:
ETL Tools:
Azure Synapse Analytics:
Power BI:
- A minumum of a Bachelor's Degree + 4-6 years of related experience.
- Core Technical Skills:
- 5+ years of experience in SQL Server (2008/2012/2016) development and administration.
- Proficient in writing complex queries, stored procedures, and optimizing performance.
- Experience with database backup, recovery, replication, and performance tuning.
- Strong experience with Azure services including Azure SQL Database (PaaS) and Azure SQL VM (IaaS).
- Hands-on experience in migrating on-premises databases to Azure.
- Proficient with Azure Data Factory (ADF) for orchestrating data pipelines.
- Experience with other ETL tools like Matillion, including error handling, logging, and monitoring.
- Experience with building and optimizing data warehouses in Azure Synapse.
- Proficient in performance tuning, data partitioning, and loading large data sets.
- Experience in developing reports and dashboards using Power BI for business analytics.
Additional Technical Skills
- System Administration:
Scripting:
Database Monitoring & Tools:
Migration Experience
Data Management & Reporting
Ø Experience in designing, developing, and deploying SSIS packages in data warehouse environments.
Ø Performance tuning in SSIS packages using Row Transformations and involved in error handling and logging.
Ø Developed custom SQL Server Reporting Services (SSRS) reports for monitoring job performance.
Ø Experience in creating interactive reports and dashboards using Power BI.
Health Sector-Specific Requirements
Ø Experience working with highly sensitive and protected health information (PHI) in a secure cloud environment.
Behavioral and Situational Competencies
Ø Demonstrated experience in identifying and resolving performance bottlenecks in SQL and Azure environments.
Ø Expertise in troubleshooting complex issues and implementing solutions in high-pressure scenarios.
Ø Proven ability to collaborate with DevOps, data engineering, and development teams in a fast-paced, cross-functional environment.
Ø Experience working with business stakeholders to gather requirements and translate them into technical solutions.
Ø Experience in leading small-to-medium-sized database migration or cloud infrastructure projects.
Ø Ability to manage timelines, deliverables, and resource allocation for Azure migration projects.
Certifications (Preferred)- Proficient in managing Windows Servers (2016/2012/2008) in a production environment.
- Experience in SAN network environments and RAID configuration.
- Experience with scripting languages such as PowerShell, Bash, and Python for automation and configuration management.
- Experience with monitoring tools like Redgate, Idera Diagnostic Manager, SolarWinds for SQL Server performance monitoring and troubleshooting.
- Expertise in using SQL Server Profiler, SSMS, and SQL Sentry.
- Cloud Migrations:
- Experience in cloud migration projects, specifically migrating databases from on-premises environments to Azure.
- Familiarity with DevOps procedures for cloud architecture in both AWS and Azure environments.
- SSIS and Data Warehousing:
- Reporting:
- Regulatory Compliance:
- Understanding of health sector regulations and compliance standards such as HIPAA for data security and privacy.
- Data Sensitivity:
- Problem-Solving and Troubleshooting:
- Collaboration and Teamwork:
- Project Management and Leadership:
- Microsoft Certified: Azure Data Engineer Associate
- Microsoft Certified: Azure Fundamentals
- Microsoft Certified: SQL Server Specialist
Apply for this job