Data Specialist

Techyard are seeking a skilled and motivated Databricks professional. In this role, you’ll play a key part in designing, developing, and optimising data solutions using the Databricks Lakehouse Platform. You’ll work closely with cross-functional teams to deliver high-performance data pipelines, enable advanced analytics, and drive business value through data

Key Responsibilities:

• Build and maintain scalable data pipelines using Databricks and Apache Spark.
• Develop efficient data models and implement Lakehouse architecture (bronze/silver/gold layers).
• Optimise data processing workflows for performance, reliability, and cost-efficiency.
• Collaborate with data analysts, scientists, and stakeholders to deliver clean, trusted datasets.
• Integrate data from various sources using native Databricks tools and/or orchestration frameworks.
• Maintain high standards of code quality through version control, testing, and CI/CD practices.
• Monitor, troubleshoot, and continuously improve data workflows in production environments.

Essential Skills & Experience:

• Hands-on experience with Databricks and Apache Spark.
• Proficiency in PySpark and SQL.
• Strong understanding of data lakehouse concepts, data modelling, and data warehousing.
• Familiarity with one or more cloud platforms (Azure, AWS, or Google Cloud).
• Knowledge of orchestration tools (e.g., Airflow, dbt, or similar).
• Awareness of data quality, security, and governance best practices.
• Strong analytical, communication, and problem-solving skills.

Desirable:

• Databricks certifications (e.g., Data Engineer Associate / Professional).
• Experience with tools such as dbt, MLflow, or Unity Catalog.
• Exposure to BI platforms (e.g., Power BI, Tableau) or machine learning workflows.
• Experience working within Agile or DevOps environments.

…