Data Engineering

‘Data Engineering’ is a customized but focused course to help students and developers achieve their potential for Data Engineering. Data engineers build systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. While working with Python, you’ll also grow your skill in other languages, such as Shell and SQL, create data engineering pipelines, automate common file system tasks, and build a high-performance database.

Who should attend:

• Students from CS, IT or Software background are welcome.
• Anyone interested in this cutting-edge science..

Objectives:

• Build skills in programming
• Deploying application on multiple clouds
• Data Acquisition from various platforms

Learning Outcome:

Upon successful completion of this program, participants will be able to:

Knowledge

• Practical understanding of the Data Engineering Ecosystem and Lifecycle.
• Apply critical concepts of data acquisition techniques.
• Understand different data storage tools and options e.g files, SQL and NoSQL.
• learn the scale of applications in terms of deployment.
• Guidance and counseling from professionals in managing a career in this field.

Skills

• Python programming basics including data structures, logic, working with files, invoking APIs, using libraries such as Pandas and Numpy, doing ETL.
• SQL query language, SELECT, INSERT, UPDATE, DELETE statements, database functions, stored procs, working with multiple tables, JOINs, & transactions.
• Schedule, automate, and monitor data pipelines using Apache Airflow. Run data quality checks, track data lineage, and work with data pipelines in production.
• Orchestrate Data with Airflow.