Hours With Experts - Cloud
1904labs' Hours With Experts Data Engineering in the AWS cloud environment.
Installing Discord (Windows)
Installing Python (Windows)
Installing Git (Windows)
Installing Java (Windows)
Setup Development Environment (Windows)
Setup Development Environment (Mac)
Running multiple programs in VS Code
Data
Data Engineering Lifecycle
The Data Engineer
Course Introduction
Applying The Data Engineering Lifecycle
Introduction to Apache Spark
Spark Basics
Spark Basics Demo
Introduction to SQL
SQL Cheat Sheet
The Python DataFrame API
Python DataFrame API Cheat Sheet
Object Storage
File Formats
Object Storage Demo
Introduction to Athena
Athena Demo
AWS Account Setup
Introduction to Kafka
Topics, Consumers, and Producers
Working with Kafka
Kafka Demo
Starting over by wiping out data and checkpoints
Spark Batch vs. Spark Streaming Syntax Cheat Sheet
Spark Reads from Kafka Cheat Sheet
Database Architectures
Medallion Architecture
Spark DataFrame Schemas
Spark Schemas Cheat Sheet
Week 5 Lab