Understanding Modern Data Architecture
Discover modern data architecture's key components, from ingestion and serving to governance and orchestration.
Start Course for Free2 hours16 videos56 exercises10,466 learnersStatement of Accomplishment
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.Training 2 or more people?
Try DataCamp for BusinessLoved by learners at thousands of companies
Course Description
Introduction to modern data architectures
The world is constantly changing thanks to data, and every day the amount of data to be processed is even bigger. Thus, this course gives you an overview of modern data architectures, guiding you through the fundamental components of data solutions.
Components of modern data architectures
This is an introductory course to modern data solutions. During the first half of this course you'll learn what modern data architectures are, will review some popular solutions like Data Mesh and Data Fabric, and will explore the different elements of data architectures such as storage, data ingestion methods, processing options, and serving techniques.
Transversal Components of modern data architectures
In the second half, the course shows the transversal components that ensure the cohesion and enable the management of data architectures; explaining concepts like data governance, orchestration, observability, and even cost optimization within cloud providers. With all these elements, you'll be able to design data solutions that highly align with your organization's needs.
Designing and evaluating modern data architectures
The final part of the course integrates the knowledge you've gained, using a business case to design a data solution that satisfies the requirements of the business.
Training 2 or more people?
Get your team access to the full DataCamp platform, including all the features.In the following Tracks
Professional Data Engineer in Python
Go To Track- 1
Introduction to Modern Data Architecture
FreeAfter seeing this chapter, you will be able to explain what modern data architecture is, and what are their key requirements and components. You'll also review different types of existing data architectures, ranging from Lambda to Data Mesh.
Modern Data Architectures50 xpWhat is a modern data architecture?50 xpModern vs traditional architectures100 xpExamples of the modern data architecture50 xpLambda architecture50 xpLambda architecture layers100 xpKappa architecture50 xpKappa: What about the batch layer?50 xpData Mesh and Data Fabric50 xpData Mesh50 xpData Fabric50 xpData architecture & its use cases100 xp - 2
Modern Data Architecture Components
Learn about the different components and layers of modern data architectures; from data ingestion to serving. Get familiar with the different paradigms and approaches in each layer, and some of the cloud services used to implement modern data solutions to get field experience!
Data Storage: Storage types50 xpWhat are blob storages?50 xpWhen should you use blob storage?50 xpSQL vs. NoSQL50 xpStorage use cases100 xpData Ingestion50 xpWhat is ingestion?100 xpWhat is the landing zone?50 xpIngesting new data50 xpData Processing50 xpBatch vs. Streaming100 xpWhat is a window in streaming?50 xpProcessing paradigms50 xpData serving50 xpServing technologies100 xpSort the layers100 xp - 3
Transversal Components of Data Architectures
Dive into the transversal components that modern data architectures consist of, covering key aspects like data governance, metadata, data security, and observability. By understanding these vital elements, you'll be prepared to design robust, secure, and efficient data platforms while maintaining high standards for the data and overall service.
Data governance50 xpWhat is data governance?50 xpData governance roles100 xpMetadata Management50 xpThe role of metadata50 xpMetadata types100 xpThe business glossary is composed of...50 xpData Security50 xpWhat is a data breach?50 xpVirtual Private Clouds (VPC)50 xpEffective data security50 xpClassifying Data Security Measures100 xpObservability50 xpTroubleshooting the source of a problem50 xpData security measures100 xp - 4
Putting it All Together
Learn how to manage workflows, understand costs in cloud services, and create modern data designs. This chapter helps you work with tools like Apache Airflow and teaches you ways to save money on cloud use. By the end, you'll know how to build and judge strong data systems for businesses.
Orchestration50 xpOrchestration vs. Scheduling100 xpOrchestration sensors50 xpStorage & processing costs best practices50 xpReserved capacity in cloud services50 xpStorage classes100 xpDesigning a modern data architecture50 xpAsking the right questions50 xpIngesting the application data50 xpEvaluating modern data architecture solutions50 xpWhy a quarantine zone?50 xpDesigning a modern data platform100 xpWrap-up50 xp
Training 2 or more people?
Get your team access to the full DataCamp platform, including all the features.In the following Tracks
Professional Data Engineer in Python
Go To Trackcollaborators
Miller Andrés Trujillo AchurySenior Software Engineer at Microsoft
Miller is a senior software Engineer at Microsoft and a professor at Universidad de los Andes, with a previous role at Google, that allowed him to sharpen his skills in data engineering, employing tools like Python, Java, and SQL, and cloud providers like AWS, Azure, and GCP. Miller's work approach is rooted in collaboration and constant learning. His work includes creating various data solutions for diverse clients including healthcare ones. Those solutions range from simple pipelines to libraries for data validation or platform tools for other data engineers, reflecting his commitment to practical, user-focused solutions.
Join over 15 million learners and start Understanding Modern Data Architecture today!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.