Introduction
The CompTIA DataX DY0-001 certification covers a wide range of domains that would take multiple courses to cover. Instead, we're going to cover the core concepts needed to pass the certification:
Mathematics and Statistics: 17%Modeling, Analysis, and Outcomes: 24%Machine Learning: 24%Operations and Processes: 22%Specialized Applications of Data Science: 13%
In this first video, we'll review everything covered in this course.
If you're already familiar with some or most of these domains, that's great! If you need to learn or refresh your skills on areas like pandas, machine learning, or deep learning, here's a list of related courses to help you get up to speed:
- Introductory Python for Data Analysts
- SQL for Data Practitioners
- Python and Pandas for Data Manipulation
- ISACA Data Science Fundamentals
- Introduction to Machine Learning
- Introduction to Deep Learning
Knowledge Check
Which learning technique is emphasized to help identify and address knowledge gaps?
The Origin of Data Science?
Knowledge Check
The term Data Scientist is a mashup of 'data analyst' and 'research scientist' created to describe a role that combined data analytics, computer science, and research.
What is Data Science?
Knowledge Check
What is the main distinction between machine learning and deep learning as discussed in the video?
Data Science Lifecycle & Machine Learning Approaches
Knowledge Check
Which of the following is the initial step in the end-to-end data lifecycle?
Machine Learning & Data Science Application Examples
Knowledge Check
Which data science application area involves forecasting future values based on historical data?
Get Google Colab
4.1 Prediction
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
# Synthetic daily data
np.random.seed(42)
days = np.arange(1, 31).reshape(-1, 1) # 30 days
temps = 20 + 0.2 * days.flatten() + np.random.normal(scale=1, size=days.shape[0])
model = LinearRegression()
model.fit(days, temps)
future_days = np.arange(31, 41).reshape(-1,1)
temp_preds = model.predict(future_days)
plt.figure(figsize=(8,5))
plt.scatter(days, temps, color='blue', label='Historical Temp')
plt.plot(days, model.predict(days), color='red', label='Trend Line')
plt.plot(future_days, temp_preds, '--', color='green', label='Forecast')
plt.title('Simple Temperature Forecast')
plt.xlabel('Day')
plt.ylabel('Temperature (°C)')
plt.legend()
plt.show()
4.2 Pattern Mining
import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)
import pandas as pd
from mlxtend.frequent_patterns import apriori, association_rules
# Input Data
df = pd.DataFrame([
{'milk': 1, 'bread': 1, 'egg': 1, 'chocolate': 0},
{'milk': 1, 'bread': 0, 'egg': 0, 'chocolate': 1},
{'milk': 1, 'bread': 1, 'egg': 0, 'chocolate': 1},
{'milk': 0, 'bread': 1, 'egg': 1, 'chocolate': 0}
])
# Frequent Itemset Generation
itemsets = apriori(df, min_support=0.5, use_colnames=True)
# Association Rules with Compatibility Adjustment
rules = association_rules(itemsets, metric="confidence", min_threshold=0.7, num_itemsets=len(itemsets))
# Output Results
print("Frequent Itemsets:\n", itemsets)
print("\nAssociation Rules:\n", rules)
4.3 Segmentation
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
np.random.seed(42)
cluster_1 = np.random.normal(loc=[2, 2], scale=0.8, size=(50, 2))
cluster_2 = np.random.normal(loc=[8, 8], scale=0.8, size=(50, 2))
data = np.vstack((cluster_1, cluster_2))
kmeans = KMeans(n_clusters=2, random_state=42).fit(data)
labels = kmeans.labels_
centroids = kmeans.cluster_centers_
plt.figure(figsize=(8,5))
plt.scatter(data[:,0], data[:,1], c=labels, cmap='coolwarm')
plt.scatter(centroids[:,0], centroids[:,1], marker='X', s=200, c='black', label='Centroids')
plt.title('K-Means Segmentation (A vs. B)')
plt.xlabel('Feature A')
plt.ylabel('Feature B')
plt.legend()
plt.show()
4.4 Natural Language Processing (NLP)
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.model_selection import train_test_split
corpus = [
"I love data science",
"Data science is awesome",
"I hate spam emails",
"Spam emails are annoying",
"I enjoy machine learning"
]
labels = [1, 1, 0, 0, 1] # 1=positive, 0=negative
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(corpus)
X_train, X_test, y_train, y_test = train_test_split(X, labels, test_size=0.4, random_state=42)
clf = MultinomialNB()
clf.fit(X_train, y_train)
print("Training Accuracy:", clf.score(X_train, y_train))
print("Test Accuracy:", clf.score(X_test, y_test))
4.5 Network Analysis
import networkx as nx
import matplotlib.pyplot as plt
G = nx.DiGraph()
G.add_nodes_from(["Root", "Child_A", "Child_B", "Grandchild_A1", "Grandchild_A2", "Grandchild_B1"])
G.add_edges_from([
("Root", "Child_A"),
("Root", "Child_B"),
("Child_A", "Grandchild_A1"),
("Child_A", "Grandchild_A2"),
("Child_B", "Grandchild_B1")
])
pos = nx.spring_layout(G, seed=42)
plt.figure(figsize=(8,5))
nx.draw(G, pos, with_labels=True, node_size=2000, node_color="skyblue", arrowsize=15)
plt.title("Tree-Like Graph")
plt.show()
4.6 Signal Processing & Computer Vision (CV)
import tensorflow as tf
(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.fashion_mnist.load_data()
tf.random.set_seed(42)
train_labels = tf.keras.utils.to_categorical(train_labels)
test_labels = tf.keras.utils.to_categorical(test_labels)
model = tf.keras.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(4, activation="relu"),
tf.keras.layers.Dense(4, activation="relu"),
tf.keras.layers.Dense(10, activation="softmax")
])
model.compile(loss=tf.keras.losses.CategoricalCrossentropy(),
optimizer=tf.keras.optimizers.Adam(),
metrics=["accuracy"])
history = model.fit(train_images, train_labels, epochs=10, validation_data=(test_images, test_labels))CHALLENGE
Here's a quick challenge to test your knowledge across these domains by answering a few questions for each. Keep in mind that this is not an in-depth quiz but rather a brief check to wrap up this introductory skill and kick off the DataX course.
Knowledge Check
Which of the following best describes the application of Data Science shown in the image below?
Knowledge Check
Which of the following best describes the application of Data Science shown in the image below?
Knowledge Check
Which of the following best describes the application of Data Science shown in the image below?
Knowledge Check
Which of the following best describes the application of Data Science shown in the image below?
View Transcript
Introduction
0:00Hello and welcome to the first skill in the DataX certification course, where
0:06you're going to get the chance to apply learning strategies for the DataX
0:10certification.
0:11My name is Jonathan Barrios and I'm excited that you're here.
0:14Before we get started, let's check out some of the domains that we're going to
0:18be working with in this course.
0:20So here we have the five domains.
0:22The first one here is Mathematics and Statistics, and you can see that it's 17
0:26%.
0:27So it's not a lot of math and statistics, but it is mostly focused around
0:31machine learning.
0:33And then we have Modeling, Analysis, and Outcomes in Machine Learning, which
0:37also includes deep learning.
0:39We'll talk more about that.
0:41Operations and Processes, and then finally just a sliver for specialized
0:45applications of data science.
0:47Now that we've gotten that out of the way, let's talk about everything that we
0:50're going to cover in this skill.
0:52Number one, we reviewed the domains, the five domains.
0:56And the reason that we did that is because I have some existing adept courses,
1:01and that's going to be number two,
1:03and these adept courses cover a whole range of domains.
1:08Not everything, not so much the math, but we're going to get into this and I'm
1:12going to show you each the course
1:15because they are deep dives resources, or I can say courses.
1:20And why am I mentioning that here?
1:22Well, think about it.
1:23When we're talking about the five domains, these are quite in depth.
1:27That's the thing about data science, right?
1:29It encompasses a lot.
1:31Not only we're talking about machine learning and deep learning, but there's
1:33math, and then there's analysis, data wrangling.
1:37I mean, we can go on and on, so I think you get the idea here.
1:40But we're not going to be able to go into deep dives in all of those domains
1:45because this course would be super long.
1:48So I'm trying to leverage the existing courses that I have where I cover all of
1:52this stuff in a deep dive format.
1:54I thought it would be really appropriate to list all of those out and show you
1:58how to navigate adept to find my courses
2:01in case you need to deep dive.
2:03Chances are you're going to understand most of this, or if you don't, then you
2:07're going to have some additional resources.
2:09Whatever the range of your skill set is right now, I hope that this pipeline of
2:15data science will be an invaluable resource to you.
2:18And it's so cool that I just finished building that resource.
2:22So I call it the zero to hero data science pipeline, and it just finished it.
2:27So that means that now that we're diving into data X, you have that as a
2:31resource.
2:32And number three, we're going to talk about the Feynman technique, and this is
2:36going to also work with this deep dive, and also an assessment skill for skill
2:44gaps.
2:44And this is coming next.
2:46And let me explain a little bit about this Feynman technique.
2:49So Richard Feynman is a physicist and educator, very well known in physics, but
2:55a wonderful educator.
2:56And if you don't know Richard Feynman, definitely look him up on YouTube, he
2:59explains quantum physics like nobody does with a lot of excitement.
3:04But this technique is really awesome.
3:06And what it does is it's a learning method where you deeply understand a
3:10concept by pretending to teach it to somebody else.
3:14Ideally, a child, maybe even a five year old, right?
3:18And by doing this, you're going to be explaining it in simple terms, which
3:22forces you to identify and address any gaps that you might have.
3:26And we're also going to identify gaps with the assessment skill.
3:30And I'm going to give you answers and explanations at the very end of that
3:33skill, so you're not going to be guessing if, you know, I'm not just going to
3:37leave you hanging with a bunch of quiz questions and then no resolution.
3:41We're going to cover those, and I'll even go as far as to explain each one of
3:45the answers.
3:46And so again, why are we doing the Feynman technique?
3:49Well, not only is it going to help you identify the gaps in your understanding,
3:53but it's going to give you an invaluable skill, which is using teaching to
3:58learn.
3:59And that's one of the learning strategies I'm going to share with you.
4:02And it really has to do with the five domains.
4:05You can actually use the Feynman technique for that.
4:08And you can also, once you identify those gaps, either using the Feynman
4:12technique or the assessment skill coming up next,
4:15then you can go to the adept courses and do the deep dive right here.
4:20So all of this works together, and you might be asking yourself, great, but how
4:23does the Feynman technique work?
4:26Let's cover that right now.
4:28So number one, you're going to choose a concept.
4:31And what you're going to do is just whatever topic, or it could be one of the
4:34domains, or one of the domain objectives, which I'm going to share with you in
4:38the next skill.
4:39And then step two, you write an explanation. And this is where you pretend that
4:49you're explaining this concept to someone with no prior knowledge, like a child
4:49.
4:49And then you write down everything you know about it.
4:52And one good example here, and you've probably seen this before, when you have
4:55a five-year-old or any child, really, that's asking, why is the sky blue?
5:00And then the parent says, because, and then the child says, why? And it's like,
5:05well, because it's, and then maybe they'll attempt some kind of an explanation.
5:10And then after that surely comes, why? And that's the process here by asking
5:15why you'll get to the point where you don't know something, right?
5:19When you're talking about the natural world, well, of course you're going to
5:22get into physics and ultimately quantum physics and go, it's a rabbit hole.
5:26So you can't know everything about everything, but this technique is really
5:30simple, and I use it all the time.
5:32And then number three, you identify gaps. And that's the most important part.
5:38You review your explanations, and then you identify areas where your
5:42understanding is unclear.
5:44Or where you use complex terminology, because we don't want to use any jargon.
5:50When you start to do that, that's a red flag.
5:52Step number four, you want to research and refine.
5:57So go back to your source material and fill in any knowledge gaps and rephrase
6:02your explanations to make it simple and accessible.
6:06You want this to be something that a child can even understand.
6:10And why is that?
6:11When you can explain it in simple terms without using jargon, then you're going
6:15to have it stored in long-term memory, and it means that you have a pretty good
6:18understanding of that, which is perfect for taking the data out.
6:21And you're taking the data X certification.
6:23That works well with the concepts because they want to make sure that you
6:26understand them.
6:27And if you understand the concepts, then you have a much better chance of
6:30getting each of the questions right in a situation where there's added pressure
6:35.
6:36Right?
6:37So that's the whole idea here.
6:38And this is a useful tool.
6:40And this is actually how the physicist Richard Feynman taught himself as he was
6:44studying physics.
6:46So it's a proven technique, not just with myself.
6:50You use it all the time, and a lot of people use it, but Richard Feynman used
6:53this technique to great avail.
6:55And to recap, you're going to gain a deep understanding because you're using
6:59active learning, and you're going to understand all of the topics rather than
7:03memorizing facts, which is terrible, especially for certification like data X.
7:09And you're going to identify knowledge gaps.
7:12And this is going to help you pinpoint areas where you need to further study
7:15and also save time if you already know something.
7:18And the goal here is improved retention because the act of explaining a concept
7:22is going to reinforce your understanding and improve memory retention, which is
7:28crucial.
7:29All right.
7:30So that's it for this video because now we know almost everything that we're
7:33going to do.
7:34And in the next video, we're going to dive into my adept courses, which I call
7:38the zero to hero data science pipeline.
7:41And these are going to offer deep dive resources for you.
7:45And there's more than what I have listed here.
7:47here. See you there!
Introduction
0:00Welcome back. All right, so now let's talk about each one of these courses. So
0:04the best way to do
0:05this, if you're new to CBT nuggets, then all of this will just seem like, well,
0:09this is adept.
0:10Let me make this a little bit bigger. But if you're used to the old CBT nuggets
0:14website,
0:15then this is going to be something that is new to you. And the reason I
0:18mentioned this is because
0:20this is something new and it's changing. So let me show you how to find these
0:23courses, because if
0:24you just type in this search area, it's a little tricky. The easiest way is to
0:29go to author and then
0:31go show more and then find my name, Jonathan Barrios, click on that. And then
0:36you'll see all of these
0:37courses. So my most popular one right now is Introduction to Machine Learning.
0:42But this is also a new one
0:43that just came out and its data science fundamentals. Now going back to this
0:47list, we're starting from
0:48the very beginning here. And this is introductory Python for data analysts. So
0:53if you don't have a
0:54programming background, this is a great place to start. And real quick, in the
0:58next video, we're
0:58going to talk about the origins of data science, because a data scientist can
1:03be someone that has a
1:05math background, or somebody that has a business analysis or business
1:09intelligence background, or
1:11somebody who's a programmer, and they're working with computer vision and deep
1:15learning and things
1:16like this. We're going to talk about where that term came from, because the
1:20reality of data science
1:22is that it's a lot of different paths. And the reason I'm showing you this is
1:26because
1:27depending on your path and your background, you may not have a programming
1:31background,
1:31or you may not have the math background, you may not be aware of what a t test
1:37is, or how to form
1:39a hypothesis for a hypothesis testing. And if you don't have that programming
1:43background, I have a
1:44lot of these resources here to share with you. So we're going to go over that
1:47in the next video.
1:48But that's why I'm showing you all of these different resources right now,
1:52because
1:52you could leverage these. And then we'll talk about the origins of data science
1:56, and then get into
1:57what is data science by talking about the domains. All right, so going back to
2:01this, this starts at
2:02the very beginning. So let's say you have a math background, but you don't have
2:06a programming
2:06background, this is a great place to start. If you don't have experience
2:11working with databases,
2:12SQL for data practitioners is a great place to get started. And then here we
2:17get into pandas and
2:18Python, which is really data manipulation and data wrangling. And the iSACA
2:24data science
2:25fundamentals, this is really for conceptual information. So a lot about the
2:31data science life
2:32cycle and things like that a little bit about machine learning, but really
2:36machine learning is
2:38going to be here. So the introduction to machine learning is a pretty in depth
2:41introduction,
2:42but it's definitely in an introduction. So it's not going to be alienating, it
2:46's going to be quite
2:46accessible. And if you wanted to go into deep learning, I would suggest having
2:50a little bit of
2:51machine learning under your belt before diving into neural networks, which is
2:56deep learning.
2:57And I really think about deep learning as the newer version of machine learning
3:02. So let's take
3:03a look at each one of these. So let's go to, oh, and I didn't mention the CompT
3:07IA data plus.
3:08I would describe this as data analysis. So data analysis is also a wide field,
3:14not as much as
3:15data science. And I think the difference here is really machine learning. But
3:19they share,
3:20there's a lot of overlapping concepts. Again, we're going to talk about this in
3:23the next video.
3:24And here, if you don't have a programming background, Python development
3:27environments,
3:28which environments are going to use, this is really a great way to start
3:33because you're not a
3:34programmer, right? You're a data scientist or data analyst. And we use a
3:38slightly different set of
3:39tools compared to a programmer, you can use programming tools. And just go
3:44ahead and use
3:45something like Visual Studio code. But we're going to be focusing on things
3:49like Jupyter Notebooks
3:51and Google Colab, and these kinds of IDEs, just to, you know, clarify that. And
3:56great,
3:57if you don't have a programming background. Let's say you have a programming
4:01background and data
4:02science background, but you're not great with data wrangling and data
4:05manipulation. Again,
4:07this starts with Jupyter Notebook. And then it gets into how I mean, it goes
4:11really deep into
4:12working with data frames. And the data frame is the programmatic equivalent to
4:17a spreadsheet. So
4:18it's a table. And then we have a series, which is like a column. So we're
4:22really working with this
4:24pandas environment, which is a very common tool for data science. And here we
4:28have SQL for data
4:29practitioners. And we're going to use Postgres SQL and PG admins. So you can
4:35set that up. And
4:36actually, it's kind of fun. We work with a movie database in this skill, or
4:40this course,
4:40which is a collection of skills. And again, it's not too big of a course, but
4:44it's really a great
4:45resource if you don't have a background working with databases and SQL in
4:49particular. And conversely,
4:51we're working with Excel. So using Excel is maybe not the most optimal moving
4:57forward,
4:57because Python is such a popular path to take when working with data science
5:01and data analytics.
5:02But what's great about this is that we dive into descriptive and influential
5:07statistics and
5:08hypothesis testing, t tests and things like that very accessible. And then here
5:13's where we turn
5:14up the heat a little bit. So we get into AI agents as an introduction. And then
5:18we get into probability.
5:20And then we define what machine learning is. And as we get into this, we
5:24started getting into
5:25data pipelines, linear regression, and then gradient descent for linear
5:29regression. This is a super
5:31important topic for machine learning. It's a very in depth course. And but it
5:36is an introduction.
5:37And then finally, neural network basics with the perceptron. And we start using
5:42PyTorch. And
5:42then we get into, let me show you down here, TensorFlow. These are the two of
5:48the most popular
5:48frameworks that we'll use when working with machine learning and deep learning.
5:52So I've covered
5:53them both. And then moving on to introduction to deep learning, this is a still
5:57an accessible
5:58course. But now we're getting into neural networks. So here we start with
6:02computer vision. And then
6:03we get into convolutional neural networks, which is basically for computer
6:07vision. And then we start
6:09working with real world image data. And then how to avoid overfitting. And when
6:14you start to identify
6:15any of these gaps that you might have, especially when you take the assessment
6:19scale, you're going
6:20to notice a lot of these terms are going to jump out at you. Like if you find a
6:26knowledge gap,
6:27for example, what is overfitting? Maybe you're not, you need to brush up on
6:30overfitting. You just,
6:32they're going to be in the title. So I try to use semantic titles for these,
6:36knowing that we would be searching for these courses if we had a knowledge gap.
6:40And last but
6:41not least, if you really want to get into TensorFlow, this is where we start to
6:44work with Jupyter Notebook.
6:46The main distinction here is that we're working here with Google Colab. And I'm
6:50going to show you
6:50that in just a second. Actually, let me show you right now. So if you go to col
6:54ab.research.google.com,
6:57you get this example. Let me open this example here. And this is just like Jupy
7:01ter Notebook, but
7:02it's, it's part of Google. So all you need is a Google account, a Gmail account
7:07. And once you have
7:08that, you can actually just go to colab.research.google.com. And these will be
7:13saved into your drive. And so
7:15these are text cells and then code cells. And you can execute these. And then
7:19you get the output right
7:20here. So the difference between the introduction to deep learning and the
7:25professional TensorFlow
7:26developer is this is going to be a lot more technical and not as accessible as
7:31the introduction to
7:32deep learning. We're going to be using Google Colab here. And really just, you
7:35know, it's more of
7:36an introduction. But here we're going to really just turn up the heat and use J
7:41upyter Notebook
7:42locally, which is a really interesting way to do that because you may not have
7:47a GPU powerful
7:50enough. So if that's the case, that's why we use Google Colab because you can
7:53use Google's GPU.
7:55And for $999 a month, you can get a pretty powerful, you know, an array of GPUs
8:00to use. All
8:01right. So I think that's about it. We covered a lot of information. And the
8:04reason that I'm doing
8:05this is just to give you an overview of what's available to you. Because in the
8:10next scale,
8:10you're going to get an opportunity to answer a bunch of questions. And you can
8:14also on top of that,
8:15use the Findment technique to identify any skill gaps that you may have. And
8:20when you identify them,
8:21then you might think, well, you know, I don't really feel very comfortable with
8:26SQL. Let me go
8:27to SQL for data practitioners. Or maybe you identify that you need some, you
8:32need to brush up on
8:33some computer's vision. So you would go to introduction to machine learning or
8:37deep learning. So the
8:39way that I would approach these two is to have some foundation of machine
8:42learning before going
8:43into deep learning. All right, in the next video, we're going to talk about the
8:47foundations or the
8:49origins of the word in the term data science. See you there.
The Origin of Data Science?
0:00Welcome back. So the origin of data science. This is a really interesting
0:05exploration because
0:08there's no really correct answer here. There's a lot of historical evidence for
0:12different terms
0:13being used a long time ago. But really, this is something quite new. And the
0:17story that I heard
0:18that is unconfirmed, and I'm going to tell you this story, and then I'm going
0:21to refer to a
0:22YouTube video that we have, there's something that you can reference. But the
0:25story that I heard,
0:27which is really kind of similar, which is the hiring manager at Facebook was
0:31looking for a data
0:32analyst, and they were doing all of these interviews, but they really were
0:36turning up the heat in the
0:37data analysis department. They wanted to do some really innovative things. So
0:41they were looking for
0:42the really top talent. And when they found this person, they were like, this
0:47data analyst is
0:48perfect. And then they made a job offer. The term data analyst didn't sit well
0:53with a candidate.
0:54They're like, well, I'm a lot more than a data analyst, right? Because they had
0:58a lot of computer
0:59vision or not computer vision, computer science and machine learning techniques
1:03. So they were
1:04applying in statistics. So there was a lot of science happening. So I think you
1:10can know where
1:11I'm going with this. So they actually really brainstormed and then came up with
1:15this, they combined data,
1:18the data from data analysis and research scientist, and then smash those
1:23together to a mashup and
1:25his data science. And so that's a story that I heard. And I saw a video on it
1:28and it was the,
1:29I just don't know where that video is. It was a hiring manager at Facebook. But
1:33let's, I'm going
1:34to show you this other one and link the video below this video so that you can
1:38actually take
1:39a look at it for yourself. It's pretty interesting. So the one that I'm
1:42referencing is from LinkedIn,
1:44and it's DJ Patil, who was the data scientist under Obama. And what they did is
1:51that they were
1:52trying to actually come up with a similar situation, come up with a term, and
1:57they didn't know what
1:58to call this new role because it was a mixture of data analytics and we can put
2:03computer science
2:05over here and research scientists. So when you take all of these, you can see
2:11there's a lot of
2:12terminology and they were coming up with different ones. So what they did is
2:16they created a bunch of
2:17these job roles because well, they said it is LinkedIn. So we can do this. And
2:21the one that
2:22everyone applied to that everyone gravitated toward was data scientist. All
2:28right. So that's
2:28the story. So why am I talking about it like this? Like I said before,
2:33everybody has a different role.
2:35For example, in my case, I started, I have a programming background and I was a
2:42full stack
2:42engineer, but I was learning machine learning. That was something that I was
2:47really passionate about.
2:48So that was really easy for me to jump into from computer science and full
2:52stack development
2:54because of Python. I already knew Python and the Python libraries really make
2:58machine learning
2:59quite easy to get into. And very early on, I found a course on Stanford online
3:04from Andrew Ng on
3:07machine learning, one of the very, very first courses. And it was something
3:11that I just watched
3:12and I was immediately into machine learning. And so if you combine computer
3:17science and machine
3:18learning, all you need is domain knowledge to be a data scientist. And so my
3:24education path,
3:25because I knew these things, I was asked to continually make courses on data
3:31analytics,
3:31data science, and then machine learning. And so that's why I'm teaching the
3:36data X certification
3:38course here at CBT nuggets. That was just my natural path. But if you think
3:42about it,
3:43let's talk about different kinds of paths, people that I've worked with and
3:47paths that I know
3:48that are very, very distinct and not based in computer science. So I'm going to
3:53say computer
3:54science over here. And over here, we can say math and statistics. And here we
3:59can say BI for
4:00business intelligence. And you can also have a researcher. And I guess we can
4:05put data practitioner.
4:07All right, so we have 12345. And this is being a little conservative, because
4:15going back to DJ
4:16Patel, they were actually trying to find a terms like what do we call these
4:20people that do
4:21everything with data. So that's who we're talking about. This is a data
4:26scientist,
4:26people that do everything with data. So we have all of these different roles.
4:31And where do you
4:32lie here? So this is my path. So I was coming out of computer science, but also
4:37research. Okay,
4:39so these two apply to me. Now, I've worked with a lot of statisticians. And I
4:44've worked with a
4:45lot of business intelligence folks, and people that I would just call data
4:48practitioners that
4:49were wizards with data, but they were working with Excel. And quite a few of
4:53the math and
4:54statisticians were also working with Excel. Here, I was working with Jupiter
4:58notebook. But you can
5:00also have IDEs, like VS code or pie charm, or whatever you use for programming.
5:06And that was
5:07something that I was familiar with because I was doing programming before I got
5:11into machine
5:12learning. And then as a result, was called a data scientist, because I do
5:16everything with data.
5:18So a researcher can use a combination of these tools. And business intelligence
5:22might use things
5:23like power BI or Tableau. Right. So there's not programming going on here. They
5:29create visualizations
5:31and they make presentations to stakeholders. So really, we have a wide array of
5:36paths for what
5:37is a data scientist. And in closing, the reason I'm bringing up these
5:41distinctions is first,
5:43I want you to feel comfortable with your path. Number two, I want you to
5:47understand that you can
5:49actually jump around and augment your other skills that you may not have. It's
5:54rare for somebody to
5:55have all of these different paths under their belt. And the data X
5:59certification is a pretty
6:01thorough and pretty wide array of domains, as I showed you before. So it's
6:07going to be common
6:08for you not to have all of those under your belt. And as a result, I've created
6:13these courses that
6:14you can use for deep dives. Okay. And now what we're going to do in the next
6:18video is talk about
6:19what is data science and talk about the domains and subdomains that we're going
6:23to be working with.
6:24See you there.
What is Data Science?
0:00Welcome back. In this video, we're going to explore what is data science
0:04because it's a collection
0:05of different domains. For example, if I said AI, what would that mean? If I say
0:11data science,
0:12what does that mean? Let's dive into this and break it down so we know what
0:16domains we're dealing
0:17with in the realm of data science. And then we'll segue into the data X domains
0:23. So let's start
0:24with the two largest domains. If I draw one circle here, and then another one
0:28like this here, I could
0:30say that this is data science. And we can just say that this is AI or
0:34artificial intelligence. And
0:37inside of this, what is the overlap between AI and data science? And again, AI
0:42is a large umbrella
0:44term for many subdomains. And so is data science. But the overlapping ones that
0:49we're going to talk
0:49about in the context of AI is going to be machine learning, and then deep
0:54learning. So deep learning
0:56is the newer version of machine learning. And that's just a really simple way
1:01to hang it in your mind,
1:02right? These are just simple terms without jargon that we're going to this is
1:07this one came first
1:08and this one came next. That's another way to put it. But when we think of AI,
1:13we can think of it as
1:14expert systems, knowledge graphs, advanced search, things like this. So it's a
1:19larger umbrella that
1:21also includes machine learning and deep learning. So what is machine learning?
1:25So machine learning
1:26are is a domain that includes algorithms that learn patterns from data. And so
1:31what is deep learning?
1:33So these are multi layered neural networks for automated feature extraction and
1:38other cool things.
1:39So the takeaway between machine learning and deep learning is that we're using
1:44algorithms with
1:45machine learning. And with deep learning, we're using neural networks. So what
1:49I mean by that,
1:50neural networks or deep learning is software that's modeled after the brain,
1:56how neurons work in the
1:57mind. They have these sort of activation thresholds and they'll fire and move
2:03to the next or neuron,
2:05or they won't. And that's going to be explained in much more detail when we get
2:10into machine
2:11learning and deep learning. But the main distinction is algorithms and neural
2:15networks. Now what I'd
2:17like to do is dive deeper into machine learning and deep learning. And then we
2:20're going to dive a
2:21little bit deeper into data science, because there's a bunch of sub domains
2:24that we have not
2:25discussed for data science. But first, let me just say that data science is end
2:30to end data life
2:32cycle. So this is everything from collection, cleaning, analyzing, modeling,
2:37deployment,
2:37and even communication. So there's a lot to talk about. So when we're talking
2:41about machine
2:42learning, we said that we're that we're using algorithms. So I'm just going to
2:46say algos are
2:47short. And we're learning patterns from data. And so some of the tools and
2:53techniques. So we can use
2:54side kit learn, if you have a background on machine learning, you probably
2:58heard this before,
2:59there's xg boost, spark, and others. And again, this is a subset of AI. And we
3:06're really emphasizing
3:08on predictive modeling from labeled and unlabeled data. We're going to dive
3:15into these two terms
3:16much more soon. But this is the, I would say the overview. And we're talking
3:21about deep learning,
3:23we're using neural networks and N for short. And I would say that this is a or
3:28deep learning is a
3:30subset of machine learning. And some of the tools and frameworks and techniques
3:35. So we've talked
3:36about TensorFlow, there's PyTorch, there's Keras, and things like the
3:40transformer architecture. So
3:42let's talk about that. So some of the relationships might include computer
3:48vision or CV, natural
3:49language processing or NLP and pattern recognition. And when we said CNN, these
3:55are convolutional
3:56neural networks and RNN recurrent neural networks. And when we said transform
4:02ers, that's an architecture
4:03that's widely used with large language models or LLMs. Again, we're going to
4:09get into all of this
4:10in the course and define all of this. But remember that you have that resource.
4:15If you need to deep
4:15dive into machine learning, there's a great skill or course introduction to
4:20machine learning that
4:21starts from the beginning. And then if you want to get into deep learning, you
4:25take that other one.
4:26And you can cherry pick, right? Using the Feynman technique, find out where you
4:30have gaps and then
4:32really just focus on those. But what I would say for both of these introduction
4:36to machine
4:36learning and deep learning, check out the very first videos because it sets you
4:41up with a development
4:42environment and kind of like lets you know what's going to happen through the
4:45rest of this course.
4:46And then you can cherry pick what you need to fill in your knowledge gaps. Okay
4:51, so we've knocked
4:51out AI and machine learning and deep learning. Now let's talk about data
4:56science. But I think we
4:58should do that in another video. So let's do that next.
Data Science Lifecycle & Machine Learning Approaches
0:00All right, so now we're going to talk about data science.
0:03This is a much bigger one.
0:04And again, this is a subset or an umbrella.
0:07Let me say that, which includes part of AI.
0:11And by that, we're talking about machine learning and deep learning.
0:16OK, so this is a very broad umbrella term.
0:18And so the tools include pandas.
0:20We've talked about that briefly.
0:22SQL for working with data, Spark, Jupiter, which we talked about that.
0:27That's also used in machine learning and deep learning as well.
0:30There's visualization.
0:32And we use things like matplotlib and seaborn for visualization.
0:38And this also includes MLOps tools.
0:41Talk about that as well.
0:42That's part of the data X certification.
0:44OK, so now we've talked about some of these tools and techniques.
0:47But what's the primary focus?
0:48So again, end to end data lifecycle.
0:52So let me put that here.
0:53End to end data lifecycle.
0:56All right, so what is the end to end data lifecycle?
1:00We talked about that briefly and we're going to get into it together.
1:02Let's just go ahead and do that now in this section.
1:05So this is the lifecycle part.
1:07So one is going to be data collection.
1:10And that also includes warehousing.
1:13Two would be-- and this isn't that in any particular order.
1:17It's a loose order because data governance, which is the next one,
1:21it can be implemented in many different places.
1:24But the reason I'm including it here is that it's part of,
1:28after you collect the data, you clean the data.
1:31And so there's some data governance that
1:33goes with cleaning the data.
1:35And number three, we get into EDA.
1:37So once you've collected the data and then you've cleaned it
1:41and making sure that you're following the rules and regulations
1:44because there's a lot to it, you can't just take data from any country
1:48and process it anywhere.
1:49And also, you need to make sure that you're protecting information
1:52and using security, all these things.
1:54And so EDA is exploratory data analysis.
1:59So this is a term that you may have heard before.
2:02And what it is, is just really the initial step.
2:05When you've collected it, you've cleaned the data.
2:08What do I mean by clean the data?
2:10Let's say there's a bunch of stuff that you don't need,
2:13maybe some garbage columns or something like that,
2:15or some missing data that you need to handle as part of cleaning.
2:19Once you've got the data sort of prepared,
2:21you do some exploratory data analysis, usually visualization.
2:25You check out the data and you're like, okay, this is what we're working with.
2:28And we're trying to answer this question or solve this problem.
2:32This kind of gets you started.
2:33And then we'll get into things like feature engineering.
2:38Maybe you want to create or combine a few columns to create
2:41a new column of information, but it goes a lot deeper than that.
2:46And then number five, we get into things like modeling.
2:49And you would build a model and then you would use validation,
2:53measure the performance of that model, make sure that it's performing.
2:56There's a lot to this, but I'm going to say that you want to have a certain
3:01accuracy
3:01and make sure that the model is not, I'm going to say,
3:05and this is including data, machine learning and deep learning, right?
3:09When you're building these models, they can be statistical models too.
3:13So modeling is a larger term, but you want to make sure that you have an
3:17accuracy
3:18that you're working with and that you're not including bias or that you're not
3:22overfitting.
3:23So these are terms that we're going to go into a little bit later,
3:26but this is the part of the life cycle for right now.
3:29And so modeling and validation on to say, VAL.
3:32And then number six, this is where you would do the deployment,
3:35deployment and monitoring.
3:38And so this monitoring part is what we call MLOps,
3:42and we'll dive more into that as well.
3:44Now what we need to do is dive back into machine learning
3:47and break it down a little bit more because there's some more granularity
3:51that we didn't talk about.
3:52We just talked about it really just in a bird's eye view.
3:56So let's dive a little deeper.
3:57So now diving into machine learning, number one, we have supervised learning.
4:03And this is something that we're going to talk about in depth,
4:06but you can also look at this in the introduction to deep learning.
4:09And when you're talking about this, this is normally things like classification
4:15,
4:15regression, and then we have unsupervised learning.
4:19And here we're talking about clustering and something called dimensionality
4:23reduction.
4:24You've heard of, maybe you've heard of PCA, principal component analysis,
4:28as part of that, and also association rules.
4:32And number three, we have something called reinforcement learning.
4:36And here what we're doing is that we have rewards and penalties
4:41in some kind of environment.
4:43So let's say that we have like a maze just to make sure that this is super
4:47clear.
4:47And this is the goal and this is sort of a dead end.
4:51And as the agent goes through this environment,
4:54it'll get rewards or penalties based on the decisions that it makes
4:59in a nutshell, reinforcement learning.
5:01And the introduction to machine learning, I go through this kind of an example
5:06using a lot of different use cases.
5:09And I call it grand search auto because it's really a search problem.
5:14And then we have number four, semi-supervised learning.
5:19And here what we're talking about is we might have a small labeled data set
5:25and maybe a large unlabeled data set.
5:28And let's say number five, you want to create a model to create labels to
5:34predict
5:36or to predict tokens.
5:37That would be self supervised.
5:40And again, this one number five, that's where we create models that create
5:44labels
5:44or we want to create generative toking or generative models that create tokens.
5:50And that would be things like GPT.
5:52And then number six, we have multimodal ML.
5:55So multimodal machine learning is combining text, images, audio and sensor data
6:01.
6:01So computer vision, natural language processing, all of that good stuff.
6:06So this is all under machine learning.
6:09And the subset of that is deep learning.
6:12So all of this is sort of interrelated.
6:14All right, so that's a lot of information.
6:16And what we're doing here, don't worry, you don't have to memorize this.
6:20This is just really breaking down the domain.
6:22So we understand what's inside of these larger umbrella terms.
6:25The largest one being, or the largest two is AI and data science.
6:30And in the next video, I'm going to show you some code examples
6:34and some outputs and some visualizations.
6:36So this starts to make a little bit more sense because right now, this is all
6:39really abstract knowledge, right?
6:41And I'm telling you that you don't have to memorize it.
6:44But when you see these images, you're going to start to get the picture.
6:47So I'm really assuming that, you know, I'm talking to you like you don't know.
6:52But if you know, then, you know, please excuse me and just move on.
6:56But I'm trying to make this accessible because as we discussed earlier,
7:00there are so many different paths for data science.
7:03I don't know what you know, what you don't know.
7:05So just take that with a grain of salt and keep that in mind that I'm really
7:09trying to make this accessible.
7:11And once we move out of this first skill and the next skill, we're going to be
7:14cooking pretty fast.
7:16I'm going to make sure to use every not to use any jargon to make things really
7:20accessible.
7:21But we're really just going to dive into the data extertification course.
7:26What I'm doing here is I'm covering some of the domains, some of the
7:29information you're going to need to know
7:31that's going to be on that test right now.
7:33But I'm also doing it in a way that is accessible.
7:36All right, see you in the next video.
Machine Learning & Data Science Application Examples
0:00In this section, we're going to explore a few of the major data science
0:04application areas you'll need to master for the data X certification.
0:09Each section starts with an intro and ties it back to real-world problem
0:13solving followed by a short code example to really make sure that this is
0:18tangible and concrete, right?
0:20We're moving away from these abstract ideas.
0:23And what I've done is let me move over here to Google Colab. So now you know
0:26about Google Colab. If you have a Gmail account, great. If not, get one set up.
0:32And once you do, all you have to type is colab.research.google.com.
0:36And I've included each one of these code snippets below this video.
0:40So you can just copy and paste them into this notebook and then play with the
0:44code.
0:45But you don't need to even do that because I'm going to show you what they are.
0:48Or actually, I'm going to explain exactly what's happening and how this relates
0:53to the different subdomains that we just discussed.
0:56So the first one here, as you can see, we're making some using simple
1:00temperature forecasts.
1:01So we're using some historical data over this like time series data.
1:06So this prediction is a task that usually revolves around forecasting future
1:11values based on historical data.
1:13In this case, we're using temperature.
1:15And in this case, it's pretty straightforward because what we're doing is
1:18predicting temperature.
1:20Or it could be as complicated as a demanding forecast in, let's say, a supply
1:26chain management.
1:27That can be really complex.
1:29Even if you're working with advanced neural networks in production, the
1:33fundamentals of setting up a supervised learning task is probably going to be
1:37about the same.
1:38You select some features, you train a model, you evaluate it, and then you iter
1:44ate.
1:44So what we're doing here is you can see this green part is the prediction.
1:48And the reason that this is a very simple, basically, this is just a linear
1:54regression model that's very, very simple using synthetic data.
1:59And what we're doing is that we're measuring the distance from all of these
2:03points.
2:03And it has this line that's the prediction line.
2:06And then here is going to predict that this is where the temperature is going
2:10as the days progress.
2:11So this is a prediction example.
2:14And again, this is simple linear regression, but you might also use a RIMA
2:19profit or long, short term memory when you're working with models in production
2:24.
2:24All right, so now let's move on to pattern mining.
2:27So here, pattern mining is going to focus on discovering non-trivial
2:31relationships and frequent patterns and large data sets.
2:36So typically, this might involve some kind of market basket analysis, anomaly
2:41detection, or sequential pattern discovery.
2:45And this is particularly useful for recommendation engines or identifying
2:49relationships among seemingly unrelated data points.
2:53Right?
2:54So let's say that you're trying to find items that are purchased together.
2:58Right?
2:58And so here, it's going to show you that these two milk and bread are purchased
3:03together more often than these.
3:05And this is giving you a basically a number between zero and one.
3:10And I don't want to go too deep into this, but this is an example of pattern
3:14mining.
3:15The next one is segmentation.
3:17Segmentation typically falls under unsupervised learning because you aim to
3:23group similar observations together.
3:26And common examples include customer segmentation, clustering of documents.
3:32That's NLP and the customer segmentation that would be marketing or even
3:37partitioning images based on color profiles.
3:40And that would be for computer vision.
3:42So really, the takeaway here is that the goal is to expose hidden structures in
3:47the data.
3:48So here, we're not telling it what this is or what this is.
3:52It's just saying that it's finding these two clusters by using K-means segment
3:57ation.
3:57We'll talk about K-means clustering and segmentation at a later date.
4:01But this is just an example of segmentation.
4:04Alright, so now moving into natural language processing or NLP,
4:09we have an example that handles textual or linguistic data.
4:12So this could be everything from sentiment analysis, like let's say movie
4:16reviews, you know, or right here.
4:18I love data science.
4:20Data science is awesome.
4:21I hate spam emails.
4:23So we can say that this is going to be positive or negative.
4:26And so you can see here we have made a comment there.
4:29So these two are going to be positive.
4:32These two are negative and this one is positive.
4:35And as you can see, that's pretty straightforward.
4:37It also includes things like named entity recognition.
4:42And this is an even large language models like chat GPT.
4:47They all use natural language processing.
4:49The core tasks revolve around extracting meaning from text.
4:53So I like to say teaching computers how to read.
4:56And then this could be unstructured or semi structured data.
5:00And we need to understand things like tokenizing and beddings and model
5:05selections and also things like naive base and transformer architectures.
5:10These are all essential.
5:12So here we're using a naive base model with scikit learn and you can see that
5:17we imported scikit learn right here as SK learn and here's naive base.
5:22Alright, so now let's look at network analysis.
5:26Network analysis explores how entities that could be people nodes or systems
5:32how they're interconnected.
5:33This is crucial for things like social media analytics fraud detection.
5:37In that case, it would be transaction graphs or biological networks.
5:42Those would be things like protein interactions key concepts include graph
5:46theory, centrality measures and community.
5:49So when we're talking about centrality, we're talking about distributions.
5:54And this is a pretty simple graph because most of the real world networks are
5:59large and complex.
6:00But here we have a pretty similar simple graph illustrating some sort of
6:05hierarchy.
6:06Right. So we have child a and or we have root and then child a and child B and
6:12then we have a grandchild of called B one.
6:15And when you have more than one, you'd have grandchild a one and grandchild a
6:18two.
6:19So this is just a tree like graph.
6:21And now moving on to signal processing.
6:24This includes things like images, audio, sensor readings, etc.
6:29This is modern computer vision and we leverage things like deep learning.
6:33And that includes convolutional neural networks or things like transformers.
6:39Right. There's even vision transformers.
6:42So there's a lot of code here, but this is an example of some classification.
6:47So you feed it this image and it classifies what it is.
6:51So here we have a sandal. This is a sneaker trousers pull over and so forth.
6:56And this would be an object detection task.
6:59And it could also include image segmentation in contrast to modern computer
7:04vision,
7:04which is what we're talking about classic signal processing relies on
7:08techniques like four year transforms or wavelet analysis for pattern detection
7:14in time series or acoustic data.
7:16So audio and I can give you a better example of this.
7:19I have this is in one of the, let me see where it is right here in the
7:24introduction to deep learning.
7:27I believe we get into this emnis fashion data set.
7:30And here you can't really see it in this image, but this is actually showing
7:35you the pixels and you can kind of see a shape and that shape is this shoe.
7:40And so all of these would be zeros all around it. And if I were to extract this
7:45image or make this wider, you would see that.
7:48But what we're doing here is that we have this 28 by 28 tensor that are
7:53converted into numbers and we're using that and we have these classes here.
7:57So it's labeled data and then we can make predictions.
8:00And so in this case, we're looking at these and here is the code where we go
8:04through a bunch of training epochs and here is the validation accuracy and the
8:09validation loss.
8:10So there's two different accuracies. So you can think of this as training data
8:14and then data it's never seen.
8:16So this is what it uses to learn and this is what we, how we test it because
8:21this is data it's never seen.
8:23And this is just a good example of a multi class classifier, meaning that it's
8:29not just one image.
8:30So is it a sneaker or is it not a sneaker? So that would be binary multi
8:35classes when you're dealing with several multiple classes.
8:39In this case, each type of fashion or garments in this case, which also
8:44includes shoes and sandals.
8:46Alright, so that's it for this video. And the next section is going to be a
8:50short challenge.
8:50And what that is, something that you're going to get used to at the end of
8:54every scale we have this sort of challenge section.
8:57And it's going to ask you questions and based on what you saw or learned in the
9:02skill, you should be able to answer those.
9:04And if not, you can just go ahead and review.
9:06So it's just going to be a quick review, everything that we just, these
9:10examples of data science in application that we discussed.
9:13So I'm going to show you images and then you're going to answer what type of
9:19application is that in data science.
9:21Alright, see you there.
Team training path
Turn this skill into assignable team training
This free skill is a preview of the courses your team can assign, track, and report on with CBT Nuggets.
$708
seat / year