Monitor & Measure Resources Usage

0:00[MUSIC PLAYING]

0:12Hey, guys, and welcome to the LPIC-2 course

0:15here on CBT Nuggets.

0:17So before we begin actually digging

0:19into the details of what we actually

0:21have to learn for this examination,

0:23let's just briefly take a word about the exam itself.

0:26Now, the first thing to state about this examination

0:30is that you will have heard of the LPIC-1 examination

0:34if you are interested in taking the LPIC-2 exam no doubt.

0:38That is because the LPIC-2 actually

0:41builds upon the LPIC-1 exam.

0:44Pretty much what you learn in LPIC-1

0:47is a more foundational level of information,

0:50and then you take that information

0:52and build upon it on LPIC-2.

0:55Now, the LPIC-2 examination ultimately

0:58attempts to validate the candidate's ability

1:02to administer small to medium networks.

1:05Now, when you're getting to administer medium size

1:09networks, for example, you really

1:10do have to be fairly comfortable with the Linux system.

1:14Now, one thing to note about the LPIC-2 examination is that any

1:19candidates seeking to take this exam must actually have

1:23an LPIC-1 certification--

1:26an active certification.

1:28So that means that your LPIC-1 certification cannot be

1:31expired.

1:32I do believe the validity period for the LPIC-1 certification

1:36is five years.

1:38So within five years of passing the LPIC-1 exam,

1:41you will be able to take the LPIC-2 exam.

1:44Now, another thing to note is that just

1:47like with the LPIC-1 exam, the LPIC-2 certification actually

1:52is comprised of two separate exams.

1:55The first exam is known as the 201 exam, and the second exam,

2:02this is the 202 exam.

2:05Now, as it transpires, what you can do

2:07is take these examinations completely separately.

2:10So you could just focus on the content of the 201 exam,

2:15pass that, and then focus your attention on the 202.

2:18This is often the recommended way

2:20to undertake the LPIC-2 examination.

2:23And just like what the LPIC-1 examination,

2:26when you happen to pass your LPIC-2 exam,

2:29you will have this certification valid for five years

2:34unless, of course, you happen to gain

2:35an even higher level of certification-- say,

2:39for example, an LPIC-3 exam.

2:41Now, one thing to really note about the LPIC-2 examination

2:45is that there is going to be such a common theme

2:48with the contents of what you learned within LPIC-1.

2:52Like I say, it does heavily build upon that knowledge.

2:55That means that you can expect to see many of the same tools

2:59that you've used within LPIC-1 again revisited in LPIC-2,

3:05and very often actually expanded upon--

3:08i.e., we dig a little bit deeper into these tools.

3:11But as a high level overview, some

3:14of the things that you should be expected

3:16to be able to do when you go to sit the LPIC-2 certification

3:20exam is that you should be able to perform fairly advanced

3:24system administration tasks.

3:26And that should be things like relating to the Arch Linux

3:29kernel as well as the system startup and its maintenance.

3:34Now, not only that, you should also

3:37be able to perform fairly advanced networking

3:41and authentication as well as, by the way,

3:44being able to handle and manage particular security

3:47features such as dealing with firewalls or Virtual Private

3:52Networks, otherwise known as VPNs.

3:55Now, with respect to talking about networking,

3:57some of the things that you can expect

3:59to be able to have to know for this examination

4:03is being able to be comfortable with configuring things

4:06like DHCP--

4:08this is the Dynamic Host Configuration Protocol.

4:11Now, again, if you happen to be a little bit

4:14hazy on this from what you learned in the LPIC-1

4:17examination, do not worry.

4:18You will get to cover the concepts once again

4:21and, like I say, dive a little bit deeper.

4:25But we'll get to see things like DHCP, like I say,

4:27as well as DNS, which is Domain Name System which

4:31we use to resolve our domain names to particular IP

4:34addresses, as well as dealing with things

4:36such as Secure Shell for secure, encrypted remote management

4:41of servers.

4:42As well as dealing with things like web servers

4:45and fail servers, using things such as FTP--

4:49the File Transfer Protocol-- as well as

4:52dealing with email delivery and things such as Samba.

4:56And as always, with Linux, we also

4:58want to dip our toe into a little bit

5:00of automated management-- things like being

5:03able to use scripting to help us manage our systems.

5:07So really, what I'm saying is that the LPIC-2 examination

5:09really is a fun one.

5:10There's a lot of things it again too.

5:12And if you have recently passed the LPIC-1 examination,

5:16so much of what we're going to cover

5:18should be fresh in your head.

5:20And that will make it much, much easier

5:22for you to take that fresh knowledge

5:24and build upon it within this course in LPIC-2.

5:27But like I say, if you happen to be a little bit

5:30rusty on that information, do not worry.

5:32We will absolutely cover all the essentials and core

5:36fundamentals we are bringing along with us from the LPIC-1,

5:39and make sure we are definitely up to speed

5:41and where we need to be.

5:43So that really is our little introduction

5:45into the LPIC-2 certification exam.

5:48How about now we actually dig in to some of the details of what

5:51we have to know?

5:52The first subject I want to talk to you

5:54about is all about monitoring CPU and memory utilization.

5:59And, well, that is what we're going

6:00to be covering in the very next Nugget.

6:02So I hope this has been informative for you,

6:04and I'd like to thank you for viewing.

0:12Hey, guys.

0:13And welcome back.

0:14So with the LPIC-2 examination, naturally it

0:17follows we do have to have access

0:19to some type of Linux-based system.

0:22In my case, I actually have a Windows machine.

0:25But the way I can access a Linux machine

0:28is that I just happen to use some type of virtualization

0:31software such as VMware, or in my case right here, VirtualBox.

0:36So within my own VirtualBox, I have

0:38a whole ton of different Linux distributions.

0:42And this case right now I happen to be using Xubuntu.

0:45Now if you were following along from the LPIC-1 examination,

0:49I actually went through how we could install things

0:52like VMware and VirtualBox.

0:55And how we could install within those tools things

0:58like Xubuntu or Red Hat Rocky Linux.

1:01So what I'm saying is that if you don't have

1:03a virtual machine up and running,

1:04then I definitely would recommend

1:06you do that just now to make sure you're up and running.

1:09My preference here is to be using the Xubuntu Distribution,

1:13which is a lightweight version of the Ubuntu Distribution.

1:17But if you would prefer to use something like, say, Ubuntu

1:19itself, then that is absolutely OK.

1:22So with that little word at the beginning, just to get us up

1:25and running, now let's begin and log into my system.

1:29So right here I have my virtual machine.

1:31What I'm going to do is I'm going to log

1:33in to this virtual machine.

1:34And here I am right here.

1:36So this should be a familiar sight.

1:39I have my Linux Terminal.

1:40In fact, let me just open this up afresh.

1:43So we can see here, just go with the Terminal Emulator,

1:45and this is all I'm doing.

1:47So let's begin talking about our subject then.

1:50So what we've planned to do is to talk

1:52about how we can monitor CPU and memory utilization.

1:56Now why would that be important?

1:58Well, think about it.

1:59What would be the number one complaint

2:02that your users would happen to mention when looking

2:06for help with their computers?

2:07Pretty much-- and I feel fairly safe in saying

2:11this-- is that people are probably

2:12going to be complaining that for whatever reason,

2:15their computer, their system, their network happens

2:19to be slow.

2:20And nobody likes a slow, sluggish system.

2:23It just really brings down productivity.

2:26This is something that we as system administrators

2:29would want to be able to solve quite quickly.

2:32So very often what you would have to do

2:34would be to identify the actual Bottleneck going on here.

2:39So what do I mean when I say the Bottleneck?

2:41Well, I'm talking about the part of the system,

2:44the component that is actually slowing things down.

2:48So you might just have, let's see a computer here,

2:50and you're connected to a router or a router, if you prefer.

2:54And that connects to your ISP, which is a massive network.

2:58And then you can go to then internet.

3:00Now this is obviously a masterpiece

3:02of artwork right here.

3:03Clearly my drawing skills have not really improved.

3:06So if we happen to experience some real sluggish behavior

3:10on our machine right here, we would

3:12want to be able to determine is there something,

3:15because the server we're reaching over the internet

3:18is very slow.

3:19Or maybe your ISP is having some issues,

3:23maybe our local router is experiencing some issues,

3:27maybe there's too many users within the local network

3:30actually using this router and overloading it.

3:33Or perhaps the problem lies directly on our system itself.

3:37Maybe we are taxing the CPU too much,

3:40maybe we are taxing the memory too much.

3:42So ultimately this is something we would

3:44want to be able to investigate.

3:46Now, we will actually look at things such as network input,

3:49outputs, and how we can test such things.

3:52And that may be, indeed, it is our local network itself

3:56that is slow.

3:57But let's just talk from now about our own system,

3:59our own local computer.

4:01Let's dive in and try to monitor the CPU usage, as well

4:06as the memory usage.

4:08Now, check this out, what I'm going to do here--

4:10so let me just full screen this for now.

4:12Now the first thing I will say is

4:14that on this little virtual machine,

4:16I'm not really taxing my computer too hard right now.

4:19So in order to be able to simulate such problems

4:22whereby my memory is being really harshly used or abused

4:26or my CPU is really overloaded, what I want to do

4:30is to install the stress package.

4:33This is going to allow to simulate such conditions.

4:36So what I'll do is I will say sudo apt install stress.

4:39Now if I hit Enter, I'll have to type in my administrator

4:43password, and hit Enter again.

4:45And this package is now going to begin installing.

4:48Cool.

4:49So I just wanted to get that installed on the system

4:52because we're going to be using it very, very shortly.

4:55But before we do that, one tool I

4:57want to talk to you about straightaway is one

4:59called VM Start.

5:02This is going to allow us to report our virtual memory

5:04statistics.

5:05And this will give us information about processes,

5:08about memory, about paging CPU activity, so on and so forth.

5:12So what I'll do is I'll clear the screen,

5:13and I will just issue the command vmstart.

5:17If I hit Enter here, we can actually

5:20see some information right now.

5:22And at the very top, we can see our headings,

5:24we can see processes, we can see memory,

5:27we can see information relating to swap,

5:29as well as input, output system, and our CPU.

5:34If we check out the memory here, we

5:36can see our swap memory, as well as our free memory, our buffer

5:41and our cache.

5:43So when we see swap memory, what we're talking about

5:46is when the hard drive is being used as a source of memory.

5:52Now this is a concept we did talk about in the LPIC-1

5:55examination.

5:55But if you don't remember, what happens

5:58when we use so much memory, i.e. we

6:01have so much demands on a RAM, that a RAM is fully exerted.

6:06What the system could do in that case,

6:09it could just crash if you try to open another program,

6:12because there is no more memory left.

6:14But, really, that is not a good solution.

6:16Instead of crashing, what we can do is use this swap memory.

6:21So this is going to allow us to temporarily dump

6:25that information, which would automatically be held in RAM,

6:28and use the hard drive instead.

6:31Now really, this is not a good sign for your system.

6:34If you happen to be seeing your swap memory being used,

6:38this tends to mean that your system is

6:39being taxed quite heavily when it comes to memory.

6:42I.e.

6:43There are some conditions whereby you're actually

6:46using more memory than you do have available on your system.

6:50This is something to watch out for.

6:51But we can also see information relating to our free memory,

6:54as well as memory cache.

6:56Now another command we could use to get similar outputs

6:59is the free command.

7:01So what I'll do is I'll say free.

7:02And, again, we're seeing similar information.

7:05We can see our memory that is used, how much memory is free,

7:09as well as what is available for our cache.

7:13Similarly, we can see our swap memory, which

7:16is coming from a hard drive.

7:17And we can actually see how much swap memory has been used.

7:21Now, if we want to happen to see this information, perhaps,

7:24in a little bit more readable format,

7:26we could do free -m to get these values and megabytes.

7:30This can make things a little bit more readable.

7:33But, honestly, one of my favorite commands

7:35to use, with respect to monitoring things such as CPU

7:39utilization and memory utilization,

7:42is the top command.

7:44This is going to allow you to see your running processes,

7:47and even more.

7:48So what I'll do is I'll just clear my screen,

7:51and just use the command top.

7:54So check this out.

7:54We actually get a lot of cool information right here.

7:58We can see the number of tasks we have, 176, two are running,

8:03that has now changed to one on the fly.

8:05We have 175 sleeping, and so on.

8:08Other things we can see, though, are these statistics

8:11right here.

8:12Check this out right there.

8:13We can see our CPU percentage that is being used,

8:16as well as the percentage of memory

8:20that is actually being used.

8:21And we can actually correlate that with a particular process.

8:25So the process ID on the left, we

8:28can actually correlate that to the CPU usage or the memory

8:32usage.

8:32And what it can actually do here is use the greater

8:35than sign, which is going to allow

8:37me to sort by the memory that is being used.

8:41You see that the very top, you see the process with the most

8:44memory or, rather, that is demanding the most memory,

8:47should I say, which happens to be update-manager.

8:50And then we have xorg, and then xfdesktop.

8:54And if I press the less than symbol,

8:57that's going to flip once again.

8:59So we're actually going to see it

9:00by CPU utilization, the top being

9:04the process using the most CPU.

9:07So the top command really is very useful.

9:09And the respect that it allows you to drill in and see

9:11the processes themselves, that happen

9:14to be swallowing your CPU and swallowing your memory if they

9:17happen to be getting taxed really quite heavily.

9:20So check this out right now, I have the top command running.

9:24What I'm going to do is I'm going to actually create

9:26a new terminal.

9:28So I'll just open another terminal window

9:30right here for now.

9:31And actually let me just temporarily full screen this.

9:35And what I will say is I will say stress --help.

9:39This is the till we just happen to install.

9:41So we can see here some information relating to how

9:44we can actually use this tool.

9:46Say, for example, I want to put some stress on the CPU.

9:50I can use the -c flag or --cpu.

9:54And this is going to spawn some CPU processes.

9:57So check this out then, I should clear the screen.

9:59I will say stress -c, and then 1.

10:03I.e.

10:04Spin up one CPU process.

10:07And hit Enter, so we're now dispatching hogs, as it says.

10:10We are going to be increasing the CPU demands.

10:12If I go back to my other terminal right here,

10:15notice the CPU right here is being utilized at 100%.

10:19You see that at the very top, CPU.

10:21And we can tell straight away that it happens

10:24to be the process ID of 14603.

10:28And the command is actually using all the CPU

10:31as the stress command.

10:33Now you might not be able to hear this because

10:34of the microphone filters that I do happen to be using,

10:38but my computer is beginning to work.

10:40That is the fans coming on because the demands

10:43on the computer is beginning to heat this computer up.

10:46So right now what I'll do is I'll go back

10:48to my other terminal, press ControL-C to stop the stress

10:52command.

10:52And if I go back to top, notice we are no longer

10:55utilizing 100% of the CPU.

10:58Now if I wanted to simulate stressing

11:02some of the memory, what I could do is I could go back here.

11:06And, again, if we look at the man page,

11:08or rather, not even the man page,

11:09just go back to the help page, rather, we

11:12can use the -m flag, which will allow us to tax our memory.

11:17So let's try this then.

11:18So I'll say clear, and I'll see stress -m,

11:21let's just give it a value of, maybe, say 4.

11:23So we're running 4 VM.

11:26If I go back to the top output, right now

11:29we are seeing what is consuming the most

11:31CPU at the very top, which is this process right here.

11:35But we can, again, press the greater than sign.

11:38So that would be the right arrow in my case.

11:40Now we are actually seeing what is causing or consuming

11:44the most memory at the top.

11:46So in this case, it is this process for now,

11:48consuming 9.9%.

11:51And it has just changed to 7.5.

11:53This is obviously very fluid output

11:55and always changing in real time.

11:57But, nevertheless, you're always going

11:59to see, at least in this view, what is consuming the most

12:03memory in real time.

12:04In this case right now we can see it as the stress command.

12:08So I'll go back here, I'll press Control-C, clear the screen.

12:11And, in fact-- let me just close this terminal.

12:13So as we can see here, we do have different tools

12:15available to us that we can use to interrogate our CPU

12:19usage, our memory usage.

12:21Things like the free command, the VM stack command,

12:24as well as my favorite, the top command.

12:27And it's using these types of tools

12:29that is going to allow you as a Linux administrator

12:32to be able to diagnose when you have a CPU problem or a memory

12:37problem.

12:38And as you can see here, particularly

12:39from the top command, you can actually

12:42identify the process that is actually causing that problem

12:45or contributing to that problem.

12:48And you can take the appropriate action

12:50needed to begin to stabilize your system.

12:52So that is all for our little introduction

12:55into monitoring CPU and memory.

12:57The next thing I want to talk to you

12:59about is monitoring desk and put and outputs.

13:02And, well, that is what we're going

13:04to be talking about in the very next Nugget.

13:06So I hope this has been informative for you,

13:08and I'd like to thank you for viewing.

0:12Hey, guys.

0:13And welcome back.

0:14So in the previous Nugget, we explored

0:16some solutions that we could use in order

0:19to monitor our CPU utilization, as well as

0:23our memory utilization.

0:25Now, in this Nugget right here, we're

0:27going to be looking at some additional tools

0:29that we can use with respect to monitoring.

0:32And we'll also focus on Disk I/O, i.e.

0:37Disk Input, Output monitoring.

0:39Now, the first command which I just want to touch upon

0:41is the uptime command.

0:43So if we go into the man page here, we can see here,

0:46this is actually going to allow us

0:47to see how long our system has been running.

0:52It's also going to tell us our load averages.

0:55And we're going to see three different values

0:58within our output with the uptime command,

1:01and these three different values are

1:03going to correlate to three different time spans.

1:07The first one is 1 minute, the second one

1:11is 5 minutes, and the last value you correlates

1:14through the system load average over the last 15 minutes.

1:18Now, we're actually talking about the system load averages

1:22right here, it says here this is the average number

1:27of processes that are either runnable

1:30or in uninterruptible state.

1:32Now ultimately what we're really talking about here

1:34is CPU usage.

1:36So a runnable state is when the CPU is being used

1:41or a process is waiting to use a CPU.

1:44Now, with respect to an uninterruptible state,

1:48this is when the process is waiting

1:50for input, output access.

1:52I.e.

1:53The hard--disk may be fully utilized, therefore,

1:57a particular process has to wait before it can access the disk

2:01in order to execute a particular task.

2:03Now, these load averages we are seeing

2:06within this uptime command relates to this type

2:09of information right here.

2:11Now, one thing to note about the output with respect

2:14to the uptime command--

2:16and let me just press Q to clear my screen.

2:19Now, when we're talking about our three different values

2:21of 1 minute, 5 minutes, and 15 minutes,

2:25we're actually going to see the values denoted

2:27a particular way.

2:29Now if we happen to see the value 0.5,

2:33what this tells us is that the CPU is ultimately half

2:38used, OK.

2:39Kind of makes sense, 0.5, half used.

2:42If the CPU is at the value of 1, that

2:46means that it's being fully utilized.

2:49Whereas if it's at 0.-- let's say maybe 0.25, then only

2:54a quarter of the CPU is being used.

2:57And, again, you can see this at different time intervals.

3:00Now, a few important points to note

3:02is that you may actually see a value such as 1.5.

3:06This is going to tell you that your CPU really is being taxed.

3:10Because whilst it may not make sense that how

3:13can 1.5 of the CPU being used if 1 is the entire CPU being used.

3:19What this is actually indicating to us

3:22is that some processes are having to wait.

3:25And as such, the CPU is in effect being overloaded.

3:29So if we get a value greater than 1,

3:32well, we may have problems to address.

3:35Now, one thing I just want to point out here

3:37is that the values I'm giving you

3:39relates to if you happen to have one CPU available.

3:43If you happen to have, let's say for example,

3:46four CPUs on your system, this actually changes the values.

3:51So think about this, if I happen to see

3:53the value from the uptime command, and the value is at 1.

3:57So 1.0, you may automatically think that the CPU

4:02is at full utilization.

4:05In actuality, what this means in the context

4:09of having four CPUs, only 25% of the CPU is being used.

4:14Ultimately, what you're doing is you're

4:16dividing this number here by the number of CPU.

4:19So it's 1 divided by 4 CPUs is actually 0.25.

4:25So only 25% is being used.

4:27Again, if we happen to see something like, say,

4:310.5 being used over 4 CPUs, the CPUs

4:36are actually not at half utilization.

4:38Instead it's going to be 0.5 divided by 4, instead.

4:43This is only going to come out as 12.5% utilization.

4:48Now if we actually happen to run this command here, uptime,

4:52we can see these values right here, 0.09, 0.07, and 0.01.

5:00Again, these are values over 1 minute, in this case

5:03here, 5 minutes in this case, and 15 minutes in this case.

5:08So the uptime command really is a valuable command

5:11that we can use to assess these types of values.

5:14It really does give us insight into our system.

5:17Now the next thing I want to talk to you

5:19about is relating to I/O Wait, i.e.

5:24Input, Outputs, Wait.

5:26So where am I going with this?

5:28So what I/O Wait actually is, it's

5:31ultimately a percentage of CPU actions

5:35that are awaiting for disk access.

5:38So like I happen to say, when we talk about I/O waiting,

5:42if you had, maybe say, a program, perhaps

5:45might be a browser, and your system is already

5:48using 100% of the disk, then your browser,

5:51which needs disk access in order to perform

5:54its actual task, this process, i.e. the process

5:58being used for the browser opening,

6:00this is actually going to go into a state of I/O Blocked.

6:05Input, output has been blocked.

6:07In real terms, what this means for us

6:10as the user is that the system is overloaded,

6:12at least in respect to the disk utilization.

6:16So when you try to open that browser,

6:18you double click that icon, the process may actually hang, i.e.

6:23the browser is just not going to pop up on your screen.

6:25You may be waiting a while because the process really

6:29does need to wait until the resource is free.

6:32And here's the thing, the actual percentage

6:35of CPU processes that have to wait,

6:39that is referred to as the I/O wait percentage.

6:44Now one of the ways we can interrogate

6:46this type of information comes from my favorite command,

6:49we just talked about it in the previous Nugget.

6:51That happens to be the top command.

6:54Let's try and check this out and see the information

6:56that we can actually derive.

6:58So what I'll do here is I'll just clear my screen right now,

7:01and I'll use that good old command, top.

7:04So check this out.

7:05There is a particular piece of information

7:09we can focus in relating to our I/O wait.

7:13If you can spot it here, it is this one right here, which is--

7:17in fact, let me get my drawing pen because it's not

7:19showing quite so well.

7:20This one right here, wa.

7:23So this number is ultimately a percentage,

7:26and right now it is at 0.0.

7:28That is good, meaning that we don't have any CPU processes

7:33actually waiting.

7:34So we know we actually have a tool downloaded

7:37on our system which we can use to stress our system,

7:42to simulate this type of problem.

7:44So that commands is going to be the stress command.

7:47So what I will do actually is I'll go to my options,

7:51and I will open a new terminal.

7:53So we have a new terminal right here,

7:55what I'm going to do is use the stress command.

7:58If I happen to say man stress, we

8:00can see we have the option of -d or --hdd.

8:05This is going to spawn the number of workers

8:08spearing on rates.

8:09Basically this is going to allow us to simulate

8:12a lot of disk utilization.

8:15So what I'll do here then is I'll

8:17press Q to quite, clear the screen, and obviously

8:19stress --hdd.

8:22And I'll choose the number 14, one hard-drive process.

8:26If I hit enter-- in fact, before I actually do this,

8:29let me minimize this-- keep an eye on this value here,

8:33the wa value, OK.

8:36So if I hit Enter now, that is going

8:39to actually spin up a process.

8:41And you can automatically see, or rather, instantly

8:43see, should I say, the waiting value,

8:46the percentages are actually beginning to speak.

8:48That is due to the fact that we're actually stressing

8:51our hard-disk, so to speak.

8:53We are really using that resource.

8:55And as such, if I actually try to go

8:58and maybe say open a browser, click here,

9:00it may take a little bit of time to actually open up.

9:03It's not going to be quite so snappy.

9:05That is because a lot of my processes

9:08are actually having to wait right now.

9:10They're not getting access to the disk,

9:12therefore they can't operate as they ordinarily would.

9:15So right now we can see here we're

9:17still waiting on this browser, and waiting

9:19and waiting and waiting.

9:20Sadly, our browser is kind of at the back of the queue,

9:24it can't access this resource as being hogged.

9:27So if I actually just stop the stress by pressing Control-C,

9:31now straight away the browser is going

9:34to load up because we are no longer overloading

9:36the hard-disk.

9:37Therefore, the process could access

9:39the resource, and lo and behold, we can open up the browser.

9:43But, again, think about what we're actually seeing here.

9:45In this case, we had a problem whereby our browser

9:48was very, very slow to load.

9:50If we just happened to be looking at the top commands,

9:53we would have seen that the I/O wait

9:55value was around 80% or 90%.

9:59And we could actually tell or infer

10:01that the hard-disk was being particularly tax.

10:04So straight away with our troubleshooting mind on,

10:07we would have quite a good idea of where

10:09to go to solve that problem, or the type

10:12of action we would need to take in order to solve that problem.

10:16Now, honestly, as helpful as top is

10:20and as very helpful, with respect to input, output,

10:22it is a little bit limited.

10:25So what I want to do is to actually show you

10:28some additional tools that we can use.

10:30So if I press QT quits--

10:32I'll just close this terminal for now,

10:34and I'll also close my browser for now.

10:37I'll clear the screen.

10:38And what I'm going to do is install something called iotop.

10:42This is like top but up for input, output.

10:44So I'll say sudo apt install io--

10:48if we can get it-- iotop.

10:49Hit Enter, type in the password, this

10:52is going to begin installation.

10:54Just give a little minute or so.

10:55Now if I try to run the command iotop, watch what happens,

10:59operation not permitted.

11:01You can probably guess where I'm going with this.

11:03The way to actually use this command

11:06requires superuser privileges.

11:08So I'll clear the screen, and I'll say sudo iotop,

11:11and I will hit Enter.

11:13So what we're seeing is top like information.

11:15But we are really focusing on this right

11:18here, the IO utilization, and what

11:20processes happen to be using a lot of disk resources.

11:24So what I can actually do here is

11:26I can go and open up a new terminal.

11:28And we'll do a similar thing to what

11:29we saw before, we'll use the stress command to actually put

11:33some stress on the disk.

11:36So I'll say stress --hdd, and I'll just say 1.

11:40And now the stress command should actually

11:42be putting some pretty hard work on the IO.

11:45But look at this, we're actually getting

11:47to see the input, output, and getting

11:49to see what is taking up a lot of that.

11:51So we can see the stress hdd1 taking up

11:54quite a lot of the resources.

11:55And, again, if we go back here, I just

11:58press Control-C to stop this.

12:00Instantly that's going to come back down,

12:02and the iotop command shows us that.

12:05So a really valuable tool right here, as opposed

12:08to just using the regular top command.

12:11Now, what I will do here is I'll press Q

12:13to quite this, clear the screen, I'll now say sudo apt install.

12:18And the package I want to install is called sysstat.

12:21So if I hit Enter here, this is going

12:23to install the sysstat package.

12:25Just give it a little minute or so.

12:27So now I should have access to a command called iostart.

12:31If I go into the page here, this is

12:33going to report CPU statistics for input, output.

12:37So this is going to give us, again,

12:39just like with iotop, IO related information and statistics.

12:44Now this is, in my opinion, not quite as good as iotop.

12:48But, nevertheless, the examination

12:50does mention iostart within the exam objectives.

12:54So it does actually make sense that we are actually

12:57familiar with it.

12:58So I'll clear the screen, and I'll just say iostat.

13:00And what this is ultimately doing is giving us a snapshot.

13:03So we're getting to see a lot of information

13:05about data being written to disk and read from disk.

13:08This is ultimately what is taxing the particular disk,

13:11and we can see the iowait percentage

13:14right here, which in this case is 1.77, quite low.

13:18But like I say, this as really just a snapshot.

13:20Whereas if you recall, the iotop command

13:23was giving us real time updates, and much more expansive.

13:28So I can clear the screen.

13:29Another command we want to be able to be

13:31familiar with, again, not quite as good as just using iotop,

13:35but the exam does require we are familiar.

13:38If we go in to the man page for the sar command,

13:41this is going to allow you to collect, report, or save

13:44system activity information.

13:47So you can scroll right through this man page,

13:50and you will see plenty of options and plenty descriptions

13:53of what this command can actually do.

13:55That is quite a lot here.

13:56For now I'll press Q. What I'm going to do

13:59is I'm just going to say sar 1, and then 5.

14:03So what this command is going to do,

14:05it's going to allow us to take five different data

14:09points, one second apart.

14:11So this is quite good to check how things progress over a time

14:14frame, in this case here, 5 seconds,

14:17because we have five recordings every single second.

14:21So it's a little bit better of a snapshot than iostart gives us.

14:25We have more data points, for example,

14:27but it's not quite, like I say, as

14:29good as iotop, giving us that streaming information

14:33and continually updating.

14:34But, nevertheless, let's hit Enter and see how we go.

14:37We're going to get five pieces of information.

14:39So we start at 27 seconds, we get our first data point at 28.

14:43So that's number 1, then 2, 3, 4, and 5.

14:48So 5 data points one second apart,

14:50we can see the percentage.

14:52So we get statistics over a whole bunch

14:54of different types of information, such as system

14:57or idle.

14:58But really the one we are focusing on right now

15:01is iowait.

15:02Now as it transpires, we are not actually taxing the io,

15:06so the values right here all happen to be 0.

15:10But as you can expect, we could, like we say,

15:12actually stress the --hdd once again,

15:15and we can rerun this command by hitting Enter.

15:18We get five different values right here.

15:21And at the very bottom, we get the average

15:24of those five values.

15:27And, again, we could change these values

15:28to be maybe 2 and 3, meaning we will get three data

15:33points two seconds apart.

15:35If we hit Enter, we can see here the actual times.

15:38The first one, on 00, the next one on 02, the next one on 04.

15:43And at the bottom, we get the average of those three

15:46over that time period.

15:48So, again, another really cool command to be aware of.

15:51Now what I'll do here is I'll just

15:52press Control-C to stop this stress test.

15:55In fact, let me just close this terminal right here,

15:58and I'll clear the screen.

15:59Now, another command I want to show you is one called LSOF.

16:04This is going to allow us to last our open file.

16:06That should be one you will recall from the LPIC-1

16:09examination, no doubt.

16:11And like I say, it's going to allow

16:12us to track the open files on our system,

16:15being all the open files.

16:17Now, if we just happen to use this command on its own LSOF,

16:21we're going to be bombarded with an absolute megaton

16:25of information, just far too much to actually read.

16:28But we can do things such as filtering

16:31to make this information much more palatable.

16:34So what I'll do actually-- let me just quickly open terminal,

16:38and we'll just rerun that stress command.

16:40What I can actually do here is I can run this command, OK,

16:44that would just show me absolutely everything.

16:46But what if I happen to grep for stress itself?

16:50Now, we can actually see here what files are being

16:53opened by the stress program.

16:56So if we happen to note a particular process

16:59or a particular command is causing a lot of IO issues,

17:03we can actually correlate what actual files are

17:06being opened or being used by that particular command

17:10or process.

17:11So this just gives us more granular information

17:13that we can use to interrogate what is exactly going on.

17:17Now, one point I should actually add--

17:19and, in fact, again, let me just stop this right here.

17:22One point I should add is that with respect to Disk I/O

17:26problems, one of the ways, of course,

17:28we can stop such a thing is by not

17:31running so many processes that are actually

17:34requiring that resource.

17:35But as you know in the world of enterprise computing,

17:38it's not quite so easy just to make sure

17:41that everyone is giving up processes

17:43that they may actually need in case

17:46we may need better solutions altogether.

17:48Now, one of those solutions may be

17:51that the rate and reading of the hard-disk

17:54is kind of slow and inefficient.

17:56And this can actually cause things to be bogged down.

17:59Because the longer it takes a particular process

18:02to write and to read, the longer it's

18:04going to engage that resource, i.e.

18:07the more of a backlog on that resource is going to happen.

18:10One of the best ways you can actually

18:13ease this type of problem is by changing things

18:16from a hard-disk to an SSD, a solid state drive.

18:20This has got far, far, far, faster and more efficient read

18:24and write operations.

18:25And this will actually alleviate a lot of the problems

18:29that you are seeing if you do happen to be encountering

18:31a lot of iowait on your system.

18:34Okey doke, so that is all for our introduction

18:37into being able to monitor your Disk I/O utilization.

18:41Like I say, we saw a handful of different commands, some

18:44more useful than others.

18:45As we know in these exams, very often you still

18:48have to be able to be familiar with perhaps some less

18:52efficient and older commands, such as what

18:55we saw with respect to the iostart command, for example.

19:00So, really, what I'm seeing is don't just

19:02skip over the commands, it may not be so obviously useful.

19:05Definitely dig in and be familiar with them,

19:08at least just for the purposes of the examination.

19:11Okey doke, so that is all for monitoring disk input, output.

19:14The next thing we have to tackle is network input, output,

19:17and how we can actually monitor that too.

19:20And, well, that is what we're going to be

19:22doing in the very next Nugget.

19:24So I hope this has been informative for you,

19:26and I'd like to thank you for viewing.

0:12Hey, guys.

0:13And welcome back.

0:14Now what I want to talk to you about

0:16is how we can actually use tools to monitor our Network I/O.

0:22One of the primary tools that we want

0:24to use in order to monitor such a thing is the Netstat command.

0:29So let's go and check this out then.

0:32So what I will do is I will open up a terminal right here.

0:35OK, so if I do it man netstat, we'll

0:37actually see the manual page.

0:39It's going to allow us to print network connections, as well

0:42as things such as routing tables, interface statistics,

0:45and others.

0:46So this command really is very, very useful.

0:50Now, one of the best switches to use

0:53when using the netstat command is the -s option.

0:57This is going to give you a nice summary of network statistics

1:01based on a particular protocol.

1:03So what I will do here is I'll say netstat-s.

1:06And check this out, you can actually

1:08see per protocol our network statistics.

1:12We can see our IP packets right here, the amount received,

1:15the amount forwarding, ICMP, which

1:18we use for ping messages, TCP, UDP, so on and so forth.

1:22So think about this.

1:23If you happen to be maybe experiencing

1:26some poor connections, with respect to Video, what

1:29you may know is that Video, just like Voice,

1:32is going to use UDP.

1:34So straight away by using netstat-s,

1:38we can target return and have a look at this then.

1:40We can actually see are our UDP packets being received,

1:44are they being sent, are we receiving

1:46a lot of packet errors.

1:48So we've been able to use this command

1:50and neatly get a summary of the particular statistics,

1:54such as the amount of packets received

1:56and the amount of errors received or sent,

1:58and been able to pass out this information per protocol.

2:02This can be really invaluable troubleshooting command.

2:05And we also have additional switches, of course,

2:08that we can use.

2:09Let's clear the screen.

2:10Say, for example, we can say netstat -l.

2:14So we scroll on up, we can see all of these sockets

2:17and the state of Listen.

2:19So if we happen to be having a server that should be accepting

2:23connections, one of the ways we could check that it actually

2:26is listening for connections, and there

2:28that may be we have connectivity issues, is to log in

2:31and to check that that server does in fact--

2:34or rather, is in fact, should I say, listening

2:37for a particular connection.

2:39So this filter right here, the netstat -l,

2:41this is going to give us that information.

2:44But, again, we can go even further yet again.

2:47As we can see here, we can see we are

2:49listening for UDP connections.

2:51If I wanted to filter just for let's say,

2:53maybe, TCP connections, I could actually change this filter.

2:57I could actually say -lt to see TCP connections that

3:02are listening.

3:02So if I hit Enter now, notice on the left hand side,

3:06the protocol is only TCP-based connections, as opposed to UDP.

3:11And, of course, we can actually invert that logic by saying lu,

3:15and we just see the UDP connections

3:18that happen to be listening.

3:20Now, one cool thing we can actually do also

3:22is to say netstat -c.

3:24What this is going to do, this is

3:26going to allow us to get real time information.

3:29And we'll actually see, as we saw there

3:31with the stream changing, continual updates

3:34of the netstat display.

3:36So when you want to actively be seeing

3:38what is happening right now, this

3:40is going to give you a window into what is actually

3:42traversing your networks.

3:43So a really, really valuable tool.

3:46Now, if you happen to experience some type of network

3:49connectivity issue, maybe you can't reach a particular server

3:52or you can't reach the internet, one

3:54of the commands you can do it is say netstat -r.

3:57Which is actually going to allow you to see your routing table.

4:02So if you happen to have some type of error configured,

4:04we don't have a gateway so that we actually

4:06can't reach external networks such as the internet,

4:10this netstat -r command will give you

4:13an insight into that by giving you the routing table.

4:15So you can actually check that these entries do exist and are

4:19correctly configured.

4:20So the Netstat command really is one of the main commands

4:23that I would use in order to monitor and troubleshoot

4:27any network issues.

4:28But as always, I would recommend you go through the main page

4:32and explore the different options that this command has

4:34available to us, and really familiarize yourself with them.

4:38Okey doke, so I hope this has been informative for you,

4:40and I'd like to thank you for viewing.

0:12Hey, guys.

0:13And welcome back.

0:14So now what I want to briefly talk to you about

0:17is the concept of being able to predict future resource needs.

0:21Now, to begin one of the first tools

0:23I want to talk to you about is something called collectd.

0:27Now, what this is is ultimately a daemon.

0:31And this daemon has the purpose of collecting system

0:36statistics.

0:37Now, the way collectd actually does

0:39this is that it uses something called plug-ins.

0:42And these plug-ins actually determine what

0:45collectd will actually collect.

0:48Now one thing to note about the collectd daemon

0:52is that there are some limitations here.

0:53This is not an all purpose tool.

0:56For example, the purpose is really limited to collecting.

1:00And it would help if I could spell that correctly,

1:02collecting data.

1:04That means that it's not going to display

1:07the data for you and in a graphical form or anything

1:10like that, or analyze the data.

1:13Instead, collectd is just going to, like I say,

1:15collect that data, and you can then

1:17feed that data, if you so wish, into another program.

1:20Now collectd typically is not installed by default

1:24on most Linux systems.

1:26So we do actually have to run through the installation.

1:29So what I will see here on my Xubuntu machine,

1:32I can just use the app package manager and install collectd.

1:36Just type in my password.

1:38And hit Y, and Enter.

1:41OK, perfect.

1:42So now that we have collectd installed--

1:44let me just clear the screen--

1:45one of the things that we want to be

1:47aware of, for the purposes of the examination,

1:50is where we can actually configure the collectd daemon.

1:54Now we are going to be able to do this

1:56by modifying a file in a particular location.

1:59So let me show you what this location is.

2:01Is if I go into the etc directory,

2:03and then go into collectd, If I do an ls,

2:07we should now see this collectd.conf file.

2:11So try to remember that path, etc collectd, collectd.conf.

2:15If I go in and modify this, let's say nano,

2:17for example, and hit Enter, this as the main configuration file.

2:22Now within here, we can do many cool things.

2:26We can actually determine how often

2:28we should be able to collect particular data.

2:31Say, for example, here we can see we have this value here,

2:34interval 10.

2:36This means that every 10 seconds we

2:39will have a new query collecting more data.

2:43Now this is the default value.

2:44If we want to happen to change this,

2:46we would uncomment this configuration

2:49by removing the what is now called the hashtag.

2:52And we can see the color change.

2:53And we could change this interval

2:55to maybe say every 5 seconds or every 15 seconds,

2:59however you choose.

3:01But for now I'm actually not too balled with this,

3:04so I'll just comment this out, so it's inactive.

3:06And we'll still have the default of 10.

3:09Now if we scroll on down, another really important thing

3:12to notice is this Plugin section right here.

3:15This is going to allow you to specify

3:18which features we actually want to collect information on.

3:21Now if you happen to scroll down,

3:23you'll notice that most of these plugins are commented out.

3:27The ones which are actually on by default

3:29are the battery, the CPU, the disk, entropy, interfaces,

3:34memory, so on and so forth.

3:35But the vast majority, like I say, are commented out.

3:38But if we wanted to decide that, hey, we actually

3:42want to collect information on our network,

3:44we could go to the plugin network,

3:46and we could just remove the hashtag to uncomment out

3:50this configuration.

3:51And this would allows us to activate this configuration.

3:54Say, for example, I could just save this.

3:57Oh, and notice, I should have pointed this out.

3:59This is a global configuration file within the etc directory,

4:03therefore I'm going to have to use superuser privileges.

4:07So I would say sudo nano collectd.conf.

4:11And scroll on down.

4:12Now I could remove and save the changes.

4:16And in order for these changes to take effect,

4:19we would actually have to reload the service.

4:22So could say sudo systemctl restart collectd.service.

4:28If I hit Enter, that should now reload the service.

4:32And the new configurations, i.e.

4:34the changes we have just made, will take effect.

4:37And we can actually use collectd to monitor our network

4:42statistics.

4:42So, really, for the purposes of the examination,

4:45understand what collectd does and what it does not do.

4:48And really try to remember the location of the configuration

4:52file, and understand that we have these concepts of plugins.

4:56And if we want to activate a particular plugin so

4:59that we can measure a particular resource, such as a CPU

5:03or what is going on in on network,

5:05we can just simply uncomment that plugin

5:08so that it will be active after we reload the configuration

5:11file.

5:12Now another thing I want to just briefly touch upon

5:15as the subject of monitoring solutions.

5:18There are some solutions we have to have an awareness of,

5:21at least at a high level for the purposes of the examination.

5:25But like I say, this is just a high level topic.

5:29Meaning that you don't have to really concern yourself

5:32with how to configure these tools specifically,

5:35just more have an awareness of what they actually do.

5:39So the first monitoring solution I want to point out

5:42is one called Nagios or Nagios.

5:45I've heard it said both ways, I'm not quite sure

5:47the correct way.

5:48Now Nagios is ultimately a family of projects.

5:52So we could have a Nagios log server or a Nagios network

5:56analyzer or a Nagios incident manager, so on and so forth.

6:01Now the cool thing about Nagios is

6:03that it can provide a plugin to interact with collectd.

6:07We can actually use collectd-nagios,

6:11and that can actually be used to query the collectd daemon

6:15for information.

6:16So the information that is collected by collectd,

6:20we can feed it into Nagios, and use the Nagios network analyzer

6:25to analyze the network traffic or what statistics,

6:29should I say, that we have gathered using collectd.

6:32Another solution we have is one called MRTG.

6:35This is the Multi Router Traffic Grapher.

6:39And this is really used to monitor network traffic loads.

6:43It basically reads information from your router and your logs.

6:47And you can actually take this information and graph it.

6:50So this can provide a nice visual interpretation

6:53of that information to make it much more digestible

6:56for you as a system administrator.

6:59Another one we want to be aware of is one called cacti.

7:02This too, like MRTG, also focuses on network monitoring.

7:08Now this tool is basically a front end interface

7:12for something called RRD tool.

7:16This stands for the Round Robin Database tool.

7:20So simply put, information collected by the Round Robin

7:23Database tool can be ingested by cacti.

7:27And cacti will provide to you a nice, easy interface so

7:31that you can display information or graph particular data

7:34received via this tool.

7:36Now the key thing to note about all of these types of tools

7:40is that we can take this information, monitor our data,

7:43and we can use that information to observe and use

7:48for capacity planning.

7:50Say, for example, you happen to note

7:51on your graph you're having a very large and prolonged spike

7:55in, maybe say, network traffic.

7:57You may think, hey, we're actually

7:59getting much more network use, maybe it's now

8:02time to increase our bandwidth.

8:04Because if we keep following this trajectory of network

8:08use just getting higher and higher and higher,

8:11suddenly we're going to get a bottleneck,

8:13and the users are going to suffer.

8:16And we definitely do not want that.

8:18So having this type of monitoring information

8:20allows us to plan ahead.

8:22And we can see trends on the horizon

8:24as they are approaching.

8:25So really these tools altogether is

8:27going to help you identify resource exhaustion,

8:31and allow you to predict growth, and even diagnose

8:35particular problems.

8:37If you happen to see a spike in some particular resource, CPU,

8:41maybe it is network traffic, you can then

8:44use that information to potentially identify

8:47a particular problem on your network.

8:49So the ability to collect data, to be

8:51able to feed that data into a tool which can graph that data

8:56or analyze that data, this is obviously

8:58very valuable for anyone administering and managing

9:03the systems on our network.

9:05Now one thing to say about these tools

9:07just as a disclaimer is that there is no way

9:10you can get 100% accuracy with these tools.

9:13You may be able to graph a particular trend,

9:16but there is no way for you to know that that trend is

9:19going to continue.

9:19Say, for example, you've got a nice graph which

9:21is showing a very steady rate of growth

9:25over a particular period of time,

9:27there is no way for you to know that suddenly you're

9:30going to encounter an abnormal spike, OK.

9:33So just be aware that these tools definitely

9:36have a use, they definitely provide valuable information,

9:39but there are limitations to their abilities.

9:42Okey doke, so I hope this has been informative for you,

9:44and I'd like to thank you for viewing.