Disk Space Usage Trend Analysis – Part 2

green

Following from Part 1 in this series, this section focuses on using the previously collected disk usage data and applies a ‘line of best fit’ linear trend-line (which will give us it’s “average” growth in GB/day).

There is a decent amount of maths going on here, to manually determine a linear trend in the format of y = mx + c. You don’t really need to understand how it works, but I’ll leave a link at the bottom for anyone interested in pursuing the methods to this madness.

To use the script below, you just need to ensure the $disklogdirectory is pointing to the same folder that you previously saved the disk usage logs to (c:\scripts\disklog from Part 1).

By default, this script looks at the last 30 days worth of data to determine the disk usage trend.  This is set by the $cutoffdate variable – and you can easily change the -30 to a larger negative number if you want to look for more long-term trends.

Also note, that there is a place-holder towards the very bottom of the script for the graph generation.  This is where the script from Part 3 will be inserted; though you can use just the Part 1 and Part 2 scripts without implementing the scripts from the next part.

 

 

When you run the above, you should see results similar what’s below. A message indicating which server is being processed, followed by the details of each disk (drive letter, slope, and C value).

The slope in this case is actually the average change per day (in GB).  So in this example, the D: drive on SQL01 is growing by an average of .31 GB per day – or around 9.4GB every month.

disktrend3

The C value is used in Part 3 when we plot the values and trend-line in a graph.

If you were just using Part 1 and Part 2 of this series, you could easily wrap the output above inside an If statement to only show you the disks where growth rate was greater than a certain value.

 

References
http://en.wikipedia.org/wiki/Linear_regression

green

4 Comments Disk Space Usage Trend Analysis – Part 2

  1. Marc

    Hi Kamal,
    Great script, many thnx for that!

    I only get a failure in the part 2 script.

    Attempted to divide by zero.
    At D:\scripts\DPP\WFF\StorageTrendAnalyzing\StorageTrendAnalyzing_2.ps1:86 char:13
    + $slope = ($nsumxy – $sumxy) / ($nsumsquaredx – $sumxsquared);
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], RuntimeException
    + FullyQualifiedErrorId : RuntimeException

    I only lowerd the $cutoffDate to -6 days, further everything is the same as youre script. The txt file of the server collected more then 6 day’s of data.

    Can you please help me. I realy would like to use this script for analyzing.

    regards

    Reply
  2. Kamal

    Hi Marc,

    hard to say without seeing your files with the hard disk usage data.
    All of the variables are initialized to zero – so, if you get a “divide by zero” it means the variables are still the default values.

    So, without seeing your exact setup, it might be that your data files are not in the correct directory ($filename variable) – OR – your data files don’t contain enough data for you $cutoffdays selected – OR – your date format settings are not consistent and it’s comparing DD/MM to MM/DD format (or vice-versa).

    Reply
  3. Karl

    Thanks for this! I used your scripts as a template for monitoring disk usage on a few servers.

    I implemented a few modifications because I wanted a much more granular monitoring for our DB server. Therefore I’m also tracking the time in my CSV files and I’ve scheduled a check every 30 minutes.

    When calculating the trend I’m using the actual timestamp.
    Your script ignores the date saved in the CSV file when calculating the trend. It just relies on consistent logging of one line a day because $sample is used as the x-value . This is a bit misleading I think and could introduce a slight error in the calculated trend if logging of one line each day was skipped or halted for a time.

    Because I’m generating much more data this way I’ve also implemented a third script that regularly cleans the CSV files. I could write down some of my modifications if anyone is interested.

    Thanks again and best regards!

    Reply
    1. Kamal

      That’s great Karl!
      Looking over it again (it’s been a while), I can definitely see some room for improvement and extension – so it’s great you’ve been able to use and build on this.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *