Blog

SCOM: Configure a monitor recovery task for a healthy state

During a recent project a client had a small request to create a monitor and run a command when a device was not accessible anymore. Easy right! But (yep there’s always a but) they wanted to run a command when the monitor was returning back to a healthy state to restart a service when the device came back online… Hmmm and all in 1 monitor.

So the conditions were as follows:

Monitor:

  • Action: Run a PowerShell based monitor to test the connection with the device
  • BAD: Device is down => Run recovery task to remediate
  • GOOD: Device is up again => Run recovery task to restart service

(note: Always do this small matrix of a monitor design to exactly know what the customer wants)

I don’t have the device to simulate but came up with a small example in my lab to show you how to get this working with just 1 monitor. The situation in my lab is very simple. I want to turn on my desk lighting when my pc is on (and I’m working) and turn it off when my pc is not online.

My conditions:

Monitor:

  • Action: Run Powershell based monitor to test the connection and pass the result to SCOM
  • BAD: PC is offline: => turn off my desk lighting
  • GOOD: PC is online:=> turn on my desk lighting

So first things first we need to test the connection to see whether my pc is running. To check this I’m using this small script:

[xml]

param ([string]$target)
$API = New-Object -ComObject "MOM.ScriptAPI"
$PropertyBag = $API.CreatePropertyBag()

$value = Test-connection $target -quiet

$PropertyBag.AddValue("status", $value)

$PropertyBag
$API.Return($propertybag)

[/xml]

So I’m testing the connection and sending the response to SCOM. The  PowerShell “Test-Connection $target –quiet” command will just return true or false as a result whether the target is accessible or not

Creating the Monitor with Silect MP Author

The creation of this monitor consists of 2 parts:

  • Defining the class where the monitor will be targeted to and therefore the machine which will test the connection to the desktop
  • Passing the status from the machine to SCOM and take action by using a monitor

Defining a class:

To properly target this monitor we need to create a class in SCOM which identifies the servers that need to test the connection. In this case I’ve added a reg key to all servers who need to ping the desktop so I’m starting a Registry Target to create my class:

printscreen-0254printscreen-0255

I fill in a server that has the key already in there to make it much easier to browse the registry instead of typing it in with an increased margin for errors.

printscreen-0256

Select the Registry key you want to look for

printscreen-0257

In my case I’ve added a key under HKEY_LOCAL_MACHINE\Software\pingtestwatchernode

printscreen-0258

Select the key and press add and ok

printscreen-0259

Identify your registry target:

printscreen-0260

Identify your discovery for the target

printscreen-0261

In my case I just check whether the key is there. No check on the content.

printscreen-0263

The discovery will run once a day.

printscreen-0264

Review everything and press finish

printscreen-0265

At this point our class is ready to be targeted with our script monitor.

Next up is to create the monitor:

Create a new script monitor:

printscreen-0266

Browse to the PowerShell script and fill in the parameters. In this case I have 1 parameter which is “target” and will hold the IP of the desktop.

printscreen-0267

Define the conditions:

Healthy condition is when the status is true and type boolean

printscreen-0268

Critical condition is when the status is False

printscreen-0269

Note: I’m using a “boolean” Type

Configure the script and select the target you have created earlier on and the availability parent monitor

printscreen-0270

Identify your script based monitor

printscreen-0271

Specify a periodic: run every 2 minutes

printscreen-0272

No alert generation necessary.

printscreen-0273

Review all the parameters and create the script based monitor.

printscreen-0274

Load the management pack in your environment and locate the monitor:

printscreen-0278

Check the properties => recovery tasks and create 2 recovery tasks for the Health state “critical”.

Note that the screenshot below already shows the correct healthy state after config of the mp.

printscreen-0279

Export the managment pack and open it in an editor and locate the “recoveries” section to find your recovery tasks we just created:

printscreen-0280

scroll to the right and locate the “ExecuteOnState” parameter and change the one you want to run when the monitor goes back to healthy from “Error” to “Success”

Save the management pack and reload it in your environment.

printscreen-0281

So all we need to do is test it…

My pc is on: IT-Rambo has his cool backlight:

20141130_230930098_iOS

My pc is off and the light is automatically turned off…

20141130_230904267_iOS

Final Note: If you use this method you need to make sure to NOT save the recovery tasks in the console anymore otherwise the different settings we just changed in our management pack will be again overwritten as SCOM can’t natively configure a recovery task for a healthy state.

You can use this basically for anything where you want to run 2 conditions on the same monitor or even 3 if you have a 3 state monitor.

SCOM: Creating a PowerShell script monitor with Silect MPAuthor

Sometimes it’s necessary to create a monitor to monitor something which is not included in the standard management packs. Unfortunately it’s not possible in SCOM  to use PowerShell to crerate a script monitor in the scom console. Although it’s not a good idea to start authoring in the operations console it sometimes can be a quick and easy way to create a monitor.

Recently Silect Sofftware released a free version of MPAuthor to create your management packs. I’m using this to create my script monitors to collect and monitor the data which I use in my monitoring my home series: http://scug.be/dieter/2014/02/19/monitor-your-home-with-scom/

Download the tool here: http://www.silect.com/mp-author

Below is an example of how I monitor the target temperature set on my Nest Thermostat.

So open the tool and create a new management pack => Create New Script Monitor…

SNAG-0243

Name the script (if you have the script somewhere as a PS1 file it will load the script body automatically.

SNAG-0246

This is the script I’m using:

[xml]

param([int]$maxtarget)
[void][system.reflection.Assembly]::LoadFrom(“C:\Program Files (x86)\MySQL\MySQL Connector Net 6.8.3\Assemblies\v2.0\MySQL.Data.dll”)

#Create a variable to hold the connection:

$myconnection = New-Object MySql.Data.MySqlClient.MySqlConnection

#Set the connection string:

$myconnection.ConnectionString = "Fill in the connection string here"

#Call the Connection object’s Open() method:

$myconnection.Open()

$API = New-Object -ComObject "MOM.ScriptAPI"
$PropertyBag = $API.CreatePropertyBag()

#uncomment this to print connection properties to the console
#echo $myconnection

#The dataset must be created before it can be used in the script:
$dataSet = New-Object System.Data.DataSet

$command = $myconnection.CreateCommand()
$command.CommandText = "SELECT target FROM data ORDER BY timestamp DESC LIMIT 1";
$reader = $command.ExecuteReader()
#echo $reader
#The data reader will now contain the results from the database query.

#Processing the Contents of a Data Reader
#The contents of a data reader is processes row by row:

while ($reader.Read()) {
#And then field by field:
for ($i= 0; $i -lt $reader.FieldCount; $i++) {
$value = $reader.GetValue($i) -as [int]
}
}
#echo $value
$myconnection.Close()
#$value = $value -replace ",", "."

if($value -gt $maxtarget)
{
$PropertyBag.addValue("State","ERROR")
$PropertyBag.addvalue("Desription","Target temperature currently set to " + $value + ": is higher than the maximum target temp " + $maxtarget)
}
else
{
$PropertyBag.addValue("State","OK")
$PropertyBag.addvalue("Desription","Target temperature currently set to " + $value + ": is lower than the maximum target temp " + $maxtarget)
}

$PropertyBag

[/xml]

Note that you need to pass the parameters through to SCOM via the propertybags. I also am a fan of doing the logic in the script itself as shown above to avoid any logic in SCOM afterwards. It’s far more easy to do the comparison in the PowerShell script. In this case I’m setting State to either ERROR or OK. This also avoids the format conflict of the output whether it’s a string or an integer.

I’m setting the maxtarget parameter to 19

SNAG-0245

Next you need to create the conditions for the monitor states:

SNAG-0247

As I’m only using a 2 state monitor I’m deleting the OverWarning state and only using UnderWarning (= Healthy state) and OverError (= Error state).

SNAG-0248

For the Healthy state I’m detecting the “State” property value as OK (note that I’m defining the Type as a String as the state is just plain text)

SNAG-0249

For the Error state I’m detecting the “State” property value as ERROR

SNAG-0250

Now we need to target the monitor. In my case it’s the watcher node target I’ve created earlier on.

 

SNAG-0251

Naming and enabling the rule

SNAG-0252

Set the schedule how many time to check the status of the max temp

SNAG-0253

Speciffy the alert that needs to be raised if any:

SNAG-0255

And create.

SNAG-0256

Now save the management pack and test it in your environment.

Enough talk, let’s build
Something together.