Datasource plugin (data points)

We’ve already created a data source plugin that creates Zenoss events for weather alerts. Now we want to use the Weather Underground Conditions API to monitor current weather observations for each station. The purpose of this is to illustrate that these Python data source plugins can also be used to collect datapoints.

Create conditions datasource plugin

Follow these steps to create the Conditions data source plugin:

Add the following contents to the end of $ZP_DIR/dsplugins.py.

class Conditions(PythonDataSourcePlugin):

    """National Weather Service conditions data source plugin."""

    @classmethod
    def config_key(cls, datasource, context):
        return (
            context.device().id,
            datasource.getCycleTime(context),
            context.id,
            'nws-conditions',
            )

    @classmethod
    def params(cls, datasource, context):
        return {
            'station_id': context.id,
            'station_name': context.title,
            }

    @inlineCallbacks
    def collect(self, config):
        data = self.new_data()

        for datasource in config.datasources:
            try:
                response = yield getPage(
                    'https://api.weather.gov/stations/{station_id}/observations/latest'
                    .format(
                        station_id=datasource.params['station_id']))

                response = json.loads(response)
            except Exception:
                LOG.exception(
                    "%s: failed to get conditions data for %s",
                    config.id,
                    datasource.params.get('station_name'))
                continue

            current_observation = response.get('properties')
            for datapoint_id in (x.id for x in datasource.points):
                if datapoint_id not in current_observation:
                    continue

                try:
                    value = current_observation[datapoint_id]['value']
                    if isinstance(value, basestring):
                        value = value.strip(' %')
                    value = float(value)
                except (TypeError, ValueError):
                    # Sometimes values are NA or not available.
                    continue

                dpname = '_'.join((datasource.datasource, datapoint_id))
                data['values'][datasource.component][dpname] = (value, 'N')

        returnValue(data)

Most of the Conditions plugin is almost identical to the Alerts plugin so I won’t repeat what can be read back in that section. The main difference starts at the following line of the collect method:

current_observation = response.get('properties')

It grabs the properties data from the response then iterates over every datapoint configured on the datasource. This is a nice approach because it allows for some user-flexibility in what datapoints are captured from the Observations API. If the API made temp_c and temp_f available, we could choose to collect temp_c just by adding a datapoint by that name.

The following line is the most important in terms of explaining how to have your plugin return datapoint values.

data['values'][datasource.component][dpname] = (value, 'N')

We just stick (value, 'N') into the component’s datapoint dictionary. The 'N' is the timestamp at which the value occurred. If you know the time it should be specified as the integer UNIX timestamp. Use 'N' if you don’t know. This will use the current time.

Restart Zenoss.

After adding a new datasource plugin you must restart Zenoss. While developing it’s enough to just restart zenhub with the following command.
```
serviced service restart zenhub
```
That’s it. The datasource plugin has been created. Now we just need to do some Zenoss configuration to allow us to use it.

Add conditions to monitoring template

To use this new plugin we’ll add a new datasource and corresponding graphs to the existing Station monitoring template defined in zenpack.yaml.

Follow these steps to update the monitoring template:

Update $ZP_DIR/zenpack.yaml to add the conditions entry within the existing datasources section.

zenpack.yaml

device_classes:
  /NWS:
    templates:
      Station:
        description: Weather monitoring using the NWS API.
        targetPythonClass: ZenPacks.training.NWS.NwsStation

        datasources:
          alerts:
            type: Python
            plugin_classname: ZenPacks.training.NWS.dsplugins.Alerts
            cycletime: "600"

          conditions:
            type: Python
            plugin_classname: ZenPacks.training.NWS.dsplugins.Conditions
            cycletime: "600"

            datapoints:
              temperature: GAUGE
              dewpoint: GAUGE
              windSpeed: GAUGE
              windGust: GAUGE
              barometricPressure: GAUGE
              visibility: GAUGE
              relativeHumidity: GAUGE
              heatIndex: GAUGE
              windChill: GAUGE
              barometricPressure: GAUGE
              precipitationLastHour: GAUGE

        graphs:
          Temperatures:
            units: "degrees C."

            graphpoints:
              Temperature:
                dpName: conditions_temperature
                format: "%7.2lf"

              Heat Index:
                dpName: conditions_heatIndex
                format: "%7.2lf"

              Wind Chill:
                dpName: conditions_windChill
                format: "%7.2lf"

              Dewpoint:
                dpName: conditions_dewpoint
                format: "%7.2lf"

          Relative Humidity:
            units: percent
            miny: 0
            maxy: 100

            graphpoints:
              Relative Humidity:
                dpName: conditions_relativeHumidity
                format: "%7.2lf%%"

          Pressure:
            units: pascals
            miny: 0

            graphpoints:
              Pressure:
                dpName: conditions_barometricPressure
                format: "%7.0lf"

          Precipitation:
            units: meters
            miny: 0

            graphpoints:
              1 Hour:
                dpName: conditions_precipitationLastHour
                format: "%7.2lf"

          Wind Speed:
            units: mps
            miny: 0

            graphpoints:
              Sustained:
                dpName: conditions_windSpeed
                format: "%7.2lf"

              Gust:
                dpName: conditions_windGust
                format: "%7.2lf"

          Visibility:
            units: meters
            miny: 0

            graphpoints:
              Visibility:
                dpName: conditions_visibility
                format: "%7.2lf"

You can refer to Monitoring templates for more information on creating monitoring templates in YAML.

Reinstall the ZenPack to update the monitoring template.
```
zenpack --link --install $ZP_TOP_DIR
```
Navigate to Advanced > Monitoring Templates in the web interface to verify that the Station monitoring template has been updated with the conditions datasource and corresponding graphs.

Test monitoring weather conditions

Run the following command to collect from api.weather.gov and test weather condition monitoring:

zenpython run -v10 --device=api.weather.gov

There will be a lot of output from this command, but we’re mainly looking for at least one datapoint being written. If one works, it’s likely that they all work. Look for a line similar to the following:

2019-05-29 21:36:42,734 DEBUG zen.MetricWriter: publishing metric api.weather.gov/conditions_barometricPressure 100710.0 1559165802 {'device': 'api.weather.gov', 'contextUUID': 'd2fd005c-b9d7-4ab8-a02b-5e3fc4b2ecba', 'key': 'Devices/api.weather.gov/nwsStations/KY63'}

You can use a commmand like the following to see just these log lines:

zenpython run -v10 --device=api.weather.gov | grep "MetricWriter" | grep "conditions"