Skip to content

Web Transactions ZenPack

ZenPacks.zenoss.PS.WebTransactions

A Web Transaction is an interaction of one or multiple steps between a web browser and a server. A web transaction example might be submitting authentication credentials and validating the resulting page.

This ZenPack provides the following functions:

  • Perform synthetic user web transactions using a web browser

  • Monitoring of the health and user experience of a web application through scripted transactions

  • Web transaction testing performed from the Zenoss Collector or customer Selenium Grid instance

  • Web Transaction testing results for the following:

    • Metric for time taken to perform the full transaction and transaction steps

    • Events for failed web page element location and their validation

Don't use this ZenPack for the following tasks:

  • Multiple location web transaction testing platform

  • Multiple browser web transaction testing platform

  • An extractor of values from web pages to datapoints

Releases

Version 1.2.1

  • Released on 2025/11/04

Prerequisites

Note the following Zenpack dependencies:

Prerequisite Restriction
Product Zenoss 6.2.1 or higher
Required ZenPacks ZenPacks.zenoss.ZenPackLib >= 2.1.2, ZenPacks.zenoss.PythonCollector >= 1.11.0, ZenPacks.zenoss.PS.Util >= 1.11.2
Other dependencies Selenium Grid, Chromium Web Browser

Installation

If you are not a Cloud Zenoss user, then note that Docker images need to be installed with the ZenPack. You can find installation media at delivery.zenoss.io. After logging in to the Support Portal, navigate to ZenPacks > 6x-zenpacks > WebTransactions.

Selenium Image Installation

Install the images on the ControlCenter Master by running the image run commands.

chmod a+x *.run
sudo ./install-zenoss-selenium-hub_XX.YY.ZZ-QQQQQQQ.run
sudo ./install-zenoss-selenium-node-chrome_XXX.YYY.ZZZ.QQQ.run

Where XX.YY in the file name represents the version of the Selenium/Browser contained in the image. These image run commands provide the following images:

zenoss/selenium-hub:latest
zenoss/selenium-node-chrome:latest

To manually check on the software versions contained in the image, run the following command.

docker run --rm zenoss/selenium-hub:latest sudo /opt/selenium/selenium-server.jar hub --version
docker run --rm zenoss/selenium-node-chrome:latest /usr/bin/google-chrome --version

Upgrade Selenium service definitions

To update an existing Zenpacks.zenoss.PS.WebTransaction installation, do the following:

  1. Call the newer image run commands on the ControlCenter Master to update the images in the Docker repository.

  2. Extract and run the serviced update scripts to update the collector's Selenium services.

Example:

unzip -p ZenPacks.zenoss.PS.WebTransactions-VERSION-py2.7.egg ZenPacks/zenoss/PS/WebTransactions/migrate/upgradeSeleniumGridHub_ServiceDefinition.txt > upgrade.txt
sudo serviced script run --service localhost/SeleniumGridHub upgrade.txt

unzip -p ZenPacks.zenoss.PS.WebTransactions-VERSION-py2.7.egg ZenPacks/zenoss/PS/WebTransactions/migrate/upgradeSeleniumGridNodeChrome_ServiceDefinition.txt > upgrade.txt
sudo serviced script run --service localhost/SeleniumGridNodeChrome upgrade.txt

sudo serviced service restart localhost/SeleniumGridHub
sudo serviced service restart localhost/SeleniumGridNodeChrome

Note

In this example, the Selenium services are updated on the localhost collector. You need to perform this task for any additional collectors.

Usage

The goal of this ZenPack is to provide a method to gauge the performance and experience of a web application.

Previously, in the 1.0.x version series of this ZenPack this was achieved using a concept called recipes. The 1.2.x version series deprecates recipes and allows for multi-step synthetic transaction monitoring. Recipes are a single-step transaction monitoring measuring the total time a simple transaction would take to run.

This ZenPack delivers web transaction monitoring using Selenium Grid. According to Selenium's documentation, Selenium Grid allows the execution of WebDriver scripts on remote machines by routing commands sent by the client to remote browser instances. By using Selenium, web transactions are run through an actual web browser, simulating the user experience. This ZenPack currently makes exclusive use of the Chromium browser.

1.0.x - Single-step Transaction Recipes

Transactions are defined at the device level and defined in the datasource.

1.2.x - Multi-step Transactions

Transactions are defined at the Web Transaction component level that the user adds in the Add Component device menu option in the Zenoss user interface. Unlike 1.0.x Single-step transaction recipes, Multi-step transactions are defined using YAML. The YAML can be defined and modified from the Web Transaction component's Details screen.

Transaction steps are defined within the Web Transaction component YAML. Transaction Steps components are automatically created or updated.

Note

Transaction Step components are never automatically deleted. When the number of steps are reduced in a Web Transaction YAML definition, delete the extra Transaction Step component(s) manually.

Configuration

Configure the following components and data sources.

Components

  • Web Transactions are manually created using the Add Component Device menu. This component has an summary of the transaction: status, total time, and relation to its steps. The transaction YAML definition format is documented further below.

  • Transaction Steps are automatically created when creating or modifying the Web Transaction. This component displays details specific to the transaction step: related transaction, step status, step order, and step runtime.

Datasource

Configure the following option for Selenium, web transactions, and recipes.

Selenium options

  • User Browser is the browser to use to perform the transaction test. the default is Chromium.

  • Command Executor is the Selenium Grid URL Endpoint used to perform the test. If not overridden, the Grid instance running on the collector is used. The default is ${here/zSeleniumGridCmdExecutor}.

  • Page Load Strategy is the strategy used for determining web page load completion. The default is normal. You can choose one of the following options:

    • normal: Waits for all resources to download.

    • eager: DOM access is ready, but other resources like images may still be loading.

    • none: Doesn't block WebDriver at all.

For more information, see the Selenium documentation.

WebTransaction options

  • WebPage Load Time: Maximum time to wait for web pages to load. The default is ${here/zPageLoadTimeout}.

  • Find Page Timeout: Maximum time to wait for the Element to exist in loaded page. The default is ${here/zFindElementTimeout}.

  • Verify Element Text Value Timeout: Maximum time to wait for the Element value to match the value you want. The default is ${here/zVerifyElementValueTimeout}.

Recipe detail options

  • URL: The initial URL to perform the transaction recipe against.

  • Transaction Recipe: The transaction recipe consists of the following:

    • HTTP - Get: Perform a simple HTTP Get request and optional content verification.
    • Login - Basic Auth: Perform a simple Basic Auth test and optional content verification.
    • Login - Form: Perform authentication by submitting login credentials using a web form. For this form, provide the form elements locator information and data to submit in the recipe configuration.
    • Custom: This is not a specific recipe, but instead allows you to use Python and Web Transaction methods to define a customized Web Transaction test.

1.2.x - Multi-step Transactions

Transactions are defined in YAML at the Web Transaction component. The YAML definition is a nested format with the different heirarchal levels that define the following:

  • Step Name

  • Step action

  • Parameters needed for Step action

  • Step action, sub-action. Some step actions are special and have actions, for example, findElement.

  • Parameters needed for sub-actions

Example format:

- Step 1:
  - Step Action 1:
      Parameter 1: value 1
      Parameter 2: value 2
  - Step Action 2:
      Parameter 1: value 1
      actions:
        - sub-action 1:
            Parameter 1: value 1
- Step 2:
  - Step Action 1:
      Parameter 1: value 1

YAML can also contain Zenoss TALES expressions to access zProps & cProps values.

Steps, step actions, and sub-actions are performed in a serial sequence to provide control of when actions are performed, thus ensuring actions are performed after previous actions that are dependent.

  • Step Name: The Transaction Step component name. The component ID will be the ID of the Web Transaction combined with the step number.

  • Step Action: Actions to take as part of the step. Actions can be any of the following:

    • getUrl
    • addToRequestHeader
    • findElement
    • basicAuthUrl

findElement is the primary worker for web transactions. You can use it to just find an element on a page, verify element values and additionally perform an action on the found element. Element actions are dependent on the Element type found, such as text box, button, and such. The findElement step action takes two parameters.

  • The first parameter dictates the locator used to find the Element. The locator parameter key is variable and can be one of the following; tag_name, id, xpath, name, class_name, css_selector, link_text, partial_link_text. Its value is then desired value desired for the locator. Examples: id: Submit Button, tag_name: title , xpath: //a[text()="sign in.

  • The second parameter is optional, defining a verification and any actions to be taken on the found element. Element actions are dependent on the found Element type.

Element verification is a built-in action. You can use it to verify an Element's value or the value of an element's attribute. Element value verification can use a regular expression, for example:

- findElement:
    id: doohickey
    actions:
    - verify:
        value: thingamajig
        attribute:
           color: green

Migrating from Recipes

Login - Basic Auth

- Auth:
  - basicAuthUrl:
      url: ${here/cWebTransactionUrl}
      userName: ${here/cWebTransactionUser}
      password: ${here/cWebTransactionPassword}
  - findElement:
      tag_name: title

Login - Form

- Auth:
  - getUrl: ${here/cWebTransactionUrl}
  - findElement:
      id: username
      actions:
      - send_keys: ${here/cWebTransactionUser}
  - findElement:
      id: passwrd
      actions:
      - send_keys: ${here/cWebTransactionPassword}
  - findElement:
      id: loginButton
      actions:
      - click
  - findElement:
      tag_name: TITLE
      actions:
      - verify:
          value: Logged In

Examples

Token Auth

- Auth:
  - addToRequestHeader:
      Authorization: Bearer ${here/cToken}
  - getUrl: ${here/cWebTransactionUrl}
  - findElement:
      tag_name: TITLE
- Events Console:
  - findElement:
      id: Events-nav-button
      actions:
        - click
  - findElement:
      id: lastupdated
      actions:
        - verify:
            value: Last updated at.*

1.0.x - Single-step Transaction Recipes

To be considered Legacy, the preferred method is to use the Multi-step functionality.

Web Transaction Success Validation

To validate the success of a Web Transaction, use the Datasource Recipe verify fields. All Web Transaction Recipes have the same four verification fields:

  • Verify Element
  • Verify Text Value
  • Verify Element Attribute
  • Verify Element Attr Value

An element is a part of a web page, which can be seen by looking at the web page source. The verify Element value can be a HTML tag name, tag 'id' attribute value, HTML DOM XPath, or tag class value. If content validation is desired, Verify Element will be a required field while the remaining verify fields are not all required.

Web Transaction recipes that interact with an element, such as filling an input box or clicking, follow the same logic as Verify Element.

Recipe fields that define a Value to be verified can also use zProperty or cProperty fields, for example, ${here/cWebTransactionPassword}. This is especially useful for password type values or reuse of monitoring templates by not hard-coding values into the template.

Note

Transaction validation does not include SSL certificate validation. All certificate errors are ignored.

Custom Recipe

When there is not a Recipe that fits the testing need, there is the Custom Recipe option. A Custom recipe is basically using the special methods in a Python script. These methods are special in that they combine and enhance the basic capability provided by the Selenium-Python driver.

Before any Custom Recipe script is performed, the maximum wait time for a web page to load is set to the zPageLoadTimeout value and a driver.get() is performed on the URL defined in the datasource configuration. The defined custom recipe actions are then performed.

ZenPack provided methods include the following:

  • findElement: Find the first matching element in the HTML DOM. Search by tag name, tag 'id' attribute value, XPath, tag class. Timeout to find element, defined in the zFindElementTimeout property (default value is 10). Elements can be interacted with in a number of ways; click(), send_keys(), etc. (Reference: https://www.selenium.dev/documentation/webdriver/elements/interactions/)

  • verifyElement: Uses findElement and then verifies one or both of element's text or attribute value. Some element values do change dynamically after initial page load and this method will wait to match desired result. Timeout to match element's value is defined in the zVerifyElementValueTimeout property (default is 10 seconds). Element text value evaluation can use regular expressions.

  • clickElement: Uses findElement and then waits until the element is clickable. The click wait re-uses the timeout defined in the zFindElementTimeout property.

  • addToRequestHeader: Add a header to the HTTP request.

Exposed Selenium-Python objects and methods include the following:

  • driver: Selenium web driver. The previous methods typically provide all the functionality required for a Web Transaction test. However, if they do not, the Selenium-Python webdriver is exposed here. For further information about usage and methods, see https://selenium-python.readthedocs.io/getting-started.html.

Note

XPath, also known as XML Path, is one of the most commonly used locators in Selenium WebDriver that can help you navigate through the HTML structure of a page. It can be used for HTML and XML documents to locate any element in a web page using HTML DOM structure. How to use or define an XPath to locate an element is beyond the scope of this document.

Examples

The following examples are snippets to highlight the use of the Web Transaction methods in a Custom Recipe. Don't use them as best-practice examples.

Example: Web Login Form

This example is what the "Login - Form" recipe would be as a Custom Recipe.

findElement('username').send_keys('${here/cWebTransactionUser}')
findElement('passwrd').send_keys('${here/cWebTransactionPassword}')
clickElement('loginButton')
verifyElement('TITLE', verifyText='Logged In')

In this example, note the following details:

  • username, passwrd, & loginButton are the 'id' attribute values for HTML "INPUT" tags. See the previous findElement information for more details about what is used to locate web page elements.

  • TITLE is the HTML tag to be found and then verified that its text value is 'Logged In'. The verifyElement method can take up to four parameters; ElementToFind, verifyText, verifyAttr, and verifyAttrVal. The verify parameters are optional but at least one needs to be defined.

    # Verify the Element's Text value:
    # e.g. <TITLE>Logged In</TITLE>
    verifyElement('TITLE', verifyText='Logged In')

    # Verify an Element's attribute value
    # e.g. <DIV id="header" class="tableHeader" description="Tuna Harvest Table">Charlie Tuna</DIV>
    verifyElement('DIV', verifyAttr='description', verifyAttrVal='Tuna Harvest Table')

    # Very both the Element's Text & Attribute value
    # e.g. <TABLE><TR id="firstRow"><TD id="exTarget" descr="Apples">Applesauce</TD>.....
    verifyElement('exTarget', verifyText='Applesauce', verifyAttr='descr', verifyAttrVal='Apples')

Example: Conditional action/verify

nfStatus = findElement('//div[@class="down-notification-content"]/h2')
if nfStatus.get_attribute('textContent') == 'Service is up!':
  findElement('//a[text()="sign in"]').click()
  verifyElement('//h1[@data-uia="login-page-title"]', verifyText='Sign In')

Example: Multi-Step Transaction

findElement("username").send_keys("${here/cWebTransactionUser}")
findElement("passwrd").send_keys("${here/cWebTransactionPassword}")
findElement("loginButton").click()
verifyElement("TITLE", verifyText="\n            Zenoss:\n            Dashboard\n        ")
findElement("Infrastructure-nav-button").click()
verifyElement("device_panel_header-inputEl", "/Devices")
findElement("sign-out-link").click()
verifyElement("TITLE", verifyText="Login")

While a multi-step transaction script is a possible complex test, this ZenPack isn't intended to deliver or support it. This example is more of an academic case to test and push the limits of what is possible.

Troubleshooting / Debugging Transactions

The Transaction error event's summary provides the title of the page where the issue occurred. Error events also have a field(webTransactionUrl), containing the URL the error occurred on. Also, when an error is encountered, a screenshot of the browser is taken to allow for basic debugging. The screenshot can be found in the zenpython container that tested the transaction in the /tmp directory.

For more advanced debugging of Web Transaction scripts or recipes, it is recommended to spin up a dedicated debug Selenium Grid instance. Then set its address in the datasource's Command Executor configuration parameter or as the device's zSeleniumGridCmdExecutor property.

Using a dedicated Selenium Grid instance allows for a variety of different debug options, such as connecting to the Node's VNC port and watching the Web Transaction play out in the browser. For more information, see Selenium's website.

Selenium Grid Transaction Execution

This ZenPack bundles a Selenium Grid and Chromium Node services that run at the collector level. Web Transactions can also be run from external Selenium Grid instances using the zSeleniumGridCmdExecutor zProperty. Its default value being the Zenoss Collector Selenium Grid address.

The SeleniumGridNodeChrome service is configured using its service environmental variables. To modify, use the command line of serviced service edit SeleniumGridNodeChrome and update the Environment section. Zenoss Selenium Node's default values are the following:

"Environment": [
  "SE_EVENT_BUS_HOST=127.0.0.1",
  "SE_EVENT_BUS_PUBLISH_PORT=4442",
  "SE_EVENT_BUS_SUBSCRIBE_PORT=4443",
  "SE_START_XVFB=false",
  "SE_START_VNC=false",
  "SE_NODE_HOST=nc{{ plus 1 .InstanceID }}",
  "JAVA_OPTS=-Dwebdriver.chrome.whitelistedIps=",
  "SE_NODE_MAX_SESSIONS=2",
  "SE_NODE_SESSION_TIMEOUT=30",
  "SE_DRAIN_AFTER_SESSION_COUNT=100"
],

Note

Don't change the first seven options.

Options include the following:

  • "JAVA_OPTS=-Dwebdriver.chrome.whitelistedIps=": Prevents container error messages of [SEVERE]: bind() failed: Cannot assign requested address (99)

  • SE_NODE_MAX_SESSIONS=2": Maximum transaction sessions that can run concurrently.

  • "SE_NODE_SESSION_TIMEOUT=30": Maximum time a transaction may run zProperty timeouts should not be greater than this value. This is the hard limit for how long a Web Transaction test may take to complete.

  • "SE_DRAIN_AFTER_SESSION_COUNT=100": Shut down the Node after '100' tests have been executed.

For more information, see https://github.com/SeleniumHQ/docker-selenium#configuring-the-containers.

Changes

1.2.1

  • SVC-3866 Fix SeleniumGridNodeChrome only able to process 100 tests after startup
  • SVC-3866 Updated SeleniumGrid service's healthcheck

1.2.0

  • SVC-3849 Implementation of Multi-Step Web Transactions

1.1.0

  • SVC-3847 Update verify element text value to use regular expressions
  • SVC-3845 Add method to update HTTP request header in a custom script

1.0.5

  • SVC-3750 Selenium images, revert Ubuntu's use of netfilter to iptables
  • SVC-3801 New WebTran Datasource configuration, Page Load Strategy

1.0.4

  • SVC-3665 Minor fixes and performance enhancements. Streamline Selenium & Browser Node image build process for version updates.

1.0.3

  • Fixes

    • SVC-3515 Do not override Datasource configured event severity for WebTransactionRecipeFailure exceptions.
    • SVC-3515 Added WebTransactionRecipeFailure. Distinction between SeleniumGridFailure exceptions. SeleniumGridFailure event severity level are hardcoded to 'ERROR'

  • New:

    • SVC-3515 Added 'total transaction time' text to all event messages.
    • SVC-3558 Stale Element' handling when verifying element text values.

1.0.2

  • SVC-3509 RecipeCard field values evaluated for TALES expressions
  • SVC-3509 Added simple debug logic, take a browser screenshot when encountering transaction error
  • SVC-3509 Update WebTransactionRecipeFailure Exception param passing logic

1.0.1

  • SVC-3399 Automated ZenPack build fixes

1.0.0 - Initial Release

  • SVC-3360 Datasource UI enhancements
  • SVC-3218 Datasource UI
  • SVC-3152 Build testing/polling infrastructure & framework