Skip to content

Web Transactions

Commercial

This ZenPack is developed and supported by Zenoss Inc. Commercial ZenPacks are available to Zenoss commercial customers only. Contact your Zenoss representative or click here to request more information regarding this or any other ZenPacks.

Organization

Zenoss Inc.

Name

ZenPacks.zenoss.PS.WebTransaction

About

A web transaction is an interaction, of one or multiple steps, between a web browser and a server. One common example of a web transaction is submitting login credentials and validating that the expected page is displayed after login.

What this ZenPack delivers:

  1. Use an actual web browser to perform web transactions
  2. Perform simple static, scripted transaction testing of web applications
  3. Perform web transaction testing from a Zenoss collector or Selenium Grid instance
  4. Web transaction testing results with:
    1. Metric for time taken to perform the full transaction
    2. Events for failed web page element location and validation of web page element values and/or attribute values

What this ZenPack does not intend to be:

  1. Multiple location web transaction testing platform
  2. Multiple browser web transaction testing platform
  3. A complex web transaction tester
  4. A mechanism to extract datapoint values from web pages

Prerequisites

This table lists the Zenpack dependencies.

Prerequisite Restriction
Product Zenoss Cloud, or Zenoss Resource Manager 6.2.1 or higher
Required ZenPacks ZenPacks.zenoss.ZenPackLib >= 2.1.2, ZenPacks.zenoss.PythonCollector >= 1.11.0, ZenPacks.zenoss.PS.Util >= 1.11.2
Other dependencies Selenium Grid, Chromium web browser

Installation

Zenoss Serviced Dynamics (on-premises) users should be aware that Docker images will also need to be installed with this ZenPack. Installation media can be found at delivery.zenoss.io / ZenPacks / 6x-zenpacks / WebTransactions

Selenium Image Installation

Install the images on the ControlCenter Master by running the image run commands.

chmod a+x *.run
sudo ./install-zenoss-selenium-hub_XX.YY.ZZ-QQQQQQQ.run
sudo ./install-zenoss-selenium-node-chrome_XXX.YYY.ZZZ.QQQ.run

Where 'XX.YY'' in the file name represents the version of the Selenium/Browser contained in the image. These image run commands provide the following images:

zenoss/selenium-hub:latest
zenoss/selenium-node-chrome:latest

To manually check on the software versions contained in the image.

docker run --rm zenoss/selenium-hub:latest sudo /opt/selenium/selenium-server.jar hub --version
docker run --rm zenoss/selenium-node-chrome:latest /usr/bin/google-chrome --version

Upgrade Selenium service definitions

To update an existing 'Zenpacks.zenoss.PS.WebTransaction' installation, ensure to first call the newer image run commands on the ControlCenter Master. This will update the images in the Docker repository. Then extract and run the serviced update scripts to update the collector's Selenium services.

Example:

unzip -p ZenPacks.zenoss.PS.WebTransactions-VERSION-py2.7.egg ZenPacks/zenoss/PS/WebTransactions/migrate/upgradeSeleniumGridHub_ServiceDefinition.txt > upgrade.txt
sudo serviced script run --service localhost/SeleniumGridHub upgrade.txt

unzip -p ZenPacks.zenoss.PS.WebTransactions-VERSION-py2.7.egg ZenPacks/zenoss/PS/WebTransactions/migrate/upgradeSeleniumGridNodeChrome_ServiceDefinition.txt > upgrade.txt
sudo serviced script run --service localhost/SeleniumGridNodeChrome upgrade.txt

sudo serviced service restart localhost/SeleniumGridHub
sudo serviced service restart localhost/SeleniumGridNodeChrome

Note

The above example is updating the localhost collector Selenium services. This will need to be done for any additional collectors.

Usage

The goal of this ZenPack is to provide a method to perform a known, defined test against web applications in order to validate and monitor their health and performance. Defined web transaction tests are encapsulated as recipies for easy re-use. These recipes are selectable and configured from the "web transaction" data source.

This ZenPack delivers web transaction monitoring using Selenium Grid. "Selenium Grid allows the execution of WebDriver scripts on remote machines by routing commands sent by the client to remote browser instances" (Source: https://www.selenium.dev/documentation/grid/). By using Selenium, web transactions are run through an actual web browser. This ZenPack currently makes exclusive use of the Chromium browser.

Data source recipes

  • HTTP - Get - Perform a simple HTTP Get request and optional content verification.
  • Login - Basic Auth - Perform a simple Basic Auth test and optional content verification.
  • Login - Form - Perform authentication by submitting login credentials via a web form. This is achieved by providing the form elements locator information and data to submit in the recipe configuration.
  • Custom - Not a specific recipe, allows you to use python and Web Transaction methods to define a customized web transaction test.

Web transaction success validation

To validate the success of a web transaction, use the data source recipe verify fields. All web transaction recipes have the same four verification fields: "Verify Element", "Verify Text Value", "Verify Element Attribute", "Verify Element Attr Value". An element is a part of a web page, which can be seen by looking at the web page source. The Verify Element value can be an HTML tag name, tag 'id' attribute value, HTML DOM XPath, or tag class value. If content validation is desired, the "Verify Element" will be a required field while the remaining verify fields are not all required.

Web transaction recipes that interact with an element (e.g., filling an input box, clicking) follow the same logic as "Verify Element".

Recipe fields that define a "Value" to be verified, can also use configuration property fields including custom properties (e.g. ${here/cWebTransactionPassword}). This is especially useful for password type values or re-use of monitoring templates by not hard-coding values into the template.

Note

Transaction validation does not include SSL certificate validation. All certificate errors are ignored.

Custom recipe

When there is not a recipe that fits the testing need, there is the "Custom" recipe option. A custom recipe consists of a Python script that invokes the "special" methods. These methods are special in that they combine and/or enhance the basic capability provided by the selenium-python driver.

Before any custom recipe script is performed, the maximum wait time for a web page to load is set to the "zPageLoadTimeout" value and a driver.get() is performed on the URL defined in the data source configuration. The defined custom recipe actions are then performed.

ZenPack provided methods:

  • findElement - Find the first matching element in the HTML Document Object Model (DOM). Search by tag name, tag 'id' attribute value, XPath, tag class. Timeout to find element, defined in the "zFindElementTimeout" property (default value is 10). Elements can be interacted with in a number of ways; click(), send_keys(), etc. (Reference: https://www.selenium.dev/documentation/webdriver/elements/interactions/).
  • verifyElement - uses "findElement" and then verifies one or both of element's text or attribute values. Some element values do change dynamically after initial page load and this method will wait to match the desired result. Timeout to match an element's value is defined in the "zVerifyElementValueTimeout" property (default is 10 seconds).
  • clickElement - uses "findElement" and then waits until the element is clickable. The click wait re-uses the timeout defined in the "zFindElementTimeout" property.

Exposed Selenium-python Objects/Methods:

  • driver - Selenium web driver. The above methods should provide all the functionality required for a web transaction test. But if they do not, the Selenium Python web driver is exposed here. Usage and its methods are outside the scope of this document but can be found at https://selenium-python.readthedocs.io/getting-started.html.

Note

XPath, also known as XML Path, is one of the most commonly used locators provided by the Selenium WebDriver that can help you navigate through the HTML structure of a page. It can be used for HTML and XML documents to locate any element in a web page using the HTML DOM structure. How to use or define an XPath to locate an element is beyond the scope of this document.

Examples

The examples provided are snippets to highlight the use of the web transaction methods in a custom recipe. They should not be seen as best-practice examples.

Web Login Form example

This example is what the "Login - Form" recipe would be as a Custom Recipe.

findElement('username').send_keys('${here/cWebTransactionUser}')
findElement('passwrd').send_keys('${here/cWebTransactionPassword}')
clickElement('loginButton')
verifyElement('TITLE', verifyText='Logged In')

"username", "passwrd", & "loginButton" are the 'id' attribute values for HTML "INPUT" tags. Refer to the "findElement" info above for more info on what is used to locate web page elements.

"TITLE" is the HTML tag to be found and then verified that its text value is 'Logged In'. The "verifyElement" method can take up to four parameters; ElementToFind, verifyText, verifyAttr, and verifyAttrVal. The verify parameters are optional but at least one needs to be defined.

# Verify the Element's Text value:
# e.g., <TITLE>Logged In</TITLE>
verifyElement('TITLE', verifyText='Logged In')

# Verify an Element's attribute value
# e.g., <DIV id="header" description="Tuna Harvest Table">Charlie Tuna</DIV>
verifyElement('DIV', verifyAttr='description', verifyAttrVal='Tuna Harvest Table')

# Verify both the Element's Text & Attribute values
# e.g., <TABLE><TR id="firstRow"><TD id="exTarget" descr="Apples">Applesauce</TD>....
verifyElement('exTarget', verifyText='Applesauce', verifyAttr='descr', verifyAttrVal='Apples')

Conditional action/verify example

nfStatus = findElement('//div[@class="down-notification-content"]/h2')
if nfStatus.get_attribute('textContent') == 'Service is up!':
  findElement('//a[text()="sign in"]').click()
  verifyElement('//h1[@data-uia="login-page-title"]', verifyText='Sign In')

Multi-Step Transaction example

findElement("username").send_keys("${here/cWebTransactionUser}")
findElement("passwrd").send_keys("${here/cWebTransactionPassword}")
findElement("loginButton").click()
verifyElement("TITLE", verifyText="\n Zenoss:\n Dashboard\n        ")
findElement("Infrastructure-nav-button").click()
verifyElement("device_panel_header-inputEl", "/Devices")
findElement("sign-out-link").click()
verifyElement("TITLE", verifyText="Login")

A multistep transaction script could be considered to be a "complex" test, which this ZenPack does not aspire to deliver or support. This example is more of an interesting case to test and push the limits of what is possible.

Troubleshooting / debugging transactions

The Transaction error event's summary will provide the title of the page where the issue occurred. Error events also have a field (webTransactionUrl), containing the URL the error occurred on. When an error is encountered, a screenshot of the browser will be taken. This is done to allow for basic debugging . The screenshot can be found in the zenpython container that tested the transaction, under the /tmp directory.

For more advance debugging of web transaction scripts or recipes, it is recommended to spin up a dedicated debug Selenium Grid instance. Then set its address in the data source's "Command Executor" configuration parameter or as the device's "zSeleniumGridCmdExecutor" property. Using a dedicated Selenium Grid instance allows for a variety of different debug options, e.g., connecting to the Node's VNC port and watching the web transaction play out in the browser. For more information, refer to Selenium's website.

Selenium Grid transaction execution

This ZenPack bundles Selenium Grid and Chromium node services that run at the collector level. Web transactions can also be run from external Selenium Grid instances using the "zSeleniumGridCmdExecutor" zProperty. Its default value being the Zenoss Collector Selenium Grid address.

The "SeleniumGridNodeChrome" service is configured via its service environment variables. To modify any of them, use this command serviced service edit SeleniumGridNodeChrome and update the "Environment" section. The Zenoss Selenium node's default values are:

"Environment": [
    "SE_EVENT_BUS_HOST=127.0.0.1",
    "SE_EVENT_BUS_PUBLISH_PORT=4442",
    "SE_EVENT_BUS_SUBSCRIBE_PORT=4443",
    "SE_START_XVFB=false",
    "SE_START_VNC=false",
    "SE_NODE_HOST=nc{{ plus 1 .InstanceID }}",
    "JAVA_OPTS=-Dwebdriver.chrome.whitelistedIps=",
    "SE_NODE_MAX_SESSIONS=2",
    "SE_NODE_SESSION_TIMEOUT=30",
    "SE_DRAIN_AFTER_SESSION_COUNT=100"
],

Warning

Only the values of the SE_NODE_MAX_SESSSIONS, SE_NODE_SESSION_TIMEOUT, SE_DRAIN_AFTER_SESSION_COUNT environment variables should be changed. All other variables should retain the values configured by Zenoss.

Option details:

  • "JAVA_OPTS=-Dwebdriver.chrome.whitelistedIps=" - to prevent container error messages of [SEVERE]: bind() failed: Cannot assign requested address (99)

  • "SE_NODE_MAX_SESSIONS=2" - maximum transaction sessions that can run concurrently

  • "SE_NODE_SESSION_TIMEOUT=30" - maximum time a transaction may run. zProperty timeouts should not be greater than this value. This is the hard limit for how long a web transaction test may take to complete.

  • "SE_DRAIN_AFTER_SESSION_COUNT=100" - Shut down the Node after '100' tests have been executed.

For more information, refer to https://github.com/SeleniumHQ/docker-selenium#configuring-the-containers.

Changelog

1.0.4

  • SVC-3665: Minor fixes and performance enhancements. Streamline Selenium & Browser Node image build process for version updates.

1.0.3

  • SVC-3515:

    • Fix: Do not override Datasource configured event severity for WebTransactionRecipeFailure exceptions.

      • Added WebTransactionRecipeFailure. Distinction between SeleniumGridFailure exceptions. SeleniumGridFailure event severity level are hardcoded to 'ERROR'.

    • New: added 'total transaction time' text to all event messages.

  • SVC-3558

    • New: 'Stale Element' handling when verifying element text values.

1.0.2

  • SVC-3509 RecipieCard field values evaluated for TALES expressions
  • Added simple debug logic, take a browser screenshot when encountering transaction error
  • Update WebTransactionRecipeFailure Exception param passing logic

1.0.1 - Automated build fixes [SVC-3399]

1.0.0 - Initial Release

  • SVC-3360 Data source UI enhancements
  • SVC-3218 Data source UI
  • SVC-3152 Build testing/polling infrastructure and framework