improved

New option for soft assertions

When testing an application, some features are more important to validate than others. For example, validating core functionality in your app, such as login or payment, is obviously a higher priority than validating that all the label attributes on a form are correct. To accommodate for such nuances, we've added a new "Fail at end of test" option for soft assertions.

The "Fail at end of test" option allows you to assert on non-functional aspects, such as element labels and style, without stopping the test from executing.

An assertion marked "Fail at end of test"

An assertion marked "Fail at end of test"

What's changed

Previously, the mabl Trainer had only one option for soft assertions: "Continue on failure." When this option was checked, assertions always passed regardless of whether they met the assertion criteria. To find out whether the assertion passed, you had to review the individual steps on the Test Output page.

How it works

When you make an assertion on a page element, variable, URL, or cookie in the mabl Trainer, you can tell mabl what to do if the assertion fails. The options include "fail immediately", "fail at end of test", and "continue on failure & mark warning."

OptionDetails
Fail immediatelyIf the assertion fails, the test stops running and fails. This is the default option.
Fail at end of testIf the assertion fails, the test continues running and is marked failed at the end. This is the new option for soft assertions.
Continue on failure & mark warningIf the assertion fails, the step still passes and is marked with a warning. This option was previously the "Continue on failure" checkbox.

If an assertion marked "Fail at end of test" fails in a cloud run, the Test Output page indicates which step the test failed on at the top:

Browser test output failing on one assertion

Browser test output failing on one assertion

If multiple assertions were marked "Fail at end of test" and failed in a cloud run, the Test Output page lists all the steps that failed at the top.

Browser test output failing on multiple assertions

Browser test output failing on multiple assertions

To learn more about using assertions in mabl, check out our documentation.