When we are just starting out, it is easy to think of test automation as go to url, click on link, enter text, click button, verify text etc. When we proceed in this direction, we pay attention to the actions performed and not the business intent. Very soon we find that we do not understand what the tests do and it is not easy to scale them to the thousands of test cases that are actually needed for thoroughly testing a modern product.
Just like we use object oriented programming, MVC frameworks etc. to organize and structure our application code, we need similar abstractions in test automation. Let us understand the various components of a scalable test automation solution.
There are a few fundamental building blocks in test automation. They are:
- Flows
- Entities
2.1. Attributes - UI Elements
- Screens
- Keywords
5.1. Steps
5.2. Automation Engine - Data
- Suites
- Reports
Let us understand them.
1. Flows
Given a set of pre-conditions, when we perform a few actions, we expect some specific outcome. This is a flow. Below are a couple of examples:
Example 1.1: Given that a product exists, when we add it to the shopping cart, we expect the product to be added to cart and the total of the cart to be equal to the price of the product.
Example 1.2: Given that a product exists and a coupon for it exists, when we add the product to the cart and apply the coupon, we expect the product to be added to the the cart and the total of the cart will be the price of the product minus the discount due to the coupon.
2. Entities
Now that we understand what flows are, let us understand entities. In the example 1.1 above, different “things” interact. There is a product and a shopping cart and most probably also a user who is purchasing the product. product , cart and user are the entities. Interactions happen between these entities. In example 1.2, we have one more entity - the coupon .
Entities normally have attributes. For example, a product entity would have a name, an image, a price, some quantity etc. as attributes. A coupon entity would have the coupon code, product id, discount amount etc. as attributes. In automation, attributes are associated with two important things - a) UI element identifiers and b) Data. We will explore on this more later.
3. UI Elements
In automation, we need to interact with UI elements like buttons, textboxes etc. It is best to store these UI elements in designated places so that they can be easily found and modified if the user interface changes. UI Elements can either be associated with an entity attribute or with a screen.
UI elements normally have a specific purpose. For example, a textbox for username, is used to take the input for the username field, which may then be persisted by the application in its database etc. or verified against the database for login. This textbox would not be used to, say, take the input for the user’s age. Such a textbox is “bound” to the user’s username. We can associate the UI element for username textbox with the “user” entity’s “username” attribute.
There are also other types of UI Elements. For example, a “Logout” button, or a “Checkout” button. These buttons are not related to any specific data. So instead of associating them with an entity’s attribute, we can just club them together based on functionality or on the screen where they appear.
4. Screens
Screens are a holder for multiple related UI elements. UI Elements which are not bound to any specific entity attribute can be kept in screens. For example, a register screen may have UI elements like Register button , Login link , Sign Up with Google link etc. A navbar screen may contain Logout link and Profile link .
5. Keywords
We saw that a flow is made of business actions like add the product to the cart and apply coupon . These business actions are called keywords.
5.1. Steps
For the above flows to execute in an automated way, we need to translate our keywords like “add the product to the cart” and “apply coupon” to actual steps that will be performed on the user interface. For example, in “add the product to the cart”, the steps may be, “click on product name”, “Enter value of quantity”, “click Add To Cart button”. These steps are the actual actions being performed to automate the business functionality. Normally steps comprise of an action (set value), a UI element (quantity textbox) and optionally, a value (the quantity).
5.2. Automation Engine
In order to perform the actual actions on a browser, we will need an automation engine. Examples of automation engines are Selenium, Webdriver, Sahi Pro etc.
6. Data
There are four types of data that we use during automation.
- Randomly generated, but valid data
- Valid data that the application should definitely accept
- Invalid data the application should definitely reject
- Valid specific data which needs to be verified against calculations
6.1. Randomly generated, but valid data
Repeatability is an important aspect of test automation. During automation it is needed that we can create entities like products and coupons as and when needed, repeatedly, so that the automated test case can be rerun repeatedly. For example, if we register a user with an email address once, we most probably can’t re-register with the same email id, since the application will prevent the same email id from being used twice. Similar may be the case for coupon code or product code. In such cases for repeatedly executing test cases, we need to generate data each time such that the data is unique. This can be done via Data Generators attached to entity attributes.
6.2. Valid data that the application should definitely accept
Sometimes there may be specific data that we need to ensure that the application accepts it. These could be positive boundary value conditions, some special characters in names etc. For example, if the valid age allowed in an age field is greater than or equal to 18, we need to specifically ensure that age 18 works correctly. Valid data could be defined for each entity attribute.
6.3. Invalid data the application should definitely reject
Any application would normally have validations which let the user know when an unacceptable value is entered. We need to ensure that such validations are shown for known invalid data. Example of an invalid data would be an email like abc@examplecom or abc.example.com etc. There would be invalid data associated with each entity attribute where the input field is free form like textbox or textarea (and not radio, checkbox or select dropdown, where the values are preconfigured by the application).
6.4. Valid specific data which needs to be verified against calculations
Some test cases are about verifying the state of an application. For example, can a product be created correctly, can it be edited, can it be searched, can it be deleted etc. In these tests, the input just needs to be valid data (of type 1 above). It does not matter what the specific value is. However, in some flows, we may need to verify a calculation that is performed on a particular attribute. For example, we may want to check if a coupon discount is applied correctly to the cart total. To test this, we will need to set up the product price (say Rs 100) and the discount value (say 10%) to known values so that the discounted cart total (Rs 90 here) can be verified.
Note: If the product price and discount value were randomly generated, the discounted total would have to be calculated. This is wrong because now the calculation is repeated in the application code, and in the test code, so we may not know which is correct. The right way to test calculations is to use pre-verified, known, hardcoded values in the tests.
7. Suites
Apart from the above, we need a way of clubbing together various flows and executing them as a batch. This allows integrating with Continuous Integration systems etc.
8. Reports
Once flows are executed, we need to be able to view the result of the execution. Results should be presented as easy to consume reports which highlight the health of the application quickly.