Selenium-TestNg - Notes By ShariqSP

Understanding Unit Testing and TestNG

What is Unit Testing?

Unit testing is a software testing technique where individual units or components of a software application are tested in isolation. The purpose is to validate that each unit of the software performs as designed.

Importance of Unit Testing in Selenium

In Selenium, unit testing ensures that each part of your automation scripts behaves as expected. By testing individual components, you can identify bugs early in the development process and maintain the reliability of your test suites.

Introduction to TestNG

TestNG is a testing framework inspired by JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as annotations, parameterization, and grouping of tests.

Why Use TestNG?

TestNG offers several advantages over other unit testing tools:

Annotations: TestNG provides a rich set of annotations that allow you to define the test execution order, set up and tear down methods, and handle dependencies between tests.
Parameterization: TestNG allows you to run the same test method with different sets of data, making it easy to perform data-driven testing.
Grouping: TestNG allows you to group tests based on certain criteria, making it easier to organize and execute tests selectively.
Batch Execution: TestNG allows you to group multiple test classes and execute them together as a batch, which can save time and effort.
Parallel Execution: TestNG supports parallel execution of tests across multiple threads or processes, speeding up the overall test execution time.
Helping Attributes: TestNG provides helpful attributes such as dependsOnMethods, priority, and dataProvider, allowing you to control test execution flow, prioritize tests, and supply test data dynamically.
Data Provider: TestNG's dataProvider feature enables you to supply test data from external sources such as Excel files or databases, enhancing test flexibility and maintainability.
Reporting: TestNG generates detailed test reports, making it easier to identify test failures and analyze test results.
Integration: TestNG integrates seamlessly with other tools and frameworks, such as Selenium and Maven, enhancing the testing process.

Installing TestNG in Eclipse

TestNG is a popular testing framework for Java that integrates seamlessly with Eclipse. Here are different methods to install TestNG in Eclipse:

1. Installing TestNG for a Single Project:

Right-click on the project for which you want to install TestNG.
Navigate to Properties > TestNG.
Check the box labeled Enable TestNG for this project.
Click Apply and then OK.

2. Installing TestNG for All Projects:

Open Eclipse.
Go to Help > Eclipse Marketplace.
Search for TestNG in the Marketplace search bar.
Click Go and then click the Install button next to TestNG.
Follow the on-screen instructions to complete the installation process.

3. Using TestNG Eclipse Update Site:

Open Eclipse.
Go to Help > Install New Software....
Click the Add... button.
In the Name field, enter a name for the TestNG update site (e.g., TestNG).
In the Location field, enter the following URL: https://testng.org/testng-eclipse-update-site/
Click Add and then click OK.
Select TestNG from the available software list.
Follow the on-screen instructions to complete the installation process.

Once TestNG is installed, you can create TestNG test classes, write test methods, and run tests directly from Eclipse.

Annotations in TestNG and Usage in Selenium

Annotations in TestNG are special markers that define how the method below should be run. They are used to control the flow of execution of the test methods and perform various actions before or after the test methods.

Here's an example of annotations usage in Selenium with TestNG:

                    
        import org.testng.annotations.Test;
        
        public class SeleniumTest {
        
            @Test
            public void testHomePage() {
                // Code to open the browser and navigate to the homepage
                System.out.println("Testing homepage");
            }
        
            @Test
            public void testLoginPage() {
                // Code to test login functionality
                System.out.println("Testing login page");
            }
        
            @Test(dependsOnMethods = {"testLoginPage"})
            public void testDashboard() {
                // Code to test dashboard functionality
                System.out.println("Testing dashboard");
            }
        }

In this example, @Test annotations are used to mark the test methods. The testDashboard() method is dependent on the testLoginPage() method using dependsOnMethods attribute.

Rules for Using TestNG Effectively

While using TestNG, it's essential to adhere to certain rules to ensure efficient and effective test automation:

No Need for Main Method: Unlike traditional Java programs, TestNG test classes do not require a main method for execution. TestNG relies on annotations to identify and execute test methods automatically.
Avoid Printing Statements: Instead of using System.out.println() for logging information or debugging purposes, utilize TestNG's built-in reporting mechanism or a logging framework such as Log4j or SLF4J. Printing statements can clutter the console output and make it difficult to analyze test results effectively.
Use Descriptive Test Names: Ensure that test method names are descriptive and convey the purpose of the test clearly. This helps in understanding test results and debugging failures.
Avoid Test Method Dependencies: Minimize dependencies between test methods to maintain test independence. Although TestNG provides features like dependsOnMethods, excessive method dependencies can make test suites brittle and difficult to maintain.
Group Tests Logically: Group related tests using TestNG's grouping feature to organize test suites logically. This makes it easier to execute tests selectively and perform targeted testing.
Use Annotations Wisely: While annotations provide powerful functionality in TestNG, use them judiciously. Overuse of annotations can clutter test code and make it harder to understand and maintain.
Handle Test Failures Gracefully: Implement proper error handling and reporting mechanisms to handle test failures gracefully. This includes using TestNG's built-in reporting features and integrating with logging frameworks.
Keep Tests Idempotent: Ensure that tests are idempotent, meaning they produce the same result regardless of the number of times they are executed. This helps in maintaining test reliability and repeatability.
Parameterize Tests: Parameterize tests using TestNG's @DataProvider or other techniques to test various scenarios with different input values. This promotes reusability and reduces code duplication.
Use Assertions: Include meaningful assertions in test methods to validate expected outcomes. Assertions help in verifying the correctness of application behavior and detecting defects.
Optimize Test Execution: Optimize test execution by leveraging TestNG's features for parallel execution, grouping, and prioritizing tests. This ensures efficient resource utilization and faster feedback cycles.
Continuous Integration: Integrate TestNG with continuous integration (CI) pipelines to automate test execution and streamline the testing process. This facilitates early detection of issues and promotes a culture of continuous testing.

Helping Attributes for @Test Annotation in TestNG

The @Test annotation in TestNG supports several helpful attributes that enhance test flexibility and control. Here's a brief overview of each attribute:

priority: Specifies the execution priority of the test method. Lower values indicate higher priority, and methods with the same priority are executed in alphabetical order.
invocationCount: Specifies the number of times the test method should be invoked in a single test run.
enabled: Specifies whether the test method should be executed. Default value is true. Setting to false will skip the test method.
threadPoolSize: Specifies the number of threads in the thread pool used for parallel execution of the test method. This attribute is applicable when using parallel execution modes.
timeOut: Specifies the maximum time allowed for the entire test method execution, in milliseconds.
dataProvider: Specifies the name of the data provider method supplying data to the test method. Allows for data-driven testing.
dataProviderClass: Specifies the class containing the data provider method when the data provider method is defined in a different class.
dependsOnGroups: Specifies the groups on which the test method depends. The test method will be executed only if all the groups specified in this attribute have been included for execution.
dependsOnMethods: Specifies the methods on which the test method depends. The test method will be executed only if all the methods specified in this attribute have passed.
description: Provides a description of the test method, which can be useful for documenting test cases.
groups: Specifies the groups to which the test method belongs. Allows for grouping related test methods for selective execution.
invocationTimeOut: Specifies the maximum time allowed for the test method invocation, in milliseconds.
successPercentage: Specifies the percentage of successful invocations required for the test method to be considered successful. Useful for tests with non-deterministic behavior.
expectedExceptions: Specifies the exception types that the test method is expected to throw. Test will fail if an unexpected exception occurs.
alwaysRun: Specifies whether the test method should be executed even if it depends on methods that have failed or skipped.

These attributes provide fine-grained control over test execution behavior in TestNG, allowing for comprehensive and flexible test automation.

Order of Execution of @Test Annotation in TestNG

TestNG provides flexibility in controlling the order of test method execution using various attributes and features. By default, TestNG does not guarantee the execution order of test methods within a test class. However, you can influence the execution order using the following techniques:

1. Priority Attribute:

You can specify the execution order of test methods by assigning a priority to them using the priority attribute of the @Test annotation. Test methods with lower priority values will be executed first, followed by methods with higher priority values.


            @Test(priority = 1)
            public void testMethod1() {
                // Test method logic
            }
            
            @Test(priority = 2)
            public void testMethod2() {
                // Test method logic
            }

If you use 0 as the priority value, it will have the same effect as not specifying any priority. TestNG will execute methods with priority 0 after methods with no specified priority, in alphabetical order.

Negative values can also be used as priority. However, they are treated the same as zero priority.

If you use a single character (e.g., 'A', 'B') as the priority value, TestNG will internally convert it to its ASCII value and use it as the priority. For example, 'A' will be treated as priority 65, 'B' as priority 66, and so on.

2. DependsOnMethods Attribute:

You can define dependencies between test methods using the dependsOnMethods attribute of the @Test annotation. Test methods specified in this attribute will be executed before the current test method.


            @Test
            public void loginTest() {
                // Login test logic
            }
            
            @Test(dependsOnMethods = {"loginTest"})
            public void dashboardTest() {
                // Dashboard test logic
            }

3. Alphabetical Order:

If you do not specify priorities or dependencies, TestNG executes test methods in alphabetical order based on their method names.

By leveraging these techniques, you can control the order of execution of test methods in TestNG and ensure that your test cases run in the desired sequence.

Understanding invocationCount Attribute in TestNG

The invocationCount attribute in TestNG allows you to specify the number of times a test method should be invoked in a single test run. This attribute is useful for executing the same test logic multiple times with different inputs or to validate the stability of the system under repeated executions.

Here's how you can use the invocationCount attribute:


            @Test(invocationCount = 5)
            public void testMethod() {
                // Test method logic
                System.out.println("Executing test method");
            }

In this example, the testMethod() will be executed 5 times in a single test run.

Behavior with Different Values:

If you specify a positive integer value for invocationCount, the test method will be executed the specified number of times.
If you specify 0 or a negative integer value (e.g., -1) for invocationCount, the test method will not be executed.
If you specify a non-integer value or a character (e.g., 'a') for invocationCount, TestNG will throw a configuration error, and the test will not be executed.

By using the invocationCount attribute effectively, you can conduct thorough testing and ensure the robustness of your application under various scenarios.

Understanding the enabled Attribute in TestNG

The enabled attribute in TestNG allows you to control whether a test method should be executed or skipped during a test run. By default, the value of this attribute is true, indicating that the test method is enabled and will be executed. However, you can explicitly set it to false to skip the execution of the test method.

Here's how you can use the enabled attribute:


            @Test(enabled = true)
            public void testMethodEnabled() {
                // Test method logic
            }
            
            @Test(enabled = false)
            public void testMethodDisabled() {
                // Test method logic
            }

In this example, testMethodEnabled() will be executed since its enabled attribute is set to true, while testMethodDisabled() will be skipped due to its enabled attribute being set to false.

Behavior:

If you set the enabled attribute to true, the test method will be executed.
If you set the enabled attribute to false, the test method will be skipped during test execution.

The enabled attribute provides flexibility in controlling the execution of individual test methods, allowing you to focus on specific tests while excluding others as needed.

Understanding the threadPoolSize Attribute in TestNG

The threadPoolSize attribute in TestNG allows you to specify the number of threads to be used for parallel execution of test methods within a test suite. This attribute is particularly useful when running tests in parallel to achieve faster execution times.

Here's how you can use the threadPoolSize attribute:


            @Test(threadPoolSize = 3)
            public void testMethod() {
                // Test method logic
            }

In this example, the testMethod() will be executed concurrently using 3 threads.

Behavior:

The threadPoolSize attribute controls how many threads TestNG will allocate for executing the test method concurrently. TestNG will create a pool of threads and distribute the test methods among these threads based on the specified size.

It's important to note that setting a threadPoolSize greater than 1 enables parallel execution of test methods. Ensure that your test methods are thread-safe to avoid potential race conditions and synchronization issues.

By leveraging the threadPoolSize attribute effectively, you can optimize test execution time by running multiple test methods concurrently, thereby improving overall testing efficiency.

Example with Invocation Count and ThreadPoolSize:

Here's an example demonstrating the combined usage of invocationCount and threadPoolSize attributes:


            @Test(invocationCount = 6, threadPoolSize = 2)
            public void testMethodWithInvocationCountAndThreadPoolSize() {
                // Test method logic
            }

In this example, the

testMethodWithInvocationCountAndThreadPoolSize()

will be executed 6 times in total, and TestNG will use 2 threads for parallel execution of these invocations.

Understanding the timeOut Attribute in TestNG

The timeOut attribute in TestNG allows you to specify the maximum time allowed for the execution of a test method. If the test method takes longer than the specified timeout duration to complete, TestNG will mark the test as failed.

Here's how you can use the timeOut attribute:


            @Test(timeOut = 5000) // Timeout specified in milliseconds
            public void testMethodWithTimeout() {
                // Test method logic
            }

In this example, if the testMethodWithTimeout() takes longer than 5 seconds (5000 milliseconds) to complete, TestNG will mark it as failed.

Behavior:

If the test method completes execution within the specified timeout duration, it will be considered successful.
If the test method exceeds the timeout duration, TestNG will interrupt the execution and mark the test as failed.

The timeOut attribute is particularly useful for preventing tests from hanging indefinitely due to unexpected issues such as deadlocks, infinite loops, or unresponsive application behavior.

It's important to set a reasonable timeout value based on the expected execution time of the test method and the overall test environment. Avoid setting excessively short timeouts that may lead to false positives or excessively long timeouts that defeat the purpose of timely test execution.

Understanding Batch Execution in TestNG

Batch execution in TestNG allows you to organize and execute your tests in groups or batches, providing flexibility in test execution and allowing you to control the order and grouping of tests based on your requirements. This feature is particularly useful for executing tests in a specific sequence or grouping related tests together for efficient execution and analysis.

How Batch Execution Works

TestNG provides the <methods> and <classes> tags in the testng.xml suite file to define batches of test methods or test classes, respectively. You can specify multiple batches within a suite file to organize your tests according to your testing strategy.

Benefits of Batch Execution

Batch execution offers several benefits for test management and execution:

Grouping Related Tests: You can group related tests together in batches to ensure they are executed together, facilitating better organization and analysis of test results.
Sequential Execution: Batches allow you to define the execution order of tests, ensuring that tests are executed in a specific sequence as required.
Parallel Execution: TestNG supports parallel execution at both the batch and suite levels, allowing you to run multiple batches concurrently for faster test execution.
Flexible Test Configuration: With batch execution, you can configure different settings, such as parameters, listeners, and data providers, for each batch to customize test execution based on specific requirements.

Example:

Here's an example of a TestNG XML suite file configured with batch execution:


            <suite name="BatchExecutionSuite">
                <test name="RegressionTests">
                    <classes>
                        <class name="RegressionTestClass1"/>
                        <class name="RegressionTestClass2"/>
                    </classes>
                </test>
                <test name="SmokeTests">
                    <classes>
                        <class name="SmokeTestClass1"/>
                        <class name="SmokeTestClass2"/>
                    </classes>
                </test>
            </suite>

In this example, the test suite is divided into two batches: "RegressionTests" and "SmokeTests". Each batch contains a set of test classes that belong to the respective regression or smoke testing category. You can configure additional settings, such as parallel execution and parameters, for each batch as needed.

Understanding Sequential Execution in TestNG

Sequential execution in TestNG refers to the process of executing test methods or test classes in a specific order defined by the test suite configuration. Unlike parallel execution, where tests run concurrently, sequential execution ensures that tests are executed one after the other, following a predefined sequence.

How Sequential Execution Works

TestNG allows you to specify the execution order of test methods or test classes within a test suite by defining dependencies or using priority attributes. By default, TestNG executes tests in the order they are defined in the testng.xml suite file, but you can override this behavior by explicitly specifying dependencies or priorities.

Benefits of Sequential Execution

Sequential execution offers several benefits for test management and execution:

Predictable Execution Order: Sequential execution ensures that tests are executed in a predictable order, making it easier to debug and troubleshoot test failures.
Dependency Management: TestNG allows you to define dependencies between test methods or test classes, ensuring that prerequisite tests are executed before dependent tests, which is essential for maintaining test integrity.
Customized Test Flow: You can customize the test execution flow by specifying priorities for test methods or test classes, allowing you to control the order in which tests are executed and prioritize critical tests.
Isolation of Tests: Sequential execution helps in isolating tests, ensuring that each test runs in a clean state without interference from other tests, which is particularly important for maintaining test independence and reproducibility.

Example:

Here's an example of a TestNG XML suite file configured for sequential execution using dependencies and priorities:


            <suite name="SequentialExecutionSuite">
                <test name="SequentialTests">
                    <classes>
                        <class name="SequentialTestClass1">
                            <methods>
                                <include name="testMethod1"/>
                                <include name="testMethod2"/>
                            </methods>
                        </class>
                        <class name="SequentialTestClass2">
                            <methods>
                                <include name="testMethod3"/>
                                <include name="testMethod4"/>
                            </methods>
                        </class>
                    </classes>
                </test>
            </suite>

In this example, the test suite "SequentialExecutionSuite" contains two test classes: "SequentialTestClass1" and "SequentialTestClass2". Test methods within each class are included in a specific order, ensuring that they are executed sequentially according to their definition in the suite file.

Understanding Group Execution in TestNG

Group execution in TestNG allows you to organize your tests into logical groups and execute them selectively based on group membership. This feature provides flexibility in test execution, allowing you to run specific subsets of tests based on their categorization, such as functional areas, priority levels, or test types.

How Group Execution Works

TestNG provides the @Test annotation with a groups attribute to categorize tests into different groups. You can assign one or more group names to each test method, allowing you to group related tests together based on common characteristics.

Additionally, TestNG allows you to include or exclude specific groups from test execution using configuration settings in the testng.xml suite file. This gives you fine-grained control over which groups of tests are executed during a test run.

Benefits of Group Execution

Group execution offers several benefits for test management and execution:

Selective Test Execution: Group execution allows you to execute specific subsets of tests based on their group membership, enabling targeted testing of specific functional areas or test scenarios.
Modular Test Organization: By categorizing tests into groups, you can organize your test suite into logical modules or components, making it easier to manage and maintain test cases.
Parallel Execution: TestNG supports parallel execution at the group level, allowing you to run multiple test groups concurrently for faster test execution, improving efficiency and reducing overall execution time.
Flexible Test Configuration: You can configure different settings, such as parameters, listeners, and data providers, for each test group to customize test execution based on specific requirements.

Example:

Here's an example of using group execution in TestNG with the @Test annotation and the testng.xml suite file:


            import org.testng.annotations.Test;
            
            public class GroupExecutionExample {
            
                @Test(groups = "smoke")
                public void test1() {
                    // Test logic
                }
            
                @Test(groups = "regression")
                public void test2() {
                    // Test logic
                }
            
                @Test(groups = {"regression", "sanity"})
                public void test3() {
                    // Test logic
                }
            }


            <suite name="GroupExecutionSuite">
                <test name="SmokeTests">
                    <groups>
                        <run>
                            <include name="smoke"/>
                        </run>
                    </groups>
                    <classes>
                        <class name="GroupExecutionExample"/>
                    </classes>
                </test>
                <test name="RegressionTests">
                    <groups>
                        <run>
                            <include name="regression"/>
                        </run>
                    </groups>
                    <classes>
                        <class name="GroupExecutionExample"/>
                    </classes>
                </test>
            </suite>

In this example, the GroupExecutionExample class contains three test methods, each assigned to one or more groups using the @Test annotation. The testng.xml suite file is configured to execute specific test groups ("smoke" and "regression") using the <groups> tag.

Understanding Group Execution in TestNG

How Group Execution Works

Benefits of Group Execution

Group execution offers several benefits for test management and execution:

Selective Test Execution: Group execution allows you to execute specific subsets of tests based on their group membership, enabling targeted testing of specific functional areas or test scenarios.
Modular Test Organization: By categorizing tests into groups, you can organize your test suite into logical modules or components, making it easier to manage and maintain test cases.
Parallel Execution: TestNG supports parallel execution at the group level, allowing you to run multiple test groups concurrently for faster test execution, improving efficiency and reducing overall execution time.
Flexible Test Configuration: You can configure different settings, such as parameters, listeners, and data providers, for each test group to customize test execution based on specific requirements.

Example:

Here's an example of using group execution in TestNG with the @Test annotation and the testng.xml suite file:


            import org.testng.annotations.Test;
            
            public class GroupExecutionExample {
            
                @Test(groups = "smoke")
                public void test1() {
                    // Test logic
                }
            
                @Test(groups = "regression")
                public void test2() {
                    // Test logic
                }
            
                @Test(groups = {"regression", "sanity"})
                public void test3() {
                    // Test logic
                }
            }


            <suite name="GroupExecutionSuite">
                <test name="SmokeTests">
                    <groups>
                        <run>
                            <include name="smoke"/>
                        </run>
                    </groups>
                    <classes>
                        <class name="GroupExecutionExample"/>
                    </classes>
                </test>
                <test name="RegressionTests">
                    <groups>
                        <run>
                            <include name="regression"/>
                        </run>
                    </groups>
                    <classes>
                        <class name="GroupExecutionExample"/>
                    </classes>
                </test>
            </suite>

                        
                            we can execute only a perticular method
                            <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
                            <suite name="Suite">
                                <test thread-count="5" name="Test">
                                    <classes>
                                        <class name="bike.bike">
                                            <methods>
                                                <include name="yamaha"/>
                                            </methods>
                                        </class>
                                    </classes>
                                </test> 
                            </suite>

Understanding Parallel Execution in TestNG

TestNG is a popular testing framework for Java, widely used for automated testing of applications. It provides the capability to execute tests in parallel, which helps in reducing the overall test execution time and improves efficiency. Parallel execution in TestNG can be achieved through various strategies, which are determined based on the requirements of the testing environment and the characteristics of the tests being executed.

Types of Parallel Execution in TestNG

1. Methods Parallel Execution

In this type of parallel execution, TestNG runs multiple test methods concurrently within the same class. Each test method is executed in its own thread, allowing for faster execution of tests. This type of parallelism is suitable when the tests are independent of each other and can be executed concurrently without any dependencies.

Example:

<suite name="MethodsParallelSuite" parallel="methods" thread-count="2">
                <test name="MethodsParallelTest">
                    <classes>
                        <class name="ParallelExecutionExamples"/>
                    </classes>
                </test>
            </suite>

2. Classes Parallel Execution

TestNG can also execute test classes in parallel, where each test class runs in its own separate thread. This type of parallelism is useful when the tests within a class are independent but there are dependencies between different classes. Running classes in parallel can help in maximizing resource utilization and reducing overall execution time.

Example:

<suite name="ClassesParallelSuite" parallel="classes" thread-count="2">
                <test name="ClassesParallelTest">
                    <classes>
                        <class name="Class1"/>
                        <class name="Class2"/>
                    </classes>
                </test>
            </suite>

3. Test Suites Parallel Execution

TestNG allows executing entire test suites in parallel, where multiple test suites are run concurrently. This type of parallelism is suitable for scenarios where there are dependencies between multiple test suites, and executing them sequentially would result in increased execution time. Parallel execution of test suites helps in achieving faster feedback and quicker validation of the application under test.

Example:


            <suite name="TestSuitesParallelSuite" parallel="tests" thread-count="2">
                <test name="TestSuite1">
                    <classes>
                        <class name="Class1"/>
                    </classes>
                </test>
                <test name="TestSuite2">
                    <classes>
                        <class name="Class2"/>
                    </classes>
                </test>
            </suite>

4. Distributed Parallel Execution

Distributed parallel execution involves distributing tests across multiple nodes or machines in a distributed testing environment. This allows for parallel execution of tests on different environments simultaneously, which can significantly reduce the overall test execution time. Tools like Selenium Grid or cloud-based testing platforms are commonly used to achieve distributed parallel execution.

Example:


            <suite name="DistributedParallelSuite" parallel="tests" thread-count="2">
                <test name="TestSuite1">
                    <classes>
                        <class name="Class1"/>
                    </classes>
                </test>
                <test name="TestSuite2">
                    <classes>
                        <class name="Class2"/>
                    </classes>
                </test>
            </suite>

In the example above, although both Test Suites Parallel Execution and Distributed Parallel Execution involve running tests in parallel, the latter specifically refers to distributing tests across multiple nodes or machines in a distributed testing environment, utilizing tools like Selenium Grid or cloud-based testing platforms.

Working with Parallel Execution in TestNG: A Step-by-Step Guide

Parallel execution in TestNG can significantly reduce the overall test execution time and improve efficiency. Follow these steps to implement parallel execution in your TestNG test suite:

Step 1: Configure TestNG XML Suite File

Create a TestNG XML suite file where you define your test suite structure and specify parallel execution settings. Use attributes like parallel and thread-count to configure the desired parallelism level. Here's an example of a TestNG XML suite file configured for parallel execution:


            <suite name="ParallelExecutionSuite" parallel="tests" thread-count="2">
                <test name="Test1">
                    <classes>
                        <class name="TestClass1"/>
                    </classes>
                </test>
                <test name="Test2">
                    <classes>
                        <class name="TestClass2"/>
                    </classes>
                </test>
            </suite>

Step 2: Implement Test Classes

Write your test classes and test methods using TestNG annotations. Ensure that your test methods are independent and can run concurrently without any dependencies. Here's an example of a test class:


            import org.testng.annotations.Test;
            
            public class TestClass1 {
            
                @Test
                public void testMethod1() {
                    // Test logic
                }
            
                @Test
                public void testMethod2() {
                    // Test logic
                }
            }

Step 3: Execute TestNG Suite

Run your TestNG suite file using your preferred method (e.g., IDE integration, Maven, Gradle, command line). TestNG will execute your test classes and methods in parallel according to the configuration specified in the suite file. Monitor the test execution to ensure parallelism is functioning as expected.

Step 4: Analyze Results

Once the test execution is complete, analyze the test results to ensure that all tests have passed successfully. Check for any failures or errors and troubleshoot as needed. Parallel execution may uncover concurrency-related issues that were not apparent in sequential execution, so thorough result analysis is essential.

Example:

Here's an example of a TestNG XML suite file configured for parallel execution along with a test class:


            <suite name="ParallelExecutionSuite" parallel="tests" thread-count="2">
                <test name="Test1">
                    <classes>
                        <class name="TestClass1"/>
                    </classes>
                </test>
                <test name="Test2">
                    <classes>
                        <class name="TestClass2"/>
                    </classes>
                </test>
            </suite>


            import org.testng.annotations.Test;
            
            public class TestClass1 {
            
                @Test
                public void testMethod1() {
                    // Test logic
                }
            
                @Test
                public void testMethod2() {
                    // Test logic
                }
            }

The DataProvider Attribute in TestNG

The @DataProvider attribute in TestNG is used to provide data to test methods. It allows you to run the same test method multiple times with different sets of data.

Usage

To use the @DataProvider attribute, you need to create a method annotated with it. This method must return a two-dimensional array, where each element of the outer array represents a set of test data, and each element of the inner array represents the parameters for the test method.

Example 1

Suppose you have a test method named loginTest which takes two parameters: username and password. You can create a data provider method as follows:


              import org.testng.annotations.DataProvider;
              import org.testng.annotations.Test;
              
              public class LoginTest {
                
                @DataProvider(name = "loginData")
                public Object[][] provideData() {
                  return new Object[][] {
                    {"user1", "password1"},
                    {"user2", "password2"},
                    {"user3", "password3"}
                  };
                }
                
                @Test(dataProvider = "loginData")
                public void loginTest(String username, String password) {
                  // Your test logic here
                  System.out.println("Logging in with username: " + username + " and password: " + password);
                }
              }

In this example, the provideData method is annotated with @DataProvider(name = "loginData"), which specifies the name of the data provider. It returns a two-dimensional array containing sets of username-password pairs.

The loginTest method is annotated with @Test(dataProvider = "loginData"), indicating that it will receive data from the loginData data provider. For each set of data provided by the data provider, the loginTest method will be executed.

Example 2

Another example is when the test method accepts an array of strings as input. Here's how you can implement it:


              import org.testng.annotations.DataProvider;
              import org.testng.annotations.Test;
              
              public class StringArrayTest {
                
                @DataProvider(name = "stringArrayData")
                public Object[][] provideData() {
                  return new Object[][] {
                    { new String[] {"apple", "banana", "orange"} },
                    { new String[] {"grape", "kiwi", "pineapple"} }
                  };
                }
                
                @Test(dataProvider = "stringArrayData")
                public void stringArrayTest(String[] fruits) {
                  // Your test logic here
                  System.out.println("Fruits:");
                  for (String fruit : fruits) {
                    System.out.println("- " + fruit);
                  }
                }
              }

In this example, the provideData method is annotated with @DataProvider(name = "stringArrayData"), which specifies the name of the data provider. It returns a two-dimensional array containing sets of string arrays.

The stringArrayTest method is annotated with @Test(dataProvider = "stringArrayData"), indicating that it will receive data from the stringArrayData data provider. For each set of data provided by the data provider, the stringArrayTest method will be executed.

Example 3

Another example is when the test method accepts a variable number of arguments. Here's how you can implement it:


              import org.testng.annotations.DataProvider;
              import org.testng.annotations.Test;
              
              public class VariableArgumentTest {
                
                @DataProvider(name = "variableArgsData")
                public Object[][] provideData() {
                  return new Object[][] {
                    { "Parameter1" },
                    { "Parameter1", "Parameter2" },
                    { "Parameter1", "Parameter2", "Parameter3" }
                  };
                }
                
                @Test(dataProvider = "variableArgsData")
                public void variableArgumentTest(String... parameters) {
                  // Your test logic here
                  System.out.println("Parameters:");
                  for (String param : parameters) {
                    System.out.println("- " + param);
                  }
                }
              }

In this example, the provideData method is annotated with @DataProvider(name = "variableArgsData"), which specifies the name of the data provider. It returns a two-dimensional array containing sets of parameters.

The variableArgumentTest method is annotated with @Test(dataProvider = "variableArgsData"), indicating that it will receive data from the variableArgsData data provider. For each set of data provided by the data provider, the variableArgumentTest method will be executed.

Using Data Utility Class to Create Accounts from Excel Data

In this scenario, you want to automate the process of creating accounts on the Demo Web Shop website using data stored in an Excel file. To achieve this, you can design a data utility class to read the data from the Excel file and use it to create accounts on the website.

Steps

Create an Excel file containing user details such as username, email, password, etc. Each row in the Excel file represents a set of data for creating a single account.
Design a data utility class responsible for reading data from the Excel file. You can use libraries such as Apache POI to handle Excel operations.
Implement a method in the data utility class to read data from the Excel file and return it in a suitable format, such as a list of objects or a two-dimensional array.
Write a test script that utilizes TestNG or any other testing framework of your choice to perform the following actions:
- Initialize the data utility class and call the method to retrieve data from the Excel file.
- Iterate over the retrieved data and use it to create accounts on the Demo Web Shop website.
- Verify the successful creation of accounts by performing assertions or checking the website's response.
Execute the test script and observe the automation process of creating accounts using the data from the Excel file.

Example

Assuming you have a data utility class named ExcelDataReader, your test script might look like the following:


              import org.testng.annotations.Test;
              
              public class AccountCreationTest {
                
                @Test
                public void createAccountsFromExcelData() {
                  // Initialize ExcelDataReader to read data from Excel file
                  ExcelDataReader reader = new ExcelDataReader("user_details.xlsx");
                  
                  // Retrieve data from Excel file
                  List<UserDetails> userData = reader.readUserData();
                  
                  // Iterate over user data and create accounts
                  for (UserDetails userDetails : userData) {
                    // Navigate to Demo Web Shop website
                    // Fill registration form using userDetails
                    // Submit registration form
                    // Verify successful account creation
                  }
                }
              }

In this example, ExcelDataReader is a hypothetical class responsible for reading user details from an Excel file. The readUserData() method retrieves the data and returns it as a list of UserDetails objects. The test script then iterates over this data, creates accounts on the Demo Web Shop website, and verifies the successful creation of each account.

Parameterization in TestNG

Parameterization is a powerful feature in TestNG that allows you to run the same test method multiple times with different sets of data. This is particularly useful when you need to test your application with various inputs or scenarios.

Using @Parameters Annotation

You can use the @Parameters annotation in TestNG to specify parameters for your test method. Here's an example:


@Test
@Parameters({"username", "password"})
public void loginTest(String username, String password) {
    // Test logic using username and password
}

In this example, the loginTest method takes two parameters: username and password. TestNG will inject values for these parameters from the testng.xml file.

Using @Parameters at Class Level

You can also specify parameters at the class level using the @Parameters annotation. This allows you to reuse the same set of parameters across multiple test methods within the same class. Here's an example:


@Parameters({"username", "password"})
public class LoginTests {

    @Test
    public void loginTest(String username, String password) {
        // Test logic using username and password
    }

    @Test
    public void logoutTest(String username, String password) {
        // Test logic for logout using username and password
    }
}

In this example, both the loginTest and logoutTest methods in the LoginTests class take the same set of parameters: username and password. These parameters are injected into the test methods from the testng.xml file.

Using Parameters Tag at Suite Level

You can specify parameters at the suite level in the TestNG XML suite file using the <parameter> tag. This allows you to pass parameters to all the test methods within the suite. Here's an example:


        <suite name="LoginSuite">
            <parameter name="username" value="testuser"/>
            <parameter name="password" value="testpassword"/>
            
            <test name="LoginTest">
                <classes>
                    <class name="com.example.LoginTests"/>
                </classes>
            </test>
        </suite>

In this example, the LoginSuite suite specifies the username and password parameters with default values. These parameters will be passed to all the test methods within the suite, including the loginTest and logoutTest methods in the LoginTests class.

Using DataProviders

Another way to parameterize tests in TestNG is by using DataProviders. DataProviders allow you to supply test data dynamically from a method. Here's how you can use it:


@DataProvider(name = "loginData")
public Object[][] loginData() {
    return new Object[][] {
        {"user1", "password1"},
        {"user2", "password2"},
        // Add more data as needed
    };
}

@Test(dataProvider = "loginData")
public void loginTest(String username, String password) {
    // Test logic using username and password
}

In this example, the loginTest method is executed multiple times, once for each set of data provided by the loginData DataProvider method.

Using TestNG XML Suite

You can also parameterize tests using TestNG XML suite files. Here's an example:


  <suite name="LoginTestSuite">

    <test name="LoginTest">
      <parameter name="username" value="user1"/>
      <parameter name="password" value="password1"/>

      <classes>
        <class name="com.example.LoginTest"/>
      </classes>

    </test>
  </suite>

In this XML suite file, you can specify parameters for the test method using the <parameter> tags. TestNG will inject these parameter values into the test method when it is executed.

These are just a few ways you can leverage parameterization in TestNG to write more efficient and flexible test suites.

Assertions in Testing

Assertions are statements that validate the expected behavior of your application during testing. They help ensure that the actual outcome matches the expected outcome.

Hard Assertions

Hard assertions halt the test execution immediately upon failure. They are the traditional way of validating test results.

Common methods of hard assertions include:

assertEquals: Compares two values for equality.
assertTrue: Asserts that a condition is true.
assertFalse: Asserts that a condition is false.
assertNotNull: Asserts that an object is not null.
assertNull: Asserts that an object is null.
assertArrayEquals: Asserts that two arrays are equal.
And more...

Examples of Hard Assertions:


  // Using assertEquals
  assertEquals("Expected", actual);
  
  // Using assertTrue
  assertTrue(condition);
  
  // Using assertFalse
  assertFalse(condition);
  
  // Using assertNotNull
  assertNotNull(object);
  
  // Using assertNull
  assertNull(object);
  
  // Using assertArrayEquals
  assertArrayEquals(expectedArray, actualArray);

Soft Assertions

Soft assertions allow the test execution to continue even after a failure, collecting all failures and reporting them at the end of the test.

Common methods of soft assertions include:

assertAll: Verifies all collected assertions.
assertTrue: Asserts that a condition is true.
assertFalse: Asserts that a condition is false.
assertNotNull: Asserts that an object is not null.
assertNull: Asserts that an object is null.
assertThat: Allows for more flexible assertions using Hamcrest matchers.
And more...

Examples of Soft Assertions:


  // Creating a soft assertion
  SoftAssertions softAssert = new SoftAssertions();
  
  // Using assertTrue
  softAssert.assertTrue(condition1);
  softAssert.assertTrue(condition2);
  
  // Using assertFalse
  softAssert.assertFalse(condition3);
  
  // Using assertNotNull
  softAssert.assertNotNull(object);
  
  // Using assertNull
  softAssert.assertNull(object);
  
  // Using assertThat (with Hamcrest matchers)
  softAssert.assertThat(actualValue, Matchers.equalTo(expectedValue));
  
  // Verifying all assertions
  softAssert.assertAll();

These are some of the common methods used in both hard and soft assertions to validate the behavior of your application during testing.

Assignment: Real-Time Scenario

For this assignment, you are tasked with writing test cases for a login feature of a web application. The login page should validate the following scenarios:

Scenario 1: Successful Login

Verify that a user can successfully log in with valid credentials.

Test Steps:

Open the login page.
Enter valid username and password.
Click the login button.
Verify that the user is redirected to the home page.

Assertions to Use:

Hard Assertion: Verify that the page title of the home page is correct.
Soft Assertion: Verify that the user's name is displayed on the home page.

Scenario 2: Invalid Login

Verify that an error message is displayed when a user enters invalid credentials.

Test Steps:

Open the login page.
Enter invalid username and password.
Click the login button.
Verify that an error message is displayed.

Assertions to Use:

Hard Assertion: Verify that the error message is displayed.
Soft Assertion: Verify that the error message text is correct.

Once you have written the test cases, execute them and ensure that all assertions pass.

Annotation Configuration in TestNG

Annotation configuration in TestNG refers to the use of special annotations to control the behavior of test cases. These annotations are used to set up the test environment, define test methods, and manage test execution flow. TestNG provides a variety of annotations that allow testers to specify actions to be taken before or after certain events in the test lifecycle.

Annotations like @BeforeSuite, @BeforeTest, @BeforeClass, @BeforeMethod, @Test, @AfterMethod, @AfterClass, @AfterTest, and @AfterSuite play a crucial role in configuring and orchestrating test execution.

@BeforeSuite: Method annotated with @BeforeSuite is executed before the entire test suite starts. It is typically used for setting up configurations or resources required for the entire suite.
@BeforeTest: Method annotated with @BeforeTest is executed before any test method belonging to a specific <test> tag in the testng.xml file. It's commonly used for setting up configurations or resources required for a specific test.
@BeforeClass: Method annotated with @BeforeClass is executed before the first test method in the current class is invoked. It's typically used for class-level setup operations.
@BeforeMethod: Method annotated with @BeforeMethod is executed before each test method in the current class. It's often used for setting up preconditions for individual test methods.
@Test: Method annotated with @Test denotes a test method. TestNG executes methods annotated with @Test as test cases.
@AfterMethod: Method annotated with @AfterMethod is executed after each test method in the current class. It's useful for performing cleanup tasks or releasing resources acquired during test execution.
@AfterClass: Method annotated with @AfterClass is executed after all test methods in the current class have been run. It's useful for performing cleanup tasks specific to the class.
@AfterTest: Method annotated with @AfterTest is executed after all the test methods belonging to a specific <test> tag have been run. It's useful for performing cleanup tasks or releasing resources acquired during the test.
@AfterSuite: Method annotated with @AfterSuite is executed after the entire test suite has completed. It's typically used for performing global cleanup tasks or releasing resources acquired during the test suite execution.

Now, let's see some examples of using these annotations in Java:

import org.testng.annotations.*;

public class TestNGAnnotationsExample {

    @BeforeSuite
    public void beforeSuite() {
        System.out.println("Before Suite");
        // Setup actions for the entire test suite
    }

    @BeforeTest
    public void beforeTest() {
        System.out.println("Before Test");
        // Setup actions for the test
    }

    @BeforeClass
    public void beforeClass() {
        System.out.println("Before Class");
        // Setup actions for the class
    }

    @BeforeMethod
    public void beforeMethod() {
        System.out.println("Before Method");
        // Setup actions for each test method
    }

    @Test
    public void testMethod() {
        System.out.println("Test Method");
        // Test logic
    }

    @AfterMethod
    public void afterMethod() {
        System.out.println("After Method");
        // Teardown actions after each test method
    }

    @AfterClass
    public void afterClass() {
        System.out.println("After Class");
        // Teardown actions after all test methods in the class
    }

    @AfterTest
    public void afterTest() {
        System.out.println("After Test");
        // Teardown actions after all test methods in the test
    }

    @AfterSuite
    public void afterSuite() {
        System.out.println("After Suite");
        // Teardown actions after the entire test suite
    }
}

Using TestNG Annotations in Real-Time Scenario

Let's consider a real-time scenario where we need to automate the following test flow for a web application, like Demo Web Shop:

Launch the browser
Register an account on the website
Login to the website using the registered account
Add a fictional type book under the books section to the shopping cart
Verify the book is added to the shopping cart
Logout from the website

We can use TestNG annotations to organize and execute this test flow efficiently. Here's how we can achieve this:

        import org.openqa.selenium.*;
        import org.openqa.selenium.chrome.ChromeDriver;
        import org.testng.annotations.*;
        import static org.testng.Assert.*;
        
        public class WebShopTest {
        
            WebDriver driver;
        
            @BeforeTest
            public void setUp() {
                // Launch the browser
                System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
                driver = new ChromeDriver();
                driver.manage().window().maximize();
            }
        
            @Test(priority = 1)
            public void registerAccount() {
                // Navigate to the registration page
                driver.get("https://demowebshop.tricentis.com/register");
        
                // Perform registration steps
                // ...
        
                // Assert registration success
                WebElement successMessage = driver.findElement(By.xpath("//div[@class='result' and contains(text(),'Your registration completed')]"));
                assertTrue(successMessage.isDisplayed(), "Registration was not successful");
            }
        
            @Test(priority = 2)
            public void login() {
                // Navigate to the login page
                driver.get("https://demowebshop.tricentis.com/login");
        
                // Perform login steps
                // ...
        
                // Assert login success
                WebElement loggedInUser = driver.findElement(By.xpath("//a[@class='account']"));
                assertEquals(loggedInUser.getText(), "My account", "Login was not successful");
            }
        
            @Test(priority = 3)
            public void addBookToCart() {
                // Navigate to the books section
                driver.get("https://demowebshop.tricentis.com/books");
        
                // Add book to the shopping cart
                // ...
        
                // Assert book added to the cart
                WebElement cartQuantity = driver.findElement(By.xpath("//span[@class='cart-qty' and text()='1']"));
                assertNotNull(cartQuantity, "Book was not added to the cart");
            }
        
            @AfterTest
            public void tearDown() {
                // Logout from the website
                // ...
        
                // Close the browser
                driver.quit();
            }
        }

In this example, we've used TestNG annotations to handle the setup, execution, and teardown of the test flow. Each annotated method represents a step in the test flow, making the test automation code organized, readable, and easy to maintain.

Assignments: Testing Scenarios on Demo Web Shop

Below are the assignments for testing scenarios on the Demo Web Shop webpage. Students are required to verify the following functionalities:

Verify that the user is able to register on the website.
Verify that the user is able to login, add a product to the cart, and logout successfully.
Verify that the user is able to enter an email id in the newsletter section and click on the subscribe button.
Verify that the user is able to vote and submit in the community poll.
Verify that the user is able to click on the shopping cart link.
Verify that the user is able to add a product to the cart.
Verify that the user is able to add a product to the wishlist.

Students should perform thorough testing for each scenario, ensuring that all functionalities work as expected and any potential issues or bugs are identified and documented.

Using reporter.log in TestNG

In TestNG, reporter.log is a method provided by the TestNG framework to log messages during test execution. It allows testers to add custom log messages to the test reports generated by TestNG, providing additional context and insights into the test execution process.

The reporter.log method is particularly useful for logging information that may be helpful for debugging, troubleshooting, or providing additional details about test execution. These log messages are typically included in the HTML test reports generated by TestNG, making it easier for stakeholders to analyze test results.

Here's how you can use reporter.log method in your TestNG tests:

import org.testng.Reporter;
import org.testng.annotations.Test;

public class ReporterLogExample {

    @Test
    public void testWithReporterLog() {
        // Perform test steps
        // ...

        // Log custom message using reporter.log
        Reporter.log("This is a custom log message for test reporting");
    }
}

In the above example, the Reporter.log method is used to log a custom message during the test execution. This message will be included in the HTML test report generated by TestNG, providing additional information about the test execution flow.

Additionally, you can use parameters to control the logging level and format of the log message. For example:

Reporter.log("This is a custom log message with priority", 2, true);

The above statement logs a custom message with a priority level of 2 (INFO) and specifies that the message should be displayed in the HTML test report.

Overall, reporter.log is a powerful feature in TestNG that enables testers to enhance test reports with custom log messages, providing better visibility and understanding of test execution results.

Understanding TestNG Reports: Accessing and Interpreting Test Results

TestNG generates comprehensive reports after test execution, providing valuable insights into the test results, including pass/fail status, execution time, and any errors or exceptions encountered during testing. Understanding these reports is essential for analyzing test outcomes and identifying areas for improvement.

Accessing TestNG Reports

TestNG generates reports in various formats, including HTML, XML, and PDF. These reports are typically saved in a designated directory within your project structure. Depending on your TestNG configuration, you can access the reports directly from your IDE, or you may need to navigate to the output directory manually.

The default location for TestNG reports is the "test-output" directory in your project root. Inside this directory, you'll find subdirectories containing HTML reports, XML files, and other artifacts generated during test execution.

Interpreting TestNG Reports

TestNG reports provide detailed information about the test execution, including the following key aspects:

1. Test Results Overview

The main page of the HTML report provides an overview of test results, including the total number of tests executed, the number of passed and failed tests, and the overall pass percentage. This summary gives you a quick glance at the overall test outcome.

2. Detailed Test Results

TestNG reports provide detailed information about each test method, including its name, status (pass/fail), execution time, and any associated exceptions or errors. You can drill down into individual test results to investigate failures and identify the root cause of any issues.

3. Suite Execution Details

TestNG reports also include information about test suite execution, such as the start and end time of the test run, the total duration of execution, and any parameters or groups associated with the test suites. This information helps in understanding the overall test execution process and identifying any performance bottlenecks.

Example:

Here's an example of a TestNG HTML report and its components:

In the example above, you can see the test results overview section, detailed test results for individual test methods, and suite execution details. Use these components to analyze test outcomes and gain insights into the effectiveness of your test suite.

TestNG Interview Questions

What is TestNG?
What are the advantages of using TestNG over JUnit?
How do you install TestNG in Eclipse?
What are the main annotations provided by TestNG?
Explain the difference between @Test and @BeforeTest annotations in TestNG.
What is the purpose of the @DataProvider annotation in TestNG?
Explain the usage of groups in TestNG.
How do you prioritize test methods in TestNG?
What is the purpose of the @Parameters annotation in TestNG?
How do you skip a test method in TestNG?
Explain the usage of listeners in TestNG.
What are the main types of listeners provided by TestNG?
How do you implement parallel execution of tests in TestNG?
Explain the purpose of dependency attribute in TestNG.
What is the difference between Hard and Soft assertions in TestNG?
How do you perform data-driven testing in TestNG?
Explain the usage of test groups in TestNG.xml.
What is the purpose of the and tags in TestNG.xml?
How do you generate HTML reports in TestNG?
What is the purpose of the preserve-order attribute in TestNG?

Multiple Choice Questions (MCQs)

Which annotation is used to define a test method in TestNG?

@Test
@TestMethod
@TestNG
@TestMethodNG

What is the purpose of the @BeforeMethod annotation in TestNG?

To execute a method before each test method
To execute a method before the test class
To execute a method before the test suite
To execute a method before the entire test run

Which annotation is used to define data provider methods in TestNG?

@DataProvider
@Data
@DataProviderMethod
@TestDataProvider

How do you skip a test method in TestNG?

Using @Skip annotation
Using throw new SkipException()
Using @Ignore annotation
Using skip=true attribute in @Test annotation

Which attribute is used to specify the priority of a test method in TestNG?

priority
order
sequence
precedence

How do you execute test methods in parallel in TestNG?

Using tag in testng.xml
Using @Parallel annotation
Using parallel=true attribute in @Test annotation
Using @RunParallel annotation

What is the purpose of the dependsOnMethods attribute in TestNG?

To specify dependencies between test methods
To specify dependencies between test groups
To specify dependencies between test classes
To specify dependencies between test suites

How do you generate HTML reports in TestNG?

Using TestNG default HTML reports
Using custom listeners
Using extent reports
Using Allure reports

What is the purpose of the tag in TestNG.xml?

To define a test suite
To define a test method
To define a test class
To define a test group