Top 10 differences between Alpha and Beta Testing

Alpha Testing	Beta Testing
It is conducted at developer’s site by developer or tester or end user	It is conducted by the end user (customer) or third party testers know as Beta testers at their site
Developer is performed in presence of developers	It is performed in absence of developers
It is conducted in Virtual environment	It is conducted in Real time environment
It is a combination of both white box and Black box testing	It is a completely a Black box testing
It is performed within the developing organization	It is performed outside the developing organization
It is known as User Acceptance Testing (UAT) at developers site	It is known as User Acceptance Testing (UAT) at customers site or Field Testing
It is not open to market and public	It is always open to market and public
It is the final testing within the development company before it is delivered to the customer	The application is tested for the first time within the customer’s environment when it is delivered and deployed
Any defects found during this testing, the same can be rectified immediately as the application is not yet delivered to customer	Immediate defect rectification is not possible

Positive and Negative testing

Test Data :

It is a data or value used by testers to perform testing on the application. In most of the testing projects, test data is provided by the client. If it is not provided by the client then tester needs to prepare the test data for testing.

Some projects might have a test data management team for test data preparation and updating. Test data can be either positive or negative.

Positive testing / test data:

Testing performed on the application or software using positive set of test data.In this testing approach, positive or valid values entered for input values and validations are performed.

e.g.  Suppose you are testing an online registration form, which has a field Contact number. There is a business requirement ‘Contact number field should accept 10 digit values staring with 9.’ For Positive testing user needs to enter the value for Contact number in the format of 9**********. 

Negative testing / test data:

Testing performed on the application or software using negative set of test data.In this testing approach, negative or invalid values entered for input values and validations are performed.

e.g.  To test business requirement ‘Contact number field should accept 10 digit values staring with 9’ for Negative testing user needs to enter the 10 digit value for Contact number starting with value other than 9 such as 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or user should enter 11 digit value starting with 9.

Requirement Traceability Matrix

Requirement Traceability Matrix (RTM) is a document that traces and maps user requirements with test cases or test scripts and check if all the requirements are addressed, implemented and tested. Requirement ids from requirement specification document are mapped to the test case ids. In traceability, the relationship of requirements to test cases can be one-to-one, one-to-many or many-to-one. Traceability requires unique identifiers for requirements and test cases.

Requirement Traceability Matrix document is prepared to ensure clients that the coverage done is complete as end to end. It consists of Requirement/ Base line document, Reference number, Test case or Conditions, and Defect/Bud id.

Using Requirement Traceability Matrix document the one can track the Requirement based on the Defect id.

Disadvantages of not using Requirement Traceability Matrix [some possible impact]:

• Loss track of any new or changed requirements
• Poor or unknown test coverage, more defects found in production
• It will lead to miss some bugs earlier test cycles which may arise later in test cycles
• Difficult project planning and tracking
• Misunderstanding between different teams over project dependencies, delay etc

Benefits of using Requirement Traceability Matrix:

• Make obvious to client that the software or application is being developed as per the requirements
• To make sure that all the requirements included in test cases
• To make sure that developers are not creating features that no one has requested or there is no such requirement specified by client in requirement specification document
• Easy to identify the missing functionalities
• If there is a change request for a requirement, then we can easily find out which test cases needs to update
• The completed system may have “Extra” functionalities that may have not been specified in the design document, resulting in wastage of manpower, time and effort.

Types of Requirement Traceability Matrix:

• Forward Traceability- Mapping of Requirements to Test cases
• Backward Traceability- Mapping to Test cases to Requirements
• Bi-Directional Traceability- A good traceability matrix is the reference from test cases to basis documentation and vice versa.

Tools used for Traceability Matrix:

MS Excel, HP Test Director, HP Quality Center, Rational Clear Ques

Top 10 differences between Black box and White box Testing

Black box Testing	White box Testing
Software testing method performed without the knowledge of internal code or structure	Software testing method with the knowledge of internal code or structure
It is carried out by software testers	It is carried out by developers
Implementation knowledge of the software is not required	Implementation knowledge of the software is required
Programming language such knowledge C,C++, Java is not required	Programming language knowledge such as C, C++, Java is required
It is performed at higher levels of testing E.g. System, User acceptance testing	It is performed at lower levels of testing E.g. Unit testing, Integration testing
Black box testing means functional or external testing	White box testing means structural or interior testing
Basic for test cases is Requirement Specification document	Basic for test cases is Detailed Design document
It mainly focuses on the functionality of the software or application	It mainly focuses on the code or program of the software
It focuses on What software does?	It focuses on How software works or performs?
The objective of testing is what the module or software is intended to do?	The objective of testing is How module or software works?
It only covers the testing of input values and desired output values	It covers the actual calculation logic used to calculate the desired output values

User Acceptance Testing

 User Acceptance Testing (UAT) is the testing of application from end users perspective. UAT can be done by customer or can done by testing team on behalf of the customer.

User Acceptance Testing is a final level of testing in which once can perform testing on the application within the presence of customer.

User Acceptance Testing is a black box testing technique. It plays a crucial role in project or software delivery to client. After successful completion of User Acceptance Testing, software or application is deployed to customer.

Entry Criteria for UAT:

System or System Integration testing of modules should be completed successfully and all the defects encountered should be closed.

Exit criteria for UAT:

After successful completion of user acceptance testing the technical team will deploy the same piece of code into the production environment.

There are two types or approaches of User Acceptance testing:

Alpha Testing:

It is conducted at developer’s site by end-users and is conducted in a controlled environment.

It is the final testing within the development company before it is delivered to the customer.

If any defects are found during this testing, the same can be rectified immediately as the application is not yet delivered to customer.

Beta Testing:

It is conducted at customers sites in customers environment. Developers are not generally present whereas in Alpha testing developer’s presence is there.The application is tested for the first time within the customer’s environment when it is delivered and deployed.

Beta testing is done by customer or third party testers know as Beta testers.

Beta test is a live application of the software in an environment that cannot be controlled by developer. The end-user keeps record of all problems that are encountered during this phase of testing and reports these to the developers at regular intervals. The developers make modifications on these problems and then prepare for release to the entire customer base.

The disadvantage of beta testing is that if there are any defects immediately rectification is not possible.

System Testing

System Testing is a crucial level of testing in which once the entire application is developed it is deployed into the system test environment. It is considered as full pledge testing which includes or covers both functional testing and non-functional testing such as GUI testing, load testing, stress testing, performance testing, security testing etc.

System testing is a black box testing technique and is always done by testing engineers or QA’s. During System Testing phase testing team will be involved to perform testing on the application by executing the test cases considering the entire application as single unit.

Entry Criteria for System testing:

An integration testing of modules should be completed successfully and all the defects encountered should be closed.

Exit criteria for System testing:

All planned test cases should be executed and all encountered defects should be closed.After successful completion of System testing the technical team will deploy the same piece of code into UAT environment.

System Testing

 Example of System testing:

Testing of online banks internet banking portal which includes various modules like login, balance inquiry, statement inquiry etc and integration modules like call to third party transfer screen, payment gateway etc as single unit.

How to calculate Defect Removal Efficiency

Defect Removal Efficiency  is the ratio of Total number of Pre-production defects to the sum of Total number of Pre-production Defects and Total number of Post-production Defects. It is used to derive the quality of product delivered to customer. Higher the value of Defect Removal Efficiency  greater is the quality of product.

Defect Removal Efficiency is calculated at project or stage level or both. It is measured in percentage (%).

How to calculate Defect Removal Efficiency :

Defect Removal Efficiency = (Total number of Pre-production defects) / (Total number of Pre-production defects + Total number of Post-production defects) * 100

Example:

Total number of Pre-production defects	Total number of Post-production defects	Defect Removal Efficiency %
100	5	92.23

Defect Removal Efficiency  = 100 / (100+5) * 100

                                                    = 95.23 %

How to calculate Defect Leakage ?

Defect Leakage metric is the ratio of Number of defects attributed to a stage but only captured in subsequent stages to the sum of Total number of defects captured in that stage and Total Number of defects attributed to a stage but only captured in subsequent stages. It is used to determine the % of defect leaked to subsequent stages. Lower the value to Defect Leakage better is the quality of testing.

Defect Leakage is calculated at overall project or stage level or both. It is measured in percentage (%).

How to calculate the Defect Leakage:

Defect Leakage=(Number of defects attributed to a SIT but captured in UAT) / (Total number of defects captured SIT + Total Number of defects attributed to SIT but captured in UAT)

Example:

Total number of defects captured in SIT	Number of defects attributed SIT but captured in UAT	Defect Leakage
72	8	10

Defect Leakage= 8 / (72+8) * 100

                           = 10 %

Note: In above example the current stage is considered as SIT (System Integration Testing) and subsequent stage is considered as UAT (User Acceptance Testing). The user needs to understand the current and subsequent stage and accordingly need to modify the formula.

How to calculate Defect Density ?

Defect Density metric is the ratio of Total number of defects found in the application to the Total number of test cases executed for the release. It is calculated at the test execution phase. Application Defect Density metric is used to check the stability of the application.

How to calculate the Defect Density:

Defect Density = (Total number of defects found in the application / Total Number of test cases executed for the release)

Example:

Total number of defects found in the application	Total Number of test cases executed for the release	Defect Density
16	200	0.08

Defect Density = 16 / 200

                         = 0.08 / Test Case