Software Testing Questions

While a test plan is being created, risks involved in testing the product are to be taken into consideration along with possibility of their occurrence and the damage they may cause along with solutions; if any. Detailed study of this is called as Risk Analysis.

Some of the risks could be:

New Hardware 

New Technology 

New Automation Tool 

Sequence of code delivery 

Availability of application test resources 

Identify and describe the risk magnitude indicators: High, Medium and Low

High magnitude means the effect of the risk would be very high and non-tolerable. Company may face severe loss and its reputation is at risk. Must be tested.

Medium: tolerable but not desirable. Company may suffer financially but there is limited liability or loss of reputation. Should be tested.

Low: tolerable. Little or no external exposure. Little or no financial loss. Company’s reputation unaffected. Might be tested.

Three perspectives of Risk Assessment: Effect, Cause and Likelihood.

To assess risk by Effect, identify a condition, event or action and try to determine its impact.

To asses risk by Cause is opposite of by Effect. Begin by stating an undesirable event or condition and identify the set of events that could have permitted the condition to exist.

To asses risk by Likelihood is to determine the probability that a requirement will not be satisfied.

The biggest concern is the cost incurred for test automation. Also, often s/w test automation requires skilled personnel and authentic test automation tools. Purchasing license of such tools is a costly affair.

Also the time required for test automation is more. Automation often requires recording the sequence of operations you need to perform in order to execute a particular test case and comparing the final o/p with the expected o/p. Functions/macros need also be written sometimes for some test cases. All this could be a time consuming job. 

Configuration management is not concerned with the choice of the hardware configuration for an application.

Configuration management is concerned with the development of procedures and standards for cost-effective managing and controlling charges in an evolving s/w system.

It includes

1)software versions maintenance

2)controlling changes in documentation

3)controlling changes to the source code

4) management activities

a.) Quality Assurance

Quality assurance is an activity that establishes and evaluates the processes that produce products. Quality assurance would measure processes to identify weaknesses and then correct them to continually improve the process: Eg. Code review.

b.) Quality Control

Quality control is a process in which the product's quality is compared with standards and corrective actions taken if required. Quality control activities focus on identifying defects in the actual products produced. Eg. Testing.

c.) Test Case

A test case is a unit level document describing the inputs, steps of execution and the expected result of each test condition for every requirement from the BRD. Testers determine whether the application is working correctly or not based on the test case that is being executed. A test case is marked as "Pass" if the application works as expected and is marked as "Fail" if otherwise. Test cases also aide in generating test status metrics.

d.) Test Condition

Test condition is the condition on which the test execution evaluates to pass or fail. Eg of a test condition: Set Password field should accept a mix of Alphanumeric and Special characters on mandatory basis and throw error message otherwise.

e.) Test Script

Test script is the set of instructions performed on the application to verify its functions.

f.) Test Data

Test data is the input data used to test a particular functionality. Eg: Username and password to test a login screen.

g.) Test Bed

Test bed is the environment on which the testing is done and includes the software and hardware set up that is required for testing the application.

Capability Maturity Model (CMM) is a model of 5 levels of process 'maturity' that determine effectiveness in delivering quality software. The 5 levels of CMM are described as below:

Level 1: Initial: - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Very few or none of the processes are in place.

Level 2: Repeatable: - Software Project Tracking, Requirements Management, Realistic Planning and Configuration Management Processes are in place; successful practices can be repeated.

Level 3: Defined: - Standard Software Development and Maintenance Processes are integrated throughout an organization. A Software Engineering Process Group is in place to oversee software processes, and training programs are used to ensure understanding and compliance.

Level 4: Managed: - Metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.

Level 5: Optimizing: - The focus is on continuous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.

Any organization can start from any level, but its motto is to reach level 5; where the focus is continuous process improvement. By doing this, high quality s/w delivery is assured.

From this, we can see that the whole essence of CMM or CMMI is to produce quality software. It targets the whole organizational practices (or processes), which are believed to be the best across industries. Testing is part of Quality Assurance. CMM levels play an important role in an organization’s Quality Assurance effort. Thus testing plays an important role in determining CMM level.

Key Challenges of s/w testing:

1. Testing considered late in project

2. Requirements not testable

3. Integration is done after all components have been developed: This might result into full testing not being covered.

4. Complete testing is not possible

The word assurance means ‘guarantee’. So the Quality Assurance Group’s role is to guarantee that the product is of high quality. They examine the overall s/w development process and their responsibility is to create and enforce standards and methods to improve it with the goal of preventing bugs from ever occurring.