Software Testing Stuff

Performance Testing vs Load Testing vs Stress Testing - Examples

2011-09-13T08:04:00.001-07:00

Performance testing - It is performed to evaluate the performance of components of a particular system in a specific situation. It very wide term. It includes: Load Testing, Stress Testing, capacity testing, volume testing, endurance testing, spike testing, scalability testing and reliability testing etc. This type of testing generally does not give pass or fail. It is basically done to set the benchmark & standard of the application against Concurrency / Throughput, Server response time, Latency, Render response time etc. In other words, you can say it is technical & formal evaluation for responsiveness, speed, scalability and stability characteristics.

Load Testing is subset of performance testing. It is done by constantly increasing the load on the application under test till the time it reaches the threshold limit. The main goal of load testing is to identify the upper limit of the system in terms of database, hardware and network etc. The common goal of doing the load testing is to set the SLAs for the application. Example of load testing can be:

Running multiple applications on a computer simultaneously - starting with one application, then start second application, then third and so on....Now see the performance of your computer.

Endurance test is also a part of load testing which used to calculate metrics like Mean Time Between Failure and Mean Time to Failure.
Load Testing helps to determine:

Throughput
Peak Production Load
Adequacy of H/W environment
Load balancing requirements
How many users application can handle with optimal performance results
How many users hardware can handle with optimal performance results

Stress testing - It is done to evaluate the application's behaviour beyond normal or peak load conditions. It is basically testing the functionality of the application under high loads. Normally these are related to synchronization issues, memory leaks or race conditions etc. Some testing experts also call it as fatigue testing. Sometimes, it becomes difficult to set up a controlled environment before running the test. Example of Stress testing is:

A banking application can take a maximum user load of 20000 concurrent users. Increase the load to 21000 and do some transaction like deposit or withdraw. As soon as you did the transaction, banking application server database will sync with ATM database server. Now check with the user load of 21000 does this sync happened successfully. Now repeat the same test with 22000 thousand concurrent users and so on.

Spike test is also a part of stress testing which is performed when application is loaded with heavy loads repeatedly and increase beyond production operations for short duration.
Stress Testing helps to determine:

Errors in slowness & at peak user loads
Any security loop holes with over loads
How the hardware reacts with over loads
Data corruption issues at over loads

SDLC vs STLC–Differences & Similarities Revisited

2011-09-12T09:42:00.001-07:00

Few days back, I was taking some interviews. I surprised to see some candidates were not able to explain simple concepts like SDLC & STLC. Everybody, was giving me very standards answers. SDLC - a systematic approach to develop a software and STLC - process of testing a software in a systematic and planned way. No one explains me logical Differences & Similarities between these two.

My question is very simple – How you can relate six phases Requirement, Design, Development, Testing, Deployment & Maintenance to Software Testing Life Cycle?

Let’s see how I related these six phases with STLC

S. No.	Phase	SDLC - Software Development Life cycle	STLC - Software Test Life Cycle
1	Requirements Gathering	Requirements gathering is done by business analyst. Development team analyse the requirements from the design, architecture & coding perspective.	Testing team also review & analyse the requirements. Testing team identifies the testing requirements like what types of testing will be required and review the requirements for logical functional relationship between various features / modules, so that any gaps can be caught at an early stage.
2	Design	Technical architect works for the high level & low design of the software. Business analyst works for the UI design of the application	Here, test architect generally the test lead/manager, does the test planning, identify high level testing points. Basically, requirement detailing is done in this phase.
3	Coding or development	Development team does the actual coding based on the designed architecture.	Testing team write the detailed test cases.
4	Testing	In SDLC, actual testing is carried out in this phase. It includes unit testing, integration testing & system testing etc..	Test Execution and bug reporting, manual testing, automation testing is done, defects found are reported. Re-testing and regression testing is also done in this phase. But, I don't agree with this statement. So, if I want to relate the testing phase with STLC, I would say it it is testing of test cases & test plans i.e. is basically review of test cases, test scenarios etc..
5	Deployment	Application is deployed on production environment for real end users.	Final testing and implementation is done is this phase and final test report is prepared. For this statement as well, I don't agree. For software / application deployment is basically, when it is installed for real use. So, this way, STLC, deployment would be when test when test cases getting used i.e. execution of test cases.
6	Maintenance	Basically, it includes, post production / deployment support & enhancements.	Most of people say - Maintenance testing is carried out in this phase. My definition for this is - updation & maintenance of test plans, test case required for the testing of support requests & enhancements as a part of maintenance.

There are two sides of this coin:

Testing within the software development (Here, the old definition of STLC may be applicable)
Consider testing as a independent sub project of a software development project (Here, I believe, old definition no longer sustains)

How to prioritize bugs numerically - An effective way of prioritization of bugs

2011-09-09T02:47:00.001-07:00

You must have seen developers & testers fighting with each others on priority of bugs. Testers log the bug as high priority or critical and sometimes, developers surprised to see the priority. Most of the times, both developer & tester will not agree on the priority set by each other. I've seen sometimes, even product manager / project manager / business analyst are not able to prioritize the defects or bugs EFFECTIVELY. Even in many process oriented organizations, it becomes a challenge.

Since, these bugs are risk to the application and project delivery, so, let's apply basic principle of risk management here. i.e. Risk Value = Probability x Impact

In case of bugs, Priority = Severity Value x Priority Value

Now, you can use this priority number to actually prioritize your bugs and fix them in the required order. Lets see the entire process in the below example:

1. Dear Testers - Please do not set priority of the bug. Do it only if you are interacting with the customer and thoroughly understand the business flow implemented in the application. Priority should be set based on the criticality of the business / functional requirement. Also, set severity only after doing a through impact analysis. Best is if we can have bug triage meeting at this stage, so that everyone can provide their inputs to identify the correct severity level.

2. Set four severity levels: High, Medium, Low, Lowest. Set their values as 4, 3, 2, 1. 4 as highest and 1 as lowest.

Severity	Severity Value	Example
High	4	Data loss, missing feature, security violation
Medium	3	Feature is missing but workaround is there, if high profiles users like management will get an impact, then it would be of high severity 4
Low	2	It is impacting minor things, but, user is able to do the work
Lowest	1	A cosmetic error, spelling mistake, typo etc..

3. Set four priority levels: Critical, High, Medium, Low. Set them values as 4, 3, 2, 1. 4 as Critical and 1 as low. Please ask the business analyst to set the priority.

Priority	Priority Value	Example
Critical	4	Application is getting started, Critical business flow got stuck, Data loss, business loss, loss of end user, a security violation, system is very unstable
High	3	Feature is missing which is not critical for only very few users that do not have much impact on business, if high profiles users like management will get an impact, then it would be of high severity 4
Medium	2	Feature is missing but workaround is there. If users from management group are getting impacted, then it can be of priority 3 or 4. It can also be some feature of the functionality is not working
Lowest	1	A cosmetic error, spelling mistake, typo. Here, one important point to note is that even a spelling mistake or typo can also be of high priority or critical. e.g. If there is a spelling mistake of name of a person from senior management, then it may have high priority.

Priority can be calculated as

Bug Id	Bug Description	Severity Value	Priority Value	Priority
Bug_1	Bug Description 1	2	4	8
Bug_2	Bug Description 2	3	2	6
Bug_3	Bug Description 3	1	1	1
Bug_4	Bug Description 4	4	3	12
Bug_5	Bug Description 5	2	4	8

Now, the bug with highest priority should be fixed first. In this case bug “Bug_4” should be fixed first. Then Bug_1 and Bug_5 should be fixed and so on.

This way we will have a number defining the priority of the based on the two logical facts.

You do not need to follow this practice all the time. After development, initial during testing/bug fixing phase, you may skip this process. Eventually, at that time, we have to fix all the bugs.

When you see there is a production release in next 2-4 days or more or less depending upon the size of release and there are lot of open bugs, you may prioritize bugs based on this approach. Best is do this exercise 5-7 days before the final release. Also, I would suggest this kind of exercise for all customer reported bugs during UAT or post production release.

As a pilot, I've applied this technique to one of my project. Let's see how practical is this. Please share your views as well on this approach.

Cross Browser Testing Tools–Reduce Browser Compatibility Testing Effort

2011-09-05T08:21:00.001-07:00

Sometimes, testing on various browsers becomes a challenge for software test professionals & project teams. Running the test cases on all browsers makes the testing cost very high. Specially, it becomes a challenge when we do not have expert designers in the team or when we don’t have verification/validation phase at the time of screen design. This is the bad part. Now, let’s see what is good.

The best thing is that there are many FREE as well as paid cross browser compatible testing tools available in the market. On top of it, you can do your job with most of the FREE tools. If you have very specific requirements, then you may need to have a paid cross browser compatible testing tool. Lets have quick look on some of the best tools:

1. IE Tab: This is one of my favourite & best tools available for free. This is basically an add-on of Firefox & Chrome. With a single mouse click from within Firefox & Chrome, you can see how the webpage will view in Internet Explorer. It is very light.

2. Microsoft Super Preview: This is a free tool offered by Microsoft. It can help you to check the webpage on various versions of Internet Explorer. you can use it to test and debug the issues in layout. You can download it for free from Microsoft website.

3. Spoon Browser Sandbox: You can use this testing tool to test the web application on almost all major browsers like Firefox, Chrome & Opera. Initially, it supports IE as well, but for the last few months, its support for IE has been reduced.

4. Browsershots: Using this free browser compatibility testing tool, you can test the application on any platform & browser combinations. So, it is most widely used tool. Due to large combination of browsers & platforms, it take long time to display results.

5. IE Tester: Using this tool, you can test the web pages on various versions of IE on various Windows platforms like Windows Vista, Windows 7 & XP.

6. BrowserCam: This is a paid browser compatibility online testing tool. It’s trial usage allows you to test for 24 hours only with the screen limit of 200.

7. Cross Browser Testing: This is a perfect tool for testing the website for JavaScript, Ajax and Flash features on various browsers. It offers 1 week free trial. It is available @ http://crossbrowsertesting.com/

8. Cloud Testing: If you want to test your application’s browser compatibility on various browsers like IE, Firefox, Chrome, Opera, then this tool is for you.

Apart from these tools, there are few other tools like IE NetRenderer, Browsera, Adobe Browser Lab etc. By investing some time on RnD on these tools, you can save your huge effort with excellent Quality.

Difference between Smoke & Sanity Software Testing

2011-06-07T04:04:00.000-07:00

Smoke Testing: Software Testing done to ensure that whether the build can be accepted for through software testing or not. Basically, it is done to check the stability of the build received for software testing.

Sanity testing: After receiving a build with minor changes in the code or functionality, a subset of regression test cases are executed that to check whether it rectified the software bugs or issues and no other software bug is introduced by the changes. Sometimes, when multiple cycles of regression testing are executed, sanity testing of the software can be done at later cycles after through regression test cycles. If we are moving a build from staging / testing server to production server, sanity testing of the software application can be done to check that whether the build is sane enough to move to further at production server or not.

Difference between Smoke & Sanity Software Testing:

Smoke testing is a wide approach where all areas of the software application are tested without getting into too deep. However, a sanity software testing is a narrow regression testing with a focus on one or a small set of areas of functionality of the software application.
The test cases for smoke testing of the software can be either manual or automated. However, a sanity test is generally without test scripts or test cases.
Smoke testing is done to ensure whether the main functions of the software application are working or not. During smoke testing of the software, we do not go into finer details. However, sanity testing is a cursory software testing type. It is done whenever a quick round of software testing can prove that the software application is functioning according to business / functional requirements.
Smoke testing of the software application is done to check whether the build can be accepted for through software testing. Sanity testing of the software is to ensure whether the requirements are met or not.

HP Quality Center - Defect/Bug Tracking

2011-06-06T01:37:00.000-07:00

Introduction:

HP quality center is web-based test management tool
QC is used to manage the application testing process
QC is used to define releases, specifying requirements, planning tests, executing tests, tracking defects, alerting on changes, and analyzing results
QC also shows you how to customize your project
Defect: Anything that is extra or missing or wrong in application is termed as defect
QC helps you to add defects to application when found and track the defect repair progression.
QC provides a central defect tracking system that can be used by testing and development teams to resolve defects

QC testing process includes 5 phases:

Specifying release
Specifying requirements
Planning Tests
Running Tests
Tracking Defects

Defect Modules:

QC Defect module provides complete system for logging, tracking, managing, and analyzing application defects
QC Defect tracking tools are organized into: Defects grid, Grid filter, Description, Attachments, History

Defect Logging: Steps to Log a defect

In Defect module, click New Defect
New defect dialog box contains data fields and multiple tabbed pages. This multiple tabbed pages are custom-defined by QC administrator for your project
You can also add attachments to provide more information about the defect. QC supports five types of attachments
Click Submit to save the defect to your project database

Defect Organizing - Grid filter provides two ways to organize defect grid:

Use entry boxes under each field heading in grid filter to select the criteria for filtering the data in the defects grid
Use the Filter dialog box to set a filter condition
To clear filter criteria, click the Clear Filter/Sort button

Defect Status:

New: Default status when defect is reported
Open: Indicates defect is assigned to review
Reopen: Indicates testing team reopened the defect which was closed
Fixed: Indicates defect is verified
Closed: Defect is closed and waiting fro approval by tester
Rejected: Defect is rejected , rationale fore rejecting defect to be provided

Defect association with other entities - Defects can be associated with the following entities:

Requirement
Test
Test set
Test instance
Test run
Test steps
Other Defect

This linkage of defects with entities enables to

Trace your defects from perspective of QC entities
Search defects that are related to specific QC entities
Link multiple entities of same type to the same defect

Defect Requirement:

Associating defects with test requirements will help to ensure consistency throughout the testing process
The defect-requirement association enables us to utilize the status of defects to determine whether requirements have been met
A requirement can be associated with more than one defect
Either a existing defect can be associated with the requirement or new defect can be added to requirement

Defect Test:

Association of defect with test helps to ensure defect traceability throughout the testing process. A defect may be indirectly linked to a test through other entities, such as a test instance, a test run, or a test step
Tests from Test plan module can be associated with defects that have been logged in the Defects module

Defect Matching:

Matching defects enables you to eliminate similar and duplicate defects in your project
Each time you add a new defect, Quality Center stores lists of keywords from the Summary and Description fields
When you search for similar defects, keywords in these fields are matched against other defects
Note that keywords must be more than two characters and letter case does not affect your results

Updating the defects:

A defect can be regularly updated to record all the information about an issue.
A defect can be regularly updated to record decisions made as different individuals review the defect.
While updating the defect ,click Details to update specific data fields.
Click Attachments to attach files to defect.
Click Linked entities and then click the Defects tab to link a defect to another defect.
Click Others tab to link a defect to other entities such as a test or test set.
Click History to view the changes made to defect.

Favorite view: A favorite view is a view of a Quality Center window with the settings you applied to it. Steps to create favorite view in defects grid:

Defect module should be displayed
Define a filter to view defects, Click the Set Filter/Sort button. The Filter dialog box opens.
Click the Filter Condition box, Click the browse button. The Select Filter Condition dialog box opens. Click OK to close the Select Filter Condition dialog box. Click OK to apply your chosen filter.
Add a favorite view. In the Favorites menu, choose Add to Favorites. The Add Favorite dialog box opens.

Unit Testing - Best Practices & Techniques

2011-06-05T02:33:00.000-07:00

Unit testing is a software development process in which the smallest testable parts of an application, called units, are individually and independently scrutinized for proper operation. The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as expected. Each unit is tested separately before integrating them into modules to test the interfaces between modules.By means of effective Unit testing large percentage of defects are identified. Unit testing is performed by developers.

Each module that is developed by designers need to be tested individually to verify proper operation so that any faulty module can be fixed immediately rather than let it exist and then cause some major issue in the integration phase. Once all of the units in a program have been found to be working efficiently and without any bugs, larger components of the program can be evaluated by means of integration testing. Though Unit testing may be time consuming, tedious and requires thoroughness on the part of the development team, but in the long run it can avoid major pitfalls in the software.

Benefits of unit testing:

The modular approach during Unit testing eliminates the dependency on other modules during testing.
We can test parts of a project with out waiting for the other parts to be available.
Designers can identify and fix problem immediately, as the modules are best known to them. This helps in fixing multiple problems simultaneously.
Cost of fixing a defect identified during the early stages is less compared to that during later stage.
Debugging is simplified.
Structural coverage of code is higher.
Unit testing is more cost effective compared to the other stages of testing

Misconceptions about Unit Testing:

Integration Tests will Catch all the Bugs Anyway: This is one of the common misconceptions of designers. Complexity of the issue rises while it passes through various testing cycles and then when the bug is raised during the later stages, the resolution time will be high as the scope of the bug widens. It is better to weed off the crop before it poisons the whole farm.
Programmers Dilemma: Most designers believe Unit testing is not actually required as it is time consuming. They feel they are too good programmers and their software doesn’t need Unit tests. But in the real world, everyone makes mistakes. Real software systems are much more complex. Software behavior various in different environment and with different scenarios. Coding is not a one pass process. Enhancements to the code can be made only when we know the existing module is functioning as expected.
It Consumes Too Much Time: Developers are often in a hurry to complete their code and integrate it. Unit Testing is most often considered a useless activity, as they feel anyways the code will be tested by QA. There is no point in having a system which works but not exactly as it is supposed to function and is to be full of bugs. Practically, such an approach to development will often result in software which will not even run. The net result is that a lot of time will be spent tracking down relatively simple bugs which are wholly contained within particular units. Individually, such bugs may be trivial, but collectively they result in an excessive period of time integrating the software to produce a system which is unlikely to be reliable. This can also lead to failure to meet the required deadlines.

Unit Testing Best Practices:

Ensure each Unit Test case is independent of each other. As the software is prone to changes during the Unit Testing due to enhancements/changes to the requirements. Hence any given behavior should be specified in one and only one test. Otherwise if you later change that behavior, you’ll have to change multiple tests.
Test only one code at a time. It is always recommended to test each of the modules independently and not while all are chained together. Otherwise you will have lots of overlap between tests and changes to one unit may effect all other modules and cause the software to fail.
Name your unit tests clearly and consistently. Ensure that your test cases are easily readable so that anyone picking up Unit test cases can execute them without any issues. Ensure the test case nomenclature is consistent throughout.
Before changing a module interface or implementation, make sure that the module has test cases and that it passes its tests before changing the implementation. This way you can know that your changes didn't break anything.
Always ensure the bug identified during Unit Testing is fixed before moving it to the next phase.

Unit Testing Techniques: Structural, Functional & Error based Techniques

Structural Techniques: It is a White box testing technique that uses an internal perspective of the system to design test cases based on internal structure. It requires programming skills to identify all paths through the software. The tester chooses test case inputs to exercise paths through the code and determines the appropriate outputs. Major Structural techniques are:

Statement Testing: A test strategy in which each statement of a program is executed at least once.
Branch Testing: Testing in which all branches in the program source code are tested at least once.
Path Testing: Testing in which all paths in the program source code are tested at least once.
Condition Testing: Condition testing allows the programmer to determine the path through a program by selectively executing code based on the comparison of a value
Expression Testing: Testing in which the application is tested for different values of Regular Expression.

Functional testing techniques: These are Black box testing techniques which tests the functionality of the application. Some functionality testing techniques are:

Input domain testing: This testing technique concentrates on size and type of every input object in terms of boundary value analysis and Equivalence class.
Boundary Value: Boundary value analysis is a software testing design technique in which tests are designed to include representatives of boundary values.
Syntax checking: This is a technique which is used to check the Syntax of the application.
Equivalence Partitioning: This is a software testing technique that divides the input data of a software unit into partition of data from which test cases can be derived

Error based Techniques: The best person to know the defects in his code is the person who has designed it.
Few of the Error based techniques are:

Fault seeding techniques can be used so that known defects can be put into the code and tested until they are all found.
Mutation Testing: This is done by mutating certain statements in your source code and checking if your test code is able to find the errors. Mutation testing is very expensive to run, especially on very large applications.
Historical Test data: This technique calculates the priority of each test case using historical information from the previous executions of the test case.

Careful approach to Unit testing helps detecting many bugs at a stage of the software development where they can be corrected economically. It is a tedious process when bugs are detected and corrected at later stages of software development as fixing the bugs is difficult, time consuming and costly. Efficiency and quality are best served by testing software as early in the life cycle. Whenever any changes are made to the software we need to ensure regression testing is performed. Testing strategies like thorough unit testing, good management of the testing process, and appropriate use of tools helps in maximizing the effectiveness of testing effort. Effective unit testing is all part of developing a very high quality software product which can benefit the organization on a whole.

Bug or Incident or Defect Communication & Triage

2011-06-02T01:17:00.000-07:00

Test Manager/Lead has to take the responsibility of communicating the defects to the respective Dev teams and all other stake holders. The process should be defined in such a way as depending on the severity of the defect the SLA for turnaround times should exist.

The bug tracking & management tools should trigger an e-mail to the assigned team when the defect is assigned. If no tool is used Test leads will be sending e-mail communication to the Dev teams.

While communicating bugs, always remember:

Do not share the actual Defect sheets through emails. Create share location and share the location details. This will avoid different people working on different versions of sheets.
Make sure Dev team also have write access to the share point and allow them to add comments and change the status of the defect

Depending on the time lines and the number of defects raised, the schedule for Triage has to be defined. Ideal way is to have this meeting every end of the day. Discuss all the defects and decide the actions.

For an effective bug triage, always remember:

Set up the conference call/Meeting and invite all the intended recipients.
Share the list of defects and their details to the Triage participants well in advance of meeting.
Make sure one representation is available from all groups.
Prepare an MOM for the Triage and log all the actions
Discuss on Actions pending from previous meetings.
Discuss defects and create new actions and identify the responsible person for each action.
Share the MOM with all the stake holders.

Common Problems in Bug Tracking

2011-06-01T04:02:00.000-07:00

Working with different stake holders in a project, teams would be facing difficulty in keeping transparency on bugs because of following common problems:

Lack of process for bug logging: Typical scenario like testing & development teams are not well educated on severities & priorities to be used and their importance.
No clear communication to testing team on required fields as part of bug logging. Testers may skip to mention required data like reproducible steps or may skip to attached screen shots.
Lack of a standard bug tracking / logging template. If all team members will follow their own template to describe defects, it may lead to discrepancy at a later stage.
Team will not have a dedicated SPOC communicate defects to all the stake holders
Team will follow their own communication channel to communicate defects using phone/email. Sometimes they won’t even log defect after communicating issue to Dev and getting fix on the fly.
Team will not maintain appropriate statuses for defects. Even after they retest and close some times defect still shows as ‘Ready for Test’.
Improper Defect Triage/Communication Process: Triage process allows all stake holders to gather at one place (Physically or virtually) to discuss on all open defects and decide on action items. Without this call/meeting in regular intervals it will be difficult to have a common understanding on issues/defects or reasons for blocking of test execution.
Test team some time will not have control on Test environments and they would not know when the fix is deployed in to test environment. Sometimes Dev team adhocly performs certain actions on test environment and fixes the issue. Not having complete control on Test Environment would cause so many issues in ensuring quality of defect retest.

No Training on tool(s) to the team
No clear guidelines on the process to the team
No availability of tool till the execution starts
No planning about how many licenses required for the team
No Communication process defined
No Triage process defined

C Vuser Functions in LoadRunner

2011-05-31T07:39:00.000-07:00

In LoadRunner, you can add C Vuser functions to any Vuser script in order to enhance the script. VuGen generates only a few of the general Vuser functions while you record. If required, the remaining functions can be manually programmed into a script.

As per my knowledge, below is a list of general API functions for ANSI C scripts. It includes all protocols except for Java, VB, and GUI:

Transaction Functions:

1. lr_end_sub_transaction --> Marks the end of a sub-transaction for performance analysis.

2. lr_end_transaction --> Marks the end of a transaction.

3. lr_end_transaction_instance --> Marks the end of a transaction instance for performance analysis.

4. lr_fail_trans_with_error --> Sets the status of open transactions to LR_FAIL and sends an error message.

5. lr_get_trans_instance_duration --> Gets the duration of a transaction instance specified by its handle.

6. lr_get_trans_instance_wasted_time --> Gets the wasted time of a transaction instance by its handle.

7. lr_get_transaction_duration --> Gets the duration of a transaction by its name.

8. lr_get_transaction_think_time --> Gets the think time of a transaction by its name.

9. lr_get_transaction_wasted_time --> Gets the wasted time of a transaction by its name.

10. lr_resume_transaction --> Resumes collecting transaction data for performance analysis.

11. lr_resume_transaction_instance --> Resumes collecting transaction instance data for performance analysis.

12. lr_set_transaction_instance_status --> Sets the status of a transaction instance.

13. lr_set_transaction_status --> Sets the status of open transactions.

14. lr_set_transaction_status_by_name --> Sets the status of a transaction.

15. lr_start_sub_transaction --> Marks the beginning of a subtransaction.

16. lr_start_transaction --> Marks the beginning of a transaction.

17. lr_start_transaction_instance --> Starts a nested transaction specified by its parent’s handle.

18. lr_stop_transaction --> Stops the collection of transaction data.

19. lr_stop_transaction_instance --> Stops collecting data for a transaction specified by its handle.

20. lr_wasted_time --> Removes wasted time from all open transactions.

Command Line Parsing Functions:

1. lr_get_attrib_double --> Retrieves a double type variable used on the script command line.

2. lr_get_attrib_long --> Retrieves a long type variable used on the script command line.

3. lr_get_attrib_string --> Retrieves a string used on the script command line.

Informational Functions:

1. lr_user_data_point --> Records a user-defined data sample.

2. lr_whoami --> Returns information about a Vuser to the Vuser script. Not applicable for Application Management.

3. lr_get_host_name --> Returns the name of the host executing the Vuser script.

4. lr_get_master_host_name --> Returns the name of the machine running the LoadRunner Controller or Tuning Console. Not applicable for Application Management.

String Functions:

1. lr_eval_string --> Replaces a parameter with its current value.

2. lr_save_string --> Saves a null-terminated string to a parameter.

3. lr_save_var --> Saves a variable length string to a parameter.

4. lr_save_datetime --> Saves the current date and time to a parameter.

5. lr _advance_param --> Advances to the next available parameter.

6. lr _decrypt --> Decrypts an encoded string.

7. lr_eval_string_ext --> Retrieves a pointer to a buffer containing parameter data.

8. lr_eval_string_ext_free --> Frees the pointer allocated by lr_eval_string_ext.

9. lr_save_searched_string --> Searches for an occurrence of string in a buffer and saves a portion of the buffer, relative to the string occurrence, to a parameter.

Message Functions:

1. lr_debug_message --> Sends a debug message to the Output window or the Business Process Monitor log files.

2. lr_error_message --> Sends an error message to the Output window or the Business Process Monitor log files.

3. lr_get_debug_message --> Retrieves the current message class.

4. lr_log_message --> Sends a message to a log file.

5. lr_output_message --> Sends a message to the Output window or the Business Process Monitor log files.

6. lr_set_debug_message --> Sets a debug message class.

7. lr_vuser_status_message --> Generates and prints formatted output to the Controller or Console Vuser status area. Not applicable for Application Management.

8. lr_message --> Sends a message to the Vuser log and Output window or the Business Process Monitor log files.

Run-Time Functions

1. lr_load_dll --> Loads an external DLL.

2. lr_peek_events --> Indicates where a Vuser script can be paused.

3. lr_think_time --> Pauses script execution to emulate think time—the time a real user pauses to think between actions.

4. lr_continue_on_error --> Specifies an error handling method.

5. lr_rendezvous --> Sets a rendezvous point in a Vuser script. Not applicable for Application Management.

Also See: Other LoadRunner Tutorials

How to do System Testing

2011-05-31T01:26:00.000-07:00

Testing the software system or software application as a whole is referred to as System Testing of the software. System testing of the application is done on complete application software to evaluate software's overall compliance with the business / functional / end-user requirements. The system testing comes under black box software testing. So, the knowledge of internal design or structure or code is not required for this type of software testing.

In system testing a software test professional aims to detect defects or bugs both within the interfaces and also within the software as a whole. However, the during integration testing of the application or software, the software test professional aims to detect the bugs / defects between the individual units that are integrated together.

During system testing, the focus is on the software design, behavior and even the believed expectations of the customer. So, we can also refer the system testing phase of software testing as investigatory testing phase of the software development life cycle.

At what stage of SDLC the System Testing comes into picture:

After the integration of all components of the software being developed, the whole software system is rigorously tested to ensure that it meets the specified business, functional & non-functional requirements. System Testing is build on the unit testing and integration testing levels. Generally, a separate and dedicated team is responsible for system testing. And, system testing is performed on stagging server.

Why system testing is required:

It is the first level of software testing where the software / application is tested as a whole.
It is done to verify and validate the technical, business, functional and non-functional requirements of the software. It also includes the verification & validation of software application architecture.
System testing is done on stagging environment that closely resembles the production environment where the final software will be deployed.

Entry Criteria for System Testing:

Unit Testing must be completed
Integration Testing must be completed
Complete software system should be developed
A software testing environment that closely resembling the production environment must be available (stagging environment).

System Testing in seven steps:

Creation of System Test Plan
Creation of system test cases
Selection / creation of test data for system testing
Software Test Automation of execution of automated test cases (if required)
Execution of test cases
Bug fixing and regression testing
Repeat the software test cycle (if required on multiple environments)

Contents of a system test plan: The contents of a software system test plan may vary from organization to organization or project to project. It depends how we have created the software test strategy, project plan and master test plan of the project. However, the basic contents of a software system test plan should be:

- Scope
- Goals & Objective
- Area of focus (Critical areas)
- Deliverables
- System testing strategy
- Schedule
- Entry and exit criteria
- Suspension & resumption criteria for software testing
- Test Environment
- Assumptions
- Staffing and Training Plan
- Roles and Responsibilities
- Glossary

How to write system test cases: The system test cases are written in a similar way as we write functional test cases. However, while creating system test cases following two points needs to be kept in mind:

- System test cases must cover the use cases and scenarios
- They must validate the all types of requirements - technical, UI, functional, non-functional, performance etc.

As per Wikipedia, there are total of 24 types of testings that needs to be considered during system testing. These are:

GUI software testing, Usability testing, Performance testing, Compatibility testing, Error handling testing, Load testing, Volume testing, Stress testing, User help testing, Security testing, Scalability testing, Capacity testing, Sanity testing, Smoke testing, Exploratory testing, Ad hoc testing, Regression testing, Reliability testing, Recovery testing, Installation testing, Idem potency testing, Maintenance testing, Recovery testing and failover testing, Accessibility testing

The format of system test cases contains:

Test Case ID - a unique number
Test Suite Name
Tester - name of tester who execute of write test cases
Requirement - Requirement Id or brief description of the functionality / requirement
How to Test - Steps to follow for execution of the test case
Test Data - Input Data
Expected Result
Actual Result
Pass / Fail
Test Iteration

Keyword Driven Methodology

2011-05-24T10:24:00.000-07:00

It is a Test Creation Methodology which involves selection of keywords. Keywords Indicate Operations to be Performed and Meant for Structured Test Creation. It creates Structured Tests – Easy to Update & Maintain.

It is suitable for companies having technically expert users: For Maintaining Resource Frameworks and companies having less technically proficient users for for design & maintenance of automated test steps. The objective is clear division of automation tasks.

Keywords can be come from

Built in keywords from QTP itself
Our own user defined keywords

Resources and infrastructure required:

Shared object repositories
Function libraries
Recovery scenarios
Persons with thorough knowledge of application
Expertise of QTP Software

keyword driven methodology in QTP

Analyzing the application to find out the testing needs: Find out the Application’s development environment like Web, Java or .NET. Load the desired QTP add-ins and find out the business processes and functionality we want to test.
Setting up the object repositories: Decide how to divide our actions, build resources to be used by our tests, the most widely used resource is the shared object repository.
Creation of function libraries: Creation of function libraries containing functions which extend QTP functionality and use these keywords to build keyword-driven tests.
Configuration of QTP according to the testing needs: Setting up of preferences like global testing preferences, run session preferences, any test specific preferences. Creation of recovery scenarios to instruct QTP how to proceed when a step fails and configuration of QTP window to easily access panes like the test flow pane, the resources pane and the available keywords pane.
Building of the tests: Construction of the tests by inserting calls to the relevant actions from the tests. Creation of one or more empty tests and addition of actions to them. Association of the object repositories with the relevant actions. Association of the function libraries and recovery scenarios with the relevant tests, so that steps can be inserted using keywords and configuration of test preferences.
Addition of Steps to the Test Actions: Addition of steps which use the keywords created in previous steps, enhancement of tests by inserting checkpoints and output values – to verify that the application is behaving according to expectations and addition of programmatic statements to enhance the tests further.
Running, analyzing and troubleshooting the tests: When the tests are ready, run them, view the results and troubleshoot the tests, as needed.

How to do Cookies Testing

2011-05-23T23:54:00.000-07:00

Below is a list of major scenarios for cookies testing of a website. Multiple test cases can be generated from
these scenarios by performing various combinations.

Check if the application is writing cookies properly or not.
Test to make sure that no personal or sensitive data is stored in the cookie. If it is there in cookies, it should be in encrypted format.
If the application under test is a public website, there should not be overuse of cookies. It may result in loss of website traffic if browser is prompting for cookies more often.
Close all browsers, delete all previously written cookies and disable the cookies from your browser settings. Navigate or use that part of web site which use cookies. It should display appropriate messages like "For smooth functioning of this site please enable cookies on your browser."
Set browser options to prompt whenever cookie is being stored / saved in your system. Navigate or use that part of web site which use cookies. It will prompt and ask if you want to accept or reject the cookie. Application under test should display an appropriate message if you reject the cookies. Also, check that if pages are getting crashed or data is getting corrupted.
Close all browsers windows and manually delete all cookies. Navigate various web pages and check and see if these web pages show unexpected behavior.
Edit few cookies manually in notepad or some other editor. Make modifications like alter the cookie content, name of the cookie, change expiry date etc. Now, test the site functionality. Corrupted cookies should not allow to read the data inside it.
Cookies written by one web site should not be accessible by other website.
If you are testing an online shopping portal, Check if reaching to your final order summary page deletes the cookie of previous page of shopping cart properly and no invalid action or purchase got executed from same logged in user.
Check if the application under test is writing the cookies properly on different browsers as intended and site works properly using these cookies. This test can be done on browsers like different versions of internet explorer, Mozilla Firefox, Netscape, Opera etc.
If the application under test is using cookies to maintain the logging state for users. Check if some id is being displayed in the address bar. Now, change the id & press enter. It should display an access denied message and and you should not be able to see other user's account.

Challenges in Software Testing

2011-03-16T09:39:00.001-07:00

All software engineering areas face lot of challenges during execution. So, I would say, as a tester, never ever get surprised when you face challenges in software testing. But, it is also a hard fact for tester community that most of the companies are not testing oriented. Management always do a good appraisal of development teams. Sometimes, appreciation goes to development teams only.

Few points that I want highlight to management / project managers & development teams:

If development teams are so good / intelligent, then why you need testers to test the application.
Remember - by testing a build / release, testers evaluate the Quality of work done by developers.
During every regression testing, lot of bugs got re-opened (what quality of work was done by developers)
Sometimes, a customer release requires 5-7 regression testing cycles. Think Why?
During these 5 to 7 regression cycles, developers made so many mistakes. Testers help to prevent the defect leakage to customer. But, at the end, if even if 2-3 minor bugs goes to customer, everybody from top to bottom will catch testers. Just imagine, with one mistake of a tester, everybody will catch testers from the neck. Are testers bound to make not even a single mistake? But, developers can repeat the same mistakes multiple times.
Most of the times, developers eat tester's time.
I never understand why management always keep a difference between developers and testers. You might be having answer to this question, but, I'm not.

Delivering a Quality product is a joint responsibility of developers, testers & other relevant stack holders. If some bugs, goes to customer, remember:

Developers also did a unit testing
An expert solution architect designed the system.
Some senior developer also reviewed the code / database
Customer also did a UAT - then, application goes live & bug appears
Product managers / business analysts review test cases (and test cases are executed by testers)

Now, apart from testing done by testers, these five steps were also performed which were not able to detect that bug. Then, why all blame came to testers.

The major root cause of all the above mentioned problems is only one challenge - "Lack of testing culture in the organization". To build a testing culture, management need to play an important role.

Second major challenge in the testing is the lack of skilled testers. Again, the major root cause for this is "Wrong Management Decision" at the time of selection of testers. As usual management do not want to invest in testers. This results into incomplete, insufficient and ad-hoc testing throughout the project life cycle. It is also true that sometimes, testers may add complications in the testing of a project due to their unskilled way of working.

There are few other challenges:

Developers did not communicate what the build is about. Wake up testers - If developers didn't communicate about what this build is about, ask them before testing.
Remember, complete Testing is never possible. Testing is always done on sampling basis. Selection of test data needs good proficiency in test data selection methods.
Sometimes, testers and developers fight with each other on bugs and take the things personally rather than professionally. It becomes a major challenge for project stake holders.
As project keep on growing, regression testing becomes challenge. For these kind of situations, both the testing & development teams need to work together to do a proper impact analysis.
In most of the organizations, at the time of effort estimation, testing team is not involved. Testing team is always asked to test the build in a particular time. If you are facing this challenge, raise your voice to relevant stack holders.
If requirements are not properly communicated to you, ask for detailed requirements. Never ever assume anything.
Management always expects high return on investment for automation. Do a proper study for "whether to go for test automation or not". Communicate timely to all stack holders.

How to Improve Software Testing Process / Practice

2011-03-07T06:56:00.001-08:00

Improving the testing process is not the responsibility of test team only. It is a joint effort of Development & Testing team and Management to understand the health of existing testing process and identify the necessary measures to improve it.

Points for Leads, Test Managers, Project Managers and Delivery Heads:

Identify all the platforms on which application will be run.
Improve requirements management process – business & functional requirements should be well documented. Involve testing team in requirement gathering phase.
Create a document / list of all possible scenarios before writing test cases. Include it into test planning.
Measure testing effort on periodical basis – New Bug Rate, Effort Variance, Schedule Variance, Test Case Effectiveness, Residual defect density etc.
Keep developers away from test environments.
Post mortem meetings must be planned after every release. Both testing & development teams should participate in these meetings.
Keep track of bug fixing time taken by development team. Keep informed about the time remaining for regression testing to relevant stack holders. Tracking of this point by relevant stack holders is very critical as it may result in low quality releases.

Points for Software Testers:

Try to understand the logic behind the screen and try to break that logic. Understand the internal workings of code from developers during lunch time or tea breaks.
Analyse test results thoroughly. Try to identify root cause from functional perspective.
Break the application into smaller functional modules.
First write test cases for valid conditions, then cover invalid conditions.
While writing test cases, refer design documents as well.
As soon as you complete the test case writing, share test case with development team. It could result in time saving.
During test case writing phase, group test cases using impact analysis. It will help in effective regression testing in less time.
If you are new tester for a old development team, have a look on old bug reports of modules / project where development worked previously. Generally developers, repeat similar mistakes.
Test the application for both implicit as well as explicit requirements.
Never communicate bugs verbally. For any critical / show stopper bugs, have an immediate discussion & then document via mail and share it to relevant stack holders.

Testing Business Components using QTP and Quality Center

2011-03-01T08:01:00.000-08:00

Generally, business components are created and modified in Quality Centre by Subject Matter Experts. However, you can use the Business Component Keyword View to create, view, modify, and debug a business component in QTP, if required.

In the Keyword View, business components are divided into steps in a modular, keyword-driven, table format. Each step is a row that comprises individual parts that you can easily modify. You create and modify steps by selecting items and operations and entering additional information, as required.

Each step in a business component is automatically documented as you complete it. This enables you to view a description of the step in understandable sentences. In addition, if you added a function library to the application area associated with the business component, when you define a step by selecting a user-defined operation (function), the documentation that you added in the function library will be displayed for the step.

Before you create or open a business component, you connect QTP to a Quality Centre project, which is where business components and application area resources and settings are stored. Connecting to your Quality Centre project enables QTP to create or open the business component. This also enables the business component to access all of the resources defined in the application area on which the component is based.

Important Note: Make sure you have the required Quality Centre permissions before working with business components and application areas.

If the application area you select does not yet contain all of the required resources and settings, you can still add steps using the ManualStep function or the Comment option. This enables you to type in manual steps as you would in Quality Centre or in another application, such as Microsoft Excel or Microsoft Word. You can also use comments to add information about a step or to separate sections of your business component. Each manual step and comment appears as a separate row in the Keyword View.

If you want to delete a component, you can do so only in Quality Centre, regardless of whether it was created in QTP or in Quality Centre.

Also, if needed, you can convert a business component to a scripted component.

Software Testing Openings in Gurgaon

2010-11-02T05:43:00.000-07:00

Date: 02-Nov-10
There few urgent openings for software test professionals having experience from 6 months to 1 year.
Location: Gurgaon
If you are interested, please send your cv at jobs_india@yahoo.com with the subject line as “Testing | Years of Experience”. Kindly mention the below details:

Current CTC:
Expected CTC:
Notice Period:

Below is the job description:

Experience in testing of web based applications
Experience in Manual Functional and System Testing
Experience in test case design and execution
Experience in regression testing, re-testing, integration testing, rapid testing
Bug Reporting & Tracking
Good understanding of software development life cycle models
Good communication skills
Knowledge of automation tools will be an added advantage

If you are interested, please send your cv at jobs_india@yahoo.com with the subject line as “Testing | Years of Experience”.
===========================================================
Date: May 06, 2008
There is an opening for Test Lead. Below are the job details:
Experience: 3 - 6 Years
Desired Qualification: UG - B.Tech/B.E. - Any Specialization; BCA - Computers
Candidate should have experince in Test Planning and Test Execution.
Interested candidates can send their profiles directly to narender@eminence.co.in
===========================================================
===========================================================
Date: May 03, 2008
There is an opening for Sr. Tester in United Health Group.
Skill Set: Automation QTP
If interested kindly mail your resume at Sharad@recruitingpundits.co.in ASAP. Kindly mention the following details also: -

Skill Sets:
Current CTC:
Expected CTC:
Notice Period:

===========================================================
Also See:

Software Test Management and Planning

2009-10-30T10:55:00.001-07:00

Software Testing Estimation Process – It covers some important factors which needs to be taken care while doing estimation.
Organizing the Software Test Team – One of the important tasks for the Software Test Manager.
Software Test Readiness Review Checklist – A sample checklist for Software Test Leads & Managers to evaluate the readiness of the application released for software testing.

Identify Software Testing Types and Exit Criteria – Identifying various types of software testing's required for the application and Exit Criteria, both are very critical for the success of testing effort.
Software Test Planning – The quality of the testing effort is directly proportional to the quality of the test planning.
Software Testing Bible - Software Test Plan Document – The document usually created after software test planning or parallel with the Test Planning effort.
Software Test Specification - a detailed summary of what all scenarios shall be tested and how they will be tested. It is basically, a part of software test planning process.
Software Test Strategy – Important activity to be done before starting the software test planning.
When software is ready to ship or release – It is really difficult to decide whether the software is ready for release or not. Only, an experienced software test manager or software test lead can make decision of it.
Responsibilities of a Software Test Manager / Lead – Here is a list of roles and responsibilities of a Software Test Manager or Test Lead. It will help aspiring Software Test Managers and leads.

Software Testing Types, Methods and Strategies

2009-10-30T02:39:00.001-07:00

In my various posts in this web log, I wrote about common and important Software Testing Types, Methods and Strategies. Below are these topics. Although this is not a complete list, but soon it will increase in size as well as in quality.

Here is the list:

License Errors in HP Mercury's LoadRunner

2009-10-21T22:50:00.001-07:00

License – Expired: Your LoadRunner license has expired. Please contact customer support to renew it. You have a time-limited license for your LoadRunner application. Some or all of your license features have expired.

Troubleshooting:

Check your license information of LoadRunner to verify whether your license has expired.
Contact customer support of LoadRunner to renew your license

License – Renewal: Your LoadRunner licenses are no longer valid. Contact customer support to renew your licenses of LoadRunner.

Troubleshooting: Contact customer support to renew your LoadRunner license.

License - Time limit Expired: LoadRunner out of licenses. You have a time-limited LoadRunner license. Some or all of your license features have expired.

Troubleshooting:

Check your LoadRunner license to verify which features have expired.
Contact customer support to renew your LoadRunner license.

License - Vuser Type Not Licensed: You do not have a LoadRunner license for this Vuser type. Please contact customer support to renew your LoadRunner license.

Troubleshooting:

Check your LoadRunner license information to verify whether the LoadRunner license currently installed includes the Vuser types that you are trying to use.
Contact customer support to obtain a LoadRunner license for this Vuser type, or to inform them that you have a problem trying to run a LoadRunner licensed Vuser type.

License - WAN Emulation: You need a LoadRunner license to use the WAN Emulation feature. Please contact customer support for LoadRunner licensing information.

Troubleshooting: Contact customer support for WAN Emulation LoadRunner licensing information.

Also See: Important LoadRunner Tutorials

Replay Tips for Citrix protocol in HP Mercury's LoadRunner

2009-10-08T05:55:00.001-07:00

Today, I’m going add one more tutorial in the LoadRunner section. Below are some useful tips to replay a HP Mercury's LoadRunner script for Citrix protocol while doing performance testing:

Set initialization quota: To prevent overloading by multiple Vusers in LoadRunner while connecting, set an initialization quota of 4 to 10 Vusers (depending on the capacity of the server) or apply ramp up initialization using the LoadRunner Scheduler.
Enable think time: While doing performance testing using LoadRunner; for best results, do not disable think time in the Run Time settings. Think time is especially relevant before the ctrx_sync_on_window and ctrx_sync_on_bitmap functions, which require time to stabilize.
Set consistency between machines: If you intend to replay the LoadRunner Vuser script on another machine, make sure that the following items are consistent between the record and replay machines: Window Size (resolution), Window Colors, System Font and the other Default Options settings for the Citrix client. In LoadRunner, these settings affect the hash value of bitmaps, and inconsistencies may cause replay to fail. To view the Citrix Client settings, select an item from the Citrix program group and choose Application Set Settings or Custom Connection Settings from the right-click menu. Select the Default Options tab.
Increase the numbers of Vusers per machine: Machines running Citrix Vusers may be limited in the number of Vusers that can run, due to the graphic resources available to that machine. In LoadRunner, to increase the number of Vusers per machine, open a Terminal Server session on the machine. You relate to this Terminal Server as a new injector machine. To refer to the virtual injector machine from the LoadRunner Controller, use the following format: machine_name:1, machine_name:2, ... using either the machine name or its IP address. Note that sessions on a Terminal server use, by default, a 256-color set. If you intend to use a terminal session for load testing, make sure to record on machines with a 256-color set.
Changing window names: In some Citrix sessions, the active window name changes while you are recording. If you try to replay the script as is, the LoadRunner's Vuser uses the original window name and the replay may fail. Using the recording options, you can specify a naming convention for the windows in which LoadRunner's VuGen uses a common prefix or common suffix to identify the window.

LoadRunner's VuGen

How to correlate a Winsock script in LoadRunner

2009-10-06T05:38:00.001-07:00

For LoadRunner 7.51 and later, users can correlate the dynamic value for a Winsock script from VuGen's Tree View. Below is further information on how it can be accomplished.

In LoadRunner, to use the Tree View for correlation:

Open the script in VuGen.
Choose View > Tree View.
The Tree View will open with the following structure:
- Function call in the LoadRunner script listed on the left panel.
- Snapshot of the page in the right panel which can be viewed in either Text View or Binary View
To correlate in LoadRunner:

Select the lrs_recieve function on the left panel of the Tree View. The contents of the received buffer are displayed in the right panel.
Select either Text View or Binary View. If you work in Text View, you need to clear the Read-only check box in the upper right corner.
Search for the dynamic value that you want to capture.
Highlight that value, right-click, and select Create Parameter. There are two ways to create a parameter:
- If the offset of the dynamic value is fixed, highlighting that value will set the correct values for Data Range in the Create Parameter window. This creates the function lrs_save_param.
- If the offset values are not fixed (when the send buffer is parameterized and the response of the server changes in length), look for fixed left and right boundaries to capture the value. Then check Extract Parameter data using boundaries and set the left and right boundaries correctly. This creates function lrs_save_search_string.
VuGen will pop-up with an option for applying the correlation to all occurrences of the value in the script.
- If you select Yes, VuGen will automatically search for this value in all the lrs_send buffers and replace the values with the parameter name.
- If you select No, the correlation will be applied to the current location only.

Also See: Other LoadRunner Tutorials

Important LoadRunner Tutorials

2009-10-05T09:17:00.001-07:00

Below are some the links of eXcellent loadrunner tutorials:

Recording Tips in Load Runner

Recording Problems in LoadRunner

HP Diagnostics and LoadRunner Integration

LoadRunner Automation API

Analysis API in LoadRunner

Vusers in LoadRunner

C Vuser Functions in LoadRunner

LoadRunner VuGen Environment Options

Silent Test Runner

Measuring Transactions

Inserting and Running Tests in LoadRunner or Business Process Monitor

Mercury Performance Testing and Business Availability Center Products

Mercury LoadRunner and Citrix

Citrix ICA Client Integration with LoadRunner

Citrix Load Testing Challenges

How to correlate a Winsock script in LoadRunner

Replay tips for Citrix protocol in LoadRunner

License Errors in LoadRunner

Load Runner - Interview Questions

Note: For further stuff on LoadRunner, please visit our other website: LoadRunnerStuff.com

Recording Tips in Load Runner

2009-10-05T08:56:00.001-07:00

In continuation to my previous post “Recording Problems in Load Runner”, here, I’m going to give you some good and useful tips for doing recording in Load Runner.

Use the mouse and not the keyboard: In LoadRunner, it is preferable to click on an object with the mouse rather then using the keyboard. During recording, use only GUI objects that are within the browser's pane. Do not use any browser icons, controls, the Stop button, or menu items, such as View > Refresh. You may, however, use the Refresh, Home, Back and Forward buttons and the address bar.
Do not record over an existing script : It is best to record into a newly created script—not an existing one.
Avoid context menus: Avoid using context menus during recording. Context menus are menus which pop up when clicking an item in a graphical user interface, such as right-click menus.
Avoid working in a browser while recording: During recording, do not work in any browser window other than the browser windows opened by VuGen.
Wait for downloads: Wait for all downloads to complete before doing any action, such as clicking on a button or filling in a text field.
Wait for pages to load: During recording, it is best to wait for the page to load completely before doing the next step. If you did not wait for all of the pages to load, record the script again.
Navigate to start page: If the last page in an action does not contain the links and buttons that were available at the start of the iteration, then the next iteration will fail. For example, if the first page has a text link Book A Flight, make sure to navigate to the appropriate page at the end of your recording, so that the same link will be visible at the end of the business process.
Use a higher event configuration level: Record the business process again the High Event Configuration level.
Disable socket level recording: In certain cases, the capturing of the socket level messages disrupts the application. For most recordings, socket level data is not required. To prevent the recording of socket level data, disable the option in the recording options. For more information, see the section about recording with Click and Script.
Enable the record rendering-related property values: If the client-side scripts of the application use a lot of styling activities, enable the Record rendering-related property values option before recording the script. For example, enable this option to record additional DOM objects such as offsetTop. Note that enabling this option may decrease the replay speed. To enable record rendering-related property values in Load Runner:

Open the Recording Options. Select Tools > Recording Options and select the GUI Properties:Advanced node

Also See: Other LoadRunner Tutorials

Recording Problems in LoadRunner

2009-10-05T04:49:00.001-07:00

Here, I’m going to describe some common recording problems / issues of Load Runner and how to resolve them. Normally, these problems are faced by everyone. I would request you all that leave your comments for the problems you face and how you tackle them.

Firefox is not supported: Only Internet Explorer is supported for Web (Click and Script). To record browser activity on Firefox, use the Web (HTTP/HTML) protocol.

Application behaves differently while being recorded: If your application behaves differently during recording, than it does without recording, you should determine if the recording problem is unique to Web (Click and Script). The effect may be that a Web page will not load, part of the content is missing, a popup window does not open, and so forth. Create a new Web (HTTP/HTML) script and repeat the recording. In the event that the recording is OK in Web (HTTP/HTML), we recommend that you disable socket level recording. The problem may be the result of an event listener. Use trial and error to disable event listeners in the Web Event Configuration Recording Options, and then re-record your session as a Web (Click and Script) user. To disable an event listener:

Open the Recording Options. Select Tools > Recording Options and select the GUI Properties:Web Event Configuration node.
Click Custom Settings and expand the Web Objects node. Select an object.
Select Disabled from the list in the Record column for the relevant Web object. If the recording still does not work, enable the listener you previously disabled, and try disabling another one. Repeat these steps until your recording succeeds.

Dynamic menu navigation was not recorded: A dynamic menu is a menu that dynamically changes depending on where you select it. If the dynamic menu navigation was not recorded, record again using "high" event configuration mode. To set the configuration level to high:

Open the Recording Options. Select Tools > Recording Options and select the GUI Properties:Web Event Configuration node.
Move the slider to High.

Certain user actions were not recorded: Check if there is a Java applet running inside the browser. If not, record the script with the Web (HTTP/HTML) protocol.

Also See: Other LoadRunner Tutorials

Latest QTP Training Material and Tutorials

2009-10-02T03:15:00.000-07:00

Here are the links of some good QTP tutorials. Beginners can use these tutorials as a training material. Read these and give your feedback for improvement.

Selenium - Overview

2009-08-30T05:26:00.001-07:00

Selenium is a portable software testing framework for web applications. The tests can be written as HTML tables or coded in a number of popular programming languages and can be run directly in most modern web browsers. Selenium can be deployed on Windows, Linux, and Macintosh. Selenium is used for UAT (User Acceptance Test).

Selenium consists of

- Selenium Core
- Selenium RC
- Selenium Grid
- Selenium on Rails
- Selenium IDE

Selenium IDE:

Selenium IDE is a complete Integrated Development Environment (IDE) for Selenium tests (previously known as Selenium Recorder). Firefox extension that allows recording and editing of tests. It allows easier development of tests. It can output Ruby for it’s Ruby based (Application Programming Interface) API.

Selenium IDE Features:

- Record and playback
- Intelligent field selection will use IDs, names, or XPath as needed
- Auto complete for all common Selenium commands
- Walk through test cases and test suites.
- Debug and set breakpoints
- Save tests as HTML, Ruby scripts, or other formats
- Support for Selenium user-extensions.js file
- Option to automatically assert the title of every page
- Rollup common commands

Selenium IDE License:

Selenium was developed by a team of programmers and testers at Thought Works. Selenium is open source software, released under the Apache 2.0 license and can be downloaded and used without charge. The Selenium Grid provides a hub allowing the running of multiple Selenium tests concurrently on any number of local or remote systems, thus minimizing test execution time.

Selenium Interview Questions - 1

2009-08-30T05:09:00.001-07:00

Q1. What is Selenium?

Ans. Selenium is a set of tools that supports rapid development of test automation scripts for web

based applications. Selenium testing tools provides a rich set of testing functions specifically

designed to fulfil needs of testing of a web based application.

Q2. What are the main components of Selenium testing tools?
Ans. Selenium IDE, Selenium RC and Selenium Grid

Q3. What is Selenium IDE?
Ans. Selenium IDE is for building Selenium test cases. It operates as a Mozilla Firefox add on and

provides an easy to use interface for developing and running individual test cases or entire test

suites. Selenium-IDE has a recording feature, which will keep account of user actions as they are

performed and store them as a reusable script to play back.

Q4. What is the use of context menu in Selenium IDE?
Ans. It allows the user to pick from a list of assertions and verifications for the selected location.

Q5. Can tests recorded using Selenium IDE be run in other browsers?
Ans. Yes. Although Selenium IDE is a Firefox add on, however, tests created in it can also be run in

other browsers by using Selenium RC (Selenium Remote Control) and specifying the name of the test

suite in command line.

Q6. What are the advantage and features of Selenium IDE?

Ans. 1. Intelligent field selection will use IDs, names, or XPath as needed
2. It is a record & playback tool and the script format can be written in various languages including

C#, Java, PERL, Python, PHP, HTML
3. Auto complete for all common Selenium commands
4. Debug and set breakpoints
5. Option to automatically assert the title of every page
6. Support for Selenium user-extensions.js file

Q7. What are the disadvantage of Selenium IDE tool?
Ans. 1. Selenium IDE tool can only be used in Mozilla Firefox browser.
2. It is not playing multiple windows when we record it.

Q8. What is Selenium RC (Remote Control)?
Ans. Selenium RC allows the test automation expert to use a programming language for maximum

flexibility and extensibility in developing test logic. For example, if the application under test returns

a result set and the automated test program needs to run tests on each element in the result set, the

iteration / loop support of programming language’s can be used to iterate through the result set,

calling Selenium commands to run tests on each item.

Selenium RC provides an API and library for each of its supported languages. This ability to use

Selenium RC with a high level programming language to develop test cases also allows the automated

testing to be integrated with the project’s automated build environment.

Q9. What is Selenium Grid?

Ans. Selenium Grid in the selenium testing suit allows the Selenium RC solution to scale for test suites

that must be run in multiple environments. Selenium Grid can be used to run multiple instances of

Selenium RC on various operating system and browser configurations.

Q10. How Selenium Grid works?

Ans. Selenium Grid sent the tests to the hub. Then tests are redirected to an available Selenium RC,

which launch the browser and run the test. Thus, it allows for running tests in parallel with the entire

test suite.

Q 11. What you say about the flexibility of Selenium test suite?
Ans. Selenium testing suite is highly flexible. There are multiple ways to add functionality to Selenium

framework to customize test automation. As compared to other test automation tools, it is

Selenium’s strongest characteristic. Selenium Remote Control support for multiple programming and

scripting languages allows the test automation engineer to build any logic they need into their

automated testing and to use a preferred programming or scripting language of one’s choice.

Also, the Selenium testing suite is an open source project where code can be modified and

enhancements can be submitted for contribution.

Q12. What test can Selenium do?
Ans. Selenium is basically used for the functional testing of web based applications. It can be used for

testing in the continuous integration environment. It is also useful for agile testing

Q13. What is the cost of Selenium test suite?
Ans. Selenium test suite a set of open source software tool, it is free of cost.

Q14. What browsers are supported by Selenium Remote Control?
Ans. The test automation expert can use Firefox, IE 7/8, Safari and Opera browsers to run tests in

Selenium Remote Control.

Q15. What programming languages can you use in Selenium RC?
Ans. C#, Java, Perl, PHP, Python, Ruby

Q16. What are the advantages and disadvantages of using Selenium as testing tool?
Ans. Advantages: Free, Simple and powerful DOM (document object model) level testing, can be used

for continuous integration; great fit with Agile projects.

Disadvantages: Tricky setup; dreary errors diagnosis; can not test client server applications.

Q17. What is difference between QTP and Selenium?
Ans. Only web applications can be testing using Selenium testing suite. However, QTP can be used for

testing client server applications. Selenium supports following web browsers: Internet Explorer,

Firefox, Safari, Opera or Konqueror on Windows, Mac OS X and Linux. However, QTP is limited to

Internet Explorer on Windows.

QTP uses scripting language implemented on top of VB Script. However, Selenium test suite has the

flexibility to use many languages like Java, .Net, Perl, PHP, Python, and Ruby.

Q18. What is difference between Borland Silk test and Selenium?
Ans. Selenium is completely free test automation tool, while Silk Test is not. Only web applications

can be testing using Selenium testing suite. However, Silk Test can be used for testing client server

applications. Selenium supports following web browsers: Internet Explorer, Firefox, Safari, Opera or

Konqueror on Windows, Mac OS X and Linux. However, Silk Test is limited to Internet Explorer and

Firefox.

Silk Test uses 4Test scripting language. However, Selenium test suite has the flexibility to use many

languages like Java, .Net, Perl, PHP, Python, and Ruby.

Object Repository Types in QTP

2009-08-29T08:06:00.001-07:00

Test objects can be stored in two types of object repositories—a shared object repository and a local object repository. A shared object repository stores test objects in a file that can be accessed by multiple components (via their application areas) in read-only mode. A local object repository stores objects in a file that is associated with one specific component, so that only that component can access the stored objects.

When you plan and create components, you must consider how you want to store the objects in your components. You can store the objects for each component in its corresponding local object repository, or you can store the objects in your components in one or more shared object repositories. By storing objects in shared object repositories and associating these repositories with your components’ application areas, you enable multiple components to use the objects. For each component, you can use a combination of objects from your local and shared object repositories, according to your needs. You can also transfer local objects to a shared object repository, if required. This reduces maintenance and enhances the reusability of your components because it enables you to maintain the objects in a single, shared location instead of multiple locations.

If you are new to using QTP, you may want to use local object repositories. In this way, you can record and run components without creating, choosing, or modifying shared object repositories because all objects are automatically saved in a local object repository that can be accessed by its corresponding component. If you modify an object in the local object repository, your changes do not have any effect on any other component.

If you are familiar with testing, it is probably most efficient to save objects in a shared object repository. In this way, you can use the same shared object repository for multiple components—if the components include the same objects. Test object information that applies to many components is kept in one central location. When the objects in your application change, you can update them in one location for all the components that use this shared object repository.

If an object with the same name and description is located in both the local object repository and in a shared object repository associated with the same component, the component uses the local object definition. If an object with the same name and description is located in more than one shared object repository associated with the same component, the object definition is used from the first occurrence of the object, according to the order in which the shared object repositories are associated with the component.

Local objects are saved locally with the component, and can be accessed only from that component. When using a shared object repository, you can use the same object repository for multiple components. You can also use multiple object repositories for each component.

When you open and work with an existing component, it always uses the object repositories that are specified in the application area with which the component is associated. Shared object repositories are read-only when accessed from components; you edit them using the Object Repository Manager.

Running Part of Test Script in QTP

2009-08-08T00:36:00.001-07:00

You can use the Run from Step option to run a selected part of your test. This enables you to check a specific section of your application or to confirm that a certain part of your test runs smoothly.

In the Expert View, you can use the Run from Step option to run your test from the selected step until the end of the action. Using Run from Step in this mode ignores any iterations. However, if the action contains nested actions, QTP runs the nested actions for the defined number of iterations of the nested action.

In the Keyword View, you can use the Run from Step option to run your test from the selected step until the end of the test (if the selected step is not part of a reusable action, because a reusable action needs to be called from a test, in order for the test to know from where to continue). Using Run from Step in this mode includes all iterations. The first iteration will run from the step you selected until the end of the test; all other iterations will run from the beginning of the test.

You can use the Run Current Action option to run a single action in your test. Using Run Current Action ignores any iterations. However, if the action contains nested actions, QTP runs the nested actions for the defined number of iterations.

Please note, if you only want to run one iteration of your test, select Run one iteration only from the Run pane in the Test Settings dialog box.

If you want to run your test until a specific point within the test (and not to the end of the action or test), you can insert a breakpoint. The test will then run from the selected step or action until the breakpoint.

To run an entire action, or run a test or action from a selected step:

Make sure your application is in a state matching the action or step you want to run.
Select the action or step where you want to start running the test. Make sure that the step or action you choose is not dependent on previous steps, such as a retrieved value or a parameter defined in a previous step.
Select Automation > Run from Step or Run Current Action, or right-click and select Run from Step. The Run dialog box opens.
In the Run dialog box, choose where to save the run session results, and define any input parameters you want to use.
Click OK. The Run dialog box closes and the run session starts

By default, when the run session ends, the Test Results window opens. The Test Results summary displays a note indicating that the test was run using the Run from Step or Run Current Action option.

If you cleared the View results when run session ends check box in the Run pane of the Options dialog box, the Test Results window does not open at the end of the run session.

Debugging an Action or a Function in QTP

2009-08-08T00:33:00.001-07:00

Suppose you create an action or a function that defines variables that will be used in other parts of your test or function library. You can add breakpoints to the action or function to see how the value of the variables changes as the test or function library runs. To see how the test or function library handles the new value, you can also change the value of one of the variables during a breakpoint.

This can be done by following below simple steps:

Step 1: Create a New Action or Function: Open a test and insert a new action, or open a new function library and create a new function called SetVariables.

If you are working in the Expert View, then follow Step 4 directly. If you are working in a function library, continue with Step 2 and Step 3.

Step 2: (For Function Libraries Only) Associate the Function Library with a Test: Make sure the function library is in focus. Select File > Associate Library ‘<Function Library Name>’ with ‘<Test Name>’. QuickTest associates the function library with your test.

Step 3: (For Function Libraries Only) Add a Call to the Function in Your Test: Add a call to the function by inserting a new step and typing the following in the Expert View: SetVariables.

Step 4: Add Breakpoints: Add breakpoints at the appropriate lines.

Step 5: Begin Running the Test: Run the test. The test or function library stops at the first breakpoint, before executing that step (line of script).

Step 6: Check the Value of the Variables in the Debug Viewer Pane:

Step 7: Check the Value of the Variables at the Next Breakpoint: Click the Run button to continue running the test. The test stops at the next breakpoint.

Step 8: Modify the Value of a Variable Using the Variables Tab

Step 9: Modify the Value of a Variable Using the Command Tab

Step 10: Repeat a Command from the Command History

Handling Run Errors in QTP

2009-08-08T00:30:00.001-07:00

There are two types of Run Error message boxes that can be displayed during a run session in QTP. One is displayed if the problem is a pure VBScript syntax error. When a syntax run error message box is displayed, click OK in the message box and address the error in your step.

The other message box can be displayed in a number of situations. It offers information about the error and a number of buttons for dealing with errors encountered:

Stop: Stops the run session. The run results are displayed if QTP is configured to show run results after the run.

Retry: QTP attempts to perform the step again. If the step succeeds, the run continues.

Skip: QTP skips the step that caused the error, and continues the run from the next step.

Debug: QTP suspends the run, enabling you to debug the test and any associated function library that contains a function called by the test. You can perform any of the debugging operations described in this section. After debugging, you can continue the run session from the step where the test or function library stopped, or you can use the step commands to control the remainder of the run session.

Help: Opens the QTP troubleshooting Help for the displayed error message. After you review the Help topic, you can select another button in the error message box.

The message box also recommends that you consider using Maintenance Mode if you think the error is due to intentional changes in your application and requires that you update multiple steps in your test or objects in your repository.

HP Diagnostics and LoadRunner Integration

2009-08-08T00:28:00.001-07:00

The HP Diagnostics integration with LoadRunner allows you to monitor and analyze the performance of Java 2 Enterprise Edition (J2EE), .NET-connected, SAP, Oracle, and other complex environments.

Specifying the Diagnostics Server Details: The first time you use LoadRunner to capture J2EE or .NET diagnostics data, you need to identify the machine on which the Diagnostics Server in Commander mode is running, and the port that it is using for communication with LoadRunner. You must update this information if you want to integrate with a different Diagnostics Server in Commander mode, or if you change the port it is using. To update the LoadRunner configuration settings for HP Diagnostics:

1. Select Start > Programs > HP LoadRunner > LoadRunner to open the LoadRunner launcher window.

2. From the LoadRunner launcher window menu, select Configuration > Diagnostics for J2EE/.NET Setup to open the Diagnostics for J2EE/.NET Setup dialog box. Enter the details for the Diagnostics Server in Commander mode.

3. Click Test to verify that you entered the correct information for the Diagnostics Server in Commander mode and that there is connectivity between the Diagnostics Server in Commander mode and LoadRunner. Click OK to complete the configuration process.

Configure LoadRunner Scenarios to use HP Diagnostics

1. Before configuring your scenario for Diagnostics, ensure that the application server that you are monitoring has been started.

2. In the Controller, open the relevant load test scenario (FIle > Open) or create a new scenario (File > New).

3. Select Diagnostics > Configuration to open the Diagnostics Distribution dialog box.

4. Set the percentage of Vusers to participate in the HP Diagnostics (J2EE/.NET Diagnostics) monitoring. The maximum percentage of Vusers for which HP Diagnostics (J2EE/.NET Diagnostics) data can be collected is 100%, unless you have enabled other types of diagnostics. In this case, the percentage of Vuser participation in HP Diagnostics (J2EE/.NET Diagnostics) cannot exceed the maximum of any of the other types of diagnostics that you enabled.

5. In the Online & Offline Diagnostics section of the Diagnostics Distribution dialog box, next to J2EE/.NET Diagnostics, click Configure. The J2EE/.NET Diagnostics Configuration dialog box opens.

6. Select Enable J2EE/.NET Diagnostics

7. In the Select probes list, select the probes to be included in your load test scenario.

8. If the Diagnostics Server (or a Diagnostics Server in Mediator mode in a distributed environment) is located behind a firewall, select There is a firewall between the mediator and the Controller, and enter the name of the MI listener server in the MI listener server box. If there is a firewall between the LoadRunner Controller and the Diagnostics Server involved in a load test, you must configure the Controller and the Diagnostics Server to use the MI Listener to enable the transfer of the offline analysis file.

9. To capture a percentage of server requests which occur outside the context of any Vuser transaction select Monitor server requests.

10. To investigate any issues that you have with the connections between the Diagnostics components, click the Troubleshoot Diagnostics for J2EE/.NET connectivity link. This will open the HP Diagnostics System Health Monitor in a new browser window.

During a LoadRunner load test scenario, you can view HP Diagnostics data for the whole scenario or you can drill down to HP Diagnostics data from a particular transaction. After you have run your scenario, you can use HP LoadRunner Analysis to analyze offline Diagnostics data generated during the scenario.

Also See: Other LoadRunner Tutorials

Debugging Components and Function Libraries in QTP

2009-08-07T09:46:00.001-07:00

In QTP, after you create a component or function library including registered user functions, you should check that they run smoothly, without errors in syntax or logic. To debug a function library, you must first associate it with a component via its application area and then debug it from that component.

To detect and isolate defects in a component or function library, you can control the run session using the Pause command as well as various step commands that enable you to step into, over, and out of a specific step.

You can use the Start from Step command to begin your debug session at a specific point in your component. You can also use the Run to Step command to pause the run at a specific point in your component. You can set breakpoints, and then enable and disable them as you debug different parts of your component or function library.

When the component or function library run stops at a breakpoint, you can use the Debug Viewer to check and modify the values of VBScript objects and variables. Also, if QTP displays a run error message during a run session, you can click the Debug button on the error message to suspend the run and debug the component or function library.

You can also use the Run from Step command to run your component or function library from a selected step to the end. This enables you to check a specific section of your application or to confirm that a certain part of your component or function library runs smoothly.

Important things to remember:

While the component and function libraries are running in debug mode, they are read-only. You can modify the content after you stop the debug session. If needed, you can enable the function library for editing (File > Enable Editing) after you stop the session. After you implement your changes, you can continue debugging your component and function libraries.
If you perform a file operation, the debug session is stopped.
In QTP, when you open a component, QTP creates a local copy of the external resources that are saved to your Quality Center project. So, any changes you apply to any external resource that is saved in your Quality Center project, such as a function library, will not be implemented in the component until the component is closed and reopened.

Please note, an external resource is any resource that was not created using QTP, such as, a function library created in an external editor.

LoadRunner Automation API

2009-08-07T09:38:00.000-07:00

The LoadRunner Automation API allows execution of LoadRunner scenarios without using the LoadRunner Controller Graphic User Interface. Using the LoadRunner Automation API, you create programs that define and run scenarios. You might use the API, for example, to run tests at night or to run tests as part of another program.

The central object of the LoadRunner automation API is LrEngine object. When LrEngine is created, it will connect to an existing instance of the LoadRunner Controller or launch a new one.

Use LrEngine to access and program the scenario object that determines the properties of the test to be run, and to access Events related to groups, hosts, scenarios and rendezvous.

Exceptions: LoadRunner Automation handles most errors by generating an exception rather than a return code. In general, the return codes only indicate that the function ran successfully. A return code of zero does not indicate that the action was performed successfully.

You can catch the exceptions by writing your own exception handlers or using standard error classes. Errors in creating objects can be detected by checking if the object is Nothing.

Glossary of terms used in LoadRunner Automation Library:

Controller: The LoadRunner Controller provides a graphic user interface for controlling and managing load test scenarios.

Group: A logical collection of virtual users, usually running the same script.

Host: A machine that executes one or many virtual user scripts, enabling the virtual user to emulate the actions of a human user. When you execute a scenario, the Controller distributes each virtual user in the scenario to a host.

Rendezvous: Emulates heavy user load on the application. You insert rendezvous points into virtual user scripts to ensure that multiple virtual users act simultaneously. For example, to emulate peak load on a bank application, you can insert a rendezvous point instructing 100 virtual users to transfer funds within their accounts at the same time.

Rendezvous Group: A collection of rendezvous.

Scenario: A class that includes a number of scripts to be run, and specifies the hosts that will run the scripts and the virtual user groups associated with each one.

Script: Describes the acts that a virtual user performs during scenario execution. When you run a scenario, each virtual user executes a script. The virtual user scripts include functions that measure and record the performance of your application.

Vuser: A virtual user. When you run a scenario, Vusers emulate the actions of human users, operating your application. A scenario can contain tens or thousands of Vusers. To emulate conditions of heavy user load, you create a large number of Vusers that perform a series of tasks.

Also See: Other LoadRunner Tutorials

Some Facts in Software Testing

2009-08-07T06:44:00.001-07:00

- Software Testing can not show the absence of errors

- There is no last bug in the software / application

- Maximum coverage through minimum test cases is a big challenge of testing.

- Even for simple program of loops, there can be over million paths – testable manually in million years. Domain of possible inputs is too large to test. Also, there are too many possible paths to test the program. Even if, theoretically speaking, one could design excellent test cases to detect all defects, does one have the luxury of time and resources to do so in practice?

- Preventive testing is very important. Verify documents, design, code at each stage of development to prevent errors from getting in. Use a variety of techniques for this. Code review itself throughs up many defects that may be too difficult to detect by execution of tests. On the other hand, test execution can detect errors that can not be easily detected by code reviews. Therefore, various techniques are complimentary in nature and it is only through their combined use that one can hope to detect most errors.

- Even though development tends to be given more importance than testing. A good test design is perhaps intellectually more challenging than the software design and development. Given reasonable practical limits to the quality of test design in practice, it is easy to understand that why it is difficult to uncover all defects through testing.

Analysis API in LoadRunner

2009-08-06T07:20:00.001-07:00

This API set can be used for the following purposes:

Unattended creation of an Analysis session that can then be opened in the HP LoadRunner Analysis.
Custom extraction of data from the results of a test run under the HP LoadRunner Controller

An application that creates an Analysis session can be run automatically at completion of a test run. In the LoadRunner Controller, open Tools > Options and select the Execution tab. Enter the command to run your application in the Post Execution Command box.

The API set provides the following functionality:

Convert Controller Run results to an Analysis data base file
Create, modify, and apply a global filter and Graph filters
Set graph parameters
Calculate metrics and statistics per Graph in a Run
Notifications from the API infrastructure to the API application
Logging at different severity levels
Ability to run concurrent, independent instances of API-based applications

Limitations: This version of the API set does not support:

Graph auto-correlation
Graph merging
Importing data from external monitors
XML export of data
Support Level Agreements functionality
Use of filters, graphs, and other configurations created in the Analysis user interface
J2EE Graphs are only partially supported

Prerequisites: To use this API set, you must:

Be a .NET programmer
Be familiar with the Controller and Analysis programs

The development environment requires:

Visual Studio 2005 or any other compiler that supports .NET 2.0
LoadRunner Analysis

The runtime environment requires:

LoadRunner Analysis
A .NET configuration file. A command-line utility to build the configuration file is provided in the <Analysis installation>\Additional Components\AssemblyCrawler folder.

Also See: Other LoadRunner Tutorials

Oracle Add-in for QTP

2009-08-06T07:13:00.000-07:00

After installing the Oracle Add-in, your applications will always open with Java support active. You can confirm that your Oracle environment has opened properly by checking the Java console for the confirmation message similar to: Loading Oracle Support (version x.x build xxx) (Oracle Corporation x.x.x.xx)

The QTP Oracle Add-in supports only Oracle clients that are Java-based. Oracle Developer/2000 is not supported.

Before using the Oracle Add-in to test Oracle Applications, you must first enable the Name attribute supplied by the Oracle Applications server.

The Oracle Applications server supplies a unique Name attribute for many application objects. You can also find the Oracle Applications server Name attribute in the Oracle Add-in developer name identification property. The developer name identification property is used by QTP in most test object descriptions to identify Oracle objects.

In QTP, table data is always loaded from the application itself, even if the Active Screen contains an image of the table. For this reason, you must first open the table in the application before creating a table checkpoint in a test
In some cases you may have to scroll to the last row of the table to make sure that all the data is loaded

If the table object is not open in your application when you create the checkpoint, the Table Checkpoint Properties dialog box contains only the Properties tab, and the option to select which type of information to check (content or properties) is disabled

It is not necessary to open the table in your application to edit an existing table checkpoint.

Vusers in LoadRunner

2009-08-01T21:17:00.001-07:00

Vusers emulate the actions of human users by performing typical business processes in your application. The actions that a Vuser performs during the recording session are described in a Vuser script.

HP’s tool for creating Vuser scripts is the Virtual User Generator, VuGen. You use VuGen to develop a Vuser script by recording a user performing typical business processes on a client application. VuGen records the actions that you perform during the recording session, recording only the activity between the client and the server. Instead of having to manually program the application’s API function calls to the server, VuGen automatically generates functions that accurately model and emulate real world situations.

During recording VuGen monitors the client end of the database and traces all the requests sent by the user and received from the user, to the server.

During playback, Vuser scripts communicate directly with the server by executing calls to the server API. When a Vuser communicates directly with a server, system resources are not required for the client interface. This lets you run a large number of Vusers simultaneously on a single workstation, and enables you to use only a few testing machines to emulate large server loads.

In addition, since Vuser scripts do not rely on client software, you can use Vusers to check server performance even before the user interface of the client software has been fully developed.

Using VuGen, you can run scripts as standalone tests. Running scripts from VuGen is useful for debugging as it enables you to see how a Vuser will behave and which enhancements need to be made.

VuGen enables you to record a variety of Vuser types, each suited to a particular load testing environment or topology. When you open a new test, VuGen displays a complete list of the supported protocols.

While running the Vusers, you gather information about the system’s response. Afterwards, you can view this information with the Analysis tool. For example, you can observe how a server behaved when one hundred Vusers simultaneously withdrew cash from a bank’s ATM.

Also See: Other LoadRunner Tutorials

Working with the Java Add-in in QTP

2009-08-01T10:29:00.001-07:00

When learning objects and running steps on Java applications in QTP, consider the following:

After installing the Java Add-in, Java applets and applications will always open with Java support active. You can confirm that your Java environment has opened properly by checking the Java console for a message similar to the following confirmation message: “Loading QTP Java Support (version x.x.x.x) (<App> version x.x.x.x).” (where <App> is IE, IBM, or SUN).

You can use the Object test object property to activate only public methods and to retrieve only public properties. A recommended alternative to using the Object property is to extend QTP support for the required Java object using QTP Java Add-in Extensibility.

You cannot add SWT-based JavaMenu objects directly to an object repository using the Add Objects to Local button in the Object Repository window or the Add Objects button in the Object Repository Manager. If you want to add an SWT-based JavaMenu object to the object repository, you can use the Add Objects or Add Objects to Local button to add its parent object and then select to add the parent object together with its descendants. Alternatively, you can add a JavaMenu object using the Navigate and Learn option in the Object Repository Manager.

In QTP, table data is always loaded from the application itself, even if the Active Screen contains an image of the table. For this reason, you must first open the table in the application before creating a table checkpoint in a test. In some cases you may have to scroll to the last row of the table to make sure that all the data is loaded. It is not necessary to open the table in your application to edit an existing table checkpoint.

If you load or unload an add-in that is displayed as a child of the Java add-in in the Add-in Manager, only applications that are opened after loading or unloading the add-in are affected.

Why does software have bugs

2009-08-01T10:27:00.001-07:00

- Errors of Requirements: Gap between “what an application should do” and “what it actually does”. Thanks to one or more of the many possible reasons – deficiency in the specifications or the communication or understanding.

- Errors of Design: Not a well-engineered application. Deficient / defective design.

- Programming / coding errors

- Software Complexity: A non-trivial application has an inherent complexity

- Errors are difficult to detect

- The domain of possible inputs is too large to test. There are too many possible paths through the program to test. Test design is not a simple affair. Minimal tests with maximum coverage is not an easier task.

- Insufficient time to test

- Deficiency in Documentation: Incomplete, incorrect, inadequate, vague, missing documentation – leading to differing interpretations and messing up of construction or maintenance. It is tough to maintain and modify code that is badly written or poorly documented and risky too. Lack of resources, time pressures and bad practices may mean poor documentation

- Changing Requirements: The customer may not understand the effect of changes. Improperly controlled changes may play havoc with the application

- Time Pressures: Software time estimates are after all just that estimates which involves a lot of guess work and assumptions. When deadlines loom and the crunch comes, mistakes will be made

- Software helps automate. But, software to automate software construction does not exist – there are tools to only partially assist

- Software construction is predominantly a manual process. Software is written by people. People make mistakes. Software reflects those mistakes

- Defect Masking: A defect may have remained hidden/masked on account of another defect. Only when the defect, masking the defect, is removed, the masked defect get exposed

Software Testing Services

2009-08-01T09:12:00.001-07:00

For a software company, the software testing service is one of the most critical components to maintain. The effectiveness of application or system being developed is often gauged by software testing service they offered to the project team. Poor software application quality may result in increased support costs, loss of customers or legal issues, loss of market share and a tarnished brand image. A good software testing service is required to ensure that tested applications meet all application requirements. Sometimes, achieving quality and reducing risks of residual defects using in house software testing services, may become challenging, expensive and resource intensive.

Many companies offers independent Testing and Quality Assurance consulting services which follow the best of security, integration and industry standard processes. Some of the best Software Testing Service providers are:

1. STS Consulting — Consulting and staffing company specializing in software testing and validation services

2. uTest – uTest is the world’s largest marketplace for software testing services – 18000+ QA professionals from more than 150 countries

3. Acutest - UK software testing consultancy, providing outsourced technical and business assurance services. These include performance and load testing, user acceptance, functional testing and many more.

4. Applabs - Independent software testing company offering outsourced software testing services, offshore testing & QA, outsourced testing & consultancy, onsite & offshore

5. EffectiveSoft - offers offshore software testing services. Their dedicated team of skilled software testers, Offshore QA Lab, provides a full range of quality and testing services.

6. Off shore testing services – Software testing Center offers Quality Assurance(QA) services including performance and load testing, User Acceptance, Web Application & Functional testing.

7. A1QA – offshore software testing firm offers software testing outsourcing, software testing services including web application testing and dedicated QA team

8. QA InfoTech - is a professional Quality Assurance and Software Testing service provider with qualified Testing Experts who are committed to provide creative soultions.

9. BugHuntress - Since 2001, it provides integrated software testing services which help companies to develop high quality software.

10. TestPros - it provides software testing and software quality assurance services.

Delphi Add-in in QTP 10

2009-07-05T04:17:00.001-07:00

Below is the basic information about the Delphi Add-in and how it relates to some commonly-used aspects of QTP.

General Information:

Add-in Type: This is a Windows-based add-in. Much of its functionality is the same as other Windows-based add-ins.

Checkpoints and Output Values: Bitmap, Standard, Table, Text, Text Area, XML.

Prerequisites:

Opening Your Application: You can open your Delphi application before or after opening QTP.

Add-in Dependencies: None

Other: Before running a test on a Delphi application, the application being tested must be compiled with the QTP Professional agent MicDelphiAgent.

Setting Preferences:

Options Dialog Box : Use the Windows Applications pane. (Tools > Options > Windows Applications node)

Record and Run Settings Dialog Box (tests only):

Use the Windows Applications tab.
(Automation > Record and Run Settings)

QTP recognizes only Delphi applications that have been precompiled with the MicDelphiAgent.pas module.

Custom Active Screen Capture Settings Dialog Box
(tests only): Use the Windows section. (Tools > Options > Active Screen pane > Custom Level button)

Application Area Settings Dialog Box: Use the Applications pane. (File > Settings > Applications node).

Localization Testing

2009-06-25T08:35:00.000-07:00

Localization testing is part of software testing focused on internationalization and localization aspects of software.

Localization Test is the process of adapting globalized application to a particular culture/locale. Localizing the application requires a basic understanding of the sets of character commonly employed in modern software development and an apprehension of the risks associated with them.

Localisation testing checks how well the build has been interpreted into a particular target language. This test is founded on the results of globalized validation where the functional support for that particular locale has already been validated. Whenever the product is not globalised enough to support a given language, you probably will not try to focalise it into that language in the first place.

You still have to check that the application you are delivering to a specific market actually works and the following section shows you some of the common areas on which to focus when executing a Localization of test.

Localization testing contains the translation of the application user interface and adapting graphics for a particular culture/locale. The localisation process can also include translating any help content associated with the application program into native language.

Localisation of business solutions needs that you implement the correct business processes and practices for a culture/locale. Differences in how cultures/locales conduct business are to a great extent determined by governmental and regulatory requirements. Hence, localisation of business logic can be a big task.

Things which are often altered during localization, such as the user interfaces and content files. Below is a sample localization testing checklist:

- Spelling Rules
- Sorting Rules
- Upper and Lower case conversions
- Printers
- Size of Papers
- Operating System
- Key boards
- Text Filters
- Hot keys
- Mouse
- Date formats
- Measurements and Rulers
- Available memory

Note: It is as well a good idea to test that everything you are going to distribute in a local market complies with the local laws and regulations

IBM RFT FAQ - 2

2009-05-12T07:48:00.000-07:00

Q. 28: What are the environmental prerequisites prior to playback of RFT scripts?

Ans.

- Make sure that the application under test is in the same state that it was in when you record the script.
- Ensure that applications and windows that were open or displayed or active when we started recording the script must be open or displayed or active when we start playback.
- Ensure to restore relevant network settings, active databases and system memory (if any) to the same state as when the script was recorded.

Q. 29: When we run a RFT script what actions are performed?

Ans. When script runs from Rational Functional Tester, it plays back all recorded actions. Playback Monitor starts and provides information when the script plays back. A log file gets displayed after the script runs completed. When script runs from Test Manager, Test Manager creates a temporary suite, runs the suite and removes the suite after the run is completed. We can run a script either from the Test Manager Test Scripts window or from the Test Manager File menu.

Q. 30: Out of test scripts or test cases which one should be run to generate test case results?

Ans. Test cases should be run to generate test case results instead of the script. If we run a script the Test Manager does not generate test case coverage results.

Q. 31: Describe the process of debugging of scripts in Rational Functional Tester.

Ans. If the scripts are recorded in the Java programming language - debugging scripts is the same process as debugging a Java application. If the scripts are recorded in the VB.NET Scripting - debugging scripts is the same process as debugging a Visual Basic application.

Q. 32: What is the procedure to create data pools in Rational Functional Tester?

Ans. A data pool is a test data set - a collection of related data records which supplies data values to the variables in test script during test script play back. You can create data for a data pool from scratch or we can import existing data into a new data pool from another Functional Tester data pool or from a .csv file.

Note: If you need to import the data, you will need to import the data into the same Rational Functional Tester project as the test scripts that access the data. If you need to import data from Test Manager data pool, you will need to first associate a RFT project with a Rational project to use the Test Manager data pool.

Q. 33: What type of changes we can do in a data pool?

Ans. Once we data drive a test to create a data pool or create an empty data pool, then we can edit the records and variables in the data pool. We can
1) Add – Remove – Move - Edit a record and variable
2) Edit data pool values
4) Cut - Copy or Paste a cell or record or a variable

Q. 34: Can we replace literal values in a script with data pool references?

Ans. Yes. We can replace literal values in existing script with data pool references to add realistic data to the script. We can replace all literal values or just the number or string or Boolean literals with a data pool reference. If we do not use an existing datapool variable, then the same literal values are used each time we run the script.

Q. 35: Which events take place when we run a script?

Ans. Verification point accesses the data in the datapool and creates the expected results from the baseline and the active data pool record. Datapool reference resolves and supplies data to verification point. The expected result is compared to the actual result. The expected and actual results are recorded in the log. Any deviations from the expected results are logged as a failure in the test log.

Q. 36: Name the two types of datapool record selection orders available in RFT?
Ans. Sequential record selection order and Random record selection order.

Q. 37: Explain the purpose of Test Object Inspector?

Ans. It is used to get a display of the test object information in running application. The information can be of parent hierarchy, inheritance hierarchy, test object properties, non-value properties and method information. Test Object Inspector automatically tracks the cursor and performs live updates after the application is opened. Test Object Inspector does not capture the object / objects in the application if it is not active.

Q. 38: What is the wait state feature for the objects?

Ans. In RFT, this feature is used to set wait state for an object during the playback to check its existence. It is useful when waiting for an object after starting the application or after other actions taking a long time.

Q. 39: What happens to the related files when we rename a test asset?

Ans. When we rename a script, Rational Functional Tester renames all its related files like the helper script files, private object map and any verification point files. If you rename a test object map, the associated scripts with the new name gets updated.

Note: After renaming operation, we need to change all callScript commands in scripts which reference the old script name.

Q. 40: How Data driving testing is helpful in RFT?

Ans: During data driven testing, the script uses variables for key application input fields and programs instead of literal values so that we can use external data to drive the application being tested. Data driven testing uses data from an external file called as datapool. We can add data to the data pool after finishing the recording of the test script.

Q. 41: Name two methods to select an object in an application under test in RFT?

Ans. Object Finder Tool method and Test Object Browser method

Q. 42: Name the various types of datapools in Rational Functional Tester.

Ans. There are two types of datapools: Private datapool and Shared datapool

Q. 43: In how many ways test scripts can be saved in RFT?

Ans. Three ways: Save the current test script or file, Save all test scripts & files and Save a script under another name

Q. 44: Sometimes record monitor goes blank when recording against Java or HTML application. What is the reason for that?

Ans. Record Monitor goes blank while recording against Java or HTML application when JREs or browsers are not enabled.

Q. 45: What are the actions performed by the agent computers?

Ans.

- Running the scripts on more than one computer.
- Test different machine configurations.
- Run Functional Tester scripts remotely on Linux or Windows computers.

Q. 46: What is required to run a RFT script from Test Manager?

Ans. Java Scripting / VB.NET Scripting. On a remote computer, you need to install Rational Test Agent software on the Agent computer and start the agent.
However, if we need external library to run scripts, then we need to install the software on the system on which we plan to run the script. To install a .dll, you need to add the library to directory that is in system search paths and to install a .jar file, you will need to add it to the Functional Tester customization directory. Default location for the customization directory is: C:\ Documents and Settings \ All Users \ Application Data\IBM\RFT\customization

Q. 47: What is the purpose of using Script Assure in RFT?

Ans. This feature helps the successful playback of scripts even when the application under test has been updated. We can play back scripts in RFT by using the ScriptAssure feature to control object-matching sensitivity.
For RFT to recognize an object in the application-under-test, the object properties must match the properties recorded in test object map. RFT can find the object if 1 or 2 properties do not match. If 2 properties do not match, RFT gives a weak recognition warning to the log. If more than 3 properties do not match, RFT cannot find the object in the application.

Q. 48: Ways of using ScriptAssure in RFT.

Ans. - Standard way: ScriptAssure Page - Standard controls object-matching sensitivity during playback by using a slider control.
- Advanced way: The ScriptAssure Page - Advanced sets thresholds for recognition scores. You can set a maximum score to consider a test object as a candidate for recognition.

Q. 49: What is the purpose of Hover feature during recording?

Ans. Hover feature is useful to move the mouse towards a particular place during playback in HTML application. It is helpful for clicking on menus or links in HTML testing.

Q. 50: What are the six steps involved in test development using RFT?

Ans

- Restore test environment and set the playback options.
- Play back each script against the same version of the application under test that was used for recording.
- Analyze the results in the Rational Functional Tester log.
- Use Functional Tester Verification Point Comparator for determining the cause of verification point failures.
- If the script fails; then edit it or debug it or re record it so that it runs as required.
- If we associate a Rational Functional Tester project with the current Rational project, we can group various scripts into test suite and play back the suite to verify that the scripts work properly.

Q. 51: What are the two main phases of script playback?

Ans. Test Development Phase and Regression Testing Phase

Q. 52: What are the six steps involved in regression testing phase using Rational Functional Tester?

Ans.

- Restore testing environment and set playback options.
- Play back test script against a new build of the application under test.
- Analyze the test results in the Rational Functional Tester log.
- Use the Rational Functional Tester Verification Point Comparator to determine the cause of verification point failures.
- If required, revise your test scripts to use new features in the application under test and playback the revised test scripts.

Q. 53: What is purpose of Playback Monitor in Rational Functional Tester?

Ans. Using playback monitor, during the playback of script; we can view the script name, number of the line that is executing, status icons and a description of the action in progress.

Q. 54: What are the Local and agent computers / machines in RFT?

Ans. When we associate Rational Functional Tester project with a Rational project, we can use Test Manager to run the RFT scripts on computers that are designated as Agent computers from a single Windows computer called the Local computer.

Also See:

- Rational Functional Tester Interview Questions - 1

Rational Functional Tester Interview Questions - 1

2009-05-06T18:23:00.000-07:00

Q. 1: What are the main features of Functional Tester?

Ans.
- Complete functional testing of the application
- test the state of objects through verification points.
- Testing any object in the application, including the object's properties and its data.
- Creating and editing simple object-oriented test scripts.
- Generating code through wizards, for example automatically creating a verification point through wizard.

Q. 2: What Process is followed by Rational Functional Tester for test automation?

Ans. Recording of Test Scripts, Enhancing the Test Scripts and then Playing Back the Scripts

Q. 3: For how many development platforms & Languages Functional Tester is available?

Ans.
- Java Scripting uses the Java language and the IBM Rational Software Development Platform.
- VB.NET Scripting uses the VB.NET language and the Microsoft Visual Studio .NET development environment.

Q. 4: What is benefit of test object map in Rational Functional Tester?

Ans. Test object map gets created automatically when recording the script for the application under test. It provides quick way to add objects to a script. It creates a list of test objects available in the application whether they are currently displayed or not. We can create a new test object map either by basing it on an existing map or by adding objects as required. It contains recognition properties for every object; so we can update the recognition information in one central location. Therefore, any scripts using this test object map shall be able to share the updated information. Test object maps created for HTML applications are more hierarchical than those created for Java applications. A browser is the top level of the test object map and the HTML application is inside the browser.

Q. 5: How various objects are identified by Functional Tester?
Ans. In Functional Tester object-oriented technology is used to identify various objects by their internal object properties and not by their screen coordinates. Thus even if the location or text of an object changes, the Functional Tester can still find it on playback.

Q. 6: In how many ways we can test the applications on Linux using RFT?

Ans.
First Way: We can record a script on Windows Environment and export it to Linux & then play it back on Linux.

Second Way: By writing scripts using the Test Object Insert tool. Instead of creating an object map through recording we populate it outside of the recorder by opening the map and selecting objects in the test application.
We can insert verification points without recording, by opening the Verification Point wizard from the Script Explorer.

Q. 7: What are two types of TestManagers in RFT?

Ans.
1) TestManager: It includes the Test Manager Log, running Windows & Linux remote playback agents and test management features like running scripts as part of a test suite. Functional Tester has its own logs and does not depend on the TestManager log integration.

2) ClearQuest Test Manager: It helps us in associating functional test scripts to Test Cases and executes them. Using CQTM we can analyze the received execution results and record those results for reporting and analysis.

Q. 8: What is a Rational and Functional Test project?

Ans. Rational Test project is used for storing test assets like test plans test cases, test logs, reports, builds etc. To associate a Functional Tester project with a Rational project, a Rational project needs to be created to store the test assets. Functional Test project stores all application test assets like scripts, object maps, verification point baseline files and script templates. Functional Test project is created before recording the scripts.

Q. 9: What is the use of associating a project with TestManager in RFT?

Ans. 1) Creating and playingback the scripts from within Rational TestManager.
2) Using the TestManager Agent for playing back scripts on remote computer.
3) You can view the scripts from the Test Scripts window of Rational TestManager.

Q. 10: Benefits of associating a Functional Tester project with Rational Project.

Ans.
1) Management of testing assets from one Rational project.
2) You can record and play back all Functional Tester scripts from Rational TestManager.
3) Playing back Functional Tester script on different computer using Rational Test Agents through TestManager.
4) Management Functional Tester logs using the Test Log window of TestManager.
5) Use of Test Manager reports to get information about the results of running Functional Tester test script and get coverage report for testing project.
6) Use TestManager suite to coordinate the way that scripts run.

Q. 11: Are there any restrictions when we disassociate a project from the current Rational project?

Ans. Yes. We can not play back Functional Tester script or test cases or suites that include Functional Tester scripts in disassociated project from TestManager. Test Manager deletes test script source from the disassociated project. This operation can not be undo. If we associate Functional Test project with Rational project again, TestManager creates new test script source.

Q. 12: Name three buttons available in Functional Tester Projects view banner.

Ans.
- Connect to Functional Test Project: It allows us to connect to an existing Functional Tester project.
- Refresh Projects: It enables us to repaint the display to reflect changes.
- Synchronize with Editor: It scrolls in the tree hierarchy to the name of the script currently displayed in the Java Editor.

Q. 13: Can we export Functional Test project items?

Ans. We can export project items like scripts, test object maps, Java files, Visual Basic files and data pools to another Functional Tester project. When we export a script Functional Tester includes all necessary files like shared test object maps even though it was not selected by us.

Q. 14: What is the utility of a verification points in Rational Functional Tester?

Ans. During the recording verification points can be inserted in the script to confirm the state of an object in the application.
A verification point captures the object information based upon the type of verification point and stores it in a baseline data file. The information in the base file becomes baseline of expected state of object during subsequent builds.

Q. 15: What types of verification points are provided by RFT?

Ans.: Rational Functional Tester has two types of verification points: Object properties verification point and Data verification points. Data verification point has five further points: Menu hierarchy verification point, Table verification point, Text verification point, Tree hierarchy verification point, List verification point.

Q. 16: How RFT playback depends upon the platform and the browser?

Ans.: Test playback in RFT is independent of the platform as well as independent of browser. E.g. we can record script on Windows and play it back on Linux. Also, the script contains no references to the browser during the recording, we can play back the script using any of the supported versions of Firefox, Mozilla, Internet Explorer or Netscape.

Q. 17: What host & test platforms supported by RFT?

Ans. Functional Tester runs on the following Host and Test platforms: Windows XP Professional, Windows 2000 Professional & Server, Windows 2000 Server & Advanced Server, Windows 2003 Enterprise, Standard & Server

Platforms supporting all functions of RFT except the recording: Red Hat Linux Enterprise Workstation 4.0, SuSE Linux Professional version 9.0, SuSE Linux Enterprise Server 10.0, Novel Desktop 10.0

Q. 18: What types of nesting of objects is supported by Functional Tester?

Ans. Parent-child nesting and Owner-owned nesting

Q. 19: Type of file extensions created while exporting Functional Test project items?

Ans. RFT compresses a copy of the files into a data transfer file with the name specified by us. It also appends a .rftjdtr (Functional Tester Java Scripting) or .rftvdtr (Functional Tester VB.NET Scripting) extension to it.

Q. 20: What is the helper superclass in RFT?

Ans. All Functional Tester scripts use default helper superclass. These scripts extend the RationalTestScript class and thereby inherit a number of methods. However, advanced users can create their own helper superclass (if they want to add additional methods or override the methods within RationalTestScript).

Q. 21: Name the Functional Test project items that can be imported?

Ans. Project items like scripts, test object maps, Java files or Visual Basic files, and datapools can be imported into a Functional Tester project. Functional Tester adds all the project items from the data transfer file into the project specified by us. We can use any file compression program which support the .zip to view and work with items in the data transfer file.

Q. 22: RFT provides how many type of delete Cookies methods?

Ans.
- Method of deleting all cookies for the current profile or user
- Method of deletes cookies in a specific page or domain for the current profile or user

Q. 23: How an application starts during recording the tests?

Ans. While recording tests the application can be started by using startApp command. It makes playing back the tests more reliable because startApp uses the application configuration information. RFT will open either Java applications that we specify or HTML pages in our browser, or run applications in executable form.

Q. 24: What is a test project in RFT?

Ans. Location where Functional Tester keeps all of the scripts, along with expected and actual results is called as a test project. The format of test project is governed by the environment being used.

Note: If you are using the Java language, create a Java test project. However you are using Visual Basic.NET, create a .NET test project.

Q. 25: How can we create a new RFT script without recording?

Ans. In new script > RFT includes import statements for files we need to compile the script and comments containing information of archiving. Functional Tester uses the script name as the class name and sets up testMain. Here, you can add the commands to include in the script.

Q. 26: Can we do recording in an existing RFT script?

Ans. Yes. We can do the recording in existing RFT script by bringing our cursor to the desired location & just start the recording. Start the application >> insert verification points >> and add all script support functions.

Q. 27: How can we record script for testing the HTML applications using RFT?

Ans.
- Enabling the Web browser: Enabling of the browser can be done by clicking on Configure > "Enable Environments for Testing". This will enable the environments for testing.

- Configuring the HTML applications for testing: It can be done by clicking on Configure > "Configure Applications for Testing" to specify information about our application and its environment. For testing MS HTML Applications (MSHTA), we need to run mshta.exe to configure application desired to be tested.

Agile Testing

2009-05-01T08:57:00.000-07:00

Historically, software testing has always been about finding where the software breaks. In fact, when we talk about the qualities of a good software tester we quote anecdotes about how "she was able to break my program in 3 minutes" or that "he found 27 bugs when running on Linux with Apache". In other words, software testing was about finding bugs and software testers would get rewarded for and take satisfaction from finding bugs.

Whereas agile testing is more about keeping the bugs out than finding the ones that are in there. The (automated) software tests that are written in agile testing are more about showing that the software works the way we expect than about finding where it breaks.

This doesn’t mean that agile testers should not attempt to find bugs nor that pre-agile testers never write tests to show what works. It means agile thinking has introduced a change in emphasis from “fault-finding” to “working software”.

In pre-agile days, developers would bang away on the code until they considered it done. Then they would throw the code over the wall to QA to find the bugs. QA's first task would be to find the places where it would break. They'd submit a whole bunch of bugs which developers would fix. This cycle would repeat until they couldn't find any more.

Software Testing now happens closer to the developer in space and in time. Software Testers design tests to show that the software works as expected. The developers and testers work together in a room to expand the set of cases that works. The bugs should never get introduced in the first place.

In the old days the conversation might have gone something like this:

=========================================================
Software Developer: The baby has been delivered.
Software Tester: Your baby is early,
Software Developer: Everything works.
Software Tester: C is broken
Software Developer: fixed C
Software Tester: E doesn't work with X
Developer: fixed E with X
and so on until the tester can't find any more bugs.
=========================================================

In agile environment, it would go more like this:

=========================================================
Software Developer: A is done
Software Tester: A works

Software Developer: B is done
Software Tester: B works
and so on until every case works
=========================================================

Nature of Agile Testing:
The biggest difference between agile projects and most ‘traditional’ software development projects is the concept of test driven development. With agile testing, every chunk of code is covered by unit tests, which must all pass all the time. The absence of unit level and regression bugs means that testers actually get to focus on their job: making sure the code does what the customer wanted. The acceptance tests define the level of quality the customer has specified (and paid for!).

Testers who are new to agile testing should keep in mind the agile values: communication, simplicity, feedback and courage.

Difference between Traditional Software Testing and Agile Testing:

Agile testing was different in many ways from ‘traditional’ software testing. The biggest difference is that on an agile project, the entire development team takes responsibility for quality. This means the whole team is responsible for all software testing tasks, including acceptance test automation. When software testers and programmers work together, the approaches to test automation can be pretty creative. Software Testing is integrated into software development. Having everyone in one room speeds up communication by an astonishing amount. Questions that might otherwise take a couple days to be answered over email are answered within a couple of minutes. There can be numerous occasions when hours of time could be saved by overhearing a conversation in the team room.

On traditional projects, folks with Quality somewhere in their title (Quality Assurance, Quality Engineers, et al) perform Independent Verification and Validation (IV&V) activities to assess the quality of the system. Often these teams also review design artifacts. Sometimes they also have a hand in defining and/or enforcing the process by which the software is made.

Agile project teams generally reject the notion that they need an independent group to assess their work products or enforce their process. They value the information that testing provides and they value testing activities highly. Indeed they value testing so much, they practice Test Driven Development (TDD), writing and executing test code before writing the production code to pass the tests. However, even though agile teams value testing, they don't always value independent testers. And they're particularly allergic to the auditing or policing aspects of heavyweight, formal QA.

So how can testers make themselves useful on a team that does not see much use in traditional, formal QA methodologies? Here's what they can do:

1. Supporting Programmer Testing (Technology facing):

When supplementing programmer testing the testers support the programmers in creating the software. They sit in the synergy with the programmers. If the programmers are practicing agile, it's a given that they have an extensive set of unit tests. The software tester’s role is not to do the programmers' unit testing for them. Instead, most of their work involves manual exploratory testing to discover important information about the software that the unit tests failed to reveal.

In order to do this, they have to:

- Get and build the latest source code.
- Run all the unit tests to verify they are starting from a "known good" place.
- Run the application (usually locally from the IDE).
- Minimize documentation: Test documentation can account for a large percentage of the test effort. An informal poll among 135 testers across 57 organizations revealed that testers spend about one third of their time just documenting test cases
- Occasionally add to the automated unit or acceptance test suites.
- Asking “what if” questions to programmers in the planning game. For example, “What if the migrated data has null values?” Analyzing risks and providing information early.
- Offering information about external dependencies or requirements that the team might not otherwise know about.

Unlike unit tests that use mocks and stubs to isolate the code under test, this exploratory testing is as end-to-end as possible. As a result, they are usually able to find issues and risks that the unit tests don't reveal. And they are finding them sooner than we would if we waited for the customer to try things end-to-end. (Early feedback is good.) In doing so, testers not only help the programmers improve the software, they also help them improve their unit tests.

To support programmer testing, you're going to have to be comfortable mucking about in source code. If you haven't coded in a long time (or ever) that means more than just learning Java or whatever language your organization uses. You'll also have to be comfortable:

- Working in the development environment the developers are using, whether that's Visual Studio, Eclipse, IntelliJ, or something else.
- Fetching and building the latest code from the source control system.
- Using the test frameworks the developers are using such as jUnit or NUnit.
- Configuring your own system and setting up your own data, so you may have to learn more about operating systems, networks, and databases

In short, technology-facing testers become members of the development team. You'll need to grow your technical skills accordingly.

Supporting Customer Testing (Business Facing):

The word "Customer" is used in the sense: The set of people who represent the business-facing stakeholders on a project, and not referring to the customers who pay for the software. When supplementing customer testing, they support them by helping them define and execute acceptance tests. Specifically, they:

- Walk through the existing software with them, using an interview-style conversation to surface assumptions and expectations.
- Use the information from walkthroughs and other sources to design and articulate acceptance tests.
- Use a variety of analysis techniques to discover risks and implications of decisions.
- Help the customers and programmers define what "good enough" means for their context.
- Execute acceptance tests manually where needed.
- Automate acceptance tests (either by automating them themselves or by working with a programmer or existing automation engineer)
- Provide metrics or other high level data as needed to help the Customer satisfy his management's need for numbers

In this second role, software testers spend as much of their time facilitating communication and clarifying expectations.

To help business stakeholders articulate their needs, you're going to end up doing a whole lot of requirements extraction. Business-facing testing is as much about distilling requirements and testing assumptions as it is about testing software. There's an entire body of knowledge on requirements analysis, design analysis, and modeling that can help.

- Consider brushing up on UML.
-Learn how to be an effective interviewer and facilitator.
-Grow both your analysis skills and soft skills.

Fine-Tuning the Bitmap Comparison

2009-04-17T21:12:00.000-07:00

When running a bitmap checkpoint, QuickTest compares the area that you are checking in the application with the bitmap stored in the checkpoint, pixel by pixel. By default, if any pixels are different, the checkpoint fails. The Bitmap Checkpoint Properties dialog box provides options for fine-tuning the bitmap comparison.

You can adjust the comparison to enable the checkpoint to pass even if the bitmaps are not identical by setting the RGB tolerance and Pixel tolerance options described below.

In addition, QuickTest enables you to use custom comparers for bitmap checkpoints. A custom comparer is a COM object that you or a third party developed to run the bitmap comparison in the checkpoint according to a more specific algorithm. If one or more custom comparers are installed and registered on the QuickTest computer, the Bitmap Checkpoint Properties dialog box includes a Comparer option. This option enables you to select the QuickTest default comparer or a custom comparer that performs the bitmap comparison according to your testing requirements.

If you select a custom comparer, some of the options in the Bitmap Checkpoint Properties dialog box are different.

Bitmap Checkpoint Tolerance Options:

RGB tolerance: The RGB (Red, Green, Blue) tolerance determines the percent by which the RGB values of the pixels in the actual bitmap can differ from those of the expected bitmap and allow the checkpoint to pass. (The RGB tolerance option is limited to bitmaps with a color depth of 24 bits.)

For example, a bitmap checkpoint on identical bitmaps could fail if different display drivers are used when you create your checkpoint and when you run your test. Suppose one display driver displays the color white as RGB (255, 255, 255) and another driver displays the color white as RGB (231, 231, 231). The difference between these two values is about 9.4%. By setting the RGB tolerance to 10%, your checkpoint will pass when running your test with either of these drivers.

Note: QuickTest applies the RGB tolerance settings when comparing each pixel in the actual and expected bitmaps. The Red, Green, and Blue values for each pixel are compared separately. If any of the values differs more than the tolerance allows, the pixel fails the comparison.

Pixel tolerance: The pixel tolerance determines the number or percentage of pixels in the actual bitmap that can differ from those in the expected bitmap and allow the checkpoint to pass.

For example, suppose the expected bitmap has 4000 pixels. If you define the pixel tolerance to be 50 and select the Pixels radio button, up to 50 pixels in the actual bitmap can be different from those in the expected bitmap and the checkpoint passes. If you define the pixel tolerance to be 5 and select the Percent radio button, up to 200 pixels (5 percent of 4000) in the actual bitmap can be different from those in the expected bitmap and the checkpoint passes.

Using both RGB and Pixel Tolerances: If you define both RGB and pixel tolerances, the RGB tolerance is calculated first. The pixel tolerance then defines the maximum number of pixels that can fail the RGB criteria and allow the checkpoint to pass.

For example, suppose you define an RGB tolerance of 10 percent and a pixel tolerance of 5 percent for a bitmap that has 4000 pixels.

For the checkpoint to pass, each pixel in the actual bitmap must have RGB values that are no greater than or no less than 10 percent of the RGB values of the expected bitmap. If that criterion fails, QuickTest checks that the number of pixels that failed are less than 200. If that criterion passes, the checkpoint passes.

Jumping to a Step in QuickTest 10.00

2009-04-17T21:02:00.000-07:00

You can view the step in QuickTest that corresponds to a node in the run results tree. To view the step in the test that corresponds to a node:

1. Select a node in the run results tree.

2. Perform one of the following:

- Click the Jump to Step in QuickTest button from the Run Results toolbar.
- Right-click and select Jump to Step in QuickTest from the context-sensitive menu.
- Select View > Jump to Step in QuickTest.

3. The QuickTest window is activated and the step is highlighted.

To jump to a step, the following conditions must be true:

a. QuickTest must be running and open to the test whose results are displayed in the Test Results window.

b. The test was run in a version of QuickTest that supports the Jump to Step in QuickTest functionality.

c. The node has a corresponding step in QuickTest. This feature is disabled for the Action, Iteration, Run-Time Data Table, and Test Summary nodes.

d. The step was not performed by a recovery scenario.

e. The step was not run from the Watch or Command tabs of the Debug Viewer.

f. The step is not part of an action that was run using the LoadAndRunAction statement.

g. The test was saved before the run session.

h. The test ran in Normal mode. For information on running QuickTest in Normal mode.

Installing Custom Comparer and Registering to QuickTest

2009-04-17T20:52:00.000-07:00

The custom comparer that you develop needs to be installed on any computer that runs a test that includes a bitmap checkpoint that uses the custom comparer.

Make sure that when the custom comparer is installed, the documentation that you provide for the QuickTest user is placed in the location that you specified in the GetHelpFilename method.

In addition, for QuickTest to recognize the COM object that you create as a custom comparer, you must register it to the component category for QuickTest bitmap comparers.

You register a COM object to a component category by listing the relevant component category ID as a registry key under the COM object's HKEY_CLASSES_ROOT\CLSID\Object's CLSID\Implemented Categories key.

The component category ID must be registered under the HKEY_CLASSES_ROOT\Component Categories key. When QuickTest is installed, it adds the component category ID for QuickTest bitmap comparers as a registry key in this location.

The component category ID for QuickTest bitmap comparers, CATID_QTPBitmapComparers, is defined in QuickTest installation folder\dat\BitmapCPCustomization\ComponentCategory.h.

When you design your custom comparer, you must ensure that when it is installed on the QuickTest computer, it is also registered to the component category for QuickTest bitmap comparers. This can be achieved in different ways.

If you develop your custom comparer in C++ using Microsoft Visual Studio, you can specify this name in the Type box in the ATL Simple Object Wizard.

If you develop the custom comparer in Visual Basic, this value is automatically set to the COM object's ProgID. If you want to modify the custom comparer name, you can edit it manually in the registry after the comparer is installed, or design the program that performs the installation and registration to edit this value as well.

If you develop your custom comparer in C++ using Microsoft Visual Studio, you can modify the DllRegisterServer and DllUnregisterServer methods to handle this registration. These methods are called when you run a DLL using the regsvr32.exe program.

If you develop your custom comparer in an environment that does not enable you to modify the registration methods, you can add an additional program that handles this registration and instruct users who install the custom comparer to run this program as well. You can see an example of this type of implementation in the Visual Basic sample custom comparer that QuickTest provides.

Considerations for Developing Custom Comparers

2009-04-17T20:12:00.000-07:00

In QTP, to develop a custom comparer, below are some importants to keep in mind:

1. To develop a custom comparer you must understand image processing and know how to develop COM objects.

2. You can implement a custom comparer using any language and development environment that supports creating COM objects.

3. Custom comparers run within the QuickTest context. You must therefore exercise care when developing your custom comparer, as its behavior and performance will affect the behavior and performance of QuickTest.

4. The custom comparer must be installed and registered on any computer that runs a test with a bitmap checkpoint using the custom comparer.

5. Before installing and registering a new version of a custom comparer, unregister the existing comparer.

6. More than one custom comparer can be installed and registered on the same QuickTest computer. In the Bitmap Checkpoint Properties dialog box, QuickTest displays all of the available custom comparers, and the QuickTest default comparer. The QuickTest user can select the appropriate comparer to use for each bitmap checkpoint.

7.The computer that runs the custom comparer must have installed the runtime environment associated with the configuration in which the custom comparer DLL was built.

8. You create the custom comparer DLL using a specific development environment version; the computer on which this DLL runs must have the corresponding runtime environment installed.

Developing Bitmap Checkpoint Comparison Algorithm

2009-04-17T19:48:00.000-07:00

Using QuickTest Professional 10.00 you can develop custom comparers for bitmap checkpoints. A custom comparer is a COM object that performs the bitmap comparison for the checkpoint according to your testing requirements. For example, you could define a custom comparer that allows a bitmap checkpoint to pass even if the image in the application shifts by an amount specified when the checkpoint is defined.

After you install and register a custom comparer on the QuickTest computer, your comparer is available for selection in the Bitmap Checkpoint Properties dialog box. The dialog box provides space for specifying additional configuration preferences expected by your comparer.

When QuickTest performs a bitmap checkpoint that uses a custom comparer, it supplies the expected and actual bitmaps to the custom comparer object. It then receives and reports the results that the custom comparer returns.

To develop a custom comparer, you -create a COM object that implements the QuickTest bitmap checkpoint comparer interfaces to perform the following tasks:

- Accept input from QuickTest and perform the bitmap comparison.
- Provide comparison results to QuickTest.
- Provide information that QuickTest can display in the Bitmap Checkpoint Properties dialog box when a user creates or edits a bitmap checkpoint.

Creating a custom comparer in C++ using Microsoft Visual Studio:

1. In Microsoft Visual Studio, select New > Project. The New Project dialog box opens.

2. Select the ATL Project template, enter SampleCPPCustomComparer in the Name box for the project, and click OK. The New ATL Project wizard opens.

3. In Application Settings, make sure that the Attributed option is not selected, and click Finish.

4. In the class view, select the SampleCPPCustomComparer project, right-click, and select Add > Class. The Add Class dialog box opens.

5. Select ATL Simple Object and click Add. The ATL Simple Object Wizard opens.

6. In the Short name box, enter BitmapComparer. The wizard uses this name to create the names of the class, the interface, and the files that it creates.

7. In the Type box, enter Sample Custom Comparer. This is the custom comparer name that QuickTest will display in the Bitmap Checkpoint Properties dialog box and in the test results.

8. Click Finish. The wizard creates the necessary files for the class that you added, including .cpp and .h files with implementation of CBitmapComparer class.

9. In the class view, select CBitmapComparer, right-click, and select Add > Implement Interface. The Implement Interface wizard opens.

10. In the Implement interface from option, select File. Browse to or enter the location of the QuickTest bitmap checkpoint comparer type library. The type library is located in: \dat\BitmapCPCustomization\BitmapComparer.tlb.

The wizard displays the interfaces available in the selected type library, IBitmapCompareConfiguration and IVerifyBitmap.

11. Add both interfaces to the list of interfaces to implement, and click Finish.
In the BitmapComparer.h file, the wizard adds the declarations, classes, and method stubs that are necessary to implement the interfaces. In subsequent steps you will need to add implementation to these method stubs.

Please note that in Microsoft Visual Studio 2005, the wizard generates the signature for the CompareBitmaps method in the IVerifyBitmap interface incorrectly. To enable your project to compile correctly, manually change the type of the last argument (pbMatch) from BOOL* to VARIANT_BOOL*.

12. Open the BitmapComparer.h and BitmapComparer.cpp files.

13. In BitmapComparer.h, create declarations for the bitmap checkpoint comparer interface methods (based on the function bodies that the wizard created): CompareBitmaps, GetDefaultConfigurationString, and GetHelpFilename.

14. Move the function bodies that the wizard created for the bitmap checkpoint comparer interface methods from the BitmapComparer.h file to the BitmapComparer.cpp file.

15. For QuickTest to recognize the COM object that you create as a custom comparer, you must register it to the component category for QuickTest bitmap comparers. The component category ID is defined in \dat\BitmapCPCustomization\ComponentCategory.h.

You can implement this registration in the DllRegisterServer and DllUnregisterServer methods in the SampleCPPCustomComparer.cpp file that the wizard created as part of your project. These methods are called when you run a DLL using the regsvr32.exe program.

- Add the \dat\BitmapCPCustomization folder to your project's include path.

- Open the SampleCPPCustomComparer.cpp file and add the following line: #include "ComponentCategory.h"

- In the SampleCPPCustomComparer.cpp file, modify the DllRegisterServer and DllUnregisterServer methods created by the wizard.

Your custom comparer can now be used in QuickTest for bitmap checkpoints by using the following steps:

a. Open QuickTest and create a bitmap checkpoint on the Windows Calculator application (Standard view).The Bitmap Checkpoint Properties dialog box includes the Comparer option, in which you can select the QuickTest default comparer or your sample custom comparer.

b. Change the Calculator view to Scientific. The size of the calculator object is now larger. Run the checkpoint using the default QuickTest comparer. The checkpoint fails.

c. Edit the checkpoint and select Sample Custom Comparer in the Comparer box.

Creating Portable Copies of Tests

2009-04-17T19:37:00.000-07:00

Generaly, tests and their resource files are stored on network drive or in Quality Center, as this enables the reuse of actions and other resources, and helps ease test management.

However, you may need to open or run a test when you do not have access to a network drive or Quality Center. For example, you may need to create a portable copy of a test for use when travelling to other sites. You can save a standalone copy of your test and its resource files to a local drive or to another storage device using the File > Save Test with Resources command.

When you save a test in this manner, QuickTest creates a copy of the following and saves the files in the location you specify:

1. Source test: QuickTest saves a copy of this test in the location you specify.

2. Resource files: QuickTest saves a copy of all resource files associated with the source test, such as function libraries and shared object repositories. QuickTest stores these files in sub-folders of the copied test.

3. Called actions: QuickTest saves a copy of any external actions called by the source test. For example, if Test A calls actions that are stored in Test B, QuickTest creates a local copy of the actions stored in Test B and stores them in a sub-folder of Test A. The sub-folder has the same name as the test from which the called actions were copied. In this example, the sub-folder is named Test_B. QuickTest also creates a copy of any resources associated directly with these actions, such as its local shared object repositories and action sheets in the Data Table. QuickTest does not, however, save the resource files associated with Test B, so you must ensure that these resources are associated with the source test, Test A.

This enables you to modify or run the test without access to a network drive or Quality Center.

Also, if you use QuickTest with a concurrent license but do not have access to the concurrent license server, you can install a commuter license. For more information, see the HP QuickTest Professional Installation Guide.

Comparing Shared Object Repositories

2009-04-17T19:22:00.000-07:00

QuickTest Professional enables you to compare existing assets from two different object repositories using the Object Repository Comparison Tool. The tool is accessible from the Object Repository Manager, and enables you to compare different object repository resources, or different versions of the same object repository resource, and identify similarities, variations, or changes.

You can compare two object repositories by following below steps:

1. In QuickTest Professional, select Resources > Object Repository Manager.

2. In the Object Repository Manager, select Tools > Object Repository Comparison Tool. The New Comparison dialog box opens on top of the Object Repository - Comparison Tool window.

3. In the First file and Second file boxes, enter or browse to and select the .tsr object repository files that you want to compare. The object repository files can be located in the file system or Quality Center. By default, the boxes display the last files selected for comparison using the Object Repository Comparison Tool. You can click the down arrow next to each box to view and select recently used files.

4. Click OK. The Object Repository Comparison Tool compares the objects in the selected object repositories and displays the results in the Statistics dialog box on top of the Object Repository - Comparison Tool window.

5. Review the statistics.

6. In the Object Repository - Comparison Tool window, you can:

- Filter the objects in the object repositories
- Find specific objects in the object repositories

Please note that the Object Repository Comparison Tool does not compare checkpoint or output objects. You cannot work with the Object Repository Manager or the Object Repository Merge Tool when the Object Repository Comparison Tool is open.

This Comparison Tool automatically identifies objects during the comparison process by classifying them into one of the following types:

1. Identical: Objects that appear in both object repository files. There is no difference in their name or in their properties.

2. Matching description, different name: Objects that appear in both object repository files that have different names, but the same description properties and values.

3. Similar description: Objects that appear in both object repository files that have similar, but not identical, description properties and values. One of the objects always has a subset of the properties set of the other object. This implies that it is likely to be a less detailed description of the same object. For example, an object named Button_1 in the second object repository has the same description properties and values as an object named Button_1 in the first object repository, but also has additional properties and values.

Objects that do not have a description, such as Page or Browser objects, are compared by name only. If the same object is contained in both the object repositories but with different names, they will be shown in the object repositories as two separate objects.

Please note that the Object Repository Comparison Tool gives precedence to matching object descriptions over the matching of object names. For this reason, certain object nodes may be linked during the comparison process and not others.

4. Unique to first file, or Unique to second file: Objects that appear in only one of the object repository files.

5. Does not exist: Objects that do not exist in one of the repository files, but do exist in the other file.

After the compare process, the Comparison Tool provides a graphic presentation of the objects in the object repositories, which are shown as nodes in a hierarchy. Objects that have differences, as well as unique objects that are included in one object repository only, can be identified according to a color configuration that you can select. Objects that are included in one object repository only are identified in the other object repository by the text "Does not exist". You can also view the properties and values of each object that you select in either object repository.

If you are connected to a Quality Center 10.00 project with version control enabled, you can compare two versions of the same object repository.

Asset Comparison Tool and Asset Viewer

2009-04-17T19:13:00.000-07:00

In QTP 10.00, you can view comparison of two versions of an asset either side-by-side, or one above the other.

Compare versions of an integral element: You can view a drilldown comparison of a specific element in the currently open version comparison. Elements include any resource that is an integral part of the test (not saved as an external resource), such as the Data Table or local object repository. When you check in a test, these elements are checked in, too. This enables you to view a version comparison of these elements directly from the test.

View the latest content of an associated resource file: An associated resource file is any resource file used by an asset. For example, a function library and a shared object repository are examples of resource files that can be used by a test. When you drill down in a test, you can view the last saved version of a resource file. This enables you to view the latest content. (If you want to compare different versions of the drilled-down resource, you can open the resource and perform a new comparison.)

To drill down:

1. Click the blue drilldown arrow adjacent to any asset that can be compared.
2. Double-click the element.
3. Right-click the element and select View Drilldown of Selected Asset.

View the QuickTest location of an element: You can view a screen capture depicting the QuickTest location of an element by right-clicking the relevant node and selecting View Sample Snapshot. The screen capture displays an example of the relevant dialog box. The option (or area) for the node you right-clicked is highlighted in the screen capture.

For example, suppose you are viewing a comparison of a test, and you notice that the Disable Smart Identification during the run session node is highlighted, indicating that it was changed. If you are not sure where this option is located in QuickTest, you can right-click the node in the comparison tree and select View Sample Snapshot. QuickTest then opens a dialog box showing you that this area is located in the Run pane of the Test Settings dialog box. The title bar of the dialog box lists the selected element, and a purple rectangle outlines the option.

You can modify the text and background colors for the filter types (changed, added, removed, and so on) in the Asset Comparison Tool window using the Color Settings dialog box.

When you modify the background color of a filter type, the color of the filter type in the legend at the top of the window changes accordingly. These changes remain in effect unless you change them again or restore the default settings

If the sub-elements of an element are different between versions, and you collapse the node representing that element, a legend is displayed adjacent to the node. This legend indicates the number of differences that exist under the collapsed element.

Resources and Dependencies Model

2009-04-17T19:07:00.000-07:00

The Resources and Dependencies model provides very good integration between QuickTest and Quality Center.

1. It replaces the use of attachments with linked QuickTest assets. You store your tests in the Test Plan module, and you store your resource files in the Test Resources module. When you associate a resource file to a test, these assets become linked. Linking assets improves runtime performance by decreasing download time. It also helps to ensure that the relationships between dependent assets are maintained. (Using attachments increases download time from Quality Center 10.00)

2. It also supports versioning & baselines for tests and resource files. You can create versions of these assets in QuickTest or in Quality Center. You manage asset versions and baselines in Quality Center.

3. Enables you to view and compare your QuickTest assets in both Quality Center and QuickTest. You can use the Asset Comparison Tool to compare versions of individual QuickTest assets and the Asset Viewer for viewing an earlier version of a QuickTest asset. Both of these viewers are available in Quality Center and in QuickTest.

4. Enables you to import and share assets across Quality Center projects.

Further, please note that if you are working with the Resources and Dependencies model with Quality Center 10.00, specify an absolute Quality Center path.

Enhancements in QuickTest Professional 10

2009-04-16T18:46:00.000-07:00

New version of QTP offers the following product and feature enhancements. Click a link to view more details about the selected item.

1. Upgrade from QuickTest 9.5: If QuickTest 9.5 is installed on your computer, you can choose to upgrade to QuickTest version 10.00. This enables you to continue using many of the configurations and options you already set in QuickTest 9.5. You can also use an msi silent installation command line to upgrade from QuickTest 9.5.

2. Improved IntelliSense Functionality: QuickTest now provides full IntelliSense for the following types of objects:

- Objects created by a step or function (for example, by calling the CreateObject method)
- Variables to which an object is assigned
- Reserved objects
- COM objects
- Properties that return objects
- IntelliSense sentence completion now provides lists of local variables.
- Multi-level IntelliSense information is available for all reserved objects, COM objects, and objects created using the CreateObject method. For example, method names and available properties are offered for each item in the statement below:

attr_count = XMLUtil.CreateXML.ChildElementsByPath("").ItemByName("abc").Attributes.Count

3. Added Control for Editing and Managing Actions in Automation Scripts: The QuickTest Professional Automation Object Model has a new set of objects and methods for manipulating test actions and action parameters. You can now use automation scripts to create new actions, modify and validate the syntax of action scripts, create and modify action parameters, and more.

4. Improved Debugger Pane Design and Functionality: The Debug Viewer pane has a new look, including icons to help you identify the type of information displayed.

The Watch tab and Variable tab now display the types of expressions or variables, in addition to their names and values.

The Command tab now displays the command history (in read-only format) in addition to the command line, enabling you to view previously-run commands and select commands to reuse.

In addition, a right-click context menu in the Command tab enables you to:

- copy from the command history and edit the command line using the clipboard.
- clear the command history.

5. New Object Identification Solutions in Maintenance Run Mode: In addition to helping you update your steps and object repositories when objects in your application change, the Maintenance Run Wizard can now help you solve the following problems:

- The step failed because the object in your test is missing from the action's associated object repositories.
- The object in your step exists in the application, but can be identified only through Smart Identification.

6. Additional Configuration Settings for Text Recognition Mechanism: You can now set all text recognition configuration settings from the QuickTest Options Dialog Box (Tools > Options > General > Text Recognition), including new options for selecting the text block mode and specifying the languages to be used with the OCR mechanism. This makes it easier to make any necessary adjustments and to optimize the way that QuickTest identifies text in your application.

7. New Look for Options, Settings, and File Dialog Boxes: The QuickTest Options and Settings dialog boxes have changed from their former tab-based design to a more easily navigable tree-based structure. The tree contains only the options relevant for the add-ins that are currently loaded.

QuickTest file dialog boxes such as the New, Open, and Save dialog boxes, now include a sidebar for quickly switching between the file system and Quality Center and for switching between various Quality Center resource storage locations.

8. QuickTest Toolbar Customization Options: You can use the new Customize Dialog box (Tools > Customize) to customize the appearance of existing menus and toolbars, and to create your own user-defined menus, toolbar buttons, and shortcuts.

You can also add new commands to the QuickTest Tools menu so that you can launch an application directly from the menu. For example, you can use this option to create a shortcut to the application you want to test or to an automation script.

9. Improved Web Extensibility: QuickTest Professional Web Add-in Extensibility enables you to develop packages that provide high-level support for third-party and custom Web controls that are not supported out-of-the-box by the Web Add-in.

Limited extensibility support for the ASP .NET AJAX Control toolkit is provided with the extensibility installation. You can use this package as an example for reference or as a basis for your own Ajax extensibility packages.

By creating support for a Web control using Web Add-in Extensibility, you can direct QuickTest to recognize the control as belonging to a specific test object class. You can also extend the list of available test object classes that QuickTest is able to recognize and the list of operations that are available for each class. This enables you to create tests that fully support the specific behavior of your custom Web controls.

The QuickTest Professional Web Add-in Extensibility SDK 10.00 offers the following improvements:

- The Web Add-in Extensibility SDK now provides a global object, window, that you can use in your JavaScript code to access the Internet Explorer global namespace. This enables you to access client-side JavaScript objects and functions in the application you are testing.

- You can now use the Microsoft Script Debugger or the Microsoft Visual Studio debugger to debug JavaScript code that you write for Web Add-in Extensibility.

- The LogLine method provided by the _util object can now (optionally) accept a numeric ID and a numeric Category for the log entry that it passes to the event log. Users can later use this information to filter log entries in the Event Viewer. In addition, the LogLine method can now accept string values for the severity argument, in addition to numeric values.

10. .NET Add-in and Extensibility Improvements:

- The .NET Add-in now supports learning, recording, and running on .NET Windows Forms property grids. QuickTest learns these controls using the new SwfPropertyGrid test object class.

- The .NET Add-in has a new GetErrorProviderText method and ErrorProviderText identification property, which is supported for all .NET Windows Forms test objects. You can use this method or property to retrieve the tooltip text of the error icon associated with the object.

- The C# and Visual Basic project templates and wizards provided with the .NET Add-in Extensibility SDK installation are now provided on Microsoft Visual Studio 2008 (as well as on Microsoft Visual Studio .NET and Microsoft Visual Studio 2005).

- .NET Add-in Extensibility now enables users to create support for table checkpoints on custom .NET Windows Forms table controls

11. New Terminal Emulator Configuration Validation: The Terminal Emulator pane of the Options dialog box now includes a Validate button. When you click this button, QuickTest checks the current configurations of the selected emulator. If a problem is detected, a brief description is displayed in the pane. You can also click the Troubleshoot button to view a Help page that provides additional information about the detected problem.

Test Analysis using QTP 10.00

2009-04-16T18:27:00.000-07:00

QTP 10.00 includes a excellent feature of new reporting options that help you analyze and manage your run results more thoroughly and efficiently. These include:

1. Jump to step: When you want to know more about a particular node in the Test Results window, right-click it and select Jump to Step in QuickTest. The QuickTest main window comes into focus, and the cursor jumps to the relevant step.

2. Export Test Results to Microsoft Word or PDF Formats: In addition to HTML format, you can now select to save your QuickTest run results in Microsoft Word or PDF formats. You can then share this information, together with any local system monitoring data, with your developers and performance testing teams.

3. Add Images to the Test Results: When you use the Reporter.ReportEvent method to add information to the run results, you can now specify an image file as the fourth argument. When you view the results, the image is displayed in the Result Details tab of the Test Results window. For example, you can include an image returned by a CaptureBitmap step in the run results.

4. View Bitmaps Differences in Bitmap Checkpoints: In addition to the expected and actual bitmaps that are displayed in the Test Results window, you can also select to view the differences between the two. The difference bitmap is a black-and-white image that contains one black pixel for every pixel that is different between the two images.

5. Include Images in Exported and Printed Test Results: When you choose the Detailed option for printing or exporting, the document now includes all images. These include screen captures of steps, expected, actual, and difference bitmaps for bitmap checkpoints, and any images sent to the run results using the ReportEvent method.

6. Additional Quality Center Details: The run results for tests and components that are run as part of a Quality Center test set now include the Quality Center server and project name.

Manage and Share Testing Assets in Quality Center and QTP 10

2009-04-16T08:30:00.000-07:00

QuickTest Professional (QTP) 10.00 has a powerful feature of new Quality Center 10.00 integration capabilities for QuickTest assets, Dependencies, and Versions. QuickTest assets include tests, components, application areas, and the resources associated with them, such as shared object repositories, function libraries, recovery scenarios, and external data tables.

These integration capabilities include:

- New resources and dependencies model for storing and managing shared assets
- Excellent Support for asset versioning and baselines
- Good asset Comparison Tool for comparing versions of individual QuickTest assets
- Asset Viewer for viewing an earlier version of a QuickTest asset
- A special tool for Quality Center administrators that upgrades all QuickTest assets to use these new features.

1. New resources and dependencies model: The new Quality Center resources and dependencies model enables you to store your tests, components, and resources such that you can manage the relationships between them and you can analyze the impact on all your assets when one asset changes.

If you are working with a Premier edition Quality Center server, then you can also import and share your resources across different projects. You can synchronize these assets in both projects when changes are made. This functionality also enables you to reuse your existing assets instead of creating new assets whenever you create a new project. For example, you can create a set of template assets to use as a basis for new projects.

In previous versions of QuickTest and Quality Center, a test's associated resource files (such as shared object repositories, function libraries, recovery scenarios, and external data tables), were stored in Quality Center as attachments, while files associated with an application area were stored in a standalone Resources folder in the Business Components module.

Quality Center 10.00 introduces a new Test Resources module. This module enables you to store all of these resources as separate entities that are linked to their owners and marked as dependencies. Tests or actions that call other tests are also linked as dependencies.

When you select a QuickTest test, business process test, or individual component in the Quality Center test or component tree, you can view these dependent resources and tests in a new Dependencies tab. The Dependencies tab shows all Quality Center entities that are used by your testing document as well as all entities that are using it. For example, if your QuickTest test is associated with two function libraries, and the actions in your test are associated with three object repositories, these entities will be displayed in the Using table of the test's Dependencies tab. If one of the actions in your test is called by another test, then that calling test will be displayed in the Used by table.

In QuickTest, you can also view the action-specific dependencies in the Used By tab of the Action Properties dialog box.

Quality Center recognizes assets that are linked as owned by or owner dependencies and ensures that these important relationships are maintained when you import or create baselines, rename or move resources, select to delete resources, or perform other operations that can impact these relationships.

Note: If required, you can continue using the previous attachments model for all or some of your QuickTest assets. However, if you use the old model, you will not be able to take advantage of many of the features associated with the resources and dependencies model.

2. Versioning: In previous versions of QuickTest and Quality Center, limited version control options were available if your Quality Center server had the Version Control Add-ins, which worked with third-party version control tools to perform version control. Now, version control is fully integrated into Quality Center, and the site administrator can enable version control on a per project basis.

When QuickTest is connected to a Quality Center project with version control support, you can check any QuickTest asset into or out of the version control database.

3. Baselines: In Quality Center, a project administrator can create baselines that provide "snapshots" of a project at different stages of development. In the Management module—Libraries tab, the administrator first creates a library, which specifies the root folders from which to import the data. The administrator then creates the actual baseline, which comprises the latest checked in versions of every asset included in the library. The administrator can also import and share entire baselines from other Quality Center projects.

When a project reaches an important stage in the project lifecycle, the administrator can create a new baseline of the files in the library at that point.

In Quality Center, these baselines can be viewed and compared in their entirety. In QuickTest, you can view, retrieve, or compare the individual assets saved in any baseline of a library. This enables you to review an asset as it appeared at a specific phase in the project time line.

Also, please note that baselines are supported in Quality Center 10.00 Enterprise and Premier editions only. They are not supported in Quality Center Starter edition.

4. Asset Comparison Tool and Asset Viewer: The QuickTest Asset Comparison Tool enables you to compare two versions of a particular QuickTest asset, such as a test, function library, shared object repository, or recovery scenario. For example, tests in different baselines or shared projects may be using different versions of the same resource. You can use the comparison tool to make sure that each test is using the correct version of its resources.

The QuickTest Asset Comparison Tool compares each element of the asset in a hierarchal display. The tool also enables you to drill down to view a comparison of the assets that are associated with that asset. For example, when comparing two versions of a test, the comparison may indicate that two of the function libraries, a recovery scenario, and some of the action steps changed between the two versions. You can then drill down to view a comparison of the two versions of any of these items.

The QuickTest Asset Viewer is similar to the Asset Comparison Tool, but is used to view the data for a single version of a QuickTest asset.

5. Asset Upgrade Tool: The QuickTest Asset Upgrade Tool for Quality Center enables you to upgrade, in a batch, all the QuickTest assets in a Quality Center project from an earlier QuickTest version to the current format and to convert testing document attachments to the new resources and dependencies model. QuickTest assets include:

a. QuickTest testing documents, such as tests, components, and application areas
b. Testing document attachments, such as function libraries, shared object repositories, and recovery scenario files

This tool is intended for use only by the Quality Center Administrator and must be run during the Quality Center upgrade process, before you begin working with Quality Center integration features.

The QuickTest Asset Upgrade Tool for Quality Center is the only way to upgrade QuickTest assets in a Quality Center 10.00 project, from an earlier QuickTest version to the current format.

Test Readiness Review Checklist

2009-04-06T07:13:00.000-07:00

Before starting the actual testing, it is important to check whether the system / project / environment is ready for testing. This is called as Test Readiness Review. It is better to do it with a checklist.

Below is a sample Test Readiness Review Checklist:

1. Whether all the tests are conducted according to the Test Plan / Cases ?

2. Are all problems / defects translated into Defect Reports ?

3. Are all the Defect Reports satisfactorily resolved ?

4. Is the log of the tests conducted available?

5. Is unit testing complete in all respects?

6. Is Integration testing complete ?

7. Is all the relevant documentation baselined ?

8. Is all work products products baselined?

9. Is the test plan baselined ?

10. Does the test plan contain the strategy / procedure to conduct the system test ?

11. Are baselined test designs and test cases ready?

12. Is unit/integrated test software ready ?

13. Is the user manual ready?

14. Is the installation procedure documented ?

15. Are all the product requirements implemented?

16. Is the list of known problems available? Is there any "workaround" procedure for the known bugs ?

17. Are test environment needs met for Hardware, code, procedures, scripts, test tools etc.?

18. List of exceptions in test software and test procedures and their work around if any?

19. Is the test reporting tool is available?

20. Are the designers educated on Test reporting tool?

21. Is any standard methodology / tool used and is appropriate to the type of the project?

22. Is the criteria for regression testing defined? Has the regression testing been done accordingly?

23. Is the source code available from the client for performing regression testing complete in all respects?

24. Is the source code freezed for testing?

LoadRunner VuGen Environment Options

2009-03-17T07:31:00.000-07:00

In LoadRunner, you can set up your VuGen working environment in order to customize the auto recovery settings, the loadrunner VuGen editor, and the startup preferences. You set these options from the General Options Environment tab.

Auto Recovery: The auto recovery options, allow you to restore your script’s settings in the event of a crash or power outage. To allow auto recovery, select the Save AutoRecover Information check box and specify the time between the saves in minutes.

Editor: You can set the editor options to select a font and enable LoadRunner VuGen’s Intellisense capabilities which automatically fill in words and function syntax.

Auto show function syntax: When you type the opening parenthesis of a function, VuGen shows the syntax of the function with its arguments and prototypes. To enable the showing of the syntax globally, select the check box adjacent to this option. To disable this feature, clear the check box adjacent to the Auto show function syntax option. If you disable Show Function Syntax globally, you can still bring up the syntax by pressing Ctrl+Shift+Space or choosing Edit > Show Function Syntax after typing the opening parenthesis in the editor.

Auto complete word: When you type the first underscore of a function, VuGen opens a list of functions allowing you to choose the exact function without having to manually type in the entire function. To enable word completion globally, select the check box adjacent to this option. To disable this feature, clear the check box adjacent to the Auto complete word option. If you disable this option globally, you can still bring up the function list box by pressing Ctrl+Space or choosing Edit > Complete Word while typing in the editor.

Select Font: To set the editor font, click Select Font. The Font dialog box opens. Select the desired font, style, and size and click OK. Note that only fixed size fonts (Courier, Lucida Console, FixedSys, and so on) are available.

Startup Dialog: The Show Startup Dialog option opens the Startup dialog box when you open VuGen. The Startup dialog has quick links to create a new script, open an existing script, or view a recent script. If you disable this option, VuGen opens with an empty screen.

Default Environment Settings: By default, Show Function Syntax and Auto complete word are enabled globally. Auto Recovery is set to 10 seconds. VuGen opens with the Startup dialog box.

Also See: Other LoadRunner Tutorials

Organizing the Test Team

2008-12-27T09:35:00.000-08:00

The people component includes human resource allocations and the required skill sets. The test team should comprise the highest-caliber personnel possible. They are usually extremely busy because their talents put them in great demand, and it therefore becomes vital to build the best case possible for using these individuals for test purposes. A test team leader and test team need to have the right skills and experience, and be motivated to work on the project. Ideally, they should be professional quality assurance specialists but can represent the executive sponsor, users, technical operations, database administration, computer center, independent parties, etc. The latter is particularly useful during final system and acceptance testing. In any event, they should not represent the development team, for they may not be as unbiased as an outside party. This is not to say that developers shouldn’t test. For they should unit and function test their code extensively before handing it over to the test team.

There are two areas of responsibility in testing:

1. Testing the application, which is the responsibility of the test team
2. The overall testing processes, which is handled by the test manager.

The test manager directs one or more testers, is the interface between quality assurance and the development organization, and manages the overall testing effort. Responsibilities include:

• Setting up the test objectives
• Defining test resources
• Creating test procedures
• Developing and maintaining the test plan
• Designing test cases
• Designing and executing automated testing tool scripts
• Test case development
• Providing test status
• Writing reports
• Defining the roles of the team members
• Managing the test resources
• Defining standards and procedures
• Ensuring quality of the test process
• Training the team members
• Maintaining test statistics and metrics

The test team must be a set of team players and have the following responsibilities:

• Execute test cases according to the plan
• Evaluate the test results
• Report errors
• Design and execute automated testing tool scripts
• Recommend application improvements
• Record defects

The main function of a team member is to test the application and report defects to the development team by documenting them in a defect tracking system. Once the development team corrects the defects, the test team reexecutes the tests which discovered the original defects.

It should be pointed out that the roles of the test manager and team members are not mutually exclusive. Some of the team leader’s responsibilities are shared with the team member and visa versa.

The basis for allocating dedicated testing resources is the scope of the functionality and the development time frame, e.g., a medium development project will require more testing resources than a small one. If project A of medium complexity requires a testing team of 5, project B with twice the scope would require 10 testers (given the same resources).

Another rule of thumb is that the testing costs approach 25% of the total budget. Since the total project cost is known, the testing effort can be calculated and translated to tester headcount.

The best estimate is a combination of the project scope, test team skill levels, and project history. A good measure of required testing resources for a particular project is the histories of multiple projects, i.e., testing resource levels and performance compared to similar projects.

Identify Testing Types and Exit Criteria

2008-12-27T09:28:00.000-08:00

Identifing Manual / Automated Test Types

The types of tests that need to be designed and executed depend totally on the objectives of the application, i.e., the measurable end state the organization strives to achieve. For example, if the application is a financial application used by a large number of individuals, special security and usability tests need to be performed. However, three types of tests which are nearly always required are: function, user interface, and regression testing. Function testing comprises the majority of the testing effort and is concerned with verifying that the functions work properly. It is a black-box-oriented activity in which the tester is completely unconcerned with the internal behavior and structure of the application. User interface testing, or GUI testing, checks the user’s interaction or functional window structure. It ensures that object state dependencies function properly and provide useful navigation through the functions. Regression testing tests the application in light of changes made during debugging, maintenance, or the development of a new release.

Other types of tests that need to be considered include system and acceptance testing. System testing is the highest level of testing which evaluates the functionality as a total system, its performance and overall fitness of use. Acceptance testing is an optional user-run test which demonstrates the ability of the application to meet the user’s requirements. This test may or may not be performed based on the formality of the project. Sometimes the system test suffices.

Finally, the tests that can be automated with a testing tool need to be identified. Automated tests provide three benefits: repeatability, leverage, and increased functionality. Repeatability enables automated tests to be executed more than once, consistently. Leverage comes from repeatability from tests previously captured and tests that can be programmed with the tool, which may not have been possible without automation. As applications evolve, more and more functionality is added. With automation, the functional coverage is maintained with the test library.

Identifing the Test Exit Criteria:

One of the most difficult and political problems is deciding when to stop testing, since it is impossible to know when all the defects have been detected. There are at least four criteria for exiting testing:

Scheduled testing time has expired: This criteria is very weak, since it has nothing to do with verifying the quality of the application. This does not take into account that there may be an inadequate number of test cases or the fact that there may not be any more defects that are easily detectable.

Some predefined number of defects discovered: The problems with this is knowing the number of errors to detect and also overestimating the number of defects. If the number of defects is underestimated, testing will be incomplete. Potential solutions include experience with similar applications developed by the same development team, predictive models, and industry-wide averages. If the number of defects is overestimated, the test may never be completed within a reasonable time frame. A possible solution is to estimate completion time, plotting defects detected per unit of time. If the rate of defect detection is decreasing dramatically, there may be “burnout,” an indication that a majority of the defects have been discovered.

All the formal tests execute without detecting any defects: A major problem with this is that the tester is not motivated to design destructive test cases that force the tested program to its design limits, e.g., the tester’s job is completed when the test program fields no more errors. The tester is motivated not to find errors and may subconsciously write test cases that show the program is error free. This criteria is only valid if there is a rigorous and totally comprehensive test case suite created which approaches 100% coverage. The problem with this is determining when there is a comprehensive suite of test cases. If it is felt that this is the case, a good strategy at this point is to continue with ad hoc testing. Ad hoc testing is a black-box testing technique in which the tester lets his or her mind run freely to enumerate as many test conditions as possible. Experience has shown that this technique can be a very powerful supplemental or add-on technique.

Combination of the above: Most testing projects utilize a combination of the above exit criteria. It is recommended that all the tests be executed, but any further ad hoc testing will be constrained by time.

Software Test Planning

2008-12-27T09:20:00.000-08:00

The purpose of test planning is to provide the basis for accomplishing testing in an organized manner. From a managerial point of view it is the most important document, because it helps manage the test project. If a test plan is comprehensive and carefully thought out, test execution and analysis should proceed smoothly.

The test plan is an ongoing document, particularly in the spiral environment since the system is constantly changing. As the system changes, so does it. A good test plan is one which:

• Has a good chance of detecting a majority of the defects
• Provides test coverage for most of the code
• Is flexible
• Is executed easily, repeatably, and automatically
• Defines the types of tests to be performed
• Clearly documents the expected results
• Provides for defect reconciliation when a defect is discovered
• Clearly defines the test objectives
• Clarifies the test strategy
• Clearly defines the test exit criteria
• Is not redundant
• Identifies the risks
• Documents the test requirements
• Defines the test deliverables

The planning test methodology includes three steps:

1. Building the test plan
2. defining the metrics
3. reviewing/approving the test plan.

Step 1: Build a Test Plan

- Prepare an Introduction
- Define the High-Level Functional Requirements (In Scope)
- Identify Manual / Automated Test Types
- Identify the Test Exit Criteria
- Establish Regression Test Strategy
- Define the Test Deliverables
- Organize the Test Team
- Establish a Test Environment
- Define the Dependencies
- Create a Test Schedule
- Select the Test Tools
- Establish Defect Recording / Tracking Procedures
- Establish Change Request Procedures
- Establish Version Control Procedures
- Define Configuration Build Procedures
- Define Project Issue Resolution Procedures
- Establish Reporting Procedures
- Define Approval Procedures

Step 2: Define the Metric Objectives

- Define the Metrics
- Define the Metric Points

Step 3: Review/Approve the Plan

- Schedule / Conduct the Review
- Obtain Approvals

Rational SiteCheck Tips and Techniques

2008-12-24T23:29:00.000-08:00

Below are some usefull tips and general guidelines will help you become more efficient while using SiteCheck:

1. Is your Web server getting full? Click List View Defects Orphans to see all files that have no links to them. Delete those orphans you don’t need.

2. Can you remember the title of the document you are looking for but not the file name? Use List View to find the file by title and then view its other attributes, like the file name.

3. Use the toolbar filter buttons in Page View to reduce file clutter and help you better understand the structure of your Web site.

4. Rational SiteCheck automatically fixes links when you move files. All files and their respective links are maintained during this process. When working on local sites, you can drag files from one directory to another and the links to those files are preserved.

5. Source View offers you enhanced text editing, including color-coded text and a link-sensitive right-click menu. You can make on-the-fly updates to your HTML documents from Source View.

6. Rational SiteCheck can check all your links: the internal links within your site and external links to other Web sites.

7. Is it safe to delete a file? Select the document in Page View and you'll see all links to and from the file.

8. LinkWizard instantly recognizes when a file operation will affect links and can fix them automatically.

9. Rational SiteCheck saves you time by providing customizable e-mail templates that automatically create messages that you send frequently.

10. Check your content before going live. Rational SiteCheck's list of broken links will show all the documents that need repair.

11. Switch quickly between the views offered by Rational SiteCheck. The function keys F4 and F6 through F10 are shortcuts to all the views.

12. All Rational SiteCheck views stay synchronized as you move through your Web site.

13. In List View, click on a column heading to sort the items by that column. Click again to reverse the sort order.

14. You can rearrange, add, or delete columns to create a custom report in List View. You can then print or export the List View.

15. Use SiteMonitor to automatically check for defects on your Web site.

16. To check the status of external links at any time, you can right-click on an external link and click Check Link or Recheck Link.

17. When a page uses frames and Rational SiteCheck reports a broken link, be sure to check the NOFRAMES section of the frameset page.

18. When Rational SiteCheck reports a broken link to an image map but it looks correct from your browser, be sure to check both the server-side and client-side image maps.

19. With Rational SiteCheck, you can analyze any site that you can visit with your browser, even sites that are managed by other people.

20. Rational SiteCheck fully supports access to sites through your corporate firewall. To specify your firewall settings, click the Proxy Settings button found on the HTTP Options tab of the Preferences dialog box and the HTTP Options dialog box of the Site Settings wizard.

21. Use the ActiveScan View to follow links from Forms, Java applets, JavaScript, VB scripts and ActiveX controls.

22. Want a quick summary of your Web site statistics? Use the Site Properties View to get an overview of your Web site, including the total number of pages, the total size, and the number of broken links.

23. Want to speed up your load time for large sites? Switch to Site Properties View to increase performance.

24. Do you have a multiserver domain? Rational SiteCheck offers full multiserver support. Click the Servers button in the General dialog box of the Site Settings wizard, and enter the server path. Use wildcard characters (*) to enter a group of related sites.

25. You can save your site at any time. This creates a "site map" that provides a "snapshot" of your site structure at the time of the save. This operation does not save changes or updates to actual Web files.

26. All external links that do not refer to a specific HTML file are displayed in Page View and List View using the "default page name" for the file name. This does not affect any functionality. Actual HTTP requests for this link do not include the default page name.

27. When you move HTML page files, you can choose to leave bookmarks. Bookmarks do not apply to other file types.

28. Rational SiteMonitor is a site-management tool that comes with Rational SiteCheck. With SiteMonitor, you can automatically schedule scans of any site, at any time. You can use SiteMonitor to scan your complete site for defects, check your site to make sure it is reachable, compare two Web sites for differences, or compare your site against a previously saved site map to look for changes.

29. Assertions let you to set your own test conditions. You can check for the presence or absence of display text, HTML tags, links, Java applets, ActiveX controls and other features. With assertions, you can enforce coding standards and verify that corporate standards, like meta tag content, logos and legal notices, are used consistently on your Web site.

30. You can exclude portions of a site from a scan. When opening your site, click the Exclude button on the General dialog box of the Site Settings wizard, and enter the directories you do not want to scan.

31. You can scan to a given number of levels. Simply enter the number of levels in the scan settings when opening your site.

32. You use realms to access password-protected areas of a site and you can enter multiple user names and passwords for each realm.

33. HTML reports let you view summary and detailed data about any site you load. You can generate reports detailing broken links, pages with broken links, failed assertions, slow pages, permanently moved pages, orphan pages, and files with viruses. The reporting utility creates an executive summary of the data as well as a detailed breakdown of all defects. Because the report is loaded directly into your browser, you can move directly to defective files by clicking the link.

34. When saving reports, select a dedicated directory. If you are analyzing a large site, you may end up with a large number of files.

Integration Testing - Four step procedure

2008-12-24T23:04:00.000-08:00

As I wrote in my previous posts, Integration testing is designed to test the structure and the architecture of the software and determine whether all software components interface properly. Integration testing does not verify that the system is functionally correct, only that it performs as designed.

It is the process of identifying errors introduced by combining individual program unit tested modules. Integration Testing should not begin until all units are known to perform according to the unit specifications. It can start with testing several logical units or can incorporate all units in a single integration test.

Below are the four steps of creating integration test cases:

Step 1 - Identify Unit Interfaces: The developer of each program unit identifies and documents the unit’s interfaces for the following unit operations:

- Responding to queries from terminals for information
- Managing transaction data entered for processing
- Obtaining, updating, or creating transactions on computer files
- Passing or receiving information from other logical processing units
- Sending messages to terminals
- Providing the results of processing to some output device or unit

Step 2 - Reconcile Interfaces for Completeness: The information needed for the integration test template is collected for all program units in the software being tested. Whenever one unit interfaces with another, those interfaces are reconciled. For example, if program unit A transmits data to program unit B, program unit B should indicate that it has received that input from program unit A. Interfaces not reconciled are examined before integration tests are executed.

Step 3 - Create Integration Test Conditions: One or more test conditions are prepared for integrating each program unit. After the condition is created, the number of the test condition is documented in the test template.

Step 4 - Evaluate the Completeness of Integration Test Conditions: The following list of questions will help guide evaluation of the completeness of integration test conditions recorded on the integration testing template. This list can also help determine whether test conditions created for the integration process are complete.

Q1. Is an integration test developed for each of the following external inquiries:

- Record test?
- File test?
- Search test?
- Match/merge test?
- Attributes test?
- Stress test?
- Control test?

Q2. Are all interfaces between modules validated so that the output of one is recorded as input to another?

Q3. If file test transactions are developed, do the modules interface with all those indicated files?

Q4. Is the processing of each unit validated before integration testing?

Q5. Do all unit developers agree that integration test conditions are adequate to test each unit’s interfaces?

Q6. Are all software units included in integration testing?

Q7. Are all files used by the software being tested included in integration testing?

Q8. Are all business transactions associated with the software being tested included in integration testing?

Q9. Are all terminal functions incorporated in the software being tested included in integration testing?

Software Testing Techniques and Levels

2008-12-24T19:35:00.000-08:00

In this post, I'm going to describe techniques and strategies for software testing. Techniques cover different ways testing can be accomplished. Testing techniques can be defined in three ways: Preparation, Execution and Approach.

Preparation: From preparation point of view there are two testing techniques: Formal Testing and Informal Testing.

Formal Testing: Testing performed with a plan, documented set of test cases, etc that outline the methodology and test objectives. Test documentation can be developed from requirements, design, equivalence partitioning, domain coverage, error guessing, etc. The level of formality and thoroughness of test cases will depend upon the needs of the project. Some projects can have rather informal ‘formal test cases’, while others will require a highly refined test process. Some projects will require light testing of nominal paths while others will need rigorous testing of exceptional cases.

Informal Testing: Ad hoc testing performed without a documented set of objectives or plans. Informal testing relies on the intuition and skills of the individual performing the testing. Experienced engineers can be productive in this mode by mentally performing test cases for the scenarios being exercised.

From the execution point of view, the two testings types are: Manual Testing and Automated Testing.

Manual Testing: Manual testing involves direct human interaction to exercise software functionality and note behavior and deviations from expected behavior.

Automated Testing: Testing that relies on a tool, built-in test harness, test framework, or other automatic mechanism to exercise software functionality, record output, and possibly detect deviations. The test cases performed by automated testing are usually defined as software code or script that drives the automatic execution.

From the testing approach point of view, the two testings types are: Structural Testing and Functional Testing.

Structural Testing: Structural testing depends upon knowledge of the internal structure of the software. Structural testing is also referred to as white-box testing.

Data-flow Coverage: Data-flow coverage tests paths from the definition of a variable to its use.

Control-flow Coverage

Statement Coverage: Statement coverage requires that every statement in the code under test has been executed.

Branch Coverage: Branch coverage requires that every point of entry and exit in the program has been executed at least once, and every decision in the program has taken all possible outcomes at least one.

Condition Coverage: Condition coverage is branch coverage with the additional requirement that “every condition in a decision in the program has taken all possible outcomes at least once.” Multiple condition coverage requires that all possible combinations of the possible outcomes of each condition have been tested. Modified condition coverage requires that each condition has been tested independently.

Functional Testing: Functional testing compares the behavior of the test item to its specification without knowledge of the item’s internal structure. Functional testing is also referred to as black box testing.

Requirements Coverage: Requirements coverage requires at least one test case for each specified requirement. A traceability matrix can be used to insure that requirements coverage has been satisfied.

Input Domain Coverage: Input domain coverage executes a function with a sufficient set of input values from the function’s input domain. The notion of a sufficient set is not completely definable, and complete coverage of the input domain is typically impossible. Therefore the input domain is broken into subsets, or equivalence classes, such that all values within a subset are likely to reveal the same defects. Any one value within an equivalence class can be used to represent the whole equivalence class. In addition to a generic representative, each extreme value within an equivalence class should be covered by a test case. Testing the extreme values of the equivalence classes is referred to as boundary value testing.

Output Domain Coverage: Output domain coverage executes a function in such a way that a sufficient set of output values from the function’s output domain is produced. Equivalence classes and boundary values are used to provide coverage of the output domain. A set of test cases that “reach” the boundary values and a typical value for each equivalence class is considered to have achieved output domain coverage.

Various Software Testing Levels

Although many testing levels tend to be combined with certain techniques, there are no hard and fast rules. Some types of testing imply certain lifecycle stages, software deliverables, or other project context. Other types of testing are general enough to be done almost any time on any part of the system. Some require a particular methodology. When appropriate common utilizations of a particular testing type will be described. The project’s test plan will normally define the types of testing that will be used on the project, when they will be used, and the strategies they will be used with. Test cases are then created for each testing type.

Unit Testing: A unit is an abstract term for the smallest thing that can be conveniently tested. This will vary based on the nature of a project and its technology but usually focuses at the subroutine level. Unit testing is the testing of these units. Unit testing is often automated and may require creation of a harness, stubs, or drivers.

Component Testing: A component is an aggregate of one or more components. Component testing expands unit testing to include called components and data types. Component testing is often automated and may require creation of harness, stubs, or drivers.

Single Step Testing: Single step testing is performed by stepping through new or modified statements of code with a debugger. Single step testing is normally manual and informal.

Bench Testing: Bench testing is functional testing of a component after the system has been built in a local environment. Bench testing is often manual and informal.

Developer Integration Testing: Developer integration testing is functional testing of a component after the component has been released and the system has been deployed in a standard testing environment. Special attention is given to the flow of data between the new component and the rest of the system.

Smoke Testing: Smoke testing determines whether the system is sufficiently stable and functional to warrant the cost of further, more rigorous testing. Smoke testing may also communicate the general disposition of the current code base to the project team. Specific standards for the scope or format of smoke test cases and for their success criteria may vary widely among projects.

Feature Testing: Feature testing is functional testing directed at a specific feature of the system. The feature is tested for correctness and proper integration into the system. Feature testing occurs after all components of a feature have been completed and released by development.

Integration Testing: Integration testing focuses on verifying the functionality and stability of the overall system when it is integrated with external systems, subsystems, third party components, or other external interfaces.

System Testing: System testing occurs when all necessary components have been released internally and the system has been deployed onto a standard environment. System testing is concerned with the behavior of the whole system. When appropriate, system testing encompasses all external software, hardware, operating environments, etc. that will make up the final system.

Release Testing: Release tests ensure that interim builds can successfully deployed by the customer. This includes product deployment, installation, and a pass through the primary functionality. This test is done immediately before releasing to the customer.

Beta Testing: Beta testing consists of deploying the system to many external users who have agreed to provide feedback about the system. Beta testing may also provide the opportunity to explore release and deployment issues.

Acceptance Testing: Acceptance testing compares the system to a predefined set of acceptance criteria. If the acceptance criteria are satisfied by the system, the customer will accept delivery of the system.

Regression Testing: Exercises functionality that has stabilized. Once high confidence has been established for certain parts of the system, it is generally wasted effort to continue rigorous, detailed testing of those parts. However, it is possible that continued evolution of the system will have negative effects on previously stable and reliable parts of the system. Regression testing offers a low-cost method of detecting such side effects. Regression testing is often automated and focused on critical functionality.

Performance Testing: Performance testing measures the efficiency with respect to time and hardware resources of the test item under typical usage. This assumes that a set of non-functional requirements regarding performance exist in the item’s specification.

Stress Testing: Stress testing evaluates the performance of the test item during extreme usage patterns. Typical examples of “extreme usage patterns” are large data sets, complex calculations, extended operation, limited system resources, etc.

Configuration Testing: Configuration testing evaluates the performance of the test item under a range of system configurations. Relevant configuration issues depend upon the particular product and may include peripherals, network patterns, operating systems, hardware devices and drivers, user settings.

When software is ready to ship or release

2008-12-23T17:59:00.000-08:00

Deciding when software is ready to release is a difficult. You have pressure from all sides to release perfect software, with added features. Below are some common facts:

- The engineers have said that the code was complete months ago, but are still making code changes.

- Sales people promised the software to major accounts months ago and are making commitments for the next release.

- The product manager wants a few more features added and you want to release a zero defect software.

Having no specific release criteria, many organizations wait for the manager to say "Release it", not knowing when or why the decision was made at that time. In other cases, there is a general group consensus to just push the software out the door because the tired and stressed group wants to move out from the seemingly never ending cycles.

You can never take all of the stress out of releasing good software, but by planning ahead and setting up a good defect tracking repository, predetermining acceptable bug counts, properly testing the software and effectively triaging the defects you can always know the current state of the software, but most importantly know when the software is ready to ship.

Defect Tracking Repository:

The most common as well as important tool to determine the state of the software is the defect tracking repository. The repository must be able to provide information required for the triage process. The repository must include the following info:

- The expected results of the testers test
- The steps to recreating the defect
- The version in which the defect was found
- The module in which the defect was found
- The severity of the defect
- A short descriptive name for the defect
- A description of the defect
- The actual results if the testers test
- Resolution notes

The project team should be able to query the database and gather information such as defect counts at various severity levels, and module levels, descriptions of the defects and steps to reproduce them. The repository becomes the archive for the project and it becomes important that correct and complete information be entered in order to use this information for later projects as a planning tool.

Predetermined Acceptable Bug Counts: The goal of releasing software with no defects cannot be achieved given the limited time and resources so, in the preliminary planning of the project exit criteria must be set up. The exit criteria should then be translated into the test plan outline, which should include acceptable bug count levels in order to ship the software.

The exact acceptable bug count level is not some magic number that will insure a successful product, but more a target goal that the group can use to see that they are moving forward toward a common goal. An acceptable bug count statement may look like this:

- Showstoppers: There may not be any
- High: There may not be any
- Medium: There may not be any that have a high probability of appearance to the customer
- Low: There may be a minimal amount in specified core areas and all other areas may have any amount.

In order for the entire group to know the state of the software at all timesit would be preferable to produce a weekly build grading process. The idea is that each week the build would be treated as if it were the software to be shipped and given a grade. The weekly grade keeps the team current on the software state and the team can see the software progressively improve.

This quantitative method takes a lot of the guesswork out of the equation.

Proper Testing: The key to shipping quality software is finding and fixing all defects. The testers responsibility becomes finding the defects and properly reporting them. In order for the testers to find the defects the lead tester must set up in the test plan outline, the specific stages of testing to ensure that the process is organized and covers all aspects of the software. The test plan may contain the following 7 phases:

- Test Case and Test Suite Planning
- Unit Testing
- Regression Testing
- System Testing
- Regression testing
- Performance Testing
- Release Candidate Testing

The goal is to turn up as many defects as early as possible in order to make the fixes easier on the engineers and to provide ample time for regression testing since every time the code is altered there is a greater risk of creating more defects.

As important as finding the defect, the tester must be able to correctly report it in the repository. This becomes extremely important in large projects where the defect counts may rise in the thousands.

Effective Triage:

The Test Lead and Engineer must sit down regularly to evaluate the reported defects and drive the process forward in the most efficient manner for the engineers as well as the testers. Much of the decision making here is based on queries from the defect repository, which shows the importance of the accuracy and completeness of the repository.

The creation of a "Hot List" which is a list of important bugs is a great tool to create. This can be done in an plain Excel sheet, which identifies the most important defects, their module and a brief description. This list is great to use in triage to identify the defect action items for the upcoming week.

Defects that prevent further testing in a certain area must be given precedence and the engineers can assist in the risk analysis of future modules to assist the testers in their test case development. In the determination of what to fix next generally it is advantageous to group and fix defects relating to the same module at the same time even though there may be other important defects in other modules. Grouping of defects assists the engineers finish a module and move on.

Shipping quality software is not an easy task but by being organized from the start of the project the ship date and defect counts don't have to be a "gut feeling". The project must have a Defect Tracking Repository that must be maintained and updated continually with correct and complete information. The Repository is used as the main tool for an effective triage process and in conjunction with the Test Plan Outline that must be createdat the start of the project describing in detail the exit criteria to ship the software.

Please note the entire process rests on proper testing and the use of proper test planning which is what uncovers the defects. If you do not uncover the defects the users of the software will.

Thus, by following the above described process it is possible to know the state of the software at any time so that the entire team knows the goal, sees the progress and knows when the software is ready to release.

How to do Integration Testing - writing Integration Testing test cases

2008-12-22T17:56:00.000-08:00

Integration Testing is a four step procedure. Below are the steps for creating integration test cases:

1. Identify Unit Interfaces: The developer of each program unit identifies and documents the unit’s interfaces for the following unit operations:

- External inquiry (responding to queries from terminals for information)
- External input (managing transaction data entered for processing)
- External filing (obtaining, updating, or creating transactions on computer files)
- Internal filing (passing or receiving information from other logical processing units)
- External display (sending messages to terminals)
- External output (providing the results of processing to some output device or unit)

2. Reconcile Interfaces for Completeness: The information needed for the integration test template is collected for all program units in the software being tested. Whenever one unit interfaces with another, those interfaces are reconciled. For example, if program unit A transmits data to program unit B, program unit B should indicate that it has received that input from program unit A. Interfaces not reconciled are examined before integration tests are executed.

3. Create Integration Test Conditions: One or more test conditions are prepared for integrating each program unit. After the condition is created, the number of the test condition is documented in the test template.

4. Evaluate the Completeness of Integration Test Conditions: The following list of questions will help guide evaluation of the completeness of integration test conditions recorded on the integration testing template. This list can also help determine whether test conditions created for the integration process are complete.

1. Is an integration test developed for each of the following external inquiries:

- Record test?
- File test?
- Search test?
- Match/merge test?
- Attributes test?
- Stress test?
- Control test?

2. Are all interfaces between modules validated so that the output of one is recorded as input to another?

3. If file test transactions are developed, do the modules interface with all those indicated files?

4. Is the processing of each unit validated before integration testing?

5. Do all unit developers agree that integration test conditions are adequate to test each unit’s interfaces?

6. Are all software units included in integration testing?

7. Are all files used by the software being tested included in integration testing?

8. Are all business transactions associated with the software being tested included in integration testing?

9. Are all terminal functions incorporated in the software being tested included in integration testing?

Testing ActiveX Controls using QuickTest Professional

2008-12-09T03:56:00.000-08:00

Many applications include ActiveX controls developed with third-party vendors. You can use the QuickTest Professional ActiveX Add-in to test ActiveX controls. You can create and run software tests and components on these controls, as well as check their properties. QuickTest Professional supports Software Testing on ActiveX controls in any application.

In QuickTest Professional ActiveX Add-in is considered a Windows-Based add-in.

You can check the properties of an ActiveX control as you check the properties of any other Windows object. You can also use output value steps to output property or text values from the objects in your ActiveX application.

When you create a checkpoint on an ActiveX control, QuickTest Professional captures all the properties for an ActiveX control, but it does not select any properties to check.
You use the Windows Applications tab in the Options dialog box to set global preferences for testing ActiveX applications.
You use the Windows Applications tab of the Record and Run Settings dialog box or the Application Area Settings dialog box to set preferences for testing ActiveX applications.

The ActiveX Add-in provides custom test objects, methods, and properties that can be used when testing objects (controls) in ActiveX applications.
The application containing the ActiveX controls on which you want to record must be opened after setting the Record and Run options.

In QuickTest Professional It is recommended that you begin your recording session before opening the application containing the ActiveX controls on which you want to record.
When testing ActiveX objects in a browser, the top-level ActiveX object is inserted within the standard Web object hierarchy, for example, Browser > Page > ActiveX.
QuickTest Professional can record on standard controls within an ActiveX control and if an ActiveX control contains another ActiveX control, then QuickTest Professional can record and run on this internal control as well. For example, suppose your ActiveX control is a calendar that contains a drop-down list from which you can choose the month. If you record a click in the list to select the month of May, QuickTest Professional records this step in the Expert View as:



Dialog("ActiveX Calendars").ActiveX("SMonth Control").WinComboBox("ComboBox").Select "May"

When creating a programmatic description for an ActiveX test object and the relevant run-time object is windowless (has no window handle associated with it), you must add the windowless property to the description and set its value to True. For Example:




Set ButDesc = Description.Create
ButDesc("ProgId").Value = "Forms.CommandButton.1"
ButDesc("Caption").Value = "OK"
ButDesc("Windowless").Value = True
Window("Form1").AcxButton(ButDesc).Click

HP QuickTest Professional Test Object Schema

2008-12-09T03:38:00.000-08:00

In this post, I'm going to describes the elements and attributes available for creating test object configuration XML content to be used with your QuickTest extensibility project.

In a test object configuration XML, you define the test object classes (for example, their identification properties, the test object methods they support, and so on). To do this, you define a ClassInfo element for each test object class. In addition, you define the name of the environment for which the test object classes are intended, and the QuickTest add-in which they extend. If the relevant add-in is not loaded when QuickTest opens, QuickTest does not load the information in this XML. Similarly, if the environment name is displayed in the Add-in Manager dialog box and the check box for this environment is not selected, the information in this XML is not loaded.

A test object class definition can include the following:

- The name of the new test object class and its attributes, including the base class—the test object class that the new test object class extends.

- The test object class name must be unique among all of the environments whose support a QuickTest user might load simultaneously. For example, do not use names of test object classes from existing QuickTest add-ins, such as WebButton, WebEdit, Page, etc.

- The path of the icon file to use for this test object class (Optional. If not defined, a default icon is used.) The icon file must be in an uncompressed .ico format.

- A context-sensitive Help topic to open when F1 is pressed for the test object in the Keyword View or Expert View. The definition includes the Help file path and the relevant Help ID within the file.

- The methods of the new test object, including the following information for each method:

1. The arguments, including the argument type (String or Variant) and direction (In or Out)
2. Whether the argument is mandatory, and, if not, its default value
3. Whether a dynamic list of possible values for this argument can be retrieved from the supported control and displayed in the Keyword View, Expert View, and Step Generator
4. The method description (shown as a tooltip in the Keyword View, Expert View, and Step Generator)
5. The Documentation string (shown in the Documentation column of the Keyword View and in the Step Generator)
6. The return value type
7. A context-sensitive Help topic to open when F1 is pressed for the test object method in the Keyword View or Expert View, or when the Operation Help button is clicked for the method in the Step Generator. The definition includes the Help file path and the relevant Help ID within the file.

- The test object method that is selected by default in the Keyword View and Step Generator when a step is generated for an object of this class

- The identification properties of the new test object

1. The identification properties that are used for the object description
2. The identification properties that are used for Smart Identification (Smart Identification needs to be enabled for these definitions to take effect)
3. The identification properties that are available for use in checkpoints
4. The identification properties that are selected by default for checkpoints (in the QuickTest Checkpoint Properties dialog box)

Each time you run QuickTest, it reads all of the test object configuration XMLs and merges the information for each test object class from the different XMLs into one test object definition.

Each time you open QuickTest, it reads all of the test object configuration files located in the \dat\Extensibility\ folders. QuickTest then merges the information for each test object class from the different files into a single test object class definition, according to the priority of each test object configuration file.

You define the priority of each test object configuration file using the Priority attribute of the TypeInformation element.

Please Note:

- If the priority of a test object configuration file is higher than the existing class definitions, it overrides any existing test object class definitions, including built-in QuickTest information. For this reason, be aware of any built-in functionality that will be overridden before you change the priority of a test object configuration file.

- QuickTest ignores the definitions in a test object configuration files in the following situations:

a. The Load attribute of the TypeInformation element is set to false.
b. The environment relevant to the test object configuration file is displayed in the Add-in Manager dialog box, and the QuickTest user selects not to load the environment.

When multiple test object class definitions exist, QuickTest must handle any conflicts that arise. The following sections describe the process QuickTest follows when ClassInfo, ListOfValues, and Operation elements are defined in multiple test object configuration files. All of the IdentificationProperty elements for a specific test object class must be defined in only one test object configuration file.

ClassInfo Elements:

If a ClassInfo element is defined in a test object configuration file with a priority higher than the existing definition, the information is appended to any existing definition. If a conflict arises between ClassInfo definitions in different files, the definition in the file with the higher priority overrides (replaces) the information in the file with the lower priority.

If a ClassInfo element is defined in a test object configuration file with a priority that is equal to or lower than the existing definition, the differing information is appended to the existing definition. If a conflict arises between ClassInfo definitions in different files, the definition in the file with the lower priority is ignored.

ListOfValues Elements:

- If a conflict arises between ListOfValues definitions in different files, the definition in the file with the higher priority overrides (replaces) the information in the file with the lower priority (the definitions are not merged).

- If a ListOfValues definition overrides an existing list, the new list is updated for all arguments of type Enumeration that are defined for operations of classes in the same test object configuration file.

- If a ListOfValues is defined in a configuration file with a lower priority than the existing definition, the lower priority definition is ignored.

Operation Elements:

- Operation element definitions are either added, ignored, or overridden, depending on the priority of the test object configuration file.

- If an Operation element is defined in a test object configuration file with a priority higher than the existing definition, the operation is added to the existing definition for the class. If a conflict arises between Operation definitions in different files, the definition in the file with the higher priority overrides (replaces) the definition with the lower priority (the definitions are not merged).

Implement Web Add-in Extensibility in QTP

2008-12-09T03:29:00.000-08:00

Using Web Add-in Extensibility, you can implement full support for all QuickTest features for your controls. You can implement Web Add-in Extensibility support for a set of controls (also referred to as a toolkit or custom toolkit) by developing a toolkit support set.

Implementing Web Add-in Extensibility consists of the following stages:

1. Planning the Toolkit Support Set

- Determine the set of Web controls that comprise your custom toolkit.

- Define the test object model by determining which test objects and operations you want to support based on the controls and business processes supported by your toolkit.

2. Developing the Toolkit Support Set: A Web Add-in Extensibility toolkit support set is comprised of the following files:

- One test object configuration file, which describes the test object model for your toolkit to QuickTest.

- One toolkit configuration file, which describes which test object class represents each control in the toolkit and how QuickTest interacts with each control.

- One or more files containing JavaScripts that QuickTest can call to perform operations on the custom controls.

3. Deploying the Toolkit Support Set: To deploy your toolkit support set and enable QuickTest to support your controls, you need to copy the files you created to specific locations within the QuickTest installation folder.

After you deploy the toolkit support set, when QuickTest opens, it displays your toolkit name as a child node under the Web Add-in node in the Add-in Manager. If you select the check box for your toolkit, QuickTest supports the controls in this toolkit using the toolkit support set that you developed.

4. Enhancing the Toolkit Support Set: After you have created and tested basic Web Add-in Extensibility support for your controls you can enhance your toolkit support set by using some of the more complex options to fine-tune your support.

When you develop a Web Add-in Extensibility toolkit support set, you can start by creating a simple and basic support set and deploying it to QuickTest. This enables QuickTest to recognize your controls correctly and enables QuickTest user to create and run tests on the controls. You can then enhance your support to enable more complex capabilities, such as filtering the child objects learned with a control and more advanced handling of events when recording a test.

QTP, CSTE, ISEB, ISTQB, CSQA Question Papers

2008-12-08T22:37:00.000-08:00

`
Here are some of the sample question papers of QTP HP-M016, CSTE, ISEB, ISTQB and CSQA:

CSQA Question Papers - This is the place where you will find question papers for Certified Software Quality Analyst (CSQA) exam.

QTP HP-M016 Sample Papers: For QTP certification you may refer to the below question papers.

CSTE Sample Papers: Below are four question papers for Certified Software Test Engineer (CSTE) exam.

ISTQB Sample Papers: If you are preparing for ISTQB foundation level exam, then below papers might help you.

ISEB Sample Papers: For ISEB Software Testing exams, please refer to the following links of questions:

Silent Test Runner

2008-11-29T08:18:00.000-08:00

Silent Test Runner enables you to simulate the way a QuickTest test runs from LoadRunner and Business Availability Center. When you run a test using Silent Test Runner, it runs without opening the QuickTest user interface, and the test runs at the same speed as when it is run from LoadRunner or Business Availability Center At the end of the test run, you can view information about the test run and transaction times.

You can also use Silent Test Runner to verify that your QuickTest test is compatible with LoadRunner and Business Availability Center. A test will fail when run using Silent Test Runner if it uses a feature that is not supported by LoadRunner or Business Availability Center.

Note: You cannot run Silent Test Runner if QuickTest is already open or another test is currently running. You must close QuickTest and wait for its process to end before running your test using Silent Test Runner. You cannot use the ResultDir QuickTest environment variable when running a test from Silent Test Runner.

To run a QuickTest test using Silent Test Runner:

1. To open Silent Test Runner, choose Start > Programs > QuickTest Professional > Tools > Silent Test Runner. The Silent Test Runner dialog box opens.

2. Click the Browse button to navigate to your test. The Open Test dialog box opens and displays the tests located in your \Tests folder.

3. Select the test you want to run and click Open. The Open Test dialog box closes, the test name appears in the Test box of the Silent Test Runner dialog box, and the Run Test button is enabled. If you select a test that you ran previously, the Test Run Log and Transaction Summary buttons are enabled and you can display information about the last run of the selected test. The first time you run a test, the Test Run Log and Transaction Summary buttons are disabled.

4. Click Run Test to run your test. The test runs without opening the QuickTest user interface. The text Running test... appears next to the Run Test button while the test is running. After you start a test run, you cannot stop the test run from Silent Test Runner. If you close Silent Test Runner, the test continues to run. You can end a test by ending the mdrv.exe process.

5. When the test run finishes, the text Running test is replaced with the text Test run completed. If Silent Test Runner was unable to run your test, the text Test could not be run appears. If previously disabled, the Test Run Log button is enabled. The Transaction Summary button is also enabled if you ran a test with transactions and the button was previously disabled.

Viewing Test Run Information: Silent Test Runner provides test run information in log files. Each test generates a test run log, and any test with transactions generates an additional transaction summary.

Viewing the Test Run Log: The test run log is saved as output.txt in the \Tests\ folder. A log file is saved for each test run with Silent Test Runner and is overwritten when you rerun the test. To open the log file, click Test Run Log.

The log file displays information about the test run. For example, information is shown about each iteration, action call, step transaction, failed step, and so forth. Each line displays a message or error ID.

Viewing the Transaction Summary: The transaction summary is saved as transactions.txt in the \Tests\ folder. A transaction summary is saved for each test that includes transactions and is overwritten when you rerun the test. To open the log file, click Transaction Summary. The transaction summary displays a line for each transaction in the test. For each transaction, the status is displayed together with the total duration time and any wasted time (in seconds). The transaction measurements in Silent Test Runner are exactly the same as if the test was run from LoadRunner or Business Availability Center. A transaction summary is available only for a test that contains transactions ending with an EndTransaction statement. If a transaction started but did not end because of test failure, it is not included in the transaction summary.

Distributed transactions (transactions that start in one test and end in another) are not reported in the transaction summary but are included in the test run log.

Any transaction information included in the transaction summary is also included in the test run log.

Also See: Other LoadRunner Tutorials

Measuring Transactions

2008-11-29T08:11:00.000-08:00

You can measure how long it takes to run a section of your test by defining transactions. A transaction represents the process in your application that you are interested in measuring. Your test must include transactions to be used by LoadRunner or the Business Process Monitor. LoadRunner and the Business Process Monitor use only the data that is included within a transaction, and ignore any data in a test outside of a transaction.

You define transactions within your test by enclosing the appropriate sections of the test with start and end transaction statements. For example, you can define a transaction that measures how long it takes to reserve a seat on a flight and for the confirmation to be displayed on the client's terminal.

During the test run, the StartTransaction step signals the beginning of the time measurement. The time measurement continues until the EndTransaction step is reached. The test report displays the time it took to perform the transaction.

Note: If you start a transaction while there is already open transaction with the same name, the previous transaction is ended with Fail status and then the new transaction is started.

There is no limit to the number of transactions that can be added to a test. You can also insert a transaction within a transaction.

You can insert a variety of transaction-related statements using the Step Generator or Expert View. You can also enter Start Transaction and End Transaction steps using options in the QuickTest window.

Inserting Transactions:During the test run, the Start Transaction signals the beginning of the time measurement. You define the beginning of a transaction in the Start Transaction dialog box.

To insert a transaction:

1. Click the step where you want the transaction timing to begin. The page is displayed in the Active Screen tab.

2. Click the Start Transaction button or choose Insert > Start Transaction. The Start Transaction dialog box opens.

3. Enter a meaningful name in the Name box.

4. Decide where you want the transaction timing to begin. To insert a transaction before the current step, select Before current step and To insert a transaction after the current step, select After current step.

5. Click OK. A Start Transaction step is added to the Keyword View.

Ending Transactions: During the test run, the End Transaction signals the end of the time measurement. You define the end of a transaction in the End Transaction dialog box.

Note: There may be cases in which you want to instruct QuickTest to perform all the steps in a transaction, even though an error occurs during the run session. In the Run tab of the Test Settings dialog box (File > Settings), select proceed to next step from the When error occurs during run session list.

To end a transaction:

1. Click the step where you want the transaction timing to end. The page opens in the Active Screen.

2. Click the End Transaction button or choose Insert > End Transaction. The End Transaction dialog box opens.

3. The Name box contains a list of the transaction names you defined in the current test. Select the name of the transaction you want to end.

4. Decide where to insert the end of the transaction. To insert a transaction before the current step, select Before current step and to insert a transaction after the current step, select After current step.

5. Click OK. An End Transaction step is added to the Keyword View.

Also See: Other LoadRunner Tutorials

Inserting and Running Tests in LoadRunner or Business Process Monitor

2008-11-29T08:02:00.000-08:00

Before you insert and run your QuickTest test in LoadRunner or Business Process Monitor, you should consider the guidelines below.

Inserting and Running Tests in a LoadRunner Scenario:

1. You can run only one GUI Vuser concurrently per computer. (A GUI Vuser is a Vuser that runs a QuickTest test.)

2. To insert a QuickTest test in a LoadRunner scenario, in the Controller Open Test dialog box, browse to the test folder and select Astra Tests in the Files of type box. This enables you to view QuickTest tests in the folder.

3. Ensure that QuickTest is closed on the QuickTest computer before running a QuickTest test in LoadRunner.

4. Transaction breakdown is not supported for tests (scripts) recorded with QuickTest.

5. QuickTest cannot run on a computer that is logged off or locked. In these cases, consider running QuickTest on a terminal server or Make sure that the test is finished before starting to run another QuickTest test.

6. The settings in the LoadRunner Run-time Settings dialog box are not relevant for QuickTest tests.

7. You cannot use the ResultDir QuickTest environment variable when running a test in LoadRunner.

Inserting and Running Tests from Business Process Monitor:

1. Before you try to run a QuickTest test in Business Process Monitor, check which versions of QuickTest are supported by your version of Business Process Monitor.

2. Business Process Monitor can run only one QuickTest test at a time. Make sure that the previous QuickTest test is finished before starting to run another QuickTest test.

3. Ensure that QuickTest is closed on the QuickTest computer before running a QuickTest test in Business Process Monitor.

4. Transaction breakdown is not supported for tests recorded with QuickTest.
If you make changes to your local copy of a QuickTest test after uploading it to Business Availability Center, you will need to upload the zipped test again to enable Business Process Monitor to run the test with your changes.

5. QuickTest cannot run tests on a computer that is logged off, locked, or running QuickTest as a non-interactive service.

6. You cannot use the ResultDir QuickTest environment variable when running a test in Business Process Monitor.

Note: You can simulate how the test will run from LoadRunner or Business Process Monitor by using Silent Test Runner.

Also See: Other Load Runner Tutorials

QuickTest Tests for Use with LoadRunner or Business Process Monitor

2008-11-29T07:59:00.000-08:00

QuickTest Tests for Use with LoadRunner or Business Process Monitor: The QuickTest tests you use with LoadRunner or Business Process Monitor should be simple, designed to pinpoint specific operations, and should avoid using external actions and references to other external files (including resources stored in Quality Center). Also, when working with action iterations, corresponding StartTransaction and EndTransaction statements must be contained within the same action.

Designing Tests for LoadRunner: Consider the following guidelines when designing tests for use with LoadRunner:

1. Do not include references to external actions or other external resources (including resources stored in Quality Center), such as an external Data Table file, environment variable file, shared object repositories, function libraries, and so forth. This is because LoadRunner may not have access to the external action or resource. (However, if the resource can be found on the network, QuickTest will use it.)

2. Every QuickTest test must contain at least one transaction to provide useful information in LoadRunner.

3. Make sure that the last step(s) in the test closes the application being tested, as well as any child processes that are running. This enables the next iteration of the test to open the application again.

Designing Tests for Business Process Monitor: Consider the following guidelines when designing tests for use with Business Process Monitor:

1. Every QuickTest test must contain at least one transaction to provide useful information in Business Process Monitor.

2. When measuring a distributed transaction over two different Business Process Monitor profiles, the profile with the StartDistributedTransaction statement must be run before the profile with the associated EndDistributedTransaction.

3. When measuring distributed transactions, make sure that you relate the tests to a single Business Process Monitor instance. Business Process Monitor searches for the end transaction name in all instances, and may close the wrong distributed transaction if it is included in more than one instance.

4. When measuring a distributed transaction over two Business Process Monitor profiles, make sure that the timeout value you specify is large enough so that the profile that contains the StartDistributedTransaction step and all the profiles that run before the profile that contains the EndDistributedTransaction step, will finish running in a time that is less than the value of the specified timeout.

5. Business Process Monitor does not support running QuickTest Professional tests that require access to external resources, including resources stored in Quality Center (such as a shared object repository, function library, external Data Table, external actions, and so forth). Tests that require external resources may fail to run on Business Process Monitor. (However, if the resource can be found on the network, QuickTest will use it.)

6. Make sure that the last step(s) in the test closes the application being tested, as well as any child processes that are running. This cleanup step enables the next test run to open the application again.

Mercury Performance Testing and Business Availability Center Products

2008-11-29T07:51:00.000-08:00

After you use QuickTest to create and run a suite of tests that test the functional capabilities of your application, you may want to test how much load your application can handle or to monitor your application as it runs.

Mercury LoadRunner tests the performance of applications under controlled and peak load conditions. To generate load, LoadRunner runs hundreds or thousands of virtual users. These virtual users provide consistent, repeatable, and measurable load to exercise your application just as real users would.

Mercury Business Availability Center enables real-time monitoring of the end user experience. Business Process Monitor runs virtual users to perform typical activities on the monitored application.

If you have already created and perfected a test in QuickTest that is a good representation of your users' actions, you may be able to use your QuickTest test as the basis for performance testing and application management activities. You can use Silent Test Runner to check in advance that a QuickTest test will run correctly from LoadRunner and Business Process Monitor.

QuickTest enables you to create complex tests that examine the full spectrum of your application's functionality to confirm that every element of your application works as expected in all situations.

The run mechanisms used in all Mercury Performance Testing and Mercury Business Availability Center products are the same. This means that you can create tests that are compatible with LoadRunner and Business Process Monitor, enabling you to take advantage of tests or test segments that have already been designed and debugged in QuickTest.

For example, you can add QuickTest tests to specific points in a LoadRunner scenario to confirm that the application's functionality is not affected by the extra load at those sensitive points. You can also run QuickTest tests on Business Process Monitor to simulate end user experience and ensure that your application is running correctly and in a timely manner.

QuickTest also offers several features that are designed specifically for integration with LoadRunner and Business Process Monitor. However, since LoadRunner and Business Process Monitor are designed to run tests using virtual users representing many users simultaneously performing standard user operations, some QuickTest features may not be available when integrating these products with QuickTest.

If you do plan to use a single test in both QuickTest and LoadRunner and/or Business Process Monitor, you should take into account the different options supported in each product as you design your test.

QuickTest Performance Testing and Business Availability Center Features: You can use the Services object and its associated methods to insert statements that are specifically relevant to Performance Testing and Business Availability Center. These include AddWastedTime, EndDistributedTransaction, EndTransaction, GetEnvironmentAttribute, LogMessage, Rendezvous, SetTransaction, SetTransactionStatus, StartDistributedTransaction, StartTransaction, ThinkTime, and UserDataPoint. For more information on these methods, refer to the Services section of the QuickTest Professional Object Model Reference and your LoadRunner or Business Availability Center documentation.

You can also insert StartTransaction and EndTransaction statements using the Insert > Start Transaction and Insert > End Transaction menu options or toolbar buttons to insert the statement.

Note: LoadRunner and Business Process Monitor use only the data that is included within a transaction, and ignore any data in a test outside of a transaction.

Also See: Other LoadRunner Tutorials

BPT Methodology in QTP and Quality Center

2008-11-29T06:59:00.000-08:00

Each scenario that the Subject Matter Expert creates is a business process test. A business process test is composed of a serial flow of components. Each component performs a specific task. A component can pass data to a subsequent component.

Understanding Components: Components are easily-maintained reusable scripts that perform a specific task, and are the building blocks from which an effective business process testing structure can be produced. Components are parts of a business process that has been broken down into smaller parts. For example, in most applications users need to log in before they can do anything else. A Subject Matter Expert can create one component that represents the login procedure for an application. Each component can then be reused in different business process tests, resulting in easier maintenance, updating, and test management.

Components are comprised of steps. For example, the login component's first step may be to open the application. Its second step could be entering a user name. Its third step could be entering a password, and its fourth step could be clicking the Enter button.

You can create and edit components in QuickTest by adding steps on any supported environment, parameterizing selected items, and enhancing the component by incorporating functions (operations) that encapsulate the steps needed to perform a particular task. In Quality Center, a Subject Matter Expert creates components and combines them into business process tests, which are used to check that the application behaves as expected.

Creating Components in the Quality Center Business Components Module:The Subject Matter Expert can create a new component and define it in the Quality Center Business Components module. The Business Component module includes the following:

Details: Provides a general summary of the component's purpose or goals, and the condition of the application before and after a component is run (its pre-conditions and post-conditions).

Snapshot: Displays an image that provides a visual cue or description of the component's purpose or operations.

Parameters: Specifies the input and output component parameter values for the business component. Implementing and using parameters enables a component to receive data from an external source and to pass data to other components in the business process test flow.

Design Steps: Enables you to create or view the manual steps of your business component, and to automate it if required.

Automation: Displays or provides access to automated components. For keyword-driven components, enables you to create and modify the steps of your automated business component in a keyword-driven, table format, and provides a plain-language textual description of each step of the implemented component.

Used by: Provides details about the business process tests that include the currently selected business component. The tab also includes a link to the relevant business process test in the Test Plan module.

Component Requests pane: Enables you to handle new component requests that were generated in the Test Plan module. Component requests are requests to add a new business component to the project.

Implementing Components in QuickTest Professional: Generally, components are created by Subject Matter Experts in Quality Center, although they can also be created and debugged in QuickTest.

In QuickTest, you create components by recording steps on any supported environment or by adding steps manually (if the object repository is populated and the required operations are available). You can parameterize selected items. You can also view and set options specific to components.

QuickTest enables you to create and modify two types of components: business components and scripted components. A business component is an easily-maintained, reusable unit comprising one or more steps that perform a specific task. A scripted component is an automated component that can contain programming logic. Scripted components share functionality with both test actions and business components. For example, you can use the Keyword View, the Expert View, and other QuickTest tools and options to create, view, modify, and debug scripted components in QuickTest. Due to their complexity, scripted components can be edited only in QuickTest. (If needed, you can convert test actions to scripted components. For more information, click the Help button in the Action Conversion Tool window.)

In Quality Center, the Subject Matter Expert can open components created in QuickTest. The Subject Matter Expert can then view and edit business components, but can only view the details for scripted components.

Creating Business Process Tests in the Quality Center Test Plan Module: To create a business process test, the Subject Matter Expert selects (drags and drops) the components that apply to the business process test and configures their run settings.

Note: When you run a business process test from Quality Center, the test run may also be influenced by settings in the QuickTest Remote Agent.

Each component can be used differently by different business process tests. For example, in each test the component can be configured to use different input parameter values or run a different number of iterations.

If, while creating a business process test, the Subject Matter Expert realizes that a component has not been defined for an element that is necessary for the business process test, the Subject Matter Expert can submit a component request from the Test Plan module.

Running Business Process Tests and Analyzing the Results: You can use the run and debug options in QuickTest to run and debug an individual component.

You can debug a business process test by running the test from the Test Plan module in Quality Center. When you choose to run from this module, you can choose which components to run in debug mode. (This pauses the run at the beginning of a component.)

When the business process test has been debugged and is ready for regular test runs, the Subject Matter Expert runs it from the Test Lab module similar to the way any other test is run in Quality Center. Before running the test, the Subject Matter Expert can define run-time parameter values and iterations using the Iterations column in the Test Lab module grid.

From the Test Lab module, you can view the results of the entire business process test run. The results include the value of each parameter, and the results of individual steps reported by QuickTest.

You can click the Open Report link to open the complete QuickTest report. The hierarchical report contains all the different iterations and components within the business process test run.

Understanding the Differences Between Components and Tests: If you are already familiar with using QuickTest to create action-based tests, you will find that the procedures for creating and editing components are quite similar. However, due to the design and purpose of the component model, there are certain differences in the way you create, edit, and run components. The guidelines below provide an overview of these differences.

- A component is a single entity. It cannot contain multiple actions or have calls to other actions or to other components.

- When working with components, all external files are stored in the Quality Center project to which you are currently connected.

- The name of the component node in the Keyword View is the same as the saved component. You cannot rename the node.

- Business components are created in the Keyword View, not the Expert View.
You add resources via the component's application area, and not directly to the component.

- Components use custom keywords created in function libraries to perform operations, such as verifying property values and opening the application you are testing.

Business Process Testing

2008-11-29T06:49:00.000-08:00

The Business Process Testing model is role-based, allowing non-technical Subject Matter Experts (working in Quality Center) to collaborate effectively with Automation Engineers (working in QuickTest Professional). Subject Matter Experts define and document business processes, business components, and business process tests, while Automation Engineers define the required resources and settings, such as shared object repositories, function libraries, and recovery scenarios. Together, they can build, data-drive, document, and run business process tests, without requiring programming knowledge on the part of the Subject Matter Expert.

Note: The role structure and the tasks performed by various roles in your organization may differ from those described here according to the methodology adopted by your organization. These roles are flexible and depend on the abilities and time resources of the personnel using Business Process Testing. For example, the tasks of the Subject Matter Expert and the Automation Engineer may be performed by the same person. There are no product-specific rules or limitations controlling which roles must be defined in a particular organization, or which types of users can do which Business Process Testing tasks (provided that the users have the correct permissions).

The following user roles are identified in the Business Process Testing model:

Subject Matter Expert. The Subject Matter Expert has specific knowledge of the application logic, a high-level understanding of the entire system, and a detailed understanding of the individual elements and tasks that are fundamental to the application being tested. This enables the Subject Matter Expert to determine the operating scenarios or business processes that must be tested and identify the key business activities that are common to multiple business processes.

Using the Business Components module in Quality Center, the Subject Matter Expert creates business components that describe the specific tasks that can be performed in the application, and the condition or state of the application before and after those tasks. The Subject Matter Expert then defines the individual steps for each business component comprising the business process in the form of manual, or non-automated steps.

During the design phase, the Subject Matter Expert works with the Automation Engineer to identify the resources and settings needed to automate the components, enabling the Automation Engineer to prepare them. When the resources and settings are ready, the Subject Matter Expert automates the manual steps by converting them to keyword-driven components. Part of this process entails choosing an application area for each component. The application area contains all of the required resource files and settings that are specific to a particular area of the application being tested. Associating each component with an application area enables the component to access these resources and settings.

Using the Quality Center Test Plan module, the Subject Matter Expert combines the business components into business process tests, composed of a serial flow of the components. For example, most applications require users to log in before they can access any of the application functionality. The Subject Matter Expert could create one business component that represents this login procedure. This component procedure can be used in many business process tests, resulting in easier and more cost-efficient maintenance, updating, and test management.

The Subject Matter Expert configures the values used for business process tests, runs them in test sets, and reviews the results. The Subject Matter Expert is also responsible for maintaining the testing steps for each of the individual business components.

While defining components, Subject Matter Experts continue collaborating with the Automation Engineer. For example, they may request new operations (functions) for a component or discuss future changes planned for the component.

Automation Engineer. The Automation Engineer is an expert in using an automated testing tool, such as QuickTest Professional. The Automation Engineer works with the Subject Matter Expert to identify the resources that are needed for the various business process tests.

The Automation Engineer then prepares the resources and settings required for testing the features associated with each specific component, and stores them in an application area within the same Quality Center project used by the Subject Matter Experts who create and run the business process tests for the specific application.

Each application area serves as a single entity in which to store all of the resources and settings required for a component, providing a single point of maintenance for all elements associated with the testing of a specific part of an application. Application areas generally include one or more shared object repositories, a list of keywords that are available for use with a component, function libraries containing automated functions (operations), recovery scenarios for failed steps, and other resources and settings that are needed for a component to run correctly. Components are linked to the resources and settings in the application area. Therefore, when changes are made in the application area, all associated components are automatically updated.

The Automation Engineer uses QuickTest features and functionality to create these resources from within QuickTest. For example, in QuickTest, the Automation Engineer can create and populate various object repositories with test objects that represent the different objects in the application being tested, even before the application is fully developed. The Automation Engineer can then add repository parameters, and so forth, as needed. The Automation Engineer can manage the various object repositories using the Object Repository Manager, and merge repositories using the Object Repository Merge Tool. Automation Engineers can also use QuickTest to create and debug function libraries containing functions that use programming logic to encapsulate the steps needed to perform a particular task.

Using the resources created by the Automation Engineer, the Subject Matter Experts can automate component steps, and create and maintain components and business process tests.

Automation Engineers can also create, debug, and modify components in QuickTest, if required.