Android Application Testing

How to collect  logs when you are not connected to DDMS?

Logs are very important when you are getting any crash in your app or when your app is not behaving correctly.Traditionally for Android apps, we all collect the logs by connecting to DDMS.However sometimes it becomes so difficult to dig out the log when your device do not get connected to your PC due to some cable problem or any other issues with the device.Recently I faced such problem for my Android Tablet Asus Transformer and it really made me so difficult to provide the crash logs to my developers.If you are the one who faced such problem, here is an app which can help you in this situation.

Log Collector:-

• Log collector is an app/tool  which can provide you application logs whenever required
• You are not required to connect with your PC
• You can share logs via various means like Email,Facebook,Twitter,Bluetooth,Messaging etc.
• You can edit the logs before sending these logs to your developer.

How to get Log Collector?

You can install it from Android Market(Now Google Play). Just type “Log Collector” and search in Google play.This tool/app can be install on Android 1.5 and above.

Screenshots:-

 

 

 

 

Google Increase the file size limit up to 4 GB in Android Market

Here is  yet another rejoicing news for all Android developers. Google has increased the Android app size limit  from 50 MB to 4GB to upload in Android Market. Well the maximum allowed APK (Application Package File) file size will still be limited to 50 MB. However the developers will have two expansion files, each up to 2 GB. What does this really mean? Here are some points which will make you understand about what this update from Google can do for developers and users.

Predominantly  50 MB size is enough for any kind of android applications. One of the famous android game Angry Birds Rio has the file size of 19 MB. However there are some types of apps e.g.  high-quality 3D interactive games  having advanced 3-D graphics, audio, and video which requires more resources.

Before I proceed, It should be clear that Android apps were free to exceed the size limit (50 MB) earlier also; however developers had to host the data on their own.

Traditionally, when user try to download such bigger apps or game from Android market, after downloading the core app, user is prompted to install additional files to complete the download. These additional files had to be hosted by developers .Obviously the file size of these additional files(which may be in GB’S) were not considered while showing the app size of that game/app on Android Market. Hence sometimes even a user may feel fooled as for app/game of 10 MB (as per app size shown in android market) we need to install additional 1-2 GB of data after installing the core apk.

Moreover downloading this additional data were often leading to expiry of the refund time. (Google provides a 15 min Refund window for paid apps. If you download an app and you uninstall within 15 mins, you can automatically get the refund for your purchased price. However For a bigger apps or games, downloading the additional data were often leading to expiry of this refund time as downloading of this additional data may takes several minutes.)

What’s Great Now after this update? 

• As Google has increased the  Android app size limit  from 50 MB to 4GB to upload in Android Market, these additional files will now be hosted on Google’s server and developers will not have to host these additional files on their own expenses on some third party server.
• As there will  be less restriction on the app size, we can expect to have some more good quality application in Android Market.
• As all that data can be fully integrated into the APK and the expansion file, user will be able to see the total size of the app in Android Market before purchasing and downloading it.
• Most Interestingly, unlike earlier, the refund window will begin only when you have installed all the files.
• For Most newer devices, the expansion files will be automatically downloaded when you will download the core app from android market.
• However for older devices Google will provide a downloaded library (developer tool) to download the expansion file.
• In short this update from Google is beneficial for both Developers and Users.:)

Docomo to Launch Remote Testing Center to Fight with Android Device Fragmentation

In Recently held conference on 15-02-12,NTT DOCOMO has decided to introduce an elaborate remote testing center, located at the University of Aizu in Fukushima Prefecture, Japan to help developers test their content. This Remote Testing Solution which is based on Perfecto Mobile Platform and executed by Accenture will have hundreds of handsets of all software versions, screen sizes and will help developers combat with Android device fragmentation. Here are some highlights of this proposed remote testing center:-

• App developer can upload their application on the remote devices and can test their application
• The system will allow 60 handsets to be tested at one time
• Developers will be able reserve time slots on specific handsets
• Developer can upload and test their software  on this platform
• Developers will be able to run automated batch tests.
• Developers will have Android testing Interface through which they can take desired action such as swipes, taps at specific locations and button presses.
• Remote testers will be able to use the Android testing interface, which allows for actions
• More advanced inputs, like pinching on the touch display, or GPS and accelerometer readings, will not be accessible
• This service will enable smooth portability of content adaptation across devices.

 

Mobile testing center powered by the MobileCloud platform

In short this solution will be similar as Perfecto Mobile cloud or Device Anywhere from where user can remotely test their application.

 

SeeTest & the Android multi-device challenge

Summary: there are many Android devices. When doing mobile test automation – say with SeeTest from Experitest – it is not practical to test ALL devices. The question is then – How to select a small number of Android devices that will enable to identify issues across 99% of ALL Android devices?

In this short article we will suggest two strategies – Edge strategy (testing the most extreme devices) and Commonality strategy (testing the most common, popular devices). Our suggested strategy to obtain optimal results in Android test automation would be a combination of both strategies.

The Challenge – testing thousands of Android devices

The business module behind the Android mobile platform creates huge challenge when you come to test your application. Your application will run on multiple hardware platforms and software versions.

The question is how to set your risk management strategy when you come to select the devices to perform your testing on?

Ideally you would run your entire regression test on all the possible variation of phone model / all Android OS versions / and all vendor firmware versions.

To get an idea of what we are looking at:

As of Q3 2011 there are:

• ~130 (without tablets) different HW modules.
• 7 Android Platform versions: 1.5, 1.6, 2.1, 2.2, 2.3, 3.0 and 3.1.
• The vendor firmware can also vary and can be updated from time to time  – a realistic assumption would be ~2 firmware per device.

So in order to test all the permutations of Android devices we are looking at around 1800 combinations (=130 HW device models x 7 SW OS versions x 2 firmware)

For a full list of devices: http://en.wikipedia.org/wiki/List_of_Android_devices

Needless to say this is totally impractical. No mobile automation engineer can seriously target such a large set of devices.

The Solution – identify relevant device factors

The question then is how to narrow down this huge list of thousands of Android devices and select a subset to test on?

The critical thing when narrowing the list is to identify which factors of the device might affect your application behavior and make sure to cover all the possible permutations of such factors. Tackling the factors instead of the devices themselves enables to narrow down to a subset of say 8 devices and provide coverage for ~90% of the devices.

Following is a list of the factors that can affect your application behavior:

• Screen size – One of your main concerns is the screen side of the devices that will execute your application. One of the great pitfalls of building an Android application is how to build the view layout so it will render correctly on all screen sizes.The screens sizes can vary from QVGA (240×320), WVGA (480×800), HVGA (320×480), FWVGA (480×854) , in tablets you will   find 1024×600 and what probably will be the standard for tablets 1280×800.So running the same application on 240×320 screens and in the same time on 1280×800 screens is huge challenge.

             To emphasize it please look at the relative size difference (the red rectangular representing the smaller size and the black    presenting the larger size):

• Android OS versions – The android platform is changing very fast. The difference from version 1.5 to version 2.3 is huge. Lots of new capabilities like graphics HW acceleration were added and can affect your application.
• CPU – Mobile devices in general are very sensitive to processing power. We can see phones with single core running at 600 MHz to phones that have duel core 1200 MHz.

Two other relevant factors of relatively minor effect are GPU (Low-Medium affect) and Manufacture (Low affect).

The Strategy – combined Edge & Commonality strategy

To reduce the risk you can work in 2 strategies:

1. Edge Strategy – select phones that have factors that are at the edges of the scale: minimum screen size, maximum screen size, running with OS version of 1.5 and running on the latest version and minimum CPU power.
   As we  combine them together we can end up with 2 devices:
   On the lower end:
   HTC Hero: with android 1.5, 320×480 HVGA screen and 528 MHz Qualcomm CPU

            On the highest end:
Galaxy Tab 10.1v: with android 3.0, 1280×800 screen and 1000MHz dual core.

        2. Commonality Strategy – analyze what is the probability each phone has to meet your application and select those with the                   highest probability.The analysis here is more complex and defendant on your application type, region, target age and more.

            There is a huge difference if your application is a business application that is targeted for the US, of if it’s a teenager game for girls    in  Japan.

A combination of the 2 abovementioned strategies will probably give you the best result

SOURCE: http://www.appbrain.com/stats/

Seetest-A mobile Test Automation tool for Android, iPhone, Blackberry, Symbian & WindowsPhone 7

Need for Mobile Test Automation Tool on Various Platforms (Android, iPhone, Blackberry, Symbian & WindowsPhone 7) 

Since launch of the First Mobile phone in the market to the launch of hundreds of devices a month,the market has changed drastically.Unlike earlier, now:-

• There are lots of application getting developed and launched in to the market everyday.
• Enterprises (banks, retailers etc) need to provide access to their online services via smartphones.
• Application developers need to quickly launch their apps on various platforms to cope up with competitive market.
• Hardware vendors are developing many different devices based on Android and other mobile OS.

Thousands of mobile devices in the market, different OS/Platforms with different OS versions and competitive market,all these things lead to need of a comprehensive & effective automation tool which can help in  cost effective Testing of a Mobile Application or hardware they have developed in quick time.More specifically, they need a test automation tool for real physical mobile devices (not emulators) that has full coverage, ability to integrate  into their existing testing environment , is simple & quick to operate and can run the same test  on many devices/mobile OS.AND MOST IMPORTANTLY – a downloadable trial version – because there’s nothing like trying it yourself.

Well all these and many other challenges/Issues  in Mobile Application Testing will  now be addressed by an efficient Mobile Automation tool from Experitest Inc.

Experitest (www.experitest.com) – a strategic partner of both HP and Microsoft – has developed SeeTest – a test automation tool for mobile that meets all these requirements and is deployed in Fortune 500 companies worldwide, such as Microsoft, NYSE, Marvell, Texas Instruments, Clicksoftware, BSkyB, 888, Cisco and many more.  To watch a video click here

Requirement 1:  Full coverage – all smartphone OS, all functionality

All mobile OS:

•  All types of OS:  There are 5 Android, iOS, Blackberry, WindowsPhone 7 and Symbian. To provide a real coverage of your customer base you need a tool that can test on any of these.All mobile OS versions: OS versions are constantly launched ot the market. There is need ot support all of them.
• All mobile device models: phones, tablets. Both are used today by end users. So both need to be tested.

All functionality: Smartphones have a broad range of gestures, system alerts and many other features.  All need to be supported. Otherwise no real automation can be achieved:

• Gestures:Swipe, drag &drop, zoom in and zoom out, mutlitouch.
•  System alerts:Security pop ups.
  
• Virtual keyboards:  all keyboard configurations

SeeTest is the most comprehensive mobile test tool today in the market. It covers all OS, all functionality. You name it, SeeTest has it.

Requirement 2:  Plug into QTP,TestComplete,MSTest,Junit,Perl,Python:-

Mobile testing is “the new guy on the neighborhood”. It is joining into a realty where organizations already have an existing testing environment such as QTP, TestComplete, MSTest, Java, Perl, Python. Naturally, organizations are looking for a solutions that can easily integrate into these existing environments so that they can continue working from their usual testing environment only extend it to cover mobile testing as well.

SeeTest has plugins into all testing environments such as QTP, Testcomplete, MSTest, JUnit, Perl and Pyhton.

Let’s take for example QTP. The SeeTest plugin enables to work from within QTP and simply create tests the regular and usual way tests are created in QTP (Keyword view, Expert view, data driven tests, test results and anaylsis), only that this time it is done on real physical mobile devices connected via a standard USB cable to the tester’s computer. The user can record/edit the test, run it and view reports – all in QTP. Just as he has always done. Same goes for all the HP testing & monitoring tools such as QC and LoadRunner.

 

Example screenshots: Keyword view, Expert view, data driven tests, test results

 

Requirement 3: simple & quick to Operate – a recorder

The mobile world changes quickly. And so should the ability to create tests easily and efficiently. The SeeTest recorder enables simple, quick test creation.

Requirement 4: Same script running on multiple devices and mobile OS.

There are 5 smartphone OS. There are hundreds of smartphone device models. There are constantly new OS versions. This reality mandates one clear and strong requirement – script once, run on any device/OS.

One of SeeTest’s main strengths is its ability to run the very same test script on any mobile OS. Any physical device. No exceptions. This brings clear and indisputable ROI to the mobile automation project.

Requirement 5: Why believe marketing materials? Try it yourself now – free downloadable trial.

Click here to download (download starts automatically)

Mobile Application Testing

Developing applications for the Android platform is a complicated business. You have to test with multiple operating system versions, hardware vendor interface layers, hardware configurations, and network capabilities. The testing matrix for Android-based applications can be a serious challenge, impacting your product’s quality, time-to-market, and in the end, profitability.

Impetus mAutomate efficiently addresses your Android-based application testing concerns. It is an automated testing framework that runs on a cluster of phones from a variety of vendors, operating system versions, networks, and geographies. Using mAutomate you can execute automated tests on the cloud, on the devices connected over the air to your target operator networks.

 

mAutomate enables you to:-

• ‘Record/Execute’ test cases ‘for/on’ multiple devices at a time.
• Save test cases and their various versions, for multiple mobile applications, at a single place and then retrieve them later for run on multiple devices.
• Test case runs simultaneously on multiple devices.
• Receive test results at a predetermined frequency of time or number of test cases/steps of run.
• Invoke priority-based product support for the mobile application testing product used within your enterprise.

 

 

This framework, integrated with the client application, will automatically scale itself based on application functionality. It will undertake automatic instrumentation in the client application to make it compatible with Impetus’ automation testing engine.

mAutomate framework helps you in generating information about the CPU, Memory, Services, battery, network and various other analytics from the device while running test cases. While mAutomate is currently available only on the Android OS, Impetus is developing test platforms for the iOS, BlackBerry and also supporting Mobile Web.

Another gray area that the framework helps test engineers to deal with is, acquiring performance analytics details of an application. Typically, performance testing helps you to:

• Identify bottlenecks and their causes.
• Optimize applications based on performance results so that they can run perfectly on maximum configurations.
• Verify the reliability of applications under stress.

 

The Automation Process:-

 The complete automation process consists of the following five key steps:

• Step 1: Ensures that the Mobile Client Application is re-built with a platform specific library, to create a deployable test application.
• Step 2: Executes the test application on the mobile device in record mode so that all the user events and their impact in the normal flow of the application are recorded by the library in form of test scripts.
• Step 3: Test scripts generated in step 2 above, are sent to the server and made available on the web Interface for View/editing.
• Step 4: Enables the test scripts on the web interface to add more test data or create regression scenarios around them. A push message is then sent to the selected devices on the Cloud which launches the test application, fetches the test scripts from the server and executes them on multiple devices simultaneously.
• Step 5: Ensures that all the events and their outcomes are played and sent back to the server along with performance parameters (such as time taken in execution of the test case, memory used, CPU cycles used and impact on battery), and screen shots, where they are processed by the server to produce the test results along with activity performance analysis reports.

 

Key features:-

• Add application/product space.
• Create test builds for application/product.
• Associate test builds with application/product space.
• Add your own remote devices, by getting a small service app installed on them.
• Record test cases/scripts/data on a reference device/emulator.
• Associate test cases/scripts/data with application/product space.
• Maintain test cases/scripts/data for each application/product.
• Select devices/emulators to run your test scripts.
• Get test results e-mailed to you (after completing the entire run, the fixed number of steps, and after every X units of time) – PDF format supported currently.

 

Product Benefits:-

Reliability: It allows you to perform the same operation on each iteration, eliminating human error

Repeatability: It enables you to repeat test cases, multiple times

Better quality: You can test under extreme stress scenarios that go beyond manual testing

Time savings: It helps you run test suites on multiple remote targets at high speed, thus saving time

 

Product Trial:-

Request for Trial:www.mautomate.com

Cost savings: It allows you to use devices shared on the Cloud,without having to build up a costly library of reference devices

The top 10 IT disasters of all time

  Disasters……:(:(

1.  Mariner Bugs Out (1962):

Cost: $18.5 million

Disaster: The Mariner 1 rocket with a space probe headed for Venus diverted from its intended flight path shortly after launch.  Mission Control destroyed the rocket 293 seconds after liftoff.

Cause: A programmer incorrectly transcribed a handwritten formula into computer code, missing a single superscript bar.  Without the smoothing function indicated by the bar, the software treated normal variations of velocity as if they were serious, causing faulty corrections that sent the rocket off course. (more)

2.  Hartford Coliseum Collapse (1978)

Cost: $70 million, plus another $20 million damage to the local economy

Disaster: Just hours after thousands of fans had left the Hartford Coliseum, the steel-latticed roof collapsed under the weight of wet snow.

Cause: The programmer of the CAD software used to design the coliseum incorrectly assumed the steel roof supports would only face pure compression.  But when one of the supports unexpectedly buckled from the snow, it set off a chain reaction that brought down the other roof sections like dominoes.  (more)

3.  World War III… Almost (1983)

Cost: Nearly all of humanity

Disaster: The Soviet early warning system falsely indicated the United States had launched five ballistic missiles.  Fortunately the Soviet duty officer had a “funny feeling in my gut” and reasoned if the U.S. was really attacking they would launch more than five missiles, so he reported the apparent attack as a false alarm.

Cause: A bug in the Soviet software failed to filter out false missile detections caused by sunlight reflecting off cloud-tops.  (more)

4.  Wall Street Crash (1987)

Cost: $500 billion in one day

Disaster: On “Black Monday” (October 19, 1987), the Dow Jones Industrial Average plummeted 508 points, losing 22.6% of its total value. The S&P 500 dropped 20.4%.  This was the greatest loss Wall Street ever suffered in a single day.

Cause: A long bull market was halted by a rash of SEC investigations of insider trading and by other market forces.  As investors fled stocks in a mass exodus, computer trading programs generated a flood of sell orders, overwhelming the market, crashing systems and leaving investors effectively blind.  (more)

5.  AT&T Lines Go Dead (1990)

Cost: 75 million phone calls missed, 200 thousand airline reservations lost

Disaster: A single switch at one of AT&T’s 114 switching centers suffered a minor mechanical problem and shut down the center.  When the center came back up, it sent a message to other switching centers, which in turn caused them to shut down and brought down the entire AT&T network for 9 hours.

Cause: A single line of buggy code in a complex software upgrade implemented to speed up calling caused a ripple effect that shut down the network.  (more)

6.  Pentium Fails Long Division (1993)

Cost: $475 million, corporate credibility

Disaster: Intel’s highly-promoted Pentium chip occasionally made mistakes when dividing floating-point numbers within a specific range. For example, dividing 4195835.0/3145727.0 yielded 1.33374 instead of 1.33382, an error of 0.006%.  Although the bug affected few users, it become a public relations nightmare.  With an estimated 5 million defective chips in circulation, Intel offered to replace Pentium chips only for consumers who could prove they needed high accuracy.  Eventually Intel replaced the chips for anyone who complained.

Cause: The divider in the Pentium floating point unit had a flawed division table, missing about five of a thousand entries and resulting in these rounding errors.  (more)

7.  Ariane Rocket Goes Boom (1996)

Cost: $500 million

Disaster: Ariane 5, Europe’s newest unmanned rocket, was intentionally destroyed seconds after launch on its maiden flight.  Also destroyed was its cargo of four scientific satellites to study how the Earth’s magnetic field interacts with solar winds.

Cause: Shutdown occurred when the guidance computer tried to convert the sideways rocket velocity from 64-bits to a 16-bit format.  The number was too big, and an overflow error resulted.  When the guidance system shut down, control passed to an identical redundant unit, which also failed because it was running the same algorithm.  (more)

8.  Mars Climate Crasher (1998)

Cost: $125 million

Disaster: After a 286-day journey from Earth, the Mars Climate Orbiter fired its engines to push into orbit around Mars.  The engines fired, but the spacecraft fell too far into the planet’s atmosphere, likely causing it to crash on Mars.

Cause: The  software that controlled the Orbiter thrusters used imperial units (pounds of force), rather than metric units (Newtons) as specified by NASA.  (more)

9.  Y2K (1999)

Cost: $500 billion

Disaster: One man’s disaster is another man’s fortune, as demonstrated by the infamous Y2K bug.  Businesses spent billions on programmers to fix a glitch in legacy software.  While no significant computer failures occurred, preparation for the Y2K bug had a significant cost and time impact on all industries that use computer technology.

Cause: To save computer storage space, legacy software often stored the year for dates as two digit numbers, such as “99″ for 1999.  The software also interpreted “00″ to mean 1900 rather than 2000, so when the year 2000 came along, bugs would result.  (more)

10.  Love Virus (2000)

Cost: $8.75 billion, millions of computers infected, significant data loss

Disaster: The LoveLetter worm infected millions of computers and caused more damage than any other computer virus in history.  The worm deleted files, changed home pages and messed with the Registry.

Cause: LoveLetter infected users via e-mail, Internet chat and shared file systems.  The email had an executable file attachment and subject line, “ILOVEYOU.”  When the user opened the attachment, the virus would infect the user’s computer and send itself to everyone in the address book.  (more)

An approach for Security Testing of Web Applications

Introduction

As more and more vital data is stored in web applications and the number of transactions on the web increases, proper security testing of web applications is becoming very important. Security testing is the process that determines that confidential data stays confidential (i.e. it is not exposed to individuals/ entities for which it is not meant) and users can perform only those tasks that they are authorized to perform (e.g. a user should not be able to deny the functionality of the web site to other users, a user should not be able to change the functionality of the web application in an unintended way etc.).

Some key terms used in security testing

Before we go further, it will be useful to be aware of a few terms that are frequently used in web application security testing:

What is “Vulnerability”?
This is a weakness in the web application. The cause of such a “weakness” can be bugs in the application, an injection (SQL/ script code) or the presence of viruses.

What is “URL manipulation”?
Some web applications communicate additional information between the client (browser) and the server in the URL. Changing some information in the URL may sometimes lead to unintended behavior by the server.

What is “SQL injection”?
This is the process of inserting SQL statements through the web application user interface into some query that is then executed by the server.

What is “XSS (Cross Site Scripting)”?
When a user inserts HTML/ client-side script in the user interface of a web application and this insertion is visible to other users, it is called XSS.

What is “Spoofing”?
The creation of hoax look-alike websites or emails is called spoofing.
Security testing approach:

In order to perform a useful security test of a web application, the security tester should have good knowledge of the HTTP protocol. It is important to have an understanding of how the client (browser) and the server communicate using HTTP. Additionally, the tester should at least know the basics of SQL injection and XSS. Hopefully, the number of security defects present in the web application will not be high. However, being able to accurately describe the security defects with all the required details to all concerned will definitely help.

1. Password cracking:

The security testing on a web application can be kicked off by “password cracking”. In order to log in to the private areas of the application, one can either guess a username/ password or use some password cracker tool for the same. Lists of common usernames and passwords are available along with open source password crackers. If the web application does not enforce a complex password (e.g. with alphabets, number and special characters, with at least a required number of characters), it may not take very long to crack the username and password.

If username or password is stored in cookies without encrypting, attacker can use different methods to steal the cookies and then information stored in the cookies like username and password.

For more details see article on “Website cookie testing”.

2. URL manipulation through HTTP GET methods:

The tester should check if the application passes important information in the querystring. This happens when the application uses the HTTP GET method to pass information between the client and the server. The information is passed in parameters in the querystring. The tester can modify a parameter value in the querystring to check if the server accepts it.

Via HTTP GET request user information is passed to server for authentication or fetching data. Attacker can manipulate every input variable passed from this GET request to server in order to get the required information or to corrupt the data. In such conditions any unusual behavior by application or web server is the doorway for the attacker to get into the application.

3. SQL Injection:

The next thing that should be checked is SQL injection. Entering a single quote (‘) in any textbox should be rejected by the application. Instead, if the tester encounters a database error, it means that the user input is inserted in some query which is then executed by the application. In such a case, the application is vulnerable to SQL injection.

SQL injection attacks are very critical as attacker can get vital information from server database. To check SQL injection entry points into your web application, find out code from your code base where direct MySQL queries are executed on database by accepting some user inputs.

If user input data is crafted in SQL queries to query the database, attacker can inject SQL statements or part of SQL statements as user inputs to extract vital information from database. Even if attacker is successful to crash the application, from the SQL query error shown on browser, attacker can get the information they are looking for. Special characters from user inputs should be handled/escaped properly in such cases.

4. Cross Site Scripting (XSS):

The tester should additionally check the web application for XSS (Cross site scripting). Any HTML e.g. <HTML> or any script e.g. <SCRIPT> should not be accepted by the application. If it is, the application can be prone to an attack by Cross Site Scripting.

Attacker can use this method to execute malicious script or URL on victim’s browser. Using cross-site scripting, attacker can use scripts like JavaScript to steal user cookies and information stored in the cookies.

Many web applications get some user information and pass this information in some variables from different pages.

E.g.: http://www.examplesite.com/index.php?userid=123&query=xyz

Attacker can easily pass some malicious input or <script> as a ‘&query’ parameter which can explore important user/server data on browser.

Important: During security testing, the tester should be very careful not to modify any of the following:

•  Configuration of the application or the server
•  Services running on the server
•  Existing user or customer data hosted by the application

Additionally, a security test should be avoided on a production system.

The purpose of the security test is to discover the vulnerabilities of the web application so that the developers can then remove these vulnerabilities from the application and make the web application and data safe from unauthorized actions.

Related Post:

• Open Source Performance Testing Tool
• Regression Testing
• Types Of SDLC Models
• Web Testing: Complete guide on testing web applications
• Software Testing Types
• Testing Tool
• Acceptance Testing

Website Cookie Testing, Test cases for testing web application cookies?

We will first focus on what exactly cookies are and how they work. It would be easy
for you to understand the test cases for testing cookies when you have clear understanding of how cookies work?
How cookies stored on hard drive? And how can we edit cookie settings?
What is Cookie?
Cookie is small information stored in text file on user’s hard drive by web server. This information is later used by web browser to retrieve information from that machine. Generally cookie contains personalized user data or information that is used to communicate between different web pages.

Why Cookies are used?

Cookies are nothing but the user’s identity and used to track where the user navigated throughout the web site pages. The communication between web browser and web server is stateless.

For example if you are accessing domain http://www.example.com/1.html then web browser will simply query to example.com web server for the page 1.html. Next time if you type page as http://www.example.com/2.html then new request is send to example.com web server for sending 2.html page and web server don’t know anything about to whom the previous page 1.html served.

What if you want the previous history of this user communication with the web server? You need to maintain the user state and interaction between web browser and web server somewhere. This is where cookie comes into picture. Cookies serve the purpose of maintaining the user interactions with web server.

How cookies work?
The HTTP protocol used to exchange information files on the web is used to maintain the cookies. There are two types of HTTP protocol. Stateless HTTP and Stateful HTTP protocol. Stateless HTTP protocol does not keep any record of previously accessed web page history. While Stateful HTTP protocol do keep some history of previous web browser and web server interactions and this protocol is used by cookies to maintain the user interactions.

Whenever user visits the site or page that is using cookie, small code inside that HTML page (Generally a call to some language script to write the cookie like cookies in JAVAScript, PHP, Perl) writes a text file on users machine called cookie.
Here is one example of the code that is used to write cookie and can be placed inside any HTML page:

Set-Cookie: NAME=VALUE; expires=DATE; path=PATH; domain=DOMAIN_NAME;

When user visits the same page or domain later time this cookie is read from disk and used to identify the second visit of the same user on that domain. Expiration time is set while writing the cookie. This time is decided by the application that is going to use the cookie.

Generally two types of cookies are written on user machine.

 1) Session cookies: This cookie is active till the browser that invoked the cookie is open. When we close the browser this session cookie gets deleted. Some time session of say 20 minutes can be set to expire the cookie.
2) Persistent cookies: The cookies that are written permanently on user machine and lasts for months or years.

Where cookies are stored?
When any web page application writes cookie it get saved in a text file on user hard disk drive. The path where the cookies get stored depends on the browser. Different browsers store cookie in different paths. E.g. Internet explorer store cookies on path “C:\Documents and Settings\Default User\Cookies”
Here the “Default User” can be replaced by the current user you logged in as. Like “Administrator”, or user name like “Vijay” etc.
The cookie path can be easily found by navigating through the browser options. In Mozilla Firefox browser you can even see the cookies in browser options itself. Open the Mozila browser, click on Tools->Options->Privacy and then “Show cookies” button.

How cookies are stored?
Lets take example of cookie written by rediff.com on Mozilla Firefox browser:
On Mozilla Firefox browser when you open the page rediff.com or login to your rediffmail account, a cookie will get written on your Hard disk. To view this cookie simply click on “Show cookies” button mentioned on above path. Click on Rediff.com site under this cookie list. You can see different cookies written by rediff domain with different names.

Site: Rediff.com Cookie name: RMID
Name: RMID (Name of the cookie)
Content: 1d11c8ec44bf49e0… (Encrypted content)
Domain: .rediff.com
Path: / (Any path after the domain name)
Send For: Any type of connection
Expires: Thursday, December 31, 2020 11:59:59 PM

Applications where cookies can be used:

1) To implement shopping cart:
Cookies are used for maintaining online ordering system. Cookies remember what user wants to buy. What if user adds some products in their shopping cart and if due to some reason user don’t want to buy those products this time and closes the browser window? When next time same user visits the purchase page he can see all the products he added in shopping cart in his last visit.

2) Personalized sites:
When user visits certain pages they are asked which pages they don’t want to visit or display. User options are get stored in cookie and till the user is online, those pages are not shown to him.

3) User tracking:
To track number of unique visitors online at particular time.

4) Marketing:
Some companies use cookies to display advertisements on user machines. Cookies control these advertisements. When and which advertisement should be shown? What is the interest of the user? Which keywords he searches on the site? All these things can be maintained using cookies.

5) User sessions:
Cookies can track user sessions to particular domain using user ID and password.

Drawbacks of cookies:

1) Even writing Cookie is a great way to maintain user interaction, if user has set browser options to warn before writing any cookie or disabled the cookies completely then site containing cookie will be completely disabled and can not perform any operation resulting in loss of site traffic.

2) Too many Cookies:
If you are writing too many cookies on every page navigation and if user has turned on option to warn before writing cookie, this could turn away user from your site.

3) Security issues:
Some times users personal information is stored in cookies and if someone hack the cookie then hacker can get access to your personal information. Even corrupted cookies can be read by different domains and lead to security issues.

4) Sensitive information:
Some sites may write and store your sensitive information in cookies, which should not be allowed due to privacy concerns.

This should be enough to know what cookies are. If you want more cookie info see Cookie Central page.

Some Major Test cases for web application cookie testing:

The first obvious test case is to test if your application is writing cookies properly on disk. You can use the Cookie Tester application also if you don’t have any web application to test but you want to understand the cookie concept for testing.

Test cases: 

1) As a Cookie privacy policy make sure from your design documents that no personal or sensitive data is stored in the cookie.

2) If you have no option than saving sensitive data in cookie make sure data stored in cookie is stored in encrypted format.

3) Make sure that there is no overuse of cookies on your site under test. Overuse of cookies will annoy users if browser is prompting for cookies more often and this could result in loss of site traffic and eventually loss of business.

4) Disable the cookies from your browser settings: If you are using cookies on your site, your sites major functionality will not work by disabling the cookies. Then try to access the web site under test. Navigate through the site. See if appropriate messages are displayed to user like “For smooth functioning of this site make sure that cookies are enabled on your browser”. There should not be any page crash due to disabling the cookies. (Please make sure that you close all browsers, delete all previously written cookies before performing

5) Accepts/Reject some cookies: The best way to check web site functionality is, not to accept all cookies. If you are writing 10 cookies in your web application then randomly accept some cookies say accept 5 and reject 5 cookies. For executing this test case you can set browser options to prompt whenever cookie is being written to disk. On this prompt window you can either accept or reject cookie. Try to access major functionality of web site. See if pages are getting crashed or data is getting corrupted.

6) Delete cookie: Allow site to write the cookies and then close all browsers and manually delete all cookies for web site under test. Access the web pages and check the behavior of the pages.

7) Corrupt the cookies: Corrupting cookie is easy. You know where cookies are stored. Manually edit the cookie in notepad and change the parameters to some vague values. Like alter the cookie content, Name of the cookie or expiry date of the cookie and see the site functionality. In some cases corrupted cookies allow to read the data inside it for any other domain. This should not happen in case of your web site cookies. Note that the cookies written by one domain say rediff.com can’t be accessed by other domain say yahoo.com unless and until the cookies are corrupted and someone trying to hack the cookie data.

8 ) Checking the deletion of cookies from your web application page: Some times cookie written by domain say rediff.com may be deleted by same domain but by different page under that domain. This is the general case if you are testing some ‘action tracking’ web portal. Action tracking or purchase tracking pixel is placed on the action web page and when any action or purchase occurs by user the cookie written on disk get deleted to avoid multiple action logging from same cookie. Check if reaching to your action or purchase page deletes the cookie properly and no more invalid actions or purchase get logged from same user.

9) Cookie Testing on Multiple browsers: This is the important case to check if your web application page is writing the cookies properly on different browsers as intended and site works properly using these cookies. You can test your web application on Major used browsers like Internet explorer (Various versions), Mozilla Firefox, Netscape, Opera etc.

10) If your web application is using cookies to maintain the logging state of any user then log in to your web application using some username and password. In many cases you can see the logged in user ID parameter directly in browser address bar. Change this parameter to different value say if previous user ID is 100 then make it 101 and press enter. The proper access message should be displayed to user and user should not be able to see other users account.

These are some Major test cases to be considered while testing website cookies. You can write multiple test cases from these test cases by performing various combinations. If you have some different application scenario, you can mention your test cases in comments below.

Related Post:

• Open Source Performance Testing Tool
• Regression Testing
• Types Of SDLC Models
• Web Testing: Complete guide on testing web applications
• Software Testing Types
• Testing Tool
• Acceptance Testing

QTP Tutorial For beginners

 

Today’s post is very much useful for those who are new in QTP and don’t know from where and how to start learning QTP and also for those who wants all qtp related materials, I am also going to include best QTP tutorials, QTP training materials, QTP websites and QTP blogs in this post so that all QTP learners can get fruitful information from one place.

• If you are not familiar with  basics of software testing then first clear the picture in your mind.
• Also get knowledge on test automation, different testing tools available in market, comparison of GUI Testing tools and different types of  frameworks in automation
  .
• Now you are ready to learn QTP, So first download demo version of QTP 10.0
  .
• Learn step by step that How to record and develop the script in QTP,  different types of checkpoints in qtp and how to  create checkpoints in qtp.
• Take some reference of  QTP scripts examples and develop some small scripts.
• Now its time to learn VB script  first then go back to gain further knowledge in QTP.
• Learn very much important  Object repository concepts  in qtp.
• Learn handling of    webtable in qtp,   data table in qtp,   ordinal identifier in qtp,    environment variable in qtp
  .
• Now you have enough information about QTP to start doing live projects also do lots of practice by developing small projects so that you can develop sound confidence in QTP.
• By this time you will be having enough knowledge on different types of objects and  properties in qtp
  and how  QTP identifies the objects
  now its time to get advance knowledge in qtp like How to do programming in QTP without using object repository? the answer is YES, learn basics of    descriptive programming-I,   descriptive programming-II.
• Also learn how to add object repository  through QTP scripting, run qtp test through command line
  which will be very helpful in live qtp projects.
• Now finally whenever you completes development in qtp, you will need to perform regression testing which will prove robustness and reliability of your QTP scripts. So for that develop your own
  regression testing framework
  in QTP.
• QTP help is the best qtp tutorial forever that you can use along with you can take help of following websites  and blogs for best practical knowledge on qtp.

1. Quicklearnqtp
   
2. AdvanceQTP
3. learnqtp
4. QTP blogspot
5. QTP google group
   
6. QTP tutorials
7. Motevich blogspot

• If you follow all this information sincerely you don’t need to join any QTP training classes, I have provided all required information on QTP tutorials, you all are always welcome to share your important inputs so that our friends can get best knowledge out of that.

The Bug Life Cycle Series

I have been talking about Bugs in the Software Applications in my recent posts. They play an important role in Software Development & Testing. The biggest value addition is to uncover the critical issues of the system at the early stage.

I have already written some posts on the Bug Tracking front and find the list below. They talk about how to handle the Bugs, avoid conflicts & the lessons to be learned for the testers.

• That’s Expected Behavior & Not a Bug – Avoid conflicts over the issues raised. Don’t fight but capture the required information.
• Lesson to be learned from Bugs – An insight into where things went wrong and how to improve upon same.
• Non Reproducible Bugs – How to deal with them ? – An insight into the areas missed and skills needed.

The above posts doesn’t cover all aspects of a Bug.  I feel that it’s better to write more on the Bug Life Cycle & explore the same.

The Bug Life Cycle series will have the following posts and i will be writing them in frequent intervals.

1. The Role of Software Testing

2. An insight into Bugs in the Software Applications

3. Context driven information for Bug Reports – It’s the need of the hour

4. Life Cycle of a Bug – Different Stages in it

5. Communicate the Value of Bug Reports

6. The End Game

Stay tuned & check out the blog for upcoming articles.

The Life Cycle of a Bug – Different Stages in it.

In this post, i will explore  different stages of the Bug from it’s inception to closer. The Bug has been found and logged into the Bug Tracking System.

1. The Bug has been found and logged into the Bug Tracking System. It will be treated as New Bug in the System.
2. The Bug will be assigned to the concerned Developer for a Resolution.
3. The developer looks in to the possibilities of the resoultion & takes a call on Resolution by fixing it or differing over the information provided.
4. Tester validates the resolved issue in the build & checks for the regression scenarios over the fix.
5. If the issue found fixed, then he choose to Close the issue else he / she will Re-open the same.
6. The Cycle follows again for the re-opened issue till it get’s closed.
7. It worth doing the following activities
   1. Capturing the required and re-usable info to the Bug Report at it’s each stage.
   2. Check for all the closed bugs of Severity 1 & 2 against final build for the release.

   In the next post, I will share my thoughts on the useful metrics over the Repository.

   Happy Testing.

Context driven information for Bug Reports

Context driven information is the need of the hour and there is a huge value associated with the same. It’s good to capture the context driven information in the bug reports. My initial experiences with bug reports way back in early 2000 have taught many lessons to improve upon.

Bug reports used to capture what is the problem with the system familiar to the user (tester) who reported the same. People spend very less time to capture all the details required and there are many reasons for the same. I hope some of the upcoming testers will also be in the similar situations.

Some of the Reasons people quote here

• We need to test more and less time to capture & write more information in the Bug Reports
• It’s tough to capture all the information required
• System is complex & It’s tough for the novice users to understand the bug reports
• You know capturing all the info is process driven & it may not be worth of efforts
• It some times boring activity to collate the info & push it
• I can reproduce it on my machine if developer needs it.

This list can go on…

I hope you have come across this situation at least once in your career.

This is my third post in the Bug Life Cycle Series and it’s good to know the different users of your Bugs and their context with them. The mission of your bug report is to provide details and context of the problem and convey the importance of it with a user driven stories.

Your bug report must be the voice of customer and it need play the role of an advocate against the problem. Bug Advocacy from Cem Kaner is an excellent source to begin with. If the bug report unable to specify the need of the context, then it’s better not to write any report

It’s good to explore & capture some of the following problems

• Productivity
• Performance
• Usability
• Migration
• Stability etc

Try to link your issues with most suited functions listed above. It may not be obvious to other users in the system to explore & analyze the issues in that fashion.

There is another context associated with Bug Reports. That’s with the stake holders of the project. The Bug Tracking system must give the right trends and identify the hot spots. Testers must capture the right kind of data to derive better valuable metrics over the bug repository.

Care must be taken to capture

• Capture all the Test Environment details
• Detailed classification on the feature. Classify to the maximum possible sub feature/component of the system
• Clarity on Severity & Priority
• Versions and Build Numbers (Affected & Fixed)
• Bug Classification (Requirements / Design / Implementation etc)
• Bug Types (Functional, Performance, Usability, Security etc)
• It can go on…

The above info helps a lot to identify the trends in bugs and focus on the unstable components / environments.

Final Thoughts
Push the entire context driven information to the bug repository at least for a release cycle and observe the results. Check back with your repository to identify the trends and risk associated with the release and I am sure that it will be in the similar lines of end user feedback.

Happy Testing…

An insight into Bugs in the Software Applications

Abstract

Bugs are there every where in the software applications. Almost every one who uses software applications for their day to day activities; do come across different kind of problems while working with them.  Each of these problems has different meanings to different people on it. There are many instances where in which, we (as end users of the system) do feel that how come they missed this bug, it’s very important in this context (The context where in which the user operates).

Tester deals with Bugs every day and its good know about other people get affected with them. The list includes Customers, Stake Holders, Sales, Professional Services, Technical Support, Architecture & the Development team.

  

This is my second post in the Bug Life Cycle Series. Before going more into the Bugs and their life cycle, it’s good to know, what it means to different people across the software application. Based on the contexts, the same bug might mean different things to different people. Do go through Role of Software Testing to understand my mission about Software Testing.

Wikipedia defines Software Bug as the following

A software bug (or “bug”) is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended (e.g., producing an incorrect result). Most bugs arise from mistakes and errors made by people in either a program’s source code or its design, and a few are caused by compilers producing incorrect code. A program that contains a large number of bugs, and/or bugs that seriously interfere with its functionality, is said to be buggy. Reports detailing bugs in a program are commonly known as bug reports, fault reports, problem reports, trouble reports, change requests, and so forth.

Tester Perspective

Any deviation from the expected results of the test case will be treated as a bug in the software application.

Customer Perspective

Customer uses the software application to solve his business needs. Any problem while modeling a solution to the business need will be considered as a bug in the software applications. These problems can be classified into two. They are the list of problems that they can live with and the other is the list of problems that need to be addressed to solve the business need.

The second list of problems will be sent to the vendors of the application for the fix. Some times, customer sends only the show stopper problems for his business. It’s true in the fact that as an end user, I will contact the vendor only if the problem is critical for me.

Technical Support Perspective

The Support Team classifies (though it’s critical) the customer requests into New Features, Enhancement, Bug, How to & Enquiries etc with respective severity levels. The decision will be taken by validating the problem with the features in the product.

Developer Perspective

The feature is designed this way and all the cases have passed. The end user might be using some other scenario. Yes, Some times there are bugs in my code. But it functions well if we use the way the application has built.

Management Perspective

Any problem with the application will be treated as Bug if it has impact over the revenue and customer satisfaction.

Final Thoughts

It’s tough to have same perspective across all the people (might happen in an ideal world). The bug for some one may not be the bug for the other person. However there are some set of show stopper bugs for which, the perspective will be the same.