Load Testing a Website

What is Load Testing?

Load testing is the process of putting demand on a system or device and measuring its response. Load testing is performed to determine a system’s behavior under both normal and anticipated peak load conditions. It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation

from wikipedia’s Load Testing

Typical user scenario

With load test, you want to know how many users the server can handle. It’s a vague statement. What do you mean by “can handle”? Does it mean all web response are within 15 seconds? Will you tolerate intermittent late response for heavy operations? Who are our users and what are their typical activities when they log in to our website? Though we create a typical user, and test the site with that user, would the result reflect the reality? I mean if you say the server can handle 1000 users, would the server really be able to handle 1000 real users? What about caching?

There would be lots of questions, and you would have to answer them as you go along the testing. Some of them might not be tackled in their nature. You cannot say the real load until real users come to the site. Even the pattern of the activities would be different depending on the day or the time.

So, measuring the accurate load is almost impossible, but we can have some result we can use as our yard stick.

So, I created three user persona (exactly speaking, me and my colleagues, but I’m writing this post and I would say “I”)

  1. Light user: 50%
  2. File user: 30%
  3. Heavy reader: 20%

Light user will be 50% of all our users in the testing. Heavy reader only consists of 20%.
And I wrote the user’s typical activity based on our web analytic and statistics.

For example, the typical activity of light user is like this.

  • Default page
  • Go to Workspace 1
  • Whiteboards
  • Read 1 message
  • Go to task
  • Create a task
  • Go to workspace overview
  • Go to files
  • Files – Add 1 new document (PDF)
  • Preview the file
  • Assign the document to a big group such as workspace users or company
  • Go to Workspace 2
  • Files – Read 2 document (PDF)
  • Download 1 document
  • Exit

Once you have a scenario, you can start recoding the test with MS Web Performance Test.

MS Test for Load and performance testing

Unfortunately, you need to have Visual Studio Ultimate. One of perks I have at work is I work in Infrastructure & architecture team that gives you MSDN ultimate subscription. Yay! MS Office is included.

You record the web performance test. It generates something like this.

Those requests only have static values, but we want to use dynamic values. You ran the test against your development box, but later you would need to run in against your production or staging server. We need to change the domain “dev” to something else.

You can do that by adding Data Binding to the web test.

Web Test Data Binding

You can choose three types; xml, csv, and sql server data source. For configuration data, such as environmental settings, I use xml file. For users, as I was going to have 200 users ideally, I used csv. I didn’t use sql data source as it may impact the performance of the DB server I test.

csv example

user,id,hash,work,board, ....
invite,1308,A1D93654,1396, ....

configuration xml

<?xml version="1.0" encoding="utf-8" ?>
<configs>
  <config>
    <domain>dev</domain>
  </config>
</configs>

Now, you start running the test, and soon find several failing requests. Web test recording out of the box doesn’t do much and requires your customisation. You can write your own code to sort those issues out with plugin.

Web test plugin

There are a few things you can’t do with the out-of-box recording.

  • OAuth token for your authenticated ajax api calls
  • Form post anti-forgery token in the request header
  • Missing ViewState in your request if you use ASP.NET WebForm
  • Grab the generated file Id you have just uploaded

When you make an authenticated api call, you need to include OAuth token in the header. WebTest recorder doesn’t not catch it, and I had to manually include it.

It was surprisingly easy to write a plug in. You create a C# class that inherits WebTestPlugin. If you inherit “WebTestPlugin”, it is executed only once prior the WebTest. It behaves like [SetUp] in NUnit test

public class OAuthTokenPlugin : WebTestPlugin
{
    public override void PreWebTest(object sender, PreWebTestEventArgs e)
    {
        var username = string.Empty;
        var password = string.Empty;

        username = e.WebTest.Context["DataSource.user#csv.username"].ToString();
        password = e.WebTest.Context["DataSource.user#csv.password"].ToString();

        e.WebTest.Context["OAuth"] = GetToken(username, password, e.WebTest.Context["configDs.config.domain"].ToString());
    }

    public string GetAccessTokenUsingPasswordCredentials(string username, string password, string topLevelDomain)
    {
        ....

        return "OAuth2 " + token;
    }
}

“DataSource.user.username” is the way to access your data source. But one thing you have to know is you need to mention that data source through property window on the test. Otherwise, the data source is not loaded into memory.

Anti-forgery token pluin is a WebTestRequestPlugin. WebTestPlugin runs once prior web test and WebTest

A dynamic url with data source

Many of our api calls use different ids or numbers in the url. For example, if the workspace id is 10, the api call becomes https://dev/workspace/10/tasks. You can easily replace the hard-coded value with data source.


https://dev/workspace/UsersDataSource.user.workspace/tasks

Other things to add

  1. deployment item
  2. get viewstate
  3. file upload
  4. Controller and agent

to be continued.

Load Testing a Website

How to use html test suite and cases on Selenium RC

References

Since 2009, I have used Selenium IDE and core. I love Selenium IDE. It is easy to use and is very handy as I can record test cases and also run them one by one by clicking on each. A colleague of mine used Selenium RC with C# but it did not impress me much because he started RC server first, then executed NUnit to lanuch the test suite. Also, I did not like coding in C# selenium test cases. How simple and elegant those html test caes are!

Then today, a bit shame to say, I realised I can use html test suite on Selenium RC. Oh how fantastic it is. My scrum team had scores of accumulated html test cases, and I can use them without any alteration.

First, let’s see how to install Selenium RC

  1. You download Selenium RC from http://seleniumhq.org/download/
  2. Unzip it into local hard drive, for example, c:\Selenium. Because Selenium RC is written in Java, you need to install java runtime if you do not have it.

selenium-server.jar is the main engine that runs test suite. You can run it on command prompt, but it is very handy to create a batch file to run it as the above reference link showes it.

For IE

cd \
cd C:\Selenium\selenium-server-1.0-beta-2
java -jar selenium-server.jar -port 4545  -htmlSuite *iehta "http://www.sekyee.com" "C:\Selenium\Test_Suite.html" "C:\Selenium\Results.html"
pause

For firefox


cd \
cd C:\Selenium\selenium-server-1.0-beta-2
java -jar selenium-server.jar -port 4545  -htmlSuite *chrome "http://www.totaljobs.com" "C:\Selenium\Test_Suite.html" "C:\Selenium\Results.html"
pause

A simple selenium-server.jar usage is like this.
selenium-server.jar -port {your port number} -htmlSuite {firefox or ie} {base url} {absolute path for your test suite} {absolute path for your test result file}

I’m not sure why * is required befire iehta, yet without the *, it does not find testsuite. Also, Results.html must exist before you run this even though it is an empty file.
I find {base url} very handly, so you can put relative path in your test cases. This enables you to use the same test cases to test on different environments such as Dev, Int, Staging, and Live.

Hoep this helps

How to use html test suite and cases on Selenium RC