This post is the guide for you to follow and complete the NUS orbital 2014 mission control #8 tutorial. Wish by after reading the post, you will know testing better and setup your own CI to see all your tests are passing. :P

Agenda

• Step 0 – What are we testing
• Step 1 – Doctest your model
• Step 2 – Unittest your helper method
• Step 3 – Test your handler
• Step 4 – Use behave to do browser test
• Final words – CI

Step by Step Tutorial

NOTE: from step-y to step-y-done is one topic of testing, there will be practice and homework for each section. Remember to setup your virtualenv before continue. Below are some tips during the tutorial:

• Use below command to go to correct step before reading the related section.

1

git checkout -f step-y

• Use below command to run test with nose

1

nosetests --with-doctest --with-gae --without-sandbox --gae-lib-root='/path/to/google_appengine/' -v

• Use below command to run test with pytest

1

py.test --doctest-modules --with-gae --gae-path='/path/to/google_appengine' -v

• Use below command to run test with behave

1

behave -v

• Passing the test doesn’t mean that you application doesn’t have bugs, and it’s not the case that the more test you have the better.

Step 0 (checkout to step-0)

First of all, I have assumed that you have setup your environment correctly, and you are able to test it out at step-x. This is the easiest step, I have downloaded the sample guestbook application, please go to guestbook folder and try to start the application with your google_appengine and make sure it is running correctly. If you can sign on your guestbook at http://localhost:8080/, then you are ready to go to the next step.

No practice and homework for this step.

Code Diff

Step 1 (checkout to step-1)

The first thing we are going to test is your model for your google app engine project. In guestbook, there is only one model called Greeting. And we will be using python doctest to help us on this task. As a demo, I have already write the testing for creating Greeting. Here is an explanation for the core part of the testing code:

1
2
3
4
5
6
7
8
9
10

You can create a Greeting:
>>> greeting = Greeting(parent=guestbook_key(), content='test content')
>>> greeting
Greeting(key=Key('Guestbook', 'default_guestbook', 'Greeting', None), content='test content')
>>> created_key = greeting.put()

You can query to select the greeting:
>>> greetings_query = Greeting.query(ancestor=guestbook_key())
>>> list(greetings_query.fetch(10)) # doctest: +ELLIPSIS
[Greeting(key=Key('Guestbook', 'default_guestbook', ...)]

In this doctest, we have created a new Greeting object and put it into the datastore, and then verify the saving by query Greeting by key.

Now it’s your turn to write some test. In the TODO part, I have asked you to write doctest for modification of Greeting object. Please try it out yourself before go to the answers.

Once you have finished writing your doctest, please run the test with either nose or pytest to verify the your answer.

Homework: Apply doctest to your model in your project where you feel needed.

Code Diff

Step 2 (checkout to step-2)

Sometimes, you may have some helper methods in your code, so let’s use unittest this time to help us test helper methods. At step-2, I have refactored the code and add a get_user_url helper method. Please study the method and see the test example I have done for you. Here is an explanation for the core part:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

def setUp(self):
    # First, create an instance of the Testbed class.
    self.testbed = testbed.Testbed()
    # Then activate the testbed, which prepares the service stubs for use.
    self.testbed.activate()
    # Next, declare which service stubs you want to use.
    self.testbed.init_user_stub()
    # Set up expected values
    self.user_login_url = {
        'url': 'https://www.google.com/accounts/Login?continue=http%3A//testbed.example.com/',
        'url_linktext': 'Login'
    }

def tearDown(self):
    self.testbed.deactivate()

def setCurrentUser(self, email, user_id, is_admin=False):
    email_to_set = email or ''
    id_to_set = user_id or ''
    admin_to_set = '1' if is_admin else '0'
    self.testbed.setup_env(USER_EMAIL=email_to_set, overwrite=True)
    self.testbed.setup_env(USER_ID=id_to_set, overwrite=True)
    self.testbed.setup_env(USER_IS_ADMIN=admin_to_set, overwrite=True)

def testNoLoginUser(self):
    self.setCurrentUser(None, None)
    result = get_user_url('/')
    assert_that(result, equal_to(self.user_login_url))
    assert_that(result['url_linktext'], is_not('Logout'))

The setUp method is run before the test to setup the Testbed stub and tearDown method is used to clean up after testing, setCurrentUser is a test helper method and at last testNoLoginUser is the actual test method. By the way, sorry for the java naming convention in python code. I was doing a java project at the same time. :(

Now it’s your turn to write some test. In the TODO part, I have asked you to write unittest for testing with a login user. Please try it out yourself before go to the answers.

Once you have finished writing your unittest, please run the test with either nose or pytest to verify the your answer.

Homework: Apply unittest to your helper method in your project where you feel needed.

Code Diff

Step 3 (checkout to step-3)

Now you have finished your unit testing part, you are confident that you code is doing what it is designed to do. But wait, when things integrate together, it may cause other issues. That’s why integration testing is needed. In the demo, I have write some code for the integration testing of MainPage handler. Here is the explanation of the core part:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

def check_html(self, response_body):
    assert_that(response_body, contains_string(self.submit_button))
    assert_that(response_body, contains_string(self.switch_button))
    assert_that(response_body, contains_string(self.guest_name_label))
    assert_that(response_body, contains_string(self.css_link))

def test_main_page_no_login(self):
    self.set_current_user(None, None)
    response = self.testapp.get('/')
    assert_that(response.status_int, equal_to(self.status_ok))
    assert_that(response.content_type, equal_to(self.html_content_type))
    body = response.normal_body
    self.check_html(body)
    assert_that(body, contains_string(self.login_string))

def test_main_page_with_login(self):
    self.set_current_user('test@test.com', 'test')
    response = self.testapp.get('/')
    assert_that(response.status_int, equal_to(self.status_ok))
    assert_that(response.content_type, equal_to(self.html_content_type))
    body = response.normal_body
    self.check_html(body)
    assert_that(body, contains_string(self.logout_string))

The test are quite straight forward, it just route to the correct path for the handler and verify if the html source returned is correct.

Now it’s your turn to write some test. In the TODO part, I have asked you to test Guestbook handler. Please try it out yourself before go to the answers.

Once you have finished writing your integration testing for handler, please run the test with either nose or pytest to verify the your answer.

Homework: Add integration test for handlers in your project.

Code Diff

Step-4 (checkout to step-4)

You are almost there. Remember about acceptance testing. Now let’s use behave and selenium to help us achieve the goal. Make sure you have a Firefox installed on your machine for the browser testing. You may wonder why we need browser testing as we have finished integration testing. As before, when you product is delivered to users, they will not use it as what you are testing with integration handler test. Browser testing add one more guarantee layer before you product meet the users. In the demo, I have help you to setup the features folder and environment.py, there is also a simple Scenario to test user sign on guestbook without login. Here is the explanation of the core part:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

@given('user has been sign out')
def step_user_has_sign_out(context):
    context.browser.get(HOME)
    try:
        sign_out_link = context.browser.find_element_by_link_text('Logout')
        sign_out_link.click()
    except NoSuchElementException:
        pass


@when('I go to "{url}"')
def step_go_to(context, url):
    context.browser.get(url)


@then('I should see main page html')
def step_see_main_page(context):
    guest_name_label = 'Guestbook name:'
    page_source = context.browser.page_source
    assert_that(page_source, contains_string(guest_name_label))
    elm = context.browser.find_element_by_id("submit")
    elm.value = "Sign Guestbook"
    elm = context.browser.find_element_by_id("switch")
    elm.value = "switch"


@when('I sign "{content}"')
def setp_sign_content(context, content):
    elm = context.browser.find_element_by_name("content")
    elm.send_keys(content)
    button = context.browser.find_element_by_id("submit")
    button.click()


@then('I should see "{content}" signed by "{user}"')
def setp_see_content(context, content, user):
    page_source = context.browser.page_source
    assert_that(page_source, contains_string("<blockquote>%s</blockquote>" % content))
    assert_that(page_source, contains_string(user))

With the magic from behave, by using @given, @when, @then decorators, we are able to translate our feature document into real test code. Take a note with how {} is used for gather information from the feature document.

Now it’s your turn to write some test. In the TODO part, I have asked you to test user sign on guestbook with login. Please try it out yourself before go to the answers.

Once you have finished writing your integration testing for handler, please run the test with behave to verify the your answer.

Homework: Try to add browser testing in your project if you are interested, as it is a bit hard to setup the environment compared to previous unit testing and integration testing.

Code Diff

Final words – CI

Now you have tested your project. Do you want to know how well your test is? Do you want to have a CI server to show off? :D

You can check out Travis CI or Drone CI for this project.

To setup CI server for you project, you can refer to the documentation from Travis and Drone for more details.

And you can also setup coveralls report if you want. :)

That’s all I have to share with you about python testing, wish you all are enjoying it.

Please comment or send email to me if you have any trouble or if you feel there is any error in the tutorial guide.