A sub-directory of: testing/behat/features
It is recommended that you place them in a sub directory based on the main area of Rogo you are testing, for example if you were testing that the details of a paper could be updated and had the desired effect you might place it in the papers sub-directory.
All Feature files must have a Feature definition, all tests in the file must fit into this description. We prefer more files to unrelated tests in the same file.
Indenting in the tests must be two spaces per level of indentation.
There should be no trailing white space
@feature_wide_tags Feature: Short descriptive title Description of what the users want to do As a type of user Description of what the user needs to be able to do Background: Given this step runs in all scenarios in the file @scenario_specific_tags Scenario: Descriptive scenario title Given the following "users" exist: | username | | student1 | When I login as "student1" Then I should see "No exams found" "content" |
For more details about the Gerkin language used in feature files see http://docs.behat.org/en/v3.0/guides/1.gherkin.html
Tags can be used to define the area of functionality a feature or scenario is testing, feel free to use them liberally. For example if your test is about the creation of creating Likert questions you may tag the test with @question and @likert, this would allow us to easily run any tests for questions in general or Likert questions in particular.
There are some tags that have special meanings beyond this:
Tag | Description |
---|---|
@javascript | The Scenario will use the Selenium2 webdriver and be able to use Javascript.Tests that use this tag will be slower than tests that do not. This tag should not be mixed with the @backend tag |
@backend | The test will be run by the backend TestSuite. |
@attachfile | Any test that uploads files should use this tag. It will allow us not to run them if we are doing testing in an environment that cannot do uploads (or a driver that does not support attaching files). |
@iframe | Any tests that run on pages within rogo that have iframes should have this tag. It will allow us not to run them if we are doing testing using a driver that does not support iframes. |
@jsevaluation | Any tests that use steps that require javascript evaluation. It will allow us not to run them if we are doing testing using a driver that does not support javascript evaluation. |
@mousemanipulation | Any tests that use steps that require mouse manipulation. It will allow us not to run them if we are doing testing using a driver that does not support mouse manipulation. |
There are two commands you may use to find available steps:
The just lists the commands:
vendor\bin\behat --config <path_to_rogo>\testing\behat\config\behat.yml -dl |
The output first tells you which test suite the step belongs to and then after the pipe tells you the step. note that Given, Then, And, But, etc. are interchangeable. If a step starts with /^ and ends with $/ it is a regular expression.
frontend | Given /^the following "([^"]*)" exist:$/ frontend | Given /^I login as "([^"]*)"$/ frontend | Given /^I click "([^"]*)" "([^"]*)"$/ frontend | Then /^I should see "([^"]*)" "([^"]*)"$/ frontend | Given /^I set the field "([^"]*)" to "([^"]*)"$/ backend | Given /^there are "([^"]*)" records in the "([^"]*)" table$/ backend | Given I store the database state backend | Given I reset the database state backend | Given /^the following "([^"]*)" exist:$/ |
The second gives a detailed list that includes information about what the step does:
vendor\bin\behat --config <path_to_rogo>\testing\behat\config\behat.yml -di |
The first line of each block has the same meaning as the simple list, the next few lines describe what the step does.
frontend | Given /^the following "([^"]*)" exist:$/ | Adds records to the database using an appropriate data generator. | at `RogoBehatFrontend::the_following_exist()` frontend | Given /^I login as "([^"]*)"$/ | Log the user into Rogo. | at `RogoBehatFrontend::i_login_as()` frontend | Given /^I click "([^"]*)" "([^"]*)"$/ | Click on an element on the page. | at `RogoBehatFrontend::i_click()` frontend | Then /^I should see "([^"]*)" "([^"]*)"$/ | Checks for the presense of text. | at `RogoBehatFrontend::i_should_see()` frontend | Given /^I set the field "([^"]*)" to "([^"]*)"$/ | Fill in a form field. | at `RogoBehatFrontend::i_set_field()` backend | Given /^there are "([^"]*)" records in the "([^"]*)" table$/ | Checks that there are an expected number of rows in a database table. | at `RogoBehatBackend::there_are_records_in_the_table()` backend | Given I store the database state | Saves the database state using the testing\behat\helpers\database\state class. | This step cannot be called twice without "I reset the database state" being | used between the calls. | at `RogoBehatBackend::i_store_the_database_state()` backend | Given I reset the database state | Resets the database state using the testing\behat\helpers\database\state class | The "I store the database state" step must previously have been called. | at `RogoBehatBackend::i_reset_the_database_state()` backend | Given /^the following "([^"]*)" exist:$/ | Adds records to the database using an appropriate data generator. | at `RogoBehatBackend::the_following_exist()` |
The version of Behat used in Rogo allows us to define multiple test suites we have two defined.
This is the default test suite, it uses a browser to simulate student interaction with the user interface of Rogo, either via a real browser or a headless browser
This suite is used if the test is tagged with @backend. It allows test to use PHP unit functions. It should also allow steps to share data between each other using variables that are automatically cleaned between tests with a custom data store
The backend test suite will automatically interpret any calls to any non-class defined properties as being a call to the custom data store. So for example if in a step you do:
$this->myvariable = 5; |
In a later step of the same test you could call $this->myvariable and it would return 5
isset() and unset() will work on variables stored in the custom data store
The steps definitions hold the code that makes the feature files work.
In Rogo we define our steps in PHP traits
A sub-directory of: testing/behat/steps
Currently there are 3 sub-directories:
Directory | Notes |
---|---|
backend | The steps that use PHP Unit functions to test code directly should be placed here |
database | Steps that generate data for Rogo by adding it directly into the database should be placed here |
frontend | The steps that interact with Rogo via a web browser should be placed here |
Each file in the sub directory should contain a single trait that defines steps related to a specific area, for example the frontend forms trait should contain steps for interacting with html forms.
The frontend and backend sub directory also have an include_ trait that should use all the other traits.
There is a special step that can be used to add data to the Rogo database.
Given the following "users" exist: | username | | student1 | |
The data type is passed as a parameter i.e. "users" above.
The table passed must include data for all required fields for the data generator, it may include any number of optional fields. If an optional field is omitted a default or random value should be generated for it.
Currently the following data generators are available via this method:
Name | Required fields | Optional fields |
---|---|---|
users | username | surname, first_names, title, email, roles, gender, special_needs, yearofstudy, user_deleted, password_expire, grade, initials, password |
questions | user, type + question type specific requirements | are question type specific from the following list: theme, leadin, notes, display_method, ownerID, q_media_id, q_media, q_media_width, q_media_height, q_media_alt, q_media_owner, q_media_num, creation_date, last_edited, bloom, scenario, scenario_plain, leadin_plain, checkout_time, checkout_authorID, deleted, locked, std, status, q_option_order, score_method, settings, quid, keywords, options |
modules | moduleid, fullname | active, schoolid, vle_api, sms_api, selfEnroll, peer, external, stdset, mapping, neg_marking, ebel_grid_template, sms_import, timed_exams, exam_q_feedback, add_team_members, map_level, academic_year_start, externalID |
module team members | moduleid, username | not applicable |
module keywords | moduleid | keyword |
papers | papertitle, papertype, paperowner, modulename | startdate, enddate, labs, duration, session, timezone, externalid, externalsys, calendaryear, remote, password |
Where data is expected as an error in a data generator or step we pass this in the feature file as a json string i.e.
@question_true_false Scenario: Create a true_false question Given I login as "teacher" And I follow "TEST1001" And I select a "true_false" question type And The upload source path is "questions/media" And I create a new "true_false" question: | theme | textbox theme | | notes | textbox notes | | scenario | textbox scenario | | leadin | textbox leadin | | true | 0 | | file | {"filename":"frog.jpg","alt":"alternate text"} | When I click "Add to Bank" "button" Then I should see "Module: TEST1001" "paper_title" |