Data Driving Topics

Why Data Drive?A basic introduction into the advantages data-driving.
Data Volatility LevelsThe different places to store different types of data.
Storing Data in Properties FilesHow to store data in properties files.
Using CSV Data in Test ScriptsHow to store data in CSV files.
Using a Data Stored in Databases in Test ScriptsHow to Use Data from a Relational Database Management System (RDBMS).
Referencing or Using Data in Test ScriptsHow to Reference Data Obtained from Various Data Sources in Jameleon Scripts.

Why Data Drive?

Having an automated script do one thing well is nice, but the one scenerio is still just one of thousands of possibilities. One of the powers of automation is the ability to exercise the same script with different data. Data-driving an automated script could help find bugs never before imagined.

Data Volatility Levels

In Jameleon, data can be placed in several different places, depending on the how often the data changes. This is not a requirement, only a feature that could end up saving maintenance time in the future.

When creating more than one script, it is likely there will be data that is shared across multiple scripts. This makes the test cases less maintainable when a change to that particular value occurs.

The following is a simple list of how Data Volatility Levels differ in an automated script:

Test Case Specific Data

This type of data is data that changes between iterations of the script. Some examples of this might be the username used to log onto the application being tested. Another example might be the type of product to choose when ordering something; maybe the shopping cart behaves differently, depending on the product ordered. This data type would be considered highly volatile and might exist in and external data source like a database or a CSV file.

Application Specific Data

This type of data is not likely to change. Some examples of this data type might be the screen title, form field names, or even some text on a given screen. Jameleon has a mechanism to pull this type of data outside of the script. It can be stored in a file with the same name as your appplication, followed by .properties and it must be in the root of your CLASSPATH. For example, to test a customer support application, a good name for that application might be csr. Therefore a file named would be created and put in the root of the CLASSPATH. This type of data could be considered low to non-volatile.

See the Variable Naming Standards section for more information on the format of this file.

Environment and Application Specific Data

This step can be skipped for applications existing only in one environment.

There is usually data that differs depending on the evironment the scripts are running against. For example, the way to start an application or to browse an application may differ between environments. Maybe a connection to a database to search for possible test data is required. This will likely also change when testing against the sandbox environment versus the QA test environment. This type of data will be shared across functional points that don't test the same areas of the application. Because these settings will likely be very small between applications, they are all listed in the same file.

See the Environment-specific Data section for more details and examples.

Organization or Affiliate Specific Data

This step can be skipped for applications that function the same no matter where they are deployed or who uses them.

Sometimes there are applications with behavior that can be changed or configured. These type of applications have the same basic feature set, but the business rules may change for each organization.

Let's use a shopping cart as an example. A shopping cart's basic functionality is to sell products and manage orders. However, usually there are several other features that can be turned on/off and/or must be configured against a specific environment to work properly. Most good shopping carts are designed to be configured and changed according an organization's needs and branding. Usually, the first thing businesses will do is change the title, logos, categories, products offered, shipping and sales tax calculations, and available shipping methods.

If the basic functionality and usually the workflow are the same, why write separate test scripts for each organization? Why not just have the scripts use different data that drive the test execution?

For more details and examples, see the Using Organizations section.

Applications properties

Jameleon uses several ways to send data to functional points. One of these ways is through a properties file. The rules for these files are:

  • The name of the file begins with the name of the application being tested followed by .properties. Please see the section on file naming standards for a more thourough explanation.
  • It is recommended that all properties files exist in the same directory. This makes it easy to include that directory in the CLASSPATH. If using the jameleon-test-suite distribution, the directory to place all properties files is the res directory.
  • The properties are in key=value format. For example, title=Jameleon - Home.
  • If using an organization all variables differing among organziations must start with the organziation's name followed by a period.
  • To use the same variable that differs between testing environments, create a file beginning with the name of testing environment, followed by
  • Variable substitution is supported.
  • All variables gathered from these properties files are put in the context of the test script. The best way to use these variables is within the script itself and not in the code of custom tags. These variables can be used by ${varName}.
  • To use these variables, simply set the application attribute in the session tag. This attribute tells Jameleon which file(s) to read and which variables to throw in the context. Please see the section on using variables stored in properties files below.

Keep it Simple

The following features are for use only when needed. If the application is deployed to a single environment and there is only one instance of this application in production, then don't use the organization and/or environment features.

File Naming Standards

To store application-specific static data that can be used by Jameleon scripts, the file name must be the name of the application followed by .properties. For example, if testing a customer service representive admin tool, one might call that application csr. This means that the file with the name would be created. This application name is then referenced in the session tag of the test script.

Currently, all properties files are loaded via the CLASSPATH. This means that in order to use them, they must be placed in the root of the CLASSPATH. For example, if the files end up in a directory named Build, then Build must be in the CLASSPATH. This is accomplished in the jameleon-test-suite package by placing all properties files in the res directory.

If organizations are used, it is likely that there will be data that is the same across the different organizations. To do this, simply create keys that don't begin with an organization. Keys with an organization override keys without the organization. This way a default value can be specified and overridden.

Variable Naming Standards

There is only one variable-value pair per line in the file. The variable name should be unique. An example of setting this variable in might look like:

logonTitle=Customer Service Tool
Of course, the form values could be declared as defualt values in the custom tag that represents the CSR logon screen. Or the variable values in the custom tag could be bound to the values in by using the contextName jameleon.attribute.

Using Organizations

For those applications that are basically the same, but have some modifications due to different affiliate requirements, Jameleon supports this by naming the organization and starting all keywords with the name of the organization followed by a ,. For example if the the logon title of a csr application is different between Company A and Company B, then the following would work:

A.logonTitle=Company A - Customer Admin Tool
B.logonTitle=Company B - Customer Admin Tool

To use organization-specific data, Jameleon must know about the organization. This can be set as a global variable, in the test case tag, or in the session tag. If defined in the session tag, then the global setting will not override the session tag's setting. This is useful when a test case spans multiple applications and one of those applications is deployed against only one of the organizations.

Environment-specific Data

Data can differ between the different environments the scripts run are against. For example, the way to start an application or to browse an application may differ between environments. A connection to a database to search for possible test data will likely change when testing against the sandbox environment versus the QA test environment.

Because these settings will likely be very small between applications, they are all listed in the same file. The naming standard for this file is the name of the environment followed by For example, to test an applicatoin deployed in test environment, simply create a file named, following the guidelines in File Naming Standards. Setting this variable is almost the same as explained above. The only difference is that this file represents all applications. Therefore the variable name must start with the application name followed by a .. For example, setting the baseUrlof the CSR application would look like:


To use environment-specific data, Jameleon must know about the environment This can be set as a global variableor in the test case tag.

To use environment-specific data against an organization, simply preprend the variable with the name of the organization. For example, to set the above baseUrl variable for the csr application in the test environment for the acme organization, the file name would be the same as the above example, but the variable would be defined with csr.baseUrl:


Variable Substitution

This file can also handle variable substitution. So in the above example of company A and company B, Customer Admin Tool is the same in both organizations. A new variable with that value can be created and used in a latter variable. For example:

appName=Customer Admin Tool
A.companyName=Company A
B.companyName=Company B
logonTitle=${companyName} - ${appName}
Now if another affiliate comes around, we simply add C.companyName=Company C to the file and now the same script can run against the new affiliate.

Using Properties Variables

Properties variables can be used in test case scripts by surounding the variable name with ${}. For example:

<ju-assert-equals functionId="compare ${var1} against ${var2}" expected="${var1}" actual="${var2}"/>
New variables can be set via the Jelly <set/> tag.

Properties variables can also be accessed, changed and created in Java via the getVariable and setVariable methods of the FunctionTag.

CSV Data

Jameleon can use a Comma Separated Value (CSV) file format to make test cases data-driven. This is a very simple format, but Jameleon has some features to allow for a more robust data set. Basically, if useCSV is set to true in the testcase tag, then a file with the same name as the test case name followed by a .csv is used for data values. This can also be accomplished with the csv tag. For every row of data, except the first row which simply names the variables names to be set, the test case will be executed. The test case only passes if all of the rows of data passed.

Mapping to a CSV File

Jameleon uses a uses the following rules to locate the CSV file used for a particular test case:

  1. data - The base directory for all CSV files.
  2. testEnvironment - the same directory name as the test environment name.
  3. organization - the name of the organization if set. This is not a required attribute.
  4. name - The name of the test case.
  5. .csv - This is the ending of the file name.

For a test case that is run in the testEnvironment "beta" against the organization "sourceforge" and with the name "SearchWithNoQuery", the Jameleon engine will look for data/beta/sourceforge/SearchWithNoQuery.csv and run the test case once for every row in the CSV file. The ./data directory is the directory relative to where Jameleon was executed from.

Because the CSV file name is based on the test case name, each test case should have a unique name unless it is desired that multiple test cases share the same data file. The reporting tool may be confusing since it shows the results based on the test case name and it will look like the same test case was executed multiple times.

Format of a CSV File

When using a CSV file in Jameleon, the first row must be a comma separated list of variable ( or attribute ) names needed by each functional point listed in the test case. The following rows should be a comma delimited list of the values for each of the variables in the same order as the first row. A line starting with "#" is a comment and will be ignored. So if we had a test case that used "searchTerm", "searchTitle", "resultsTitle", "numOfResults", "singleResultsText", the CSV file might look something like:

    #This is a legal comment
    Food,Google,Food,15,"Eat food, It's good"
    #And this is also a legal comment
    Gift,Google,Gift,9231,Christmas is upon us!
    Jameleon,Google,Jameleon,80,Jameleon - Home
The quotes are optional, but they are useful when the value has commas or is just long and is easier to tell the start and end because of quotes.

SQL Data

Jameleon can also use a Relation Database (RDBMS) to drive scripts from. The use of this tag is documented here..

Connecting to Environment-specific Databases.

There are no real standards or rules to accomplish connecting to a different database, depending on the environment and/or organization. This can all be accomplished by setting variables in the appropriate properties file. In other words, while it would be perfectly okay to put the database connection settings in the script itself, they could also be placed in a properties file with the correct environment name and application name.

The only cavaet to defining the database connection settings in a properties file is that, as mentioned above, in order to use a properties file, the application setting of the sesion tag must be set. This means that if properties file is used to store data in that a sql tag cannot be used outside of a session tag because Jameleon won't know which application to get the properties for. However, Jameleon does somewhat support nested session tags. The automated documentation won't be exactly correctly (the inner session tag will be ignored), but it will execute fine.

Using Data

Values retrieved from sources external to the script (properties files, CSV files, databases), are all stored in the scripting engine's context for the script.being executed. This means that you define a variable with the same name in a properties file and in a CSV file that the value will get overwritten depending on when the CSV file is read in. This is why it is best to keep the variable names distinct. One trick that I use is to end all variable names read in from different datasources with different letters. For example, variables defined in a CSV file end in _c, variables defined from a database end in an _s and variables defined in properties files have no special ending.

Using context variables in your script is very simple. Simply surround that context variable with ${}. For example, a variable named firstName_c, stored in a CSV file, would be referenced by ${firstName_c} in the script. This would look something like:

<testcase xmlns="jelly:jameleon" useCSV="true">

              functionId="Change the customer's first name to ${firstName_c}"

Using multiple variables as a single value is also easy:
<testcase xmlns="jelly:jameleon" useCSV="true">

              functionId="Find all customers with ${firstName_c} ${middleInitial_c} ${lastName_c} as their full name."
              customerFullName="${firstName_c} ${middleInitial_c} ${lastName_c}"/>

Don't forget that Jameleon simply uses Jelly for the scripting engine. This means you can also use Jelly tags to create a new context variable and use it.
<testcase xmlns="jelly:jameleon" xmlns:jl="jelly:core" useCSV="true">

    <jl:set var="fullName" value="${firstName_c} ${middleInitial_c} ${lastName_c}"/>
              functionId="Find all customers with ${fullName} as their full name."