Statistics

In the Statistics menu, accessible via the Survey menu, you can monitor your survey. It contains three main sections that can be opened by selecting the respective menu items: the Field report, the Online statistics and the Open-ended answers.

• The Field report provides information about the number of participants and the break-off rate.

• At any time during the survey, the Online statistics provides an overview of the answers to closed questions that have been collected so far. You can either access the data for specific questions or you can access the data of the whole survey.

• Open-ended answers lists the answers to open-ended questions.

Field report

The Field report provides information on the essential characteristic data of a study. It contains:

• statistical information, broken down by gross and net sample

• access statistics listing, for example, the time of days with the most accesses and the average number of participants per day and week

• drop-out statistics which show you on which questionnaire page most participants dropped out. If the drop-out rate is too high, you can react by changing the questionnaire dramaturgy. Errors, for example in filter setting, can also be detected by analyzing the drop-out statistics

• quota statistics

You can invoke the field report by selecting the desired project from the project list and then clicking on the Statistics menu item.

Diagram

The diagram belonging to the field report gives you a quick overview of the progress made in the field phase.

• The highlight colors indicate the participants’ disposition codes

• Click on the disposition codes below the chart to display or hide specific codes.

• If the field period is too long to display each day in the chart

• You can print the chart or download it in PNG, PDF or SVG.

• The most important information is shown above the chart.

• The field report shows the data of the current field period.

Restricting the field report

The field report can be limited to specific disposition codes or variable characteristics. The following options are available:

• disposition codes “completed” = 31, “completed after break” = 32, “screened out” = 37 and “all other status levels”.

• project variables and URL parameters. Exception: Variables of text question types (141, 142, 143, 144, 363), user-defined question variables (question type 911) and user-defined variables cannot be used for restriction purposes.

Please proceed as follows:

• Click the Configuration button.

• In the upper section disable, if necessary, the disposition codes not to be used to restrict the basis.

• If you wish to make restrictions using a project variable, select the latter and confirm with Continue.

• In the next step select the desired variable characteristic.

• Confirm again. The restricted field report is issued.

In order to undo the field report restriction, switch back to the Configuration dialog, select “Delete current restriction” and confirm with Continue.

Splitting the field report

You can split the field report by means of a specific variable.

• The following variables are available for all project types: project variables for closed questions and answer options (v_000n), URL parameters (c_000n), userdefined variables (p_000n), quota IDs (quota), language ID (language) and outmode (outmode).

• Personalized surveys: “Select box” and “radio button” participant variables may be used, provided that answer categories have been defined.

• Employee surveys: To protect the privacy of participants, participant variables cannot be used for splitting. The Org Processor variables in Employee surveys (org_allocation_x, org_code_x, org_function_x) are not available for splitting either: The Org Processor provides a comfortable overview on the return data of individual units.

Please proceed as follows: Click on the Split: Define link and select the desired split variable from the drop-down list. After that, confirm by clicking on Save. The data in the field report will now be split based on the variable you chose. You can export the data as an MS Excel file and process them externally (e.g. create diagrams using MS Excel).

Exporting the field report

You can download the field report in MS Excel file format, e. g. in order to pass the data on to your customers or to plot charts or graphs. This is particularly useful if you need the data of a field report that is restricted by configuration or has been split using the split function (see previous chapters): The field report will be exported in its current configuration.

In order to initiate the export, click on the Excel export tab and save the excel file to your computer. The excel file has an appealing layout, i.e. if necessary, you can directly pass on or present your data.

Setting a variable page marker

Using the so-called variable page marker allows you to determine how many people have completed the survey up to a specific questionnaire page X. Using this function is advisable, for example, if a prize draw or an order form for a newsletter which is not of interest to all respondents are offered at the end of your questionnaire. A respondent who drops out of the survey in such a situation would usually be classified as a dropout although the actual questionnaire has been completed. If you are using the variable page marker, the field report indicates the variable rate which results from the number of respondents who have reached the questionnaire page X marked with a variable page marker (variable rate = number of participants up to page X / gross 2).

To set the page marker, please switch to the Questionnaire editor. Click on the title of the desired page, then select the Properties menu. Tick the “Variable page marker” checkbox and confirm by clicking on Save.

Online statistics

As early as during the field phase, the online statistics give you an overview of the survey data gathered up to that point. In a first step, you must specify which project data the statistics are supposed to cover:

• By ticking the checkboxes for a specific question or ticking the “Tick all checkboxes” checkbox, you can specify whether you want to view all survey results, only a selection or the results for a specific question.

• If you tick the “Dump variable names and types” checkbox, the online statistics will additionally display the names of variables, question types and labels.

In this case, the online statistics will display the following information for the selected questions:

• the number of persons who have chosen a characteristic.

• the percentage of respondents is represented in numbers and as a bar diagram.

• the total number of persons who have given a valid answer to a question.

• Invalid (missing): This value indicates how many questions have been seen but not answered.

Restricting the online statistics

The online statistics can be limited to specific disposition codes or variable characteristics. This feature works analogous to restricting the field report.

• Read rights for config_fieldreport are required for accessing the configuration options.

Splitting the online statistics

The representation of the frequency counts in online statistics can be split using a variable, i.e. you can create cross tables. This feature works analogous to splitting the field report.

• Read rights for stat_split is required for accessing the split options.

Open-ended answers

In this menu, you can view the answers to open-ended questions.

Restricting the open-ended answers

The open answers can be limited to specific disposition codes or variable characteristics. This feature works analogous to restricting the field report.

• Read rights for config_fieldreport are required for accessing the configuration options.

Detail view

If you have the required access rights, you can view, edit and delete the data records of individual survey participants in the EFS admin area.

Access restrictions for accessing data records

For reasons of privacy, the Detail view menu is protected by special rights, i.e. its information and functions are not automatically available to every user:

Authorization for data access:

• If the viewer has read rights to “report_testdata”, they can open the menu Detail view and see the results of the individual participants. A user who has this right, but none of the following additional rights, can view various field data including the consecutive participant number (lfdn) and disposition code, browser information and system variables as e.g. language, flip variables and quota. They will not be able see the IP addresses or address data.

• If the viewer has additional read rights to “ip_addresses”, they will also see the IP addresses and host of participants.

• Additional read rights to “export_with_lfdn” enable you to access address data and result data combined. A user with this right can see the names, e-mail
  addresses, code and tester status of participants, in addition to the field and result data.

Authorization for editing data: Editing of result data is protected by a specific right. Writing authorization to detail_view_edit is required.

Overview of data records

The Projects → {Selected project} → Statistics → Detail view menu provides an overview of the result data records. Depending on your access authorization, the following will be visible:

• consecutive number and disposition code of the participants, and various additional field data, browser information and system variables

• IP address and host of the participants, if you have reading rights to ip_addresses

• in personalized projects, name, e-mail address, tester status and code of the participants, if you have reading authorization to export_with_lfdn

Depending on your access rights and the project type, you can also perform several actions on the individual data records via the “Actions” column:

• Viewing (Anonymous surveys, personalized surveys, panel surveys, employee surveys)

• Editing (Anonymous surveys, personalized surveys, employee surveys)

• Deleting (Anonymous surveys, personalized surveys, employee surveys)

In the upper part of the dialog, you can find the selection criteria of the search function (period of access, disposition code, display of additional variables, names and e-mail addresses of the participants). Also, using the View button you can open additional columns or hide superfluous columns.

Viewing Individual data records

To view the result data of a participant, click the Detail view icon in the overview. The following will be visible:

• consecutive number (“lfdn”) and disposition code of the participants, and various additional field data, browser information and system variables

• IP address and host of the participants, if you have reading rights for “ip_addresses”.

• in personalized projects, name, e-mail address, tester status and code of the participants, if you have reading authorization for “export_with_lfdn”.

• the survey results of the participant.

Editing individual data records

To edit a result data record, click the Edit icon in the column “Actions”. You can change open-ended answers by simply overwriting them. To change closed
answers, change the data by selecting another value in the drop-down lists.

Deleting data records

Using the action „Delete data records completely“, it is possible to delete individual or several data records. In personalized surveys and employee surveys, this will remove not only the field and result data, but the data in participant administration as well.

Deleting result data selectively

Result data can be deleted selectively, i.e. the value of a specific variable for all participants (e.g. an e-mail address determined in the course of the survey) will be completely removed from the records. This makes it possible, for example, to anonymize data later on. When deleting, the respective variable is set to its default value, i.e. “-77” for numeric fields and “NULL” for text fields. Delete operations are logged in project info under project documentation.

Please proceed as follows:

• Click on the Delete data menu item.

• Choose the variable whose content you wish to delete from the records of all participants.

• Confirm by clicking on Delete data.

For security reasons, this function is protected by a separate right: The menu item labeled Projects → {Selected project} → Statistics → Delete data is only visible to users who have read rights to report_erase_data.

Quality correction

The conscientiousness with which survey participants fill in questionnaires is critical for the quality of the results of online surveys. The results from survey participants that merely click very quickly through the questionnaire, for example to get a quick overview of the survey theme or on completion to participate in a prize draw, are of little value for further analysis in the end. The quality correction program helps you to recognize participants who simply “clicked through” by their answer behavior, to mark these data records, and if desired to delete them.

Response times as quality criterion

The response times of a user are the key criteria needed for indexing the quality of survey participants. The time taken by each participant to complete a particular page of a survey can be calculated. First of all an index is formed for each participant that gives information on his mean processing time for a questionnaire page. Please note here: The time required by a participant to complete an entire survey is not a factor in the calculation. Depending on the direction of the questionnaire the number of pages that the participant sees can also vary tremendously.

Subsequently a separate calculation for each page is made, which shows where the survey participant is with regard to the processing time of the media. The median survey time of 100 participants corresponds to the average survey duration of the 50th and the 51st survey participant, when these have been sorted by survey time.

Determining the quality index for individual users

In order to determine the “quality” of a participant in a survey, his individual processing time is set in relation to the average processing time of the entire sample. The relationship can be expressed as a number that is stored in the variable “quality”. The value of 0,5 stands for the results of  a survey of high-quality. A higher value represents a higher quality of the results of the survey. A value below 0.5 stands for lower quality.

Triggering quality correction

You can start the program that calculates the quality index under Projects → {Selected project} → Statistics → Quality correction by clicking on the button Save. It does not make any changes to the actual survey data, rather it expands the data record to include the variable “quality”.

Viewing the quality Index in the export data record

The quality index of a participant will be stored in the variable “quality”. This is contained within the export data record, and therefore you can identify the data records of participants with critical values and where necessary delete them before making an evaluation.

Data cleaning

If a respondent uses the “Back” button in the questionnaire or that of the browser while completing a questionnaire, they is possible that they will pass the same filter question more than once, giving different answers and then being routed into another filter branch, where they will answer different questions.

Example: Somebody answers the question whether they drive a car with “Yes” and is subsequently asked all car-related questions. Then, they go back in the questionnaire and state they do not drive a car. A filter lets them skip all car-related questions now. When the respondent completes the survey, they will have stated they do not drive a car. Nonetheless, all car-related variables will contain values.

In the past, such irrelevant data had to be manually cleaned from the result data. With the “Data cleaning” function, this cleaning process has been automated.

Please note:

• Data cleaning only takes completed interviews into consideration, i.e. records with status 31 or 32.

• The following routing types are supported:

  • “Filter” and “Random rotation” branches.

  • Trigger types which affect routing via changes in the survey results. The function should, however, not be used in projects involving page triggers: In this case, the routing that the respondent took erroneously or for test purposes cannot be unambiguously identified.

  • Action pages.

  • Depot questions.

  • EFS-side Back buttons. Please mind: browser-side Back buttons are not fully supported. If a respondent skips back several pages using the browser Back button first, and then skips forward several pages using the browser Forward button ignoring the system warning, these steps cannot be tracked and stored correctly. I.e. pages might be missing in page history, though valid data from these pages exist.

  • “Random rotation” branches and loops are not supported.

  • Data cleaning should not be applied to surveys which are prefilled with values. I.e. it should not be combined with the options “Transfer master data values to survey variables” → “Always prefill”, “Allow multiple participations, prefill with values of last participation” (Chapter 3.7.10, p. 92) and import of specific answers from other projects.

Activating automatic data cleaning

In the Projects module, click on the desired project, and then choose the Project properties menu item. On the tab General options, tick the checkbox in the “Enable data cleaning” field. Then, confirm by clicking on Submit. The Display link allows you to switch to the Data cleaning submenu.

Triggering data cleaning for the entire project

The documentation of all changes to the data as well as functions for restoring all or part of the original data are located in the data cleaning log. There are two ways of accessing this:

• Click on the Display link in the Project properties menu.

• The second way is via the Statistics → Data cleaning menu.

You are automatically taken to the General tab. The “General information” area shows whether data cleaning has already been performed and how many records were processed.

• Status data cleaning in this project: Data cleaning is activated from the Project properties menu.

• Number of cleaned records: The records that were changed during data cleaning.

• Number of data records that were checked but did not require cleaning: The records that would not be / are not affected in the event of data cleaning.

• Number of uncleaned records:

  • If data cleaning has not taken place, this value will contain the uncleaned records.

  • After data cleaning has taken place, all records that could not be checked and cleaned because the interviews were not completed remain in this field.

  • If further questionnaires were filled out after a manual data cleaning, this value will be the total of all uncleaned records, as well as all records that were not checked during the manual data cleaning because the interviews were not completed.

In the “Data cleaning for the whole project” area, you can trigger or undo data cleaning:

• If you select “Clean all data” and then confirm by clicking on Execute, the cleaning operation will be triggered.

• If you select “Recover all data” and then confirm by clicking on Execute, the original state will be restored.

Viewing details and editing records manually

The Extended tab provides a detailed documentation of all changes to the data as well as functions for both editing individual records and restoring all or part of the original data.

Display options

• Number of datasets per page: If you enter the corresponding value and then confirm by clicking on Apply, you can change the number of records displayed on one page.

• Display only datasets with a particular variable: This search function identifies all records containing the given string. After entering the string, you can trigger the search by clicking on Apply.

• Display routemap: Ticking the checkbox and then clicking on Apply will show the so-called routemap in the log. The routemap is a comma-separated page list reflecting the participant’s valid route through the survey (see the “Example” section below). It is extracted from the page history (phistory is the variable used in the export record, page history is the label used in the SPSS export record).

Interpreting the entries in the data cleaning log

The data cleaning log contains the following information and functions:

• Cons. no.: The respondent’s consecutive number in both the survey table and the export record. The table contains only the records of those respondents for whom data cleaning might be required.

• Cleaned data: Those data from the complete record that were / would be changed during cleaning.

• Original: The original state of those data from the complete record that were / would be changed by cleaning.

• Routemap: Can be shown and hidden using the “Display routemap” function in the “Display options” area.

• Cleaned: yes / no.

Editing individual records

You have the option of targeting individual records for cleaning or for restoring the original state. To do so, tick the respective checkbox in the “Execute data cleaning” or “Recover data” column, and confirm by clicking on Save.

Identifying cleaned data in the export record

The export record provides information if a individual record was cleaned: It is stored in the variable cleaned (labeling in the SPSS export record: “datacleaning”) which can have the following properties:

• 0 = uncleaned data records and records of incomplete interviews which are excluded from the cleaning process

• 1 = cleaned

• 2 = checked but did not require cleaning

Delete data

Result data can be deleted selectively, i.e. the value of a specific variable for all participants (e.g. an e-mail address determined in the course of the survey) will be completely removed from the records. This makes it possible, for example, to anonymize data later on.