How-to: Monitoring the Progress of my Survey

Mail Report

For personalized surveys, the Mail report is your go-to place for an overview of all the mails you have sent during your survey. Once you send a mail from the Participant administration, a report entry is created, and you can display information on the delivery process and, if necessary, also intervene. If your project is an anonymous survey, you can monitor participations directly from the Online statistics.

Monitoring and Managing the Delivery Process

If you would like to check, who you invited to your survey, what you wrote in the invitation mail and which mails were not delivered, just follow the steps below:

• Open your project.

• In the menu on the left-hand side, select the Mail report menu item.

In the upper part of the screen, a graphics shows all mails sent for the given project. In the bottom left part, you can see the total number of mails sent – in the below example, it’s 6. To the right, you can see the different statuses of the mails. In our example below, 5 mails have been sent and one is queued, i.e. it has been created, but not been sent yet. This information is also presented on a percentage basis in form of coloured bars.

Open

In the lower part of the page, a table lists all mailing processes for your project.

• Click on the internal ID or on the subject of the mailing process to display details on all individual mails sent.

Open

• In the lower part of the detail page for the selected mailing process, a list of all email recipients is displayed. The drop-down menu in the Display line allows you to filter the mails by status (Queued, Sending, Sent, Rejected, Delivery failed, Not reachable, Cancelled).

• If you would like to display the mail content for the mailing process, click on the E-mail view button in the upper right corner.

• By clicking the arrow button, you will be directed back to the summary page.

For queued mailings, you can intervene in the delivery process on the summary page. Just proceed as follows:

• If you would like to stop the distribution of a mail, click on the Play symbol in the Queued column. In case you want to resume the distribution of the mail and put it back in queue, click on the Pause symbol.

• To delete a mailing process from the queue, click on the Cross symbol.

• If you would like to change the date and time for the mail delivery, click on the Calendar symbol in the Date Column. Select the desired date and time from the two drop-down menus and click on Change date.

Statistics

The Statistics menu provides several features for reporting and assessing and editing your survey results during the field phase.

• Open your project, and select Statistics from the menu on the left-hand side.

Field report

Once you display the Statistics menu, you get to the Field report menu item. A graphics in the upper part of the page shows an overview of the course of the field phase. Highlight colours provide insight into the different disposition codes – or statuses – of participants. In the example below, you can see how many participants have not yet been invited, how many have been invited, how many have suspended the survey and how many have completed it.

Below the graphics, you can find general information on the survey results, e. g. the total sample and the completion rate, as well as statistical characteristics like the mean processing time, the average number of participants per day and the page with most drop-outs. Further down is the detailed field report with numerous submenus or sections that you can open by clicking on the respective arrow. Here you can, for example, display the access by time of day, access per day or week or the drop-outs per page (page the participant displayed before dropping-out). To display all areas at once, click on the button Open all sections.

Splitting Field Report Results

If required, you have the option of splitting the field report according to certain variables. For example, you can view the distribution of responses split according to the variable "u_gender" (gender of the participant). To do this, simply follow the steps below:

• Click on the Define link at the top right of the field report next to Split:.

• Select the desired split variable from the drop-down menu that appears – in this example: u_gender.

• Click on Save.

• The field report is split by gender (Not available, Male, Female).

• If you want to reset the splitting again, click on the x symbol at the top right of the field report next to Split: and the variable by which you have split.

Filtering Field Report Results

You can also filter the field report by disposition codes, common variables and survey variables. In this way, you can, for example, only display the participants, who suspended the survey, who started the survey at a certain point of time or who answered a question in a certain way. Proceed as follows to filter:

• Click on Filter results at the top right of the field report.

To filter by one or more disposition codes:

• Select one or more disposition codes from the Restrict to disposition codes area and click on Save status selection.

• The filtered field report is immediately displayed.

To filter by a common variable:

• In the Restrict to variable area under Common variables, select a variable you want to filter by – for example, “Beginning of survey”. If you select this variable, only participants, who started the survey in a certain time period, will be displayed.

• Click Continue a little further up.

• Now, set the period for the start of the survey by selecting a date from both drop-down menus.

• Click on Save.

To filter by a survey variable:

• In the Restrict to Variable section under Survey Variables, select a variable by which you want to filter the results – for example, by the answer to a specific question. In the example below, we’ll pick the question "How did you find out about our product?" of the question type "Single response list (vertical)".

• Then, click Continue at the bottom of the page.

• On the next page, select the answer to the question you want to filter by from the drop-down menu.

• Click on Save.

To reset the filtering:

• Click Filter results again.

• If you have filtered by disposition code: Deselect the disposition code by clicking on the selected box. Then, click on Save status selection.

• If you have filtered by a variable: Click on the Delete current restriction button just below Restrict to variable. Then, click on the Statistics breadcrumb to return to the unfiltered field report.

Exporting the Field Report

The field report is always exported in its current status. For example, if you have filtered by a disposition code, only the filtered view will be exported.

• To export the field report, click on the Excel export button at the top right. The report is immediately downloaded in Excel format.

Online Statistics

The Online statistics menu provides you with an overview of the survey results recorded so far – all while your survey is still in the field phase. You can therefore display how previous participants have responded to certain questions. To do this, proceed as follows:

• You are already in the Statistics menu. Now, select Online statistics from the left-hand menu.

• If your project is multilingual, select the desired language from the drop-down menu in the Select language line.

• If required, you can display variable names and types in the statistics.

• If you want to include all questions in the statistics, activate the checkbox in the Include all questions line, or alternatively select the individual questions to be included in the statistics.

• Click on Continue.

The following information on the selected questions is then displayed in the online statistics:

• The number of participants who selected a particular answer, presented as a number, percentage and bar chart

• Total: the total number of people who have answered a question

• Invalid/Missing: indicates how many questions were seen but not answered

• Average value

Filtering/Splitting the Online Statistics

You can also filter or split the Online statistics according to certain variables. The steps are the same as for filtering or splitting the field report. Proceed in exactly the same way as described under Splitting Field Report Results and Filtering Field Report Results.

Open-ended Answers

You can view the answers that participants have given to open questions via the Open-ended Answers menu item.

• Select the Open-ended Answers menu item in the Statistics menu.

• This menu item also allows you to filter the results according to specific disposition codes, general variables or survey variables. To do this, proceed in the same way as described in Filtering Field Report Results.

• If your project is multilingual: Select the desired language for which answers are to be displayed from the Select language drop-down menu.

• In case you want to display the answers to all open questions, activate the checkbox in the Include all questions line.

• If you only want to see the answers to certain questions, select them by clicking the corresponding checkbox on the right-hand side.

• Click on Continue.

The selected questions with open-ended answers – in this case question type 111  Single response list (vertical) with answer option + text and question type 142 Text area – are displayed. If you would like to select other questions for display or remove the filtering, click on the breadcrumb Display settings, and select the desired questions and filter options again.

Quality Correction

We've looked at the figures, now let's take care of quality. The Quality Correction menu item allows you to intervene in the quality of your survey results during the field phase. This feature helps identifying participants, who simply click through the survey and don't really bother answering the questions. These answers are not relevant for the later evaluation of the survey and can be deleted. Detailed information on Quality correction can be found here.

To activate the Quality correction:

• Select the Quality correction menu item in the Statistics menu.

• Click on the Save button to activate the quality correction.

→ The quality correction is triggered, and the original data set is extended by the variable quality, which can be used to check the quality of a survey participation. The process may take a while.

To check the quality of a survey participation:

• After you have triggered the quality correction, open the Survey menu and select the Export menu item.

• Retain the default setting – Result data (all answers, formats e. g. SPSS, CSV, Fixed Format, Quantum, MS Excel...) – and click Continue.

• On the next page, keep all default settings, and click on Export.

• Finally, click on Copy file to local PC on the next page. The export file is downloaded to your local system.

• Open the export file. The quality column provides information about the quality of a survey participation. The value for the quality variable is calculated by relating the individual processing time of a participant to the mean processing time of the entire sample and lies between 0 and 1. A value of 0.5, which is exactly on average, indicates a high quality of participation. Participants, who spent more time on the survey, are above this value. Participants, who are significantly below a value of 0.5, should be viewed critically and their survey responses might not be relevant for the further evaluation of the survey. In the example below, you can see that the participant in the 5th row has a value of 0,23 in the quality column. This would therefore be a participant, whose data records you could delete for quality purposes.

To delete data records that do not have the desired quality:

• Return to the Statistics menu, and open the Detail view menu item. A list of all participations is displayed.

• In the list, search for the participant with the poor quality value: In the previously exported Excel table, you can see that this participant has been assigned the sequential number ("lfdn") 5. This sequential number can also be found in the detail view of participations in the Participation column on the left-hand side. In this way, you can easily determine which participant has the poor quality value. In the example below, it is the participant Peter Pan.

• In the Actions column, activate the checkbox for participant Peter Pan, and select the action Delete data records completely from the drop-down menu below the list.

• Click on Execute.

• Confirm the data deletion on the next page by clicking Delete.

Related Topics

Once again, you are one step closer to completing your first survey project: you now know how to review the distribution of your emails and how to view initial results data in the Field report and in the Online statistics. You have also learned how to check and ensure the quality of your results data during the field phase. But as you may already have suspected – EFS naturally offers many more features, with which you can manage your data during the field phase and easily analyze it after it:

• To find out more about our Data cleaning feature in the Statistics menu, click here.

• Details on Reporting+, our complete solution for analyzing the results of EFS Survey and People projects, can be found here.