Surveys in panels are usually not comprehensive but rather based on smaller samples from the overall pool of panelists. EFS Panel provides a number of functions for controlling the sampling process:
Depending on the type of survey and on the situation, the sampling is either performed manually by the panel administrator or the participants are automatically added to an internal sample.
There is a number of functions available that allow you to limit the sample according to your needs.
Convenient features, already familiar from the participant administration of personalized projects, will help you manage the participants in your sample.
Before you start
You can create several samples per project.
A panelist can be allocated to only one sample per project.
sample may contain up to 500,000 panelists. Please mind: The size of the sample can affect the performance of your project. If you run a large panel and/or plan a large, complex survey project, it is recommended to draw several small samples and to send the invitations consecutively.
Samples are protected by their own “sample_access” right. With read rights, you have access to the samples already available for a panel project and can edit them. With write rights, you can create new samples.
Drawing a sample
To learn how to draw a sample in practice, please proceed as follows:
Create a panel survey. Make sure that you use the “Panel survey” project type, which is only available for EFS Panel installations.
In the Sample menu, click on the Create sample button.
Enter a title and description.
Confirm by clicking on Submit.
The edit dialog of the new sample is opened. The Overview tab contains basic information about the new sample. Further tabs contain various setting and editing functions. However, it is not yet possible to select all tabs.
Open the Define basic set tab.
In the top section of the tab, you specify which panel group shall form the basic set for drawing the sample. For detailed explanations of the various options.
First select the group category and then the panel group from which you want to draw the sample.
Optionally, you can use grouping filters and capacity filters to restrict the basic set further.
Optionally, you can restrict the sample to panelists with specific panel statuses.
Also, a maximum number of participants can be defined. Please note: this is only recommendable if you do not use stratification. If the basic set is narrowed down unnecessarily from the beginning, it might not be possible to reach the target values which are explicitly specified during the stratification process.
The “Extended settings” allow to further narrow the sample. For example, you can specify how many surveys the members of the new groups must have participated in or how many bonus points they must have. For detailed explanations of the various options.
In the “Choose the projects...” section, you can optionally exclude the participants of specific projects from the sample draw. To do so, tick the “Exclude” checkbox after the desired project.
After specifying all criteria, click on the Test filter button.
This will first test-create the sample. A bar will indicate whether panelists meet the conditions and how many of them do. Furthermore, in addition to the Test filter button, you will now see the Activate filter button.
You can make further changes to the filter conditions and test them by clicking on Test filter until the result is to your liking.
After that, click on the Activate filter button.
The Overview tab now contains information on the size of the new sample. The sample shown in the figure comprises 13 participants.
Optionally, you can define the distribution of certain attributes within the sample in detail on the Stratification tab. This option will only be available until you have created the sample by activating the Draw sample function on the following tab.
After you have applied all desired filter criteria to the members of the new sample and also - if stratification is required - performed all necessary stratifications, open the Draw sample tab. Click the Draw sample button: The sample will now be drawn.
After the sample has been drawn, the Define basic set, Stratification and Draw sample tabs are disabled, the Statistics, View tester list, Sample contents, Inform and Remind again tabs are activated.
Managing samples
The Sample menu contains an overview of all samples which exist within the respective project. The following information will be displayed:
Title
Description
Size: The number of panelists assigned to this sample.
Completed interviews: Number of sample participants with disposition code 31 and 32.
Screened-out participants: Number of sample participants with disposition code 35, 36, 37 and 41.
Tester: Number of testers assigned to this sample.
Status
Filter: Shows whether a filter has already been set.
Stratification
Accessible: Surveys that are not accessible are marked with a red traffic-light icon. Move the cursor over the icon to display the reason for non-accessibility.
Editing options
Via buttons you can call up the following actions:
Create sample:Opens the dialog for creating a new sample.
Send reminders: Allows for a centralized reminder dispatch. Opens a list of all samples from the current project that are currently available. You can select the desired sample.
By default, reminders are sent to participants with disposition codes 11, 12, 13 and 17. A drop-down list below the mail form allows you to optionally specify the disposition code of the recipients yourself.
You can choose from the codes 11, 12, 13, 14, 17, 20, 21, 22, or 23.
The following “Action” icons are available for editing individual samples:
By clicking on the title you will open the edit dialog of the respective sample.
Edit: Opens the properties dialog where you can change the title and description of an existing sample.
Add tester: Opens the View tester list tab
Save in group: Clicking on this icon will create a new panel group including all participants from the sample. Optionally, the “Include participants with completed interviews only” feature can be used to restrict the group to those participants who have completed the survey (disposition codes 31, 32).
Editing a sample
Clicking on the title of a sample will open the edit dialog. On its various tabs you will find the following information and functions:
Overview: This tab provides basic information on the sample.
Basic set: On this tab, you specify which group will serve as the basic set for drawing the sample, which additional criteria the participants are to meet, and how many participants are required.
Stratification: On this tab, you can define the distribution of certain attributes within the sample in detail.
Draw sample: After defining the basic set and optionally applying stratification, the actual drawing of the sample is performed on this tab.
Statistics: On this tab, you can create an ad hoc statistics to get an overview of how the sample is composed.
View tester list: On this tab you will find all the testers allocated to the current sample.
Sample contents: This tab lists the sample participants.
Inform / Inform again / Remind again: Via these tabs you can contact sample participants with specific disposition codes.
Which tabs may be opened and edited depends on the respective stage in the sample drawing process.
Limiting the basic set
On the Basic set tab, you specify which panel group shall form the basic set for drawing the sample and which criteria are to be used for further limiting this basic set. The criteria are explained in the following section.
Basic set
Field | Meaning |
|---|---|
Group category | Groups can be sorted thematically and arranged clearly. |
Group | People offers you various possibilities to form groups (also called "subpanels"). |
Filter | You can use grouping filters for creating samples. |
Capacity filter | The capacity filters can be used for organizing the panelists’ capacity in an optimal way. |
Panelstatus | The most important characteristic of panelists is their panel status: it describes their current role and activity. |
Maximum number of participants | Optionally, you can define how many participants are to be contained in the sample. Please note: limiting the maximum number is only recommendable if you do not use stratification. If the basic set is narrowed down unnecessarily from the beginning, it might not be possible to reach the target values which are explicitly specified during the stratification process. |
Extended settings
Field | Meaning |
|---|---|
Number of completed surveys (based on variable n_poll) | Evaluates the “n_poll” system data item. Please mind: If tracking is activated, evaluation of the variable “track_num_completed” (e.g. via grouping filter) provides you with more |
Number of bonus points | Evaluates the current sum of the bonus points in the panelist’s account. |
Last completed interview at least / not longer than x months ago | Evaluates the system data item last_poll_date. Please mind: If tracking is activated, evaluation of the variable “track_num_completed” (e.g. via grouping filter) provides you with more precise information. |
Panel member since x months | Calculates the time for which the panelist has been a panel member based on the system data item “penter_date”. |
Last updating of master data for x months | |
PCI value | The Panelist Compliance Index enables the reliability of panelists to be captured and recorded in a variable. |
E-mail verification status | Status of the double opt-in e-mail verification |
Variable used to define uniqueness | Each characteristic of this variable will be in the sample not more than once. Please note that finding unique values requires an extra internal processing step. Therefore, if you use this feature, queries will be considerably slower. |
Excluding panelists already participating in other panel surveys
Panelists who are already participating in other projects can be excluded from the sample draw. This may be useful if you want to avoid overburdening your respondents. Sometimes it may also be the case that several surveys are conducted on the same range of topics at the same time, while you want the panelists to participate in only one of these surveys.
At the bottom of the Define basic set tab, open the section “Choose the projects whose participants shall be excluded from the basic set”. This will open a list of all panel surveys. Select those whose participants you do not want to include in the basic set of the current sample.
If you create several samples in the same project, each panelist can only be allocated to one of these samples.
Controlling the attribute distribution within the sample
If necessary, you can define certain attribute distributions within the sample in detail. This process is also known as “stratification”.
You define the distribution of attributes for each combination of attributes or for each individual attribute.
You can either enter the target values by hand or import them.
You can enter either absolute numbers or percentages.
The following sections first explain the available definition methods and cell value specification modes. This is followed by a description of both the manual definition and the import.
Available definition methods
The table provides an overview of the definition methods available.
Option | Meaning |
|---|---|
Manual selection of the stratification variables and entry of the target distribution (absolute values) | The stratification is defined manually as usual, the target distribution will be specified in absolute values |
Manual selection of the stratification variables and entry of the target distribution (percentages) | The traditional stratification method involving entry of target distribution as percentage. |
Import of a file containing the stratification settings | The stratification is defined by import, the target distribution will be specified in absolute values. |
Import of a file containing the stratification settings | The stratification is defined by import, the target distribution will be specified in percentages. |
Cell definition mode
Similar to the quotas, there are two ways to define the target values for stratification:
Interlaced: The distribution is defined in detail for all possible combinations of attributes. In the example below, it is defined exactly how many male soccer fans, female soccer fans, male and female baseball fans the sample contains.
Non-interlaced: The stratification distribution is defined for each selected answer category of the stratification variables. The internal distribution is ignored. If this mode was applied in the example below, this might mean that soccer and baseball fans would each account for 50% of the participants, 50% of the participants were male, while the soccer fans could be made up of 20% men and 80% women.
Manual stratification
Create a sample as usual and define the basic set.
Switch to the Stratification tab.
Select one of the manual stratification methods and confirm by clicking on Continue.
Select the appropriate cell value specification mode. You can choose between “interlaced” and “non-interlaced”. Per default, “interlaced” is activated. If you change the setting, please confirm with Restrict.
Optionally you can restrict the list of available variables to a specific master data category. If you do so, confirm with Restrict.
The variables available for stratification are listed in the bottom section of the tab. Select the appropriate stratification variable(s) and confirm by clicking on Continue. In the example, the variables u_gender and m_favoritesports are used.
All answer categories of the selected stratification variables are listed.
Select the answer categories you wish to use for stratification. It is advisable to exclude missing categories such as “no answer”. Confirm by clicking on Continue.
An overview of the stratification cells with their current proportions will be displayed. The number of surveys completed by the prospective sample members over the past weeks will also be indicated: Thus, you can check in time whether a high percentage of the panelists might be overworked or whether many candidates have not participated in any survey for some time i.e. might be inactive.
Enter the target values in absolute numbers or percentages, as required. When expressing target values as percentages, you also have to predefine the absolute total amount to be reached. It should be defined in such a manner that the specific target values can be met based on the number of existing candidates.
Click on Check to see whether your stratification can be achieved with the current basic set.
Target values or the total number may have to be adjusted. Please bear in mind that the values may diverge due to rounding. In the example above, the sum total of the values of the individual cells (10) slightly differs from the absolute total number of the panelists in this stratification (13).
It may also be necessary to modify the basic set and the stratification criteria in order to reach the target values.
If the values are valid, this will be indicated by a green success message. In this case you can complete the process by clicking on Perform stratification.
On the Overview tab, you will now find a “Stratification” section. This section lists the stratification cells, the target values and the total number. In the course of the sample draw, the following additional data will be displayed:
Number of allocated participants: Number of participants in the sample that are allocated to a particular cell. This is displayed after the sample draw.
Completed or screened-out: Number of participants with disposition codes 31, 32, 36 and 37, allocated to a particular cell.
Completed or screened-out %: Percentage of participants with disposition codes 31, 32, 36 and 37, allocated to a particular cell.
Defining stratification via import
Formatting the import file
The import file should be formatted as follows:
Format: CSV or Excel format
For each master data variable used a corresponding column has to be created; an additional column has to be created for the target values.
The first line of the import file must contain the names of the variables and the title of the target value column, “target”.
In the following lines of the import file the target value for each single combination of master data characteristics is specified.
If the values entered have decimal places, please use a point as decimal separator.
Example:
m_0001;m_0002;target
1;1;1000
1;2;1150
2;1;950
2;2;1200
This file describes a stratification employing the variables m_0001 and m_0002. Both variables have the characteristics 1 and 2 each. Each line contains the target value for a combination of the characteristics of these variables. For example, the second line “1;1;1000” specifies that for the combination of m_0001 = 1 and m_0002 = 2 the target value 1000 is to be obtained.
Import
Please proceed as follows:
Create a sample as usual, define the basic set and switch to the Stratification tab.
Select one of the stratification methods using import and confirm by clicking on Continue.
Locate the import file on your hard drive and upload the file by clicking on Import file.
You will be directed to the dialog showing you the current proportion of characteristics. Check the selection of the variables and the target values. Then perform check and stratification as described above.
Checking the Composition of the sample
On the Statistics tab, you can create an ad hoc statistics to get an overview of how the sample is composed.
You can select the report variable and optionally two split variables.
This ad hoc statistics is always based on the current sample.
Viewing and managing the participants of a sample
The Sample contents tab contains a list of all panelists assigned to the sample.
Listed for each panelist are:
First name
Name
E-mail
Panel status
Code: This is the panelist’s survey access code. You can copy this code to an e-mail or use it to examine the survey from the perspective of this panelist.
Mails sent: Number of e-mails sent in the course of this sample.
Status: Panelist’s disposition code in this panel survey.
Date of first invitation
Last access:
Monitoring: With the monitoring function you can observe passively as another user clicks through a survey with a panelist account. Please note: The monitoring mode is deprecated and will be dropped in one of the next versions.
By clicking on the name or the Edit icon, you can open the detail view of the respective panelist. Additional information can be displayed via View.
Searching the sample
To search sample members you can use the usual simple search. However via extended search you can use additional limitation criteria:
Always available for selection are panel status and the participant’s status in the sample.
If you have used stratification, the stratification variables can be used as search criteria, too.
Editing options
A range of participant administration actions are available for editing sample members, such as, for example, the dispatching of e-mails to selected participants, or reset with or without deleting data.
Actions | Description |
|---|---|
Reset and delete data | Status of the panelist in the sample is reset to “active”. If the panelist had already participated in the survey he can participate again. Participant's results data already recorded are deleted. |
Reset without deleting data | Status of the panelist in the sample is reset to “active”. Participant's |
Delete panelist from this sample | Panelist is removed from sample. |
Send e-mail | Using this function, you can send mails to the selected panelists. |
Disposition code 15 (not available) | If invitation mails are bounced, you can set the corresponding panelists to disposition code 15. This code will be taken into account in the field report during evaluation. |
When using an action proceed in a manner analogous to participant administration: Search the desired panelists and mark them in the “Selection” column. Then set the desired action and confirm with Send request.
Inviting and reminding sample members
Dedicated tabs allow you to e-mail the sample members in a centralized manner during various work phases:
Inform: This tab is displayed after creating a new sample. You can use it to reach all participants with disposition codes 11, 12, 20 and 22.
Inform again: This tab is displayed after sending the first invitation e-mail. You can use it to reach all participants with disposition codes 11, 12, 20 and 22.
Remind again: This tab is visible after creating a new sample. After sending the first invitation e-mail, it is activated.
By default, reminders are sent to participants with disposition codes 11, 12, 20 and 22.
A drop-down list below the mail form allows you to optionally specify the disposition code of the recipients yourself. You can choose from the codes 11, 12, 13, 14, 20, 21, 22, or 23.
Managing personalized links for external survey start
When redirecting sample members to an external survey, personalized links must be used. The following features for managing these links are available:
Import links: Opens the import dialog.
The CSV file must contain the survey link in the first column and the url has to start with the protocol (http:// or https://), the placeholder #panel_url# or any other placeholder specifying the protocol. Optionally you may add the panelists’ pseudonym, u_account, u_other_id, or panelist_code in the second column. In this case the links will be assigned to the panel members accordingly.
You can only import or update links for panelists that have not yet started the survey.
Export links: Triggers the export of all links.
Delete links: All links are deleted after a confirmation prompt.
For details on external survey start, please see the dedicated manual which is available in the customer center. Please note that this document is currently in revision.