Sample

Surveys in panels are usually not comprehensive but rather based on smaller samples from the overall pool of panelists. EFS Panel provides a number of functions for controlling the sampling process:

• Depending on the type of survey and on the situation, the sampling is either performed manually by the panel administrator or the participants are automatically added to an internal sample.

• There is a number of functions available that allow you to limit the sample according to your needs.

• Convenient features, already familiar from the participant administration of personalized projects, will help you manage the participants in your sample.

Open

Before you start

• You can create several samples per project.

• A panelist can be allocated to only one sample per project.

• sample may contain up to 500,000 panelists. Please mind: The size of the sample can affect the performance of your project. If you run a large panel and/or plan a large, complex survey project, it is recommended to draw several small samples and to send the invitations consecutively.

• In the context of a master data survey, the sample can only be applied to testers. No additional sample can be created.

Drawing a sample

To learn how to draw a sample in practice, please proceed as follows:

• Create a panel survey. Make sure that you use the “Panel survey” project type, which is only available for EFS Panel installations.

• In the Sample menu, click on the Create sample button.

• Enter a title and description.

• Confirm by clicking on Submit.

• The edit dialog of the new sample is opened. The Overview tab contains basic information about the new sample. Further tabs contain various setting and editing functions. However, it is not yet possible to select all tabs.

Open

• Open the Define basic set tab.

• In the top section of the tab, you specify which panel group shall form the basic set for drawing the sample. For detailed explanations of the various options.

  • First select the group category and then the panel group from which you want to draw the sample.

  • Optionally, you can use grouping filters and capacity filters to restrict the basic set further.

  • Optionally, you can restrict the sample to panelists with specific panel statuses.

  • Also, a maximum number of participants can be defined. Please note: this is only recommendable if you do not use stratification. If the basic set is narrowed down unnecessarily from the beginning, it might not be possible to reach the target values which are explicitly specified during the stratification process.

• The “Extended settings” allow to further narrow the sample. For example, you can specify how many surveys the members of the new groups must have participated in or how many bonus points they must have. For detailed explanations of the various options.

• In the “Choose the projects...” section, you can optionally exclude the participants of specific projects from the sample draw. To do so, tick the “Exclude” checkbox after the desired project.

• After specifying all criteria, click on the Test filter button.

• This will first test-create the sample. A bar will indicate whether panelists meet the conditions and how many of them do. Furthermore, in addition to the Test filter button, you will now see the Activate filter button.

• You can make further changes to the filter conditions and test them by clicking on Test filter until the result is to your liking.

• After that, click on the Activate filter button.

• The Overview tab now contains information on the size of the new sample. The sample shown in the figure comprises 13 participants.

• Optionally, you can define the distribution of certain attributes within the sample in detail on the Stratification tab. This option will only be available until you have created the sample by activating the Draw sample function on the following tab.

• After you have applied all desired filter criteria to the members of the new sample and also - if stratification is required - performed all necessary stratifications, open the Draw sample tab. Click the Draw sample button: The sample will now be drawn.

• After the sample has been drawn, the Define basic set, Stratification and Draw sample tabs are disabled, the Statistics, View tester list, Sample contents, Inform and Remind again tabs are activated.

Managing samples

The Sample menu contains an overview of all samples which exist within the respective project. The following information will be displayed:

• Title

• Description

• Size: The number of panelists assigned to this sample.

• Completed interviews: Number of sample participants with disposition code 31 and 32.

• Screened-out participants: Number of sample participants with disposition code 35, 36, 37 and 41.

• Tester: Number of testers assigned to this sample.

• Status

• Filter: Shows whether a filter has already been set.

• Stratification

• Accessible: Surveys that are not accessible are marked with a red traffic-light icon. Move the cursor over the icon to display the reason for non-accessibility.

Editing options

Via buttons you can call up the following actions:

• Create sample: Opens the dialog for creating a new sample.

• Send reminders: Allows for a centralized reminder dispatch. Opens a list of all samples from the current project that are currently available. You can select the desired sample.

  • By default, reminders are sent to participants with disposition codes 11, 12, 13 and 17. A drop-down list below the mail form allows you to optionally specify the disposition code of the recipients yourself.

  • You can choose from the codes 11, 12, 13, 14, 17, 20, 21, 22, or 23.

The following “Action” icons are available for editing individual samples:

• By clicking on the title you will open the edit dialog of the respective sample.

• Edit: Opens the properties dialog where you can change the title and description of an existing sample.

• Add tester: Opens the View tester list tab

• Save in group: Clicking on this icon will create a new panel group including all participants from the sample. Optionally, the “Include participants with completed interviews only” feature can be used to restrict the group to those participants who have completed the survey (disposition codes 31, 32).

Searching all samples

On the Search in all samples tab of the Sample menu you can enter a term in the search field. Clicking on Extended search, opens a search mask, which allows you to perform a detailed search. You can, for instance, select a specific panel status, a specific sample or a status from the respective drop-down menus, or you can enter a name in the free text field Search for Panelists.

If necessary, you can customize the search mask to your individual needs:

1. Click on the configuration gear symbol at the very right in the extended search.

2. Select a variable from the drop-down menu, and click Save.

3. The variable is now available in the search mask of the extended search.

You have the option of configuring up to 5 additional variables for the use in the extended search. This includes the variable “Pseudonym”, which adds a free text field to the search mask. In this field you can enter several pseudonyms at the same time, which allows you to do mass searches in your panel projects.

Editing a sample

Clicking on the title of a sample will open the edit dialog. On its various tabs you will find the following information and functions:

• Overview: This tab provides basic information on the sample.

• Basic set: On this tab, you specify which group will serve as the basic set for drawing the sample, which additional criteria the participants are to meet, and how many participants are required.

• Stratification: On this tab, you can define the distribution of certain attributes within the sample in detail.

• Draw sample: After defining the basic set and optionally applying stratification, the actual drawing of the sample is performed on this tab.

• Statistics: On this tab, you can create an ad hoc statistics to get an overview of how the sample is composed.

• View tester list: On this tab you will find all the testers allocated to the current sample.

• Sample contents: This tab lists the sample participants.

• Inform / Inform again / Remind again: Via these tabs you can contact sample participants with specific disposition codes.

Limiting the basic set

On the Basic set tab, you specify which panel group shall form the basic set for drawing the sample and which criteria are to be used for further limiting this basic set. The criteria are explained in the following section.

Basic set

Extended settings

Excluding panelists already participating in other panel surveys

Panelists who are already participating in other projects can be excluded from the sample draw. This may be useful if you want to avoid overburdening your respondents. Sometimes it may also be the case that several surveys are conducted on the same range of topics at the same time, while you want the panelists to participate in only one of these surveys.

At the bottom of the Define basic set tab, open the section “Choose the projects whose participants shall be excluded from the basic set”. This will open a list of all panel surveys. Select those whose participants you do not want to include in the basic set of the current sample.

Controlling the attribute distribution within the sample

If necessary, you can define certain attribute distributions within the sample in detail. This process is also known as “stratification”.

• You define the distribution of attributes for each combination of attributes or for each individual attribute.

• You can either enter the target values by hand or import them.

• You can enter either absolute numbers or percentages.

The following sections first explain the available definition methods and cell value specification modes. This is followed by a description of both the manual definition and the import.

Available definition methods

The table provides an overview of the definition methods available.

Cell definition mode

Similar to the quotas, there are two ways to define the target values for stratification:

• Interlaced: The distribution is defined in detail for all possible combinations of attributes. In the example below, it is defined exactly how many male soccer fans, female soccer fans, male and female baseball fans the sample contains.

• Non-interlaced: The stratification distribution is defined for each selected answer category of the stratification variables. The internal distribution is ignored. If this mode was applied in the example below, this might mean that soccer and baseball fans would each account for 50% of the participants, 50% of the participants were male, while the soccer fans could be made up of 20% men and 80% women.

Manual stratification

• Create a sample as usual and define the basic set.

• Switch to the Stratification tab.

• Select one of the manual stratification methods and confirm by clicking on Continue.

• Select the appropriate cell value specification mode. You can choose between “interlaced” and “non-interlaced”. Per default, “interlaced” is activated. If you change the setting, please confirm with Restrict.

• Optionally you can restrict the list of available variables to a specific master data category. If you do so, confirm with Restrict.

• The variables available for stratification are listed in the bottom section of the tab. Select the appropriate stratification variable(s) and confirm by clicking on Continue. In the example, the variables u_gender and m_favoritesports are used.

• All answer categories of the selected stratification variables are listed.

• Select the answer categories you wish to use for stratification. It is advisable to exclude missing categories such as “no answer”. Confirm by clicking on Continue.

• An overview of the stratification cells with their current proportions will be displayed. The number of surveys completed by the prospective sample members over the past weeks will also be indicated: Thus, you can check in time whether a high percentage of the panelists might be overworked or whether many candidates have not participated in any survey for some time i.e. might be inactive.

• Enter the target values in absolute numbers or percentages, as required. When expressing target values as percentages, you also have to predefine the absolute total amount to be reached. It should be defined in such a manner that the specific target values can be met based on the number of existing candidates.

• Click on Check to see whether your stratification can be achieved with the current basic set.

  • Target values or the total number may have to be adjusted. Please bear in mind that the values may diverge due to rounding. In the example above, the sum total of the values of the individual cells (10) slightly differs from the absolute total number of the panelists in this stratification (13).

  • It may also be necessary to modify the basic set and the stratification criteria in order to reach the target values.

• If the values are valid, this will be indicated by a green success message. In this case you can complete the process by clicking on Perform stratification.

• On the Overview tab, you will now find a “Stratification” section. This section lists the stratification cells, the target values and the total number. In the course of the sample draw, the following additional data will be displayed:

  • Number of allocated participants: Number of participants in the sample that are allocated to a particular cell. This is displayed after the sample draw.

  • Completed or screened-out: Number of participants with disposition codes 31, 32, 36 and 37, allocated to a particular cell.

  • Completed or screened-out %: Percentage of participants with disposition codes 31, 32, 36 and 37, allocated to a particular cell.

Defining stratification via import

Formatting the import file

The import file should be formatted as follows:

• Format: CSV or Excel format

• For each master data variable used a corresponding column has to be created; an additional column has to be created for the target values.

• The first line of the import file must contain the names of the variables and the title of the target value column, “target”.

• In the following lines of the import file the target value for each single combination of master data characteristics is specified.

• If the values entered have decimal places, please use a point as decimal separator.

Example:

m_0001;m_0002;target

1;1;1000

1;2;1150

2;1;950

2;2;1200

This file describes a stratification employing the variables m_0001 and m_0002. Both variables have the characteristics 1 and 2 each. Each line contains the target value for a combination of the characteristics of these variables. For example, the second line “1;1;1000” specifies that for the combination of m_0001 = 1 and m_0002 = 2 the target value 1000 is to be obtained.

Import

Please proceed as follows:

• Create a sample as usual, define the basic set and switch to the Stratification tab.

• Select one of the stratification methods using import and confirm by clicking on Continue.

• Locate the import file on your hard drive and upload the file by clicking on Import file.

• You will be directed to the dialog showing you the current proportion of characteristics. Check the selection of the variables and the target values. Then perform check and stratification as described above.

Checking the Composition of the sample

On the Statistics tab, you can create an ad hoc statistics to get an overview of how the sample is composed.

• You can select the report variable and optionally two split variables.

• This ad hoc statistics is always based on the current sample.

Viewing and managing the participants of a sample

The Sample contents tab contains a list of all panelists assigned to the sample.

• Listed for each panelist are:

• First name

• Name

• E-mail

• Panel status

• Code: This is the panelist’s survey access code. You can copy this code to an e-mail or use it to examine the survey from the perspective of this panelist.

•  Mails sent: Number of e-mails sent in the course of this sample.

• Status: Panelist’s disposition code in this panel survey.

• Date of first invitation

• Last access:

• Monitoring: With the monitoring function you can observe passively as another user clicks through a survey with a panelist account. Please note: The monitoring mode is deprecated and will be dropped in one of the next versions.

By clicking on the name or the Edit icon, you can open the detail view of the respective panelist. Additional information can be displayed via View.

Searching the sample

To search sample members you can use the usual simple search. However via extended search you can use additional limitation criteria:

• Always available for selection are panel status and the participant’s status in the sample.

• If you have used stratification, the stratification variables can be used as search criteria, too.

Editing options

A range of participant administration actions are available for editing sample members, such as, for example, the dispatching of e-mails to selected participants, or reset with or without deleting data.

When using an action proceed in a manner analogous to participant administration: Search the desired panelists and mark them in the “Selection” column. Then set the desired action and confirm with Send request.

Inviting and reminding sample members

Dedicated tabs allow you to e-mail the sample members in a centralized manner during various work phases:

• Inform: This tab is displayed after creating a new sample. You can use it to reach all participants with disposition codes 11, 12, 20 and 22.

• Inform again: This tab is displayed after sending the first invitation e-mail. You can use it to reach all participants with disposition codes 11, 12, 20 and 22.

• Remind again: This tab is visible after creating a new sample. After sending the first invitation e-mail, it is activated.

  • By default, reminders are sent to participants with disposition codes 11, 12, 20 and 22.

  • A drop-down list below the mail form allows you to optionally specify the disposition code of the recipients yourself. You can choose from the codes 11, 12, 13, 14, 20, 21, 22, or 23.

Managing personalized links for external survey start

When redirecting sample members to an external survey, personalized links must be used. The following features for managing these links are available:

• Import links: Opens the import dialog.

  • The CSV file must contain the survey link in the first column and the url has to start with the protocol (http:// or https://), the placeholder #panel_url# or any other placeholder specifying the protocol. Optionally you may add the panelists’ pseudonym, u_account, u_other_id, or panelist_code in the second column. In this case the links will be assigned to the panel members accordingly.

  • You can only import or update links for panelists that have not yet started the survey.

• Export links: Triggers the export of all links.

• Delete links: All links are deleted after a confirmation prompt.