Press "Enter" to skip to content

Tutorial: Recording participant responses

FindingFive allows you to record a number of different types of participant responses. This blog post will cover the basics of how to collect your data using choice responses, ratings, free-text responses, and audio responses, along with a few ways to customize various types of responses.


Soliciting choice responses

A common response to solicit from participants is a choice among alternatives offered by the researcher (i.e., forced alternative choice). This is frequently used in surveys as well as many other psychological study paradigms.

The following varieties can all be implemented with the choice response type.

Permitting only a single choice

Use the choice response type without any special modifications of the defaults. By default, it displays a set of buttons for your participants to choose from, and only one choice is allowed. List the button labels under the choices property. Try it out by pasting the following code into the definition of a response (i.e., click the “+” button in the “Responses” section of the left sidebar):

{
    "type": "choice",
    "choices": ["yes", "no", "maybe"]
}

Allowing multiple selections in a choice response

If you want participants to be able to select more than one answer, set the multi_selectproperty to true. Participants will need to lock in their answer with a confirm button after making multiple selections. They can also reset their selections to change or re-do their response.

{
    "type": "choice",
    "choices": ["Georgia", "Colorado", "New York", "Michigan", "Washington"],
    "multi_select": true
}

Note that the reaction time recorded in the case of multi_select is the total number of milliseconds till the moment when participants confirm their selections.

Vertically arranging alternatives in a choice response

By default, the buttons in a choice response are displayed horizontally. This can be suboptimal when there are more than 3 options to choose from.

To list them vertically, simply set layout to vertical. Omitting the layout property from the response definition (which means leaving it at its default setting) or setting it any other value besides vertical will result in a horizontal layout.

{
    "type": "choice",
    "choices": ["Georgia", "Colorado", "New York", "Michigan", "Washington"],
    "multi_select": true,
    "layout": "vertical"
}

Allowing keyboard interactions in choice responses

By default, participants can respond via mouse click only. To enable keyboard responses, list the keys that correspond to the responses in the property key_mapping.

{
    "type": "choice",
    "choices": ["yes", "no", "maybe"],
    "key_mapping": ["F", "J", "B"],
    "key_only": false // setting this to true will disable clicking
}

If you want participants to respond only with key presses, set the key_only property to true.

Getting accuracy information in choice responses

To record the accuracy of your participants’ responses or to provide feedback to participants about their performance (more on this below), specify the correct answer(s) in the target property. Make sure that the correct answer is already listed in the choices of the response!

{
    "type": "choice",
    "choices": ["yes", "no", "maybe"],
    "target": ["yes", "maybe"]
}

You’ll get the accuracy information listed in the participant data, which will be available for download once a study session is complete.

Giving trial-by-trial feedback to participants

To provide trial-by-trial feedback to your participants (for example in the context of a reinforcement learning study), specify your target and set the feedback property to true. This will display trial-by-trial feedback to your participants by highlighting the correct response(s) in green. Additionally, a pleasant sound will be played for a correct answer while an unpleasant sound will alert participants of an incorrect answer.

{
    "type": "choice",
    "choices": ["Russian Blue", "Malamute"],
    "target": ["Russian Blue"],
    "feedback": true
}

For example, the following image demonstrates the feedback a participant receives when the participant’s choice is correct:

Giving incremental feedback to participants when multiple selections are allowed

If you have a choice response with multiple possible selections, you have the option of providing immediate feedback for each selection the participant makes using the incremental_feedback property. For example, here is a choice response with 8 possible choices, 4 of which are the correct targets:

{
  "type": "choice",
  "multi_select": true,
  "layout": "vertical",
  "instruction": "Please select all of the options that are cats:",
  "choices": [
    "Russian Blue",
    "Malamute",
    "Persian",
    "Poodle",
    "Ragdoll",
    "Burmese",
    "Shih Tzu",
    "Berger Picard"
  ],
  "target": [
    "Russian Blue",
    "Persian",
    "Ragdoll",
    "Burmese"
  ],
  "feedback": true,
  "incremental_feedback": true
}

Because incremental_feedback is turned on, participants will be notified of the correctness of their selections immediately after they click on each choice (even before they hit the “confirm” button).

Russian Blue is a target and turned green to signify a correct selection immediately after the participant clicked on it. But Malamute is incorrect, so it flashed orange and cannot be selected by the participant.

Note that if feedback is set to true (regardless of whether participants receive incremental_feedback), participants will be told about the correctness of all of their selections after they hit the “confirm” button. For example, in the screenshot below, a participant has selected some but not all of the correct responses when they confirmed their responses:

This participant neglected to select “Burmese” prior to confirming their selections, and was given feedback about their selections after they confirmed their answers.

If feedback is set to false (or isn’t specified and left to its default value) but incremental_feedback is set to true, participants will be told about the correctness each selection they make after they click on a choice, but will not be told about the overall correctness of their submissions when they hit the confirm button.

Soliciting participants’ opinions in the form of ratings

Rating responses allow participants to express their opinions with a range of numbers.

  • Use upper_bound to indicate the highest number on the displayed rating scale (default is 5).
  • Set zero_centeredto true if you want the rating scale to range from -upper_bound to +upper_bound instead of the default 1 to +upper_bound. When zero_centered is false, participants can also use the number keys to respond.
{
    "type": "rating",
    "upper_bound": 3,
    "zero_centered": false
}

Soliciting free-text responses

A text response allows participants to respond by typing whatever they wish to in a text box.

If desired, you can limit the number of characters participants may type with the max_char property.

{
    "type": "text",
    "max_char": 100
}

Soliciting audio responses

An audio response allows participants to use their microphone to record a response. Note that this feature works most reliably in Chrome and Firefox browsers. Before participants begin the experiment, you may wish to alert them that they will need to use a microphone during the course of the study so that they know to give their browser permission to access their microphone. (The browser should automatically request permission to access the microphone once the experiment begins, but it’s helpful to let participants know what to expect before they start your study.)

To make sure that participants don’t accidentally stop the recording before they are done responding to your prompt, FindingFive automatically continues to record for 500 milliseconds after the participant hits the stop button. If you would prefer more of a recording buffer, this value can be changed using the padding property:

{
    "type": "audio",
    "padding": 2000
}

The participant’s audio response is saved as an ogg file (an open multimedia format similar to mp3). If you plan to analyze the audio responses in a software package that requires a different file format, a tool like FFmpeg can easily take the ogg files saved by FindingFive and convert them into other file formats for you.

Customizing various types of responses

Recording multiple responses within a single trial

If you’d like your participants to answer multiple questions within a single trial (i.e., in response to the same stimulus), place those responses in a nested list within your trial template. This would be useful, for instance, if you wanted participants to select an answer (choice response) and then provide their confidence for that answer (rating response).

"multi_response_trial": {
    "stimuli": ["s1", "s2", "s3"],
    "responses": [["choice_response", "rating_response"]]
}

This trial template will generate 3 trials, one with “s1”, a second one with “s2”, and a final one with “s3”. On each trial, two responses will be recorded, first the choice response, and then the rating responses.

Note that when the responses list contains more than one set of elements (e.g., ["r1", ["r2", "r3"]]), the stimuli list cannot contain more than one set of elements (e.g., ["s1", ["s2", "s3"]]). Otherwise, it would imply simultaneously sampling from stimuli and responses, which can lead to unpredictable behavior.

Customizing instructions for responses

You can provide instructions on how participants should respond with the "instruction" property of the relevant response.

{
   "type": "text",
   "max_char": 100,
   "instruction": "Please provide English words only."
}

Note that this instruction will appear for every trial that includes this response.

Requiring response confirmation

Set the response_confirm property of a trial_template to true if you want participants to explicitly confirm their response before proceeding to the next trial.

A typical text response type
The explicit confirmation required after a participant submits their text response, if the trial_template has “response_confirm” set to “true”

This makes more sense in some scenarios than others – for example, we do NOT recommend use this feature in choice responses when multi_select is true, which would automatically trigger a confirm button already. Having two steps of confirmation doesn’t really make sense. You can experiment with that, however, and see if you agree with our recommendation. We’ll leave the decision up to you!

Response timing

By default, FindingFive displays responses immediately after all stimuli have been presented. To alter this behavior, specify the number of seconds you wish to pause before displaying the response options with the delay property of the relevant response. Note that this delay is relative to the end of the presentation of the last stimulus on the trial – that is, for stimuli without a duration property (e.g., text, images), this will be the moment when the last stimulus just becomes visible, since the presentation of the stimulus is instantaneous; for stimuli that has a duration (e.g., an audio or video file), this will be the moment when the stimulus finishes playing.

{
    "type": "text",
    "max_char": 100,
    "delay": 5
}

This is particularly useful if you’d like to prevent participants from responding too early (for example, in response to a sentence or paragraph). In all cases, responses remain on the screen until participants take the required action.


For more technical details about participant responses, check out the Responses section of the FindingFive API.

If you have questions about this post or how to collect participant responses in FindingFive, please don’t hesitate to contact us at researcher.help@findingfive.com.