Tutorial: Recording participant responses

FindingFive allows you to record a number of different types of participant responses. This blog post will cover the basics of how to collect your data using choice responses, ratings, free-text responses, and audio responses, along with a few ways to customize various types of responses.

Soliciting choice responses

A common response to solicit from participants is a choice among alternatives offered by the researcher (i.e., alternative forced choice). This is frequently used in surveys as well as many other psychological study paradigms.

The following varieties can all be implemented with the choice response type.

Permitting only a single choice

Use the choice response type without any special modifications of the defaults. By default, it displays a set of buttons for your participants to choose from, and only one choice is allowed. List the button labels under the choices property. Try it out by pasting the following code into the definition of a response:

{
    "type": "choice",
    "choices": ["yes", "no", "maybe"]
}

Allowing multiple selections in a choice response

If you want participants to be able to select more than one answer, set the multi_selectproperty to true. Participants will need to lock in their answer with a confirm button after making multiple selections. They can also reset their selections to change or re-do their response.

{
    "type": "choice",
    "choices": ["Georgia", "Colorado", "New York", "Michigan", "Washington"],
    "multi_select": true
}

Note that the reaction time recorded in the case of multi_select is the total number of milliseconds till the moment when participants confirm their selections.

Vertically arranging or wrapping alternatives in a choice response

By default, the buttons in a choice response are displayed horizontally. This can be suboptimal when there are more than 3 options to choose from.

To list them vertically, simply set layout to vertical. Omitting the layout property from the response definition (which means leaving it at its default setting) or setting it any other value besides vertical will result in a horizontal layout.

{
    "type": "choice",
    "choices": ["Georgia", "Colorado", "New York", "Michigan", "Washington"],
    "multi_select": true,
    "layout": "vertical"
}

Alternatively, in this situation, you may choose to wrap choice options, so that they do not all appear in a single row. To do so, set layout to horizontal(or omit the layout property from the response) and wrap to true. By default, choice responses are not wrapped.

{
    "type": "choice",
    "choices": ["Georgia", "Colorado", "New York", "Michigan", "Washington"],
    "multi_select": true,
    "layout": "horizontal",
  	"wrap": true
}

Allowing keyboard interactions in choice responses

By default, participants can respond via mouse click only. To enable keyboard responses, list the keys that correspond to the responses in the property key_mapping.

{
    "type": "choice",
    "choices": ["yes", "no", "maybe"],
    "key_mapping": ["F", "J", "B"],
    "key_only": false // setting this to true will disable clicking
}

If you want participants to respond only with key presses, set the key_only property to true.

Getting accuracy information in choice responses

To record the accuracy of your participants’ responses or to provide feedback to participants about their performance (more on this below), specify the correct answer(s) in the target property. Make sure that the correct answer is already listed in the choices of the response!

{
    "type": "choice",
    "choices": ["yes", "no", "maybe"],
    "multi_select": true,
    "target": ["yes", "maybe"]
}

You’ll get the accuracy information listed in the participant data, which will be available for download once a study session is complete.

Giving trial-by-trial feedback to participants

To provide trial-by-trial feedback to your participants (for example in the context of a reinforcement learning study), specify your target and set the feedback property to true. This will display trial-by-trial feedback to your participants by highlighting the correct response(s) in green. Additionally, a pleasant sound will be played for a correct answer while an unpleasant sound will alert participants of an incorrect answer.

{
    "type": "choice",
    "choices": ["Russian Blue", "Malamute"],
    "target": ["Russian Blue"],
    "feedback": true
}

For example, the following image demonstrates the feedback a participant receives when the participant’s choice is correct:

Giving incremental feedback to participants when multiple selections are allowed

If you have a choice response with multiple possible selections, you have the option of providing immediate feedback for each selection the participant makes using the incremental_feedback property. For example, here is a choice response with 8 possible choices, 4 of which are the correct targets:

{
  "type": "choice",
  "multi_select": true,
  "layout": "vertical",
  "instruction": "Please select all of the options that are cats:",
  "choices": [
    "Russian Blue",
    "Malamute",
    "Persian",
    "Poodle",
    "Ragdoll",
    "Burmese",
    "Shih Tzu",
    "Berger Picard"
  ],
  "target": [
    "Russian Blue",
    "Persian",
    "Ragdoll",
    "Burmese"
  ],
  "feedback": true,
  "incremental_feedback": true
}

Because incremental_feedback is turned on, participants will be notified of the correctness of their selections immediately after they click on each choice (even before they hit the “confirm” button).

Note that if feedback is set to true (regardless of whether participants receive incremental_feedback), participants will be told about the correctness of all of their selections only after they hit the “confirm” button. For example, in the screenshot below, a participant has selected some but not all of the correct responses when they confirmed their responses:

If feedback is set to false (or isn’t specified and left to its default value) but incremental_feedback is set to true, participants will be told about the correctness each selection they make after they click on a choice, but will not be told about the overall correctness of their submissions when they hit the confirm button.

Soliciting participants’ opinions in the form of ratings

Rating responses allow participants to express their opinions with a range of numbers.

Basic rating response questions

• Use max to indicate the highest number on the displayed rating scale (default is 5).
• Set zero_centeredto true if you want the rating scale to range from -max to +max instead of the default range (1 to +max). When zero_centered is false:
  • usemin to indicate the lowest number on the displayed rating scale (default is 1).
  • participants can also use the number keys to respond.

{
    "type": "rating",
    "max": 3,
    "zero_centered": false
}

Multi-row rating responses in a matrix

• List out the name of each question inrow_labelsif you want to present multiple questions that require rating responses (i.e., to form a rating matrix).

{
    "type": "rating",
    "instruction": "How do you like your new computer?",
    "row_labels": [
      "Overall speed",
      "Speakers",
      "Webcam",
      "Weight"
    ]
}

For more settings of properties on rating responses, please refer to our study grammar documentation.

Soliciting free-text responses

A text response allows participants to respond by typing whatever they wish to in a text box. If desired, you can limit the number of characters participants may type with the max_char property.

{
    "type": "text",
    "max_char": 100
}

Soliciting audio responses

An audio response allows participants to use their microphone to record a response. Note that this feature works most reliably in Chrome and Firefox browsers. Before participants begin the experiment, you may wish to alert them that they will need to use a microphone during the course of the study so that they know to give their browser permission to access their microphone. (The browser should automatically request permission to access the microphone once the experiment begins, but it’s helpful to let participants know what to expect before they start your study.)

To make sure that participants don’t accidentally stop the recording before they are done responding to your prompt, FindingFive automatically continues to record for 500 milliseconds after the participant hits the stop button. If you would prefer more of a recording buffer, this value can be changed using the padding property:

{
    "type": "audio",
    "padding": 2000
}

The participant’s audio response is saved as an ogg file (an open multimedia format similar to mp3). If you plan to analyze the audio responses in a software package that requires a different file format, a tool like FFmpeg can easily take the ogg files saved by FindingFive and convert them into other file formats for you.

Beyond type and padding, the audio response allows for a range of customizations (see our grammar reference for full details). For example, using the rerecording_allowed property, you can decide whether participants are permitted to rerecord themselves (overwriting the previous recording). Another important property is onset_detection — when this property is set to true, FindingFive will detect and record the time (in milliseconds) at which your participant started speaking.

Customizing various types of responses

Recording multiple responses within a single trial

If you’d like your participants to answer multiple questions within a single trial (i.e., in response to the same stimulus), place those responses in a nested list within your trial template. This would be useful, for instance, if you wanted participants to select an answer (choice response) and then provide their confidence for that answer (rating response).

"multi_response_trial": {
    "stimuli": ["s1", "s2", "s3"],
    "responses": [["choice_response", "rating_response"]]
}

This trial template will generate 3 trials, one with “s1”, a second one with “s2”, and a final one with “s3”. On each trial, two responses will be recorded, first the choice response, and then the rating responses.

Customizing instructions for responses

You can provide instructions on how participants should respond with the instruction property of the relevant response.

{
   "type": "text",
   "max_char": 100,
   "instruction": "Please provide English words only."
}

Note that this instruction will appear for every trial that includes this response.

Response timing

By default, FindingFive displays responses immediately after all stimuli have been presented. To alter this behavior, specify the number of seconds you wish to pause before displaying the response options with the delay property of the relevant response.

{
    "type": "text",
    "max_char": 100,
    "delay": 5
}

This is particularly useful if you’d like to prevent participants from responding too early (for example, in response to a sentence or paragraph). In all cases, responses remain on the screen until participants take the required action.

Note that this delay is relative to the end of the presentation of the last stimulus on the trial. For stimuli without a duration property (e.g., text, images), this will be the moment when the last stimulus just becomes visible, since the presentation of the stimulus is instantaneous. For stimuli that have a duration (audio files, video files, or tokenized text), this will be based on whether any of the stimuli have their barrier property set to true. If any stimuli are acting as barriers and have their barrier property set to true, responses will be displayed after the barrier stimuli are finished playing all the way through. If all temporal stimuli have barrier set to false, response items will be displayed while the stimuli are presented. For more information on how barriers work and how to play with the timing of temporal stimuli and responses, check out our barrier property tutorial.

For more technical details about participant responses, check out the Responses section of the FindingFive grammar documentation.

If you have questions about this post or how to collect participant responses in FindingFive, please don’t hesitate to contact us at researcher.help@findingfive.com.