Press "Enter" to skip to content

Crash Course Part 1: Building Your Study

FindingFive allows you to quickly and easily design experiments for deployment on the web. In this crash course, we will take a detailed tour through some of FindingFive’s features by designing a simple memory study: a modification of Tulving (1975). We highly recommend you follow through this example carefully before creating your own experiment. To see a working demo of the study described in this tutorial, log in to FindingFive and check out this study (only available to researchers).

In this first half of our crash course, we’ll go through the components of building a study from scratch using the FindingFive Study Specification Grammar. The second half of our crash course will cover how to deploy your study on FindingFive, collect data from participants, and review your data.

This crash course focuses on building a very simple experiment in order to jumpstart your FindingFive experience. But FindingFive can do so much more! Check out the Study Specification Grammar Documentation and our Researcher Tutorials to learn about all of the other features we aren’t covering in this crash course.


Getting started

Let’s begin by clicking “Research” in the top right corner of the screen and selecting “Studies” from the drop-down menu in order to see your list of current and past studies. Clicking the “Create a new study” button on the left side of the screen will take you to your new study’s profile:

There are a variety of options here for customizing your study

  • Name your study – you can set both a “Study name” (how your study will appear in the list of studies on your Researcher screen), and a “Display name,” which is the title of your study that participants will see.
  • Add a short description – this is a good place to explain to your participants what types of tasks they are expected to perform in your study.
  • Provide an estimate of the study duration – this is informational only.
  • Set a time limit – participants who fail to complete the study within the allowed time slot will be automatically disqualified. This feature is only relevant for studies run directly on FindingFive; for studies run on Mechanical Turk, set an assignment duration when launching a new session instead.
  • Compensation – Add the amount of monetary compensation you will provide once the participant completes the study. At this time, for FindingFive sessions, this is informational only! You’ll have to arrange payments to your participants. For Mechanical Turk sessions, this functions as intended.
  • Associate a consent form with your study – Upload a copy of your IRB-approved consent form, or select from consent forms you have already uploaded to your profile. Note that you cannot publish a study on FindingFive unless a consent form is uploaded!

After saving this basic information for your study, you can see how participants will view the study by clicking on “Management” in the left sidebar.

Building Your Study

Search, add, upload, and export stimulus and response definitions

The designing of your study takes place under the “Materials” tab. A FindingFive study is created by defining four components that should sound intuitive to anyone with some training in psychology: stimuli, responses, trial templates, and procedure.

These four components can be defined under this “Materials” tab. On the left-hand side, you can search, add, import, and export the definitions of stimuli and responses, which make up the basic building blocks of a study.

On the right-hand side, you can use the code editor to create trial templates, which combine stimuli and responses to form trials, and the procedure, which defines how trials are organized in a study. These are the two components that involve some coding (but rest assured that we will walk you through it!).

The coding screen for trial templates and procedure

Translating Tulving (1975) to the FindingFive Study Specification

Tulving (1975) in plain words

In our sample experiment, participants will be presented with word pairs, then asked to recognize and recall the word lists after a delay. The words and instructions are all stimuli which will be organized within trial_templates. These training-specific trial_templates will be organized into a training block in our procedure. There will also be a test block in our procedure. This block will contain responses, where the participant is asked if the presented pair of words is correct (a 2AFC recognition memory test), and other responses will require the participant to recall the target word and type it into a text box. These test trials will be organized into their own trial_templates.

Tulving (1975) in the FindingFive Study Specification

To build the study specification for our study, we must:

  • Define stimuli which will include our training and test instructions to the participant as well the word-pairs the participant must learn during our training phase and the test questions
  • Define responses for the test phase of the experiment, both 2AFC and free recall
  • Define trial_templates for each set of trials in our study (instruction trials, training trials, 2AFC test trials, and free recall test trials)
  • Define the procedure, which will consist of a training block then a testing block

Setting up the stimuli

We’ll build our experiment from the ground up, starting with the stimuli. Stimuli are items that are presented to the participant, including text instructions, audio clips, and images. For our experiment, we will need to define stimuli for our training and test instructions, each of our word-pairs and test questions, and a concluding statement to the participant at the end of the study.

Add the first two stimuli manually

To create our training instructions, let’s define a new stimulus called TrainingInst. This stimulus will only contain text, so we will set the type property to “text”. We will define the content property with our experiment instructions. To add this stimulus to our study, click on the + button under the Stimuli menu in the left sidebar. A new window will pop up, asking for the name of the stimulus, and the definition of the stimulus:

Type “TrainingInst” into the name field, and then paste the following code into the definition field, and click Save:

{
    "type": "text",
    "content": "In this first part of the experiment, you will see pairs of words presented on the screen. Your task is to <b>remember the capitalized words in each pair</b>. However, you should pay attention to both words and notice the relationship between them, since this relationship will help you remember the capitalized words." 
}

Notice that we can use HTML tags (like <b> for bold font) to alter the appearance of our text stimulus. As an exercise, add a second stimulus named “TestInst”, which has the following definition:

{
    "type": "text",
    "content": "In this portion of the study, you will be prompted with one of the two words in each word pair you learned. Your job is to indicate the correct match to the presented word. In some test trials, you will type the word into a text box. In others, you will indicate whether the presented match is the correct."
}

Congrats! You have successfully added the first two stimuli of your study. They should now be displayed in the Stimuli sidebar.

Add more stimuli via CSV batch uploading

Next, we’ll create stimulus definitions for each of our words to be presented in the training phase of the experiment. Doing so individually can be quite tedious. Fortunately, FindingFive supports batch uploading stimuli via a CSV file, which makes life much easier when we are dealing with potentially hundreds of stimuli.

To get started, hover your mouse over the batch upload button under Stimuli, and a brief explanation on how to format stimulus definitions in CSV will pop up:

For our current example, Tulving (1975), use your favorite spreadsheet software (Excel, Google Sheets, or other great programs) to create a CSV file that looks like this:

name type content alignment
ConCue1 text player center
ConTarget1 text BALL center
ConCue2 text smooth center
ConTarget2 text HARD center
ConCue3 text furniture center
ConTarget3 text CHAIR center
ConCue4 text fire center
ConTarget4 text COLD center
InconCue1 text fruit center
InconTarget1 text FLOWER center
InconCue2 text moth center
InconTarget2 text FOOD center
InconCue3 text cheese center
InconTarget3 text GREEN center
InconCue4 text cave center
InconTarget4 text WET center
ConTest1y text Was <b>ball</b> the word paired with <b>player</b>? center
ConTest1n text Was <b>tennis</b> the word paired with <b>player</b>? center
ConTest2y text Was <b>hard</b> the word paired with <b>smooth</b>? center
ConTest2n text Was <b>soft</b> the word paired with <b>smooth</b>? center
InconTest1y text Was <b>flower</b> the word paired with <b>fruit</b>? center
InconTest1n text Was <b>bloom</b> the word paired with <b>fruit</b>? center
InconTest2y text Was <b>food</b> the word paired with <b>moth</b>? center
InconTest2n text Was <b>eat</b> the word paired with <b>moth</b>? center
ConTest3 text What was the word paired with <b>furniture</b>? center
ConTest4 text What was the word paired with <b>fire</b>? center
InconTest3 text What was the word paired with <b>cheese</b>? center
InconTest4 text What was the word paired with <b>cave</b>? center
BreakInst text Please take a short break center

WARNING: Be sure to save your file as CSV (comma separated values), instead of a proprietary format like xls.

(For those of you who are hardcore programmers and want to create CSV files manually through a text editor, you are welcome to do so. Please be careful, however, about properly escaping commas and double quotes. FindingFive is capable of recognizing the standard CSV escaping methods, such as using double double quotes to escape double quotes.)

You’ll notice that the first 16 stimuli are the training stimuli, followed by the stimuli to display for the test trials. For the test trials, the word stimuli is somewhat a misnomer – these are used to display the questions participants will answer. The last stimulus will be used to prompt the participant to take a break before the test portion of the experiment.

Your stimuli section should look like this after the batch upload step.

Let’s upload this CSV and create those stimuli all at once! Click the batch upload button, and select the CSV file on your computer. For this particular example, these stimuli should be imported in just a second. Depending on the speed of your computer and the Internet connection, it might take a couple of minutes if you are uploading hundreds of stimuli at once. Please be patient and do not refresh the page unless you are absolutely sure something unexpected has happened. After the upload is complete, you should see the Stimuli sidebar populated with your stimuli!

Setting up the responses

Responses define the actions that participants can take in response to stimuli. They also automatically specify what types of participant data will be recorded. In our sample experiment, we will conduct both a recognition (2AFC) memory test and a recall (fill-in-the-blank) memory test. We will set up the actions participants can take for these test trials with the responses property.

First, we will set up the recognition test for two of our semantically congruous word pairs and two of our incongruous word pairs. Since we are asking participants to make a choice between two options (correct pairing, or incorrect pairing), the type of response we will collect is a “choice” response. We can define the possible choices for the participant (YES or NO) using the choicesproperty. Next, we will map these choices to keys on the participant’s keyboard using the key_mapping property and we will tell FindingFive the correct answer using the target property. Since the correct answer can either be YES or NO, we’ll make responses for each of these possibilities.

Adding responses to a study is very similar to adding a stimulus. Click the + button under Responses, and add the following two choice responses:

The first response is named “TestResponseTargetY”, with the definition:

{
    "type": "choice",
    "choices": ["YES", "NO"],
    "key_mapping": ["F", "J"],
    "target": "YES"
}

The second response is named “TestResponseTargetN”, with the definition:

{
    "type": "choice",
    "choices": ["YES", "NO"],
    "key_mapping": ["F", "J"],
    "target": "NO"
}

Next, we will set up our recall test. The type of response we will collect is a “text” response, so there’s no need to define choices or key_mapping, but we can set a character limit for participant’s responses using max_char. We will name the response “FreeRecall”, and use the following definition:

{
    "type": "text",
    "max_char": 10
}

We are all set with the responses. At this point, your left sidebar should look likes this:

Setting up the trial templates

Now that we have the stimuli and responses defined, we can set up the trial_templates which define the organizational logic of our trials. We will define trial_templates for each set of trials in our study (instruction trials, randomized training trials, randomized 2AFC test trials, and randomized free recall test trials).

Instruction trials

First, let’s set up our instruction trials. We’ll need an instruction trial (“T1”) at the very beginning of the experiment to explain the training trials to our participants. Defining the type as “instruction” allows us to simply present text to the participant for a time in seconds specified by duration. Participants must wait until the specified duration has elapsed before continuing the experiment (by clicking a button labeled “continue” that appears after the specified duration).

"T1": {
    "type": "instruction",
    "stimuli": ["TrainingInst"],
    "duration": 10
}, // you should keep the comma here if more trial template definitions follow this one

We’ll define similar trial types for presenting the break instructions, test instructions, and a final goodbye message to the participants.

"PreTestBreak": {
    "type": "instruction",
    "stimuli": ["BreakInst"],
    "duration": 30
}, // you should keep the comma here if more trial template definitions follow this one

"T2": {
    "type": "instruction",
    "stimuli": ["TestInst"],
    "duration": 10
}, // you should keep the comma here if more trial template definitions follow this one

You can copy and paste the code above to the code editor – please make sure that “Trial Templates” is ticked before doing so:

Training Trials

For the training trials, we create a template named “TrainingTrials”. Since we need to present 8 word-pair stimuli in total, the trial template is set up to generate 8 unique trials, each of which presents two stimuli. Note: to include two stimuli on each trial we create a list for each of our trials within the larger stimuli list.

"TrainingTrials": {
   "type": "basic",
   "stimuli": [["ConCue1","ConTarget1"],["ConTarget2", "ConCue2"],["ConCue3", "ConTarget3"], ["ConTarget4", "ConCue4"], ["InconCue1","InconTarget1"],["InconTarget2", "InconCue2"],["InconCue3","InconTarget3"], ["InconTarget4", "InconCue4"]],
   "stimulus_pattern": {"order": "random"},
   "duration": 3,
   "delay":1,
   "responses": []
}, // you should keep the comma here if more trial template definitions follow this one

Here, a simple per-participant randomization of the 8 trials is achieved by specifying the stimulus_pattern property. As in Tulving (1975), these stimuli will remain on the screen for 3 seconds each, achieved by the duration property. The delay property handles time between trials – here we set it to 1 to indicate a 1-second inter-trial-interval. We won’t collect any data from the participant in these training trials, so our responses property is set to an empty list [].

Test trials

Last, we create the trial templates for test trials. First, let’s handle the 2AFC test trials by making a trial template for trials where the correct response is YES and a second trial template for trials where the correct response is NO. FindingFive has a built-in trial template for alternative forced choice tasks (“AFC”) that performs additional sanity checks for this type of task, so we’ll use that here. Like the training trials, the test will present our test trials in a random order.

"TestTrials2AltY": {
    "type": "AFC",
    "stimuli": ["ConTest1y", "ConTest2y", "InconTest1y","InconTest2y"],
    "stimulus_pattern": {"order": "random"},
    "responses": ["TestResponseTargetY"]
}, // you should keep the comma here if more trial template definitions follow this one

"TestTrials2AltN": {
    "type": "AFC",
    "stimuli": ["ConTest1n", "ConTest2n", "InconTest1n","InconTest2n"],
    "stimulus_pattern": {"order": "random"},
    "responses": ["TestResponseTargetN"]
}, // you should keep the comma here if more trial template definitions follow this one

With our 2AFC test trials set, we just need to define the trial_template for the free recall test trials.

"TestTrialsFree": {
    "type": "basic",
    "stimuli": ["ConTest3", "ConTest4", "InconTest3","InconTest4"],
    "stimulus_pattern": {"order": "random"},
    "responses": ["FreeRecall"]
} // this will be last trial template definition, so there is no comma

At this point, your trial templates code should look like this:

Setting up the procedure

Now it’s time to put everything together into our procedure. We define the procedure to have two blocks: a training block and a testing block. First things first, let’s tick the procedure option to switch the code editor to work on the procedure:

In our training block, we will start by presenting our training instructions, housed in template T1. We’ll end the training block with our break instructions. These introductory and closing trials will be defined using the cover_trials and end_trials properties.

Then we need to tell the study specification which of the trial_templates houses the main trials for this block. In our study, the training trials live in the TrainingTrials template. Since we’re only presenting one main trial template in this block, we don’t need to randomize it, and we set the repeat property to 1. Our test block will look similar to our training block, except we want to randomize the 3 different trial_templates we specified for test trials. After defining both blocks, we set the order in which these blocks should appear using the block_sequence property: training first, then testing.

{
    "type": "blocking",
    "blocks": {
      "TrainingBlock": {
        "trial_templates": ["TrainingTrials"],
        "pattern": {"order": "fixed", "repeat": 1},
        "cover_trials": ["T1"],
        "end_trials": ["PreTestBreak"]
      },
      "TestingBlock": {
        "trial_templates": ["TestTrials2AltY", "TestTrials2AltN", "TestTrialsFree"],
        "pattern": {"order": "random", "repeat": 1},
        "cover_trials": ["T2"]
      }
    },
    "block_sequence": ["TrainingBlock","TestingBlock"]
}

You are now done building your study, and you’re ready to take plunge and run it! Check out part two of the crash course to learn more about deploying your study.

Don’t forget that you can learn more about the FindingFive platform by looking at the Study Specification Grammar Documentation. You can also see other examples of how to customize your studies by looking at the various tutorials we post on our blog!

Have questions about this tutorial? You can always reach out to our researcher help team at researcher.help@findingfive.com.