Crash Course Part 2: Running Your Study

FindingFive allows you to quickly and easily design experiments for deployment on the web. In this crash course, we will take a detailed tour through some of FindingFive’s features by designing a simple memory study: a modification of Tulving (1975). We highly recommend you follow through this example carefully before creating your own experiment. To see a working demo of the study described in this tutorial, log in to FindingFive and check out this study.

In this second half of our crash course, we’ll go through the steps involved in running your study on FindingFive. The first half of our crash course covered how to build your study from scratch…be sure to check out that post if you haven’t done so already. And don’t forget to check out the Study Specification Grammar Documentation and our Researcher Tutorials to learn about all of the other advanced features we aren’t covering in this crash course.

• Making your study ready to run
• Starting a session
• Managing active and scheduled sessions
• Looking at the data

---

Making your study ready to run

When you are working on the materials of a study, in the upper-right corner of the web page you’ll notice an orange button that says “preview study”. This button is how you will verify that your experiment runs as you expect it to.

Encountering syntax errors in the study specification

As you type in the code editor, some basic syntax errors, such as missing commas, mismatched parentheses, and unrecognized characters, will be checked in real time. If an error is found, the preview button’s font turns red: this means that there are syntactic errors in your trial templates and/or procedure. If this happens, you can hover your mouse over the preview button to get a message about the error, as shown below.

Common syntax errors of this type include:

• Missing commas between items in a list [] or a dictionary {}
• An extra comma at the end of a list [] or a dictionary {}
• Use of single quotation marks '' instead of double quotation marks ""
• Missing quotation marks "" where they are required
• Mismatched quotation marks "", list brackets [], or dictionary braces {} (e.g., quotes open without a corresponding closing quote)

These basic errors must be fixed before more complex errors, which often involve the logic of a study, are detected. These additional problems are investigated after you click on the “preview study” button, as they are checked by the more powerful study parser on our server. These more complex errors, if found, are reported to you in a language as plain as possible:

If no additional errors are found, you will be taken to preview the study directly.

Upload necessary resource files for stimuli

Our study parser automatically infers a stimulus’s filetype from your stimulus definitions (such as image, audio, or video stimuli). If a corresponding resource file is missing, you will receive a prompt when your mouse hovers over the stimulus (you may need to refresh your browser to see this prompt). To upload a single stimulus file, simply click on the stimulus and then the “upload file” button.

In many cases, a researcher may create different studies that are minor variations of each other. It would be a mind-numbing process to upload the same stimulus files over and over again. Fortunately, stimulus files that are uploaded to one study can be directly accessed by filename in another study within the same account. That is, two file-based stimuli sharing the same “content” property actually point to the same stimulus file uploaded to your account. This is because resource files (including stimulus files and consent forms) are uploaded to a single account-specific resource file folder shared across all of your studies. To see what stimuli are uploaded to your account, you can click on the Research menu at the top right of your screen, then click on Resource Files:

This will take you to a list of all of the resource files that have been uploaded to your FindingFive account, including consent forms, images, videos, and audio files. For example, if we look at the documents that we have saved in our account, we see two consent forms have been uploaded and are available to use across any of our studies: one consent form that is specifically for study tutorials that we create, and another generic consent form that covers most of our other adult studies.

Have lots of stimuli to upload? No problem! FindingFive allows for batch uploading of multiple stimulus files and/or consent forms. To do this, gather the desired files into a folder and compress the folder into a ZIP archive. Then, click the “Upload” button on your “Resource Files” page, and select the option to upload a ZIP archive. Please be patient and do not navigate away from the Resource Files screen while the archive is being uploaded.

Your study is now ready to recruit participants and run!

Starting a session

Once you’ve confirmed that your study is ready to run, it’s time to make it available to participants by creating a session. To do so, navigate back to your study’s page and click on the Sessions tab, located on the left side of the study page.

This will take to you to the “Sessions” page for your study. Here, you can see your active/scheduled or finished sessions (more on this later), or you can create a new session.

The step-by-step new session guide

Let’s create a new session of our study! To create a new session, click on the “new session” button in the upper-right corner of the “Sessions” page. This will open a pop-up box containing a step-by-step guide to setting up a new session. The first step is to choose the platform on which you wish to host your session.

FindingFive supports two kinds of session:

• FindingFive sessions are hosted entirely on the FindingFive platform
• MTurk sessions are sessions of FindingFive studies hosted on Amazon Mechanical Turk

For this tutorial, we will create a FindingFive session. Select the FindingFive icon and click “Next” to be taken to Step 2 of the new session guide.

Here you can provide a participant-friendly name and short task description for your experiment, as well as any additional notes that participants may need. You can also provide a session name for this session that only you and your collaborators will see. Fill in the information as you see fit, and click “Next” to advance to Step 3.

Here, you can specify the total number of participants you desire, restrictions based on past participation in sessions of this study or whether participants are researchers, how long you estimate the duration of the experiment to be for participants, and whether you want to force a timeout duration on the experiment. Fill in the information as you see fit, and click “Next” to advance to Step 4.

Here, you can specify additional restrictions on participation based on participation in sessions of other studies. This is defined using prerequisites and anti-prerequisites. Prerequisites are sessions that participants must have participated in in order to be allowed to participate in the current session. Anti-prerequisites are sessions that participants must not have participated in in order to be allowed to participate in the current session. To include prerequisites or anti-prerequisites, check the appropriate box and select a session from the available list. When you are satisfied, click “Next” to advance to Step 5.

Here, you can specify monetary compensation options for participants. This is informational to participants only. Once you have specified any compensation options you desire, click “Next” to advance to Step 6.

Here, you can choose a consent form from your “Resource Files” page or upload a new one. Every study session requires a consent form. Once you have selected a consent form for your study, click “Next” to advance to Step 7.

This step asks you whether you agree to share certain data with us at FindingFive. This data includes study design and content, meta data about participant behavior that does not include their actual responses, and actual participant response data. When you are satisfied, click “Next” to advance to the final step.

The final step is about scheduling your session. For FindingFive sessions, you may make the session available as soon as possible or you may schedule it for a future time. You can also specify how long the session is available (provided the session does not fill prior to that point).

Managing active and scheduled sessions

If you have ongoing or scheduled sessions, these will appear under the “Active and Scheduled” tab of the “Sessions” page, and will also be visible to participants who meet your specified eligibility requirements on the participant-view (visible by clicking the Participate button in the top navigation panel).

You can see basic information about the session, as well as its progress, from here. For a more detailed rundown of the session, to extend the duration of the session, or to terminate the session, click on “Manage”.

Ending a session

To end a session early, first click on “Manage” under the “Active and Scheduled” tab of the “Sessions” page. Click on the square icon to terminate the session. Terminating the session will prevent new participants from joining the study, but will not cause problems for participants who are currently taking the study. After terminating a session, you will be unable to reopen it, but you can always start a new session.

Looking at the data

After a session has been completed, it will appear under the “Finished” tab of the “Sessions” page. From here, you can download the data (a zipped folder of .csv files) by selecting the desired session(s) and clicking the Batch-download data button. Note that you can only download the data from a session once the session has ended (either by ending the session early or waiting until the requested number of participants have completed your study). More information on accessing and interpreting your data files can be found in our data file tutorial.

---

Congratulations on making it to the end of the crash course! We encourage you to check out the the working demo of the study described in this tutorial, read the Study Specification Grammar Documentation to learn about all of the advanced features we didn’t include in this crash course, and review our Researcher Tutorials for additional FindingFive tips and tricks.

Please do not hesitate to contact us at researcher.help@findingfive.com for any questions you may encounter while using FindingFive. We look forward to hearing from you!