Most of the studies run on the FindingFive platform are meant to be completed in a relatively short amount of time. In fact, an average study created on FindingFive contains 190 trials, which will take an average participant about 30 minutes to finish (assuming 10-15 seconds per trial). Our platform have been able to handle studies of such sizes pretty well.
Larger studies (400+ trials) are slow to load
However, if you have created a larger study consisting of more than 400 trials, you may have noticed that it takes more than a few seconds for the study to preview, and you may have heard complaints from Turkers who found the study slow to load. Fortunately, we have implemented a new feature that will significantly reduce the initial loading time of a large study.
What causes larger studies to be slower? Not the size of HTML itself.
In order to understand the problem and our solution, we need to first explain what exactly makes a large study slower to load than a smaller one. When researchers create a study on FindingFive, our server parses the grammar code and converts it into an HTML document. The HTML document is what participants will see – i.e., the study. The more trials does a study contain, the larger the HTML document will be, because it needs to store more information about what to show the participants.
You may be tempted to think: of course, a large HTML document is slower to load than a small one! That’s certainly true, but why would a study containing 100 trials take a split of a second to load, but a study containing 400 trials take nearly a minute? That makes no sense if the slowdown is purely due to the extra bulk in the size of the study.
Rendering is (nonlinearly) costly
In fact, the majority of the loading time for any study is spent not on sending data over the Internet, but on rendering the study in the participant’s browser. Think about all the stimuli you want presented on each trial, and the responses you want collected. The information on how to display those components is coded in the HTML document. The participant’s browser receives these instructions and figures out how to faithfully display those stimuli and responses trial by trial.
This is no small feat, especially given how FindingFive studies work. For a regular web page, there is usually a limited amount of information on the page, such as a single product page on Amazon. When you check out a different product, the browser navigates to a different page, fetching a new HTML document.
But for a FindingFive study, there’s only one HTML page throughout the course of a study. Trials progress as if they are separate pages, but it’s just a simulated effect that gives the study the same traditional feel of an in-lab study. At the same time, the cost was that all content of a study, including information about all the stimuli and responses used in the study, must be loaded and rendered upfront.
To make matters worse, the amount of time it takes to render HTML increases nonlinearly with the amount of content on a page. We won’t go into the details, but intuitively, try imagining going out to dinner with 2 friends vs 4 friends: it’s not just twice as time-consuming.
If by any miraculous chance, you have read the blog and followed the logic till this point, then the issue should be obvious to you: Figuring out how to render 500+ trials in relation to each other is a much more involved task for the browser than figuring out how to render 100 trials. Therefore, large studies seemingly take forever to load.
Our solution: loading and rendering one block at a time
FindingFive studies now, by default, load one block at a time. To researchers who preview the studies and participants alike, this new feature introduces no functional differences, except that your study should load faster since it only retrieves the first block of trials in the beginning. Once a participant reaches the end of a block, the next block of trials are automatically loaded and displayed.
Keep your blocks short / Upper limit
To fully take advantage of this feature, keep your blocks short. To avoid a seemingly stuck experience when the study is fetching new trials, we set an upper limit of 150 trials per block. That should be enough for most researchers. If you have a study design that must use a longer block, please contact us at researcher.help@findingfive.com.
You can turn it off too
You can also turn off block-by-block fetching on a block-by-block basis! 🙂 For example, if you want to load the trials of Block “A” and “B” together (as opposed to the default behavior of loading A first and then B), just set the “fetching_point” property of Block A to false:
<pre class="CodeMirror cm-s-default" data-setting="%1$s">"BlockA": {
"trial_templates": [...],
"fetching_point": false
}
</pre>
That is, setting the fetching point property to false loads the block that comes afterwards together with the current one.
Final word
How do you like this feature? We hope that you guys appreciate our continuing effort to make it easier for researchers to run studies. If there’s any questions, feel free to contact us!
Comments are closed.