Earlham IRB Online Subject Pool Ethical Guidelines

This policy only applies to paid online participants. Researchers may choose to recruit unpaid volunteers via online mechanisms (e.g., social media, email, etc.), but such plans must be clearly articulated in the IRB Review application.

These guidelines apply in addition to all standard IRB policies and procedures.

Any researcher proposing to use an online paid crowdsourcing site/platform to recruit research subjects, regardless of the type of review submitted (i.e., Exempt, Expedited, or Full), must agree to abide by the following guidelines. It is important to remember that online workers are just that – workers. If you choose to use online sources for research participants, then you must treat them fairly as temporary contractors. Fair treatment aligns with Earlham’s Principles & Practices of Respect for Persons, Integrity, and Peace & Justice.

A. In the IRB Application

When researchers using online crowdsourcing platforms apply for approval from the Earlham IRB, they must include the following sentence in their application:

I have read the full guidelines on the use of online paid crowdsourcing to gather research participants from the Earlham IRB. I am aware of the responsibilities involved in gathering data via online sources and agree to follow these guidelines.

B. In the Online Request (if applicable)

Many online crowdsourcing sites ask researchers/requesters to provide some information about the task (e.g., survey, study, HIT, etc.) that online workers will complete. In this information, make sure to clearly identify yourself (full name and email), provide the email for the Earlham IRB ([email protected]), and provide clear information about payment (and rejection of payment, if applicable).

C. In the Informed Consent Document

In addition to other required information in the Informed Consent document (see our IRB Guidelines for clarification), you must include a statement that you are abiding by the guidelines for using online research participants as outlined by the Earlham IRB. An example statement is below, though you may adjust the wording as desired.

This research followed the guidelines for using online participants as outlined by the Earlham College IRB (see http://earlham.edu/irb). If you feel that these guidelines have not been followed, please contact me directly ([email protected]) and/or contact the Earlham College IRB ([email protected]).

1. 1. Fair Pay:  To pay fairly, you must ensure that your time estimate for involvement is accurate. Requesters often underestimate how long it takes to complete a survey/task. Several students (who are not the researchers) should complete the survey, carefully reading all pages before generating a time estimate.
      • The Earlham IRB requires requesters (researchers) to pay, at a minimum, the equivalent to minimum wage, which can be prorated for For example, if the federal minimum wage is $7.25 and your survey takes 10 minutes to complete, then you should pay $1.21 for completing the survey. The federal minimum wage can be accessed here: https://www.dol.gov/general/topic/wages/minimumwage.
      • The Earlham IRB encourages requesters to pay a fair or living wage, which may be higher than the federal minimum We calculate a fair wage based on the federal poverty level for a family of four. Someone working 40 hours a week, 50 weeks a year, works 2000 hours per year. Divide the federal poverty level for a family of four ($25,750 in 2019) by 2000 to determine the wage ($12.88 in 2019). Federal poverty levels can be accessed here: https://aspe.hhs.gov/poverty-guidelines.
        • There is a movement for a $15 minimum wage; requesters may want to consider paying this rate or higher
   2. Make Requirements (and Rejection Criteria): Clearly identify the requirements for participation at every point possible (IRB Review Application, online request, Informed Consent, etc.).
      1. If you have a special population and are concerned about people being dishonest about their inclusion for your research, there are several options.
         1. Use an online platform that does demographic screening for you or allows you to select participants based on demographic information supplied by online participants.
         2. For online sources that do not provide demographic screening options, one possibility is to do a short initial demographic survey to identify participants who fit your requirement, then create a new survey/research process and open it only to those individuals (e.g., using Worker Qualifications in MTurk). Workers must be compensated fairly for the screening survey, even if they do not meet qualifications for the subsequent research.
         3. You can consult with the IRB convener to discuss/search for additional possibilities.
      2. If using a platform that allows you to reject the submissions of research participants – and thus not pay them for their time, clearly state the reasons that a worker’s submission might be rejected at every point possible (IRB Review Application, online request, Informed Consent, ). These reasons must be provided to the IRB with justification prior to study approval. Workers should not be rejected for other reasons.
         1. The Earlham IRB requires requesters to know how to reverse a rejection
            before they issue a rejection.
         2. Keep in mind that online participants are harmed when their work is rejected. It may prevent them from accessing other research opportunities (which often have requirements about how many rejections workers can receive) or may even cause them to lose their accounts, which is a source of livelihood for some.
         3. Remember that you can exclude responses from your analyses. This is distinct from rejecting work from (and not paying) a participant. If you feel a response is not valid for some reason that you had not anticipated, you might exclude their data from analysis, but you need to provide payment.
   3. Approve Work: Remember that online participants are temporary workers and the ethical practice is to pay them as soon as possible.
      1. If you have the opportunity to set the timeframe for automatic approval of work, the Earlham IRB requires you to set your auto-approve to no more than three days.
      2. The Earlham IRB encourages you to approve responses as quickly as you are able, within 24 hours, if possible.
   4. Provide a Valid Completion: In general, completion codes are a way to ensure that workers have made it all the way through (completed) a study. Not all online platforms will ask for them, but some (e.g., MTurk) will require it.
      1. If required for workers, be sure to provide a code upon completion of the study, generally on the last page. The code should be large and clear and not buried in other text. You may want to highlight it or use a bold font color. Best practices are to use a unique code for each participant; many online survey systems (e.g., Qualtrics) allow this option.
      2. On the same page that you provide a code, you must also provide the Earlham IRB email address ([email protected]). This means the IRB email address should appear at least twice: in the consent form and when the code is provided.

If the IRB receives complaint(s) from online workers/participants or from employees of online platforms, there is an ethical responsibility to follow-up about the complaints. The IRB will typically3 use the following process.

1. After the initial complaint(s):
   1. The IRB will discuss the complaint(s) with the researcher.
   2. Based on the substance of the complaint(s), the IRB will require the researcher to show how those issue(s) will be addressed in the future (e.g., requiring prior approval of wording or options for participant recruitment in a platform, permitting rejection of work only after receiving IRB approval first, etc.).
2. After addressing the issue with the researcher, if similar complaint(s) are received (that do not pertain to the original offense):
   1. The IRB may choose to submit an allegation of research misconduct.
   2. The IRB may decline to approve any future research studies using online platforms.

Please note Earlham’s Research Misconduct Policies and Procedures. Complaints that allege research misconduct, that are directly reported to the College via the research misconduct procedures, or repeated infractions by the researcher after address by the IRB, may intersect with the aforementioned policy and require deviation from the above outlined process. The IRB will cooperate fully in any required Research Misconduct Procedures

November 10, 2023 – Policy reformatted to add section and part numbering.