NC State’s IRB#

• Protocol Number: XX
• Project Title: XX

Cover Page#

1. Source of funding (provide name of funder not account number):

   Not the name of the PI, commonly “National Science Foundation”, but double check if in doubt

2. NCSU Faculty point of contact for this protocol:NB: only this person has authority to submit the protocol

   Dominik Wermke: Computer Science

3. Does any investigator associated with this project have a significant financial interest in, or other conflict of interest involving, the sponsor of this project? (Answer No if this project is not sponsored)

   Usually no

Preliminary Review Determination#

1. Category:
   • Exempt d.2
2. This research qualifies for Exemption. Review NC State’s Exemption Research SOP for studies that may qualify. If you want to apply for an Exemption, download the Exemption Request Form and complete it. To the eIRB, upload the completed Exemption Request Form, all instruments, and if applicable a Data Access and Security Plan and the edited Consent/Opt-Out forms modified to fit the study design. Only complete the “Title” and “Description” tabs in the eIRB, upload the aforementioned documentation, and submit the eIRB application. Do not complete any other tabs within the eIRB system.

   Don’t use the exempt form, NSF requires us to go through normal admin review for anything with human participants.

   • [This rarely applies because of participant study and NSF requirements]
   • [Could apply for non-paper research like theses or independent studies]

Participants#

1. In lay language, briefly describe the purpose of the proposed research and why it is important. Provide a brief synopsis of the study including who is targeted to participate and the data collection methods employed (limit text to 1500 characters)

   Don’t just copy paste your PPQ abstract Specifically address their points:

   • Who is targeted to participate (population, sampling, inclusion and exclusion criteria)
   • What are the data collection methods (Method and data collected)

2. Does any member of the project team who is responsible for the design, recruitment, consent, implementation of intervention, interaction with participants, or those handling identifiable private information under this IRB protocol - or any members of their immediate family (defined as spouse, dependent children - have any Significant Financial Interest or other types of conflict of interest (as described in SOP 14.3.a) related to the protocol?

   Usually no for our type of research

3. If the answer is “yes,” please provide the name of the investigator(s) with the potential or actual conflict and confirm that the relationship has been fully disclosed in the investigators most recent COI disclosure filed with NC State or disclosed through the collaborative research process. If there is a COI management plan in place with NC State University, please upload it with this application to ensure the IRB protocol meets the expectations of the COI plan and the COI is properly considered in the IRB review process. If you are uncertain how to respond or have questions, please contact COI-NOI-Compliance@ncsu.edu.
4. Is this research being conducted by a student?
5. Is this research for a thesis/disseration/capstone?
6. Is this research for a dissertatiion?
7. Is this independent research?
8. Is this research for a course?
9. Do you currently intend to use the data for any purpose beyond the fulfillment of the class assignment?
   • If so, please explain
10. If you anticipate additional NCSU-affiliated investigators (other than those listed on the Title tab) may be involved in this research, list them here indicating their name and department.
11. Will the investigators be collaborating with researchers at any institutions or organizations outside of NC State?
12. List collaborating institutions and describe the nature of the collaboration. If researchers from both institutions are doing any of the following activities: recruitment, consent process, data collection or handling of identifiable information/specimens a reliance agreement may be appropriate. For more information, please contact irb-coordinator-admin@ncsu.edu
13. What is NCSU’s role in this research?
14. Describe funding flow, if any (e.g. subcontractors)
15. Is this international research?
    • [Might include international participants]
16. Identify the countries involved in this research
17. An IRB equivalent review for local and cultural context may be necessary for this study. Can you recommend consultants with cultural expertise who may be willing to provide this review? Consultants may not be a part of the research team or have a stake in the research project. Provide email contact information for consultant(s). A local context review may lengthen the time it takes for your approval.
18. Adults 18 - 64 in the general population?
19. NCSU students, faculty or staff?
20. Adults age 65 and older?
21. Minors (under age 18–be sure to include provision for parental consent and/or child assent). If minors are included in your research, please read through the NC State University Regulation for your additional responsibilities. Following this regulation is a requirement of your affiliation with NC State.?
22. Could any of the children be “Wards of the State” (a child whose welfare is the responsibility of the state or other agency, institution, or entity)?
    • [Please explain]
23. Does this study involve people who are also incarcerated, involuntarily detained or committed, or are in a program or hospital as an alternative form of sentencing?
24. Pregnant women?
25. Are pregnant women the primary population or focus for this research?
    • [Provide rationale for why they are the focus population and describe the risks associated with their involvment as participants]
26. Fetuses?
27. Students?
28. Does the research involve normal educational practices?
29. Is the research being conducted in an accepted educational setting?
30. Are participants in a class taught by the principal investigator?
31. Are the research activities part of the required course requirements?
32. Will course credit be offered to participants?
33. Amount of credit?
34. If class credit will be given, list the amount and alternative ways to earn the same amount of credit.Note: the time it takes to gain the same amount of credit by the alternate means should be commensurate with the study task(s)
35. How will permission to conduct research be obtained from the school or district? IRB approval is not permission to conduct the research. You need to access a gatekeeper. If you are implementing a survey with NC State populations, please make sure you follow the NC State survey regulation.
36. Will you utilize private academic records?
    • [Explain the procedures and document permission for accessing these records.]
37. Employees?
    • [Describe where (in the workplace, out of the workplace) activities will be conducted.]
    • From whom and how will permission to conduct research on the employees be obtained?
    • How will potential participants be approached and informed about the research so as to reduce any perceived coercion to participate?

• Is the employer involved in the research activities in any way?
  • Please explain:
• Will the employer receive any results from the research activities (i.e. reports, recommendations, etc.)?
  • Please explain. How will employee identities be protected in reports provided to employers?
• Impaired decision making capacity/Legally incompetent?
  • How will competency be assessed and from whom will you obtain consent?
• Mental/emotional/developmental/psychiatric challenges?
  • Identify the challenge and explain the unique risks for this population.
  • Describe any special provisions necessary for consent and other study activities (e.g., legal guardian for those unable to consent).
• People with physical challenges?
• Identify the challenge and explain the unique risks for this population.
  • Describe any special provisions necessary for working with this population (e.g., witnesses for the visually impaired).
• Economically or educationally disadvantaged?
• Racial, ethnic, religious and/or other minorities?
• Non-English speakers?
  • Describe the procedures used to overcome any language barrier.
  • Will a translator be used?
    • Provide information about the translator (who they are, relation to the community, why you have selected them for use, confidentiality measures being utilized).
• Explain the necessity for the use of the vulnerable populations listed.
• State how, where, when, and by whom consent will be obtained from each participant group. Identify the type of consent (e.g., written, verbal, electronic, etc.). Label and submit all consent forms. Adult Non-Exempt Consent Form Template Exemption Consent Form Templates
• If any participants are minors, describe the process for obtaining parental consent and minor’s assent (minor’s agreement to participate).
  • [Parent/Guardian Permission Form]
  • [Minor Assent Forms]
• Are you applying for a waiver of the requirement for consent (no consent information of any kind provided to participants) for any participant group(s) in your study? For each participant group that you are requesting a waiver of consent for, please state what method this waiver is needed for, why it is needed and For each participant group that you are requesting a waiver of consent for, please state what method this waiver is needed for, why it is needed and address each of the above 5 criteria to justify why your study qualifies for a waiver of consent. Are you applying for an alteration (exclusion of one or more of the specific required elements) of consent for any participant group(s) in your study? No Identify which required elements of consent you are altering, describe the participant group(s) for which this waiver will apply, and justify why this waiver is needed. Are you applying for a waiver of signed consent (consent information is provided, but participant signatures are not collected)? A waiver of signed consent may be granted only if: The research involves no more than minimal riskThe research involves no procedures for which consent is normally required outside of the research context. No Would a signed consent document be the only document or record linking the participant to the research? No Is there any deception of the human subjects involved in this study? No Describe why deception is necessary and describe the debriefing procedures.Does the deception require a waiver or alteration of informed consent information?Describe debriefing and/or disclosure procedures and submit materials for review.Are participants given the option to destroy their data if they do not want to be a part the study after disclosure? For each participant group please indicate how many individuals from that group will be involved in the research. Estimates or ranges of the numbers of participants are acceptable. Please be aware that participant numbers may affect study risk. If your participation totals differ by 10% from what was originally approved, notify the IRB. The participant group for our study are developers who have experience with Software Composition Analysis (SCA) tools. The developers could have gained their experience from working on personal projects or from projects in their organization. Our participant pools include developers both from an open-source or industry background. We plan to interview participants until no new major themes emerge, we expect around 20 participants for the study based on past experiences. How will potential participants be be found and selected for inclusion in the study? Our inclusion criteria is previous experience with SCA tools, we are interested in a wide range of experiences from novice to expert. For each participant group, how will potential participants be approached about the research and invited to participate? Please upload necessary scripts, templates, talking points, flyers, blurbs, and announcements. We plan on gathering data from online code repositories to find users of SCA tools and invite them to participate through emails. We also plan on putting out ads through freelance platforms and online forums to reach more participants.

https://s3c2.org/studies/sca-usability is our recruitment page and includes the content we will use for recruiting participants. Describe any inclusion and exclusion criteria for your participants and describe why those criteria are necessary (If your study concentrates on a particular population, you do not need to repeat your description of that population here.) Inclusion and exclusion criteria should be reflected in all of your recruitment materials and consent forms. The inclusion criteria for participants is they have experience using SCA tools and are at least 18 years of age. Is there any relationship between researcher and participants - such as teacher/student; employer/employee? No What is the justification for using this participant group instead of an unrelated participant group? Please outline the steps taken to migitate risks to participants from the pre-existing relationship, including power dynamics of this relationship and/or perceived coercion.

Describe any risks associated with conducting your research with a related participant group. Describe how this relationship will be managed to reduce risk during the research. How will risks to confidentiality be managed? Address any concerns regarding data quality (e.g. non-candid responses) that could result from this relationship. In the following questions describe in lay terms all study procedures that will be experienced by each group of participants in this study.For each group of participants in your study, provide a step-by-step description of what they will experience from beginning to end of the study activities. Should you prefer, you can upload a detailed study procedure packet and refer us to that document in this text box. If you choose to upload a procedures packet, do not discuss procedures in the below text box. The study procedure is the same for all groups of participants (minor exception for the initial recruitment step):

1. Approach potential participant either via email or on freelancer recruitment platform with study information
2. If they are interested in being interviewed, we provide the consent form and schedule an interview time slot that works for both parties.
3. If they agree with the consent form, we go ahead with the interview. During the interview, we will ask participants questions about their usage, past challenges, and wished improvements for SCA tools.
4. After the interview, we will pay them the agreed amount.
5. Before publishing our research, we plan on sending our participants a pre-print with their de-identified quotes so that they can veto any quotes or clear up misunderstandings on our side. Are you requesting the use of secondary information to be used as data for this research project? The secondary information can either currently exist or be generated in the future. Discuss the following: permission to access the information (direct permission from the participant or records release), how researchers will access, transfer, store, and destroy the data. Discuss the identifiable/re-identifiable nature of the data through either direct IDs, indirect IDs, or triangulation of datasets, data points, researcher access/expertise, or analysis. . List all data categories to be requested (ex: age, race, student ID, GPA, ACT, Medical ID, diagnosis). Discuss if the data requires a Data Use Agreement. Discuss if the data are subject to FERPA, HIPAA, or the GDPR. no Social/Reputational? No Psychological/Emotional? No Financial/Employability? No Legal? No Physical? No Academic (affect grades, graduation)? No Employment (affect job)? No Financial (affect financial welfare)? No Medical (harm to treatment)? No Insurability (harm to eligibility)?

No Legal (reveals unlawful behavior)? No Private behavior (harm to relationships/reputation)? No Religious Issues/Beliefs? No Describe the nature and degree of risk that this study poses. Describe the steps taken to minimize these risks. You CANNOT leave this blank, say ‘N/A’, none’ or ’no risks’. You can say “There is minimal risk associated with this research.” For each ‘Yes’ selected above, describe the probability of the risk occuring and the magnitude of harm should the risk occur. Discuss how you are mitigating those risks through participant selection, study design, and data security. There is minimal risk associated with this research. If you are accessing private records, describe how you are gaining access to these records, what information you need from the records, and how you will receive/record data. Private records may include: educational, medical, financial, employment. Some of these private records may be subject to laws such as FERPA and HIPAA. Your content here should match what you’ve discussed on the procedures tab. Not accessing private records Are you asking participants to disclose information about other individuals (e.g., friends, family, co-workers, etc.)? No You have indicated that you will ask participants to disclose information about other individuals (see Populations tab). Describe the data you will collect and discuss how you will protect confidentiality and the privacy of these third-party individuals. If you are collecting information that participants might consider personal or sensitive or that if revealed might cause embarrassment, harm to reputation or could reasonably place the subjects at risk of criminal or civil liability, what measures will you take to protect participants from those risks? We do not plan on collecting personal or sensitive information. We will remove any accidentally provided sensitive information from the recordings. If any of the study procedures could be considered risky in and of themselves (e.g. study procedures involving upsetting questions, stressful situations, physical risks, etc.) what measures will you take to protect participants from those risks? If the participant feels at risk or stress during the study, they can skip the question or stop the interview at any time. If they would like to withdraw from the study after the interview, they can contact us and we will remove their data from the study. Describe the anticipated direct benefits to be gained by each group of participants in this study (compensation is not a direct benefit). no If no direct benefit is expected for participants describe any indirect benefits that may be expected, such as to the scientific community or to society.

• Contribution their insights for future improvements of SCA tools
• Deeper understanding about their own experiences and considerations using SCA tools
• Contributing to the scientific process Will you be receiving already existing data without identifiers for this study? No Will you be receiving already existing data which includes identifiers for this study? No Describe how the benefits balance out the risks of this study. Will data be collected in a way that would not allow you to link any identifying information to a participant? No Will any identifying information be recorded with the data (ex: name, phone number, IDs, e-mails, etc.)? Yes Will you use a master list, crosswalk, or other means of linking a participant’s identity to the data? Yes Will it be possible to identify a participant indirectly from the data collected (i.e. indirect identification from demographic information)? No Audio recordings? Yes

Video recordings? Yes Images? Yes Digital/electronic files? Yes Paper documents (including notes and journals)? No Physiological Responses? No Online survey? No Restricted Access (who, what, when, where)? Yes Password Protection (files, folders, drives, workstations)? Yes Suggestion of anonymous browsing? No Locks (office, desks, cabinets, briefcases)? Yes VPN (transfer, upload, download, access)? Yes Encryption (files, folders, drives)? Yes Describe all participant identifiers that will be collected from each data collection method (surveys, interviews, focus groups, existing data, background data collected via host site or software). Discuss why it is necessary to record identifiers at all and describe the deidentifying process

• For consent and payment: name, email address. The name and email address allows us to contact participants for compensation, respond if they have any questions with our study, and for registering their consent agreement.
• From interviews: No direct identifiers, for the transcripts we plan on replacing any direct identifiers with pseudonymous Participant IDs, redact any PII, and cluster quasi-identfiers to avoid indirect identification (e.g., refer to a tool a participant is working on only as “messenger application”). If recording identifiable information about participants, discuss any links between the data and the participants and why you need to retain them. Discuss destruction of links or removal of identifers. The collected identifiers (name, email) will be linked through a mapping file to the pseudonymous Participant IDs from the transcripts for payment and agreed consent form. The mapping file will be stored in an encrypted, secured location and separatly from the transcripts. The mapping can be destroyed after the end of this research. Discuss if you’ll be working with your departmental IT to create a data management plan and if you’re using NC State managed devices, NC State Google Drive or other NC State non-networked device. If using a personal device, discuss data protection. All data will be stored on NC State managed devices / Google Drive and will be password protected. Describe any ways that participants themselves or third parties discussed by participants could be identified indirectly from the data collected, and describe measures taken to protect identities. (Data can be reidentified by researcher access, technology employed, researcher expertise, and triangulation of data or other information. Discuss the probability of reidentification and the magnitude of harm to participants should the data be reidentified. Discuss the probability of reidentification occuring and the magnitude of harm should it occur). We apply extensive measures to the transcripts and reports to avoid indirectly identifiable data, including replacing any direct identifiers with pseudonymous Participant IDs, redacting any PII, and clustering quasi-identfiers to general categories. In the unlikely case that participants are indirectly identified, their past experience and considerations around SCA tools will be leaked. This should include a negible risk and no direct harm to the participants or of developers of the tool, as these are expected experiences from users of such tools and some of this information was likely already public and used in our recruitment selection.

We will make sure study data is indirectly identifiable and published reports do not contain any identifiers. For all recordings of any type:Describe the type of recording(s) to be made Describe the safe storage of recordings Who will have access to the recordings? Will recordings be used in publications or data reporting? Will images be altered to de-identify?Will recordings be transcribed and by whom? We plan on collection audio recordings during the interviews using standard tools like Zoom’s (local) recording or other recording tools like OBS. The recordings will be stored on NC State managed devices. We plan on transcribing the audio recording using locally hosted software tools (on NC State managed devices). Transcriptions will by manually verified, corrected, and de-identified by researchers involved with the study. After the audio recording have been transcribed and verified for correctness, we will delete the audio recordings. We will use pseudonymous Participant IDs to replace directly identifiable identifiers, redact PIIs, and cluster quasi-identifiers on the transcripts. The recordings and transcriptions will only be available to researchers involved in the study Describe how data will be reported (aggregate, individual responses, use of direct quotes) and describe how identities will be protected in study reports. Reporting data may sometimes reidentify your participants. If needed, you can adjust how you report your data to protect the identities of your participants. Discuss. We will ensure the published report does not allow for direct or indirect identification of the participants:

• Reported data will only include pseudonymous Participant IDs (P01, P02, …) for refering to participant.
• Any further PII in quotes will be redacted (“my coworker Dave” -> “[coworker]”)
• Any reported quasi-identifier will be clustered to hinder indirect identification (“I work at ACME messenger” -> “messenger”)
• We plan on providing out participants with a pre-print of the publications so that they can voice any privacy concerns before we publish the research study. Will anyone besides the PI or the research team have access to the data (including completed surveys) from the moment they are collected until they are destroyed? This includes sharing data with sponsors, journals, or using the data for future research endeavors. If you are sharing the data, this should be in your consent form. No Describe any compensation that participants will be eligible to receive, including what the compensation is, any eligibility requirements for that compensation, and how that compensation will be delivered. Examples of compensation include: monetary compensation, research credits, raffle/drawing, novel items. Make sure to check with your department regarding issues of tracking payments as your department accounting office may have requirements that affect your human subjects privacy (such as the mandatory tracking of anyone who receives compensation). This tracking may influence the confidentiality/anonymity of your research and must be addressed in this application. For participants that completed the interview, we will compensate them $60. The participants can choose to receive the compensation or choose to donate and support a selected open source software project. Explain compensation provisions if the participant withdraws prior to completion of the study. We plan on compensating valid participants for participating in the interview. Withdrawing prior to completion of the study might lead to a pro-rate compensation based on the scheduled interview