We—the members of the American Association for Public Opinion Research (AAPOR) and its affiliated chapters—subscribe to the principles expressed in this document, the AAPOR Code of Professional Ethics and Practices (“the Code”). Our goals are to support sound and ethical practice in the conduct of public opinion and survey research and promote the informed and appropriate use of research results.
The Code is based in fundamental ethical principles that apply to the conduct of research regardless of an individual’s membership in AAPOR or any other organization. Adherence to the principles and actions set out in the Code is expected of all public opinion and survey researchers.
As AAPOR members, we pledge to maintain the highest standards of scientific competence, integrity, accountability, and transparency in
designing, conducting, analyzing, and reporting our work, and in our interactions with participants, clients, and the users of our research.
We pledge to act in accordance with principles of basic human rights in research. We further pledge to reject all tasks or assignments that would require activities inconsistent with the principles of this Code.
The Code sets the standard for the ethical conduct of public opinion and survey research at the time of publication. Recommendations on best practices for research design, conduct, analysis, and reporting are beyond the scope of the Code but may be published separately by AAPOR Executive Council.
Definitions of Terms Used in the Code
1. “Public opinion and survey research” refers to the systematic collection and analysis of information from or about individuals, groups, or organizations concerning their behaviors, cognitions, attitudes or other characteristics. It encompasses both quantitative and qualitative research methods, traditional or emerging.
2. “Participants” refers to individuals whose behaviors, cognitions, attitudes, or other characteristics are measured and analyzed. Participants can include individuals representing groups or organizations, and individuals such as minors or those unable to consent directly, for whom a parent, legal guardian, or other proxy makes participation decisions or provide information.
a. “Active participants” have agreed to cooperate with the current research. They are typically survey respondents; individuals recruited for social science tests or experiments; participants in group discussions, in-depth interviews or other qualitative data collection; or individuals otherwise asked to provide information about themselves or those they represent.
b. “Passive participants” typically do not directly engage with the researcher and may not be aware of the current research. This includes those whose behavior, social media use, or other public activities are observed, those for whom information is obtained through analysis of publicly available information, or those whose existing research or administrative records are used under previously established provisions that allow for their use in the current research without need for additional consent.
3. “Personally identifiable information” refers to (i) measurements, records, or other data that can be used alone or in combination to distinguish or trace an individual’s identity and (ii) any other information that is linkable to an individual (e.g., employment information, medical history,
I. Principles of Professional Responsibility in Our Dealings with People
A. Research with Participants
2. We will not misrepresent our research or conduct other activities (such as sales, fundraising, or political campaigning) under the guise of conducting research.
3. We recognize that participation in our research is voluntary except where specified by regulation or law. Participants may freely decide, without coercion, whether to participate in the research, and whether to answer any presented question or item.
4. We will make no false or misleading claims as to a study’s sponsorship or purpose and will provide truthful answers to participants’ questions about the research. If disclosure of certain information about the research could endanger or cause harm to persons, or bias responses, or does not serve research objectives, it is sufficient to indicate, in response to participants’ questions about the research, that some information cannot be revealed.
5. We recognize the critical importance of protecting the rights of minors and other vulnerable individuals when obtaining participation decisions and conducting our research.
6. We will act in accordance with laws, regulations, and data owners’ (providers of research or administrative records previously collected for other purposes) rules governing the collection, use, and disclosure of information obtained from or about individuals, groups, or organizations.
B. Research Involving Personally Identifiable Information
2. We recognize the importance of preventing unintended disclosure of personally identifiable information. We will act in accordance with all relevant best practices, laws, regulations, and data owner rules governing the handling and storage of such information.
a. We will restrict access to identifiers and destroy them as soon as they are no longer required, in accordance with relevant laws, regulations, and data owner rules.
b. We will not disclose any information that could be used, alone or in combination with other reasonably available information, to identify participants with their data, without participant permission.
c. When disclosing personally identifiable data for purposes other than the current research, we will relay to data users any conditions of their use specified in the participant permission we have obtained.
3. We understand that the use of our research results in a legal proceeding does not relieve us of our ethical obligation to protect participant privacy and keep confidential all personally identifiable data, except where participants have permitted disclosure.
C. Clients or Sponsors
II. Principles of Professional Practice in the Conduct of Our Work
A. We will exercise due care in developing research designs, samples, and instruments, and in collecting, processing, and analyzing data, taking all reasonable steps to assure the reliability and validity of results.
1. We will recommend and employ only those tools and methods of analysis that, in our professional judgment, are fit for the purpose of the research questions.
2. We will not knowingly select research tools and methods of analysis that yield misleading conclusions.
3. We will not knowingly make interpretations of research results that are inconsistent with the data available, nor will we tacitly permit such interpretations. We will ensure that any findings we report, either privately or for public release, are a balanced and accurate portrayal of research results.
4. We will not knowingly imply that interpretations should be accorded greater confidence than the data actually warrant. When we generalize from samples to make statements about populations, we will only make claims of precision and applicability to broader populations that are warranted by the sampling frames and other methods employed.
5. We will not engage in data fabrication or falsification.
6. We will accurately describe and attribute research from other sources that we cite in our work, in terms of its methodology, content, comparability, and source.
B. We will describe our methods and findings accurately and in appropriate detail in all research reports, adhering to the standards for disclosure specified in Section III of the Code.
III. Standards for Disclosure
Good professional practice imposes the obligation upon all public opinion and survey researchers to disclose sufficient information about how the research was conducted to allow for independent review and verification of research claims. Full and complete disclosure will be made at the time results are released, either publicly or to a research client, as the case may be. As detailed below, some information, if not immediately available, will be released within 30 days of any request for such materials.
A. Disclosure Items for Surveys
Broadly defined, surveys feature a predetermined set of questions and/or other measurements administered to a set of sample units with the intention of supporting inferences to a population of interest. We will include the following items in any report of survey research results or make them available immediately upon release of that report:
1. Who sponsored the survey and who conducted it. If different from the sponsor, the original sources of funding will also be disclosed.
2. The exact wording and presentation of questions and response options whose results are reported. This includes preceding interviewer or respondent instructions and any preceding questions that might reasonably be expected to influence responses to the reported results.
3. A definition of the population under study and its geographic location.
4. Dates of data collection.
5. A description of the sampling frame(s) and its coverage of the target population, including mention of any segment of the target population that is not covered by the design. This may include, for example, exclusion of Alaska and Hawaii in U.S. surveys; exclusion of specific provinces or rural areas in international surveys; and exclusion of non-panel members in panel surveys. If possible the estimated size of non-covered segments will be provided. If a size estimate cannot be provided, this will be explained. If no frame or list was utilized, this will be indicated.
6. The name of the sample supplier, if the sampling frame and/or the sample itself was provided by a third party.
7. The methods used to recruit the panel or participants, if the sample was drawn from a pre-recruited panel or pool of respondents.
8. A description of the sample design, giving a clear indication of the method by which the respondents were selected, recruited, intercepted or otherwise contacted or encountered, along with any eligibility requirements and/or oversampling. If quotas were used, the variables defining the quotas will be reported. If a within-household selection procedure was used, this will be described. The description of the sampling frame and sample design will include sufficient detail to determine whether the respondents were selected using probability or non-probability methods.
9. Method(s) and mode(s) used to administer the survey (e.g., CATI, CAPI, ACASI, IVR, mail survey, Web survey) and the language(s) offered.
10. Sample sizes (by sampling frame if more than one was used) and a discussion of the precision of the findings. For probability samples, the estimates of sampling error will be reported, and the discussion will state whether or not the reported margins of sampling error or statistical analyses have been adjusted for the design effect due to weighting, clustering, or other factors. Disclosure requirements for non-probability samples are different because the precision of estimates from such samples is a model-based measure (rather than the average deviation from the population value over all possible samples). Reports of non-probability samples will only provide measures of precision if they are accompanied by a detailed description of how the underlying model was specified, its assumptions validated and the measure(s) calculated. To avoid confusion, it is best to avoid using the term “margin of error” or “margin of sampling error” in conjunction with non-probability samples.
11. A description of how the weights were calculated, including the variables used and the sources of weighting parameters, if weighted estimates are reported.
After survey results are reported, we will make the following items available within 30 days of any request for such materials:
12. Procedures for managing the membership, participation, and attrition of the panel, if a pool, panel, or access panel was used.
13. Methods of interviewer training, supervision, and monitoring, if interviewers were used.
14. Details about screening procedures, including any screening for other surveys that would have made sample members ineligible for the current survey must be disclosed (e.g., in the case of online surveys if a router was used).
15. Any relevant stimuli, such as visual or sensory exhibits or show cards. In the case of surveys conducted via self-administered computer- assisted interviewing, providing the relevant screen shot(s) is optimal, though not required.
16. Details of any strategies used to help gain cooperation (e.g., advance contact, compensation or incentives, refusal conversion contacts) whether for participation in a group, panel or access panel or for participation in a particular research project.
17. Procedures undertaken to ensure data quality, if any. Where applicable, this includes re-contacts to confirm that the interview occurred and/or to verify the respondent’s identity, measures taken to prevent respondents from completing the same survey more than once, and other quality control procedures (e.g., logic checks and tests for speeding and patterning). If no such efforts were undertaken, this will be disclosed.
18. Summaries of the disposition of study-specific sample records so that response rates for probability samples and participation rates for non- probability samples can be computed. If response or cooperation rates are reported, they will be computed according to AAPOR Standard Definitions. If dispositions cannot be provided, the reason(s) will be disclosed and this will be mentioned as a limitation of the study.
19. The unweighted sample size on which one or more reported subgroup estimates are based.
20. Specifications adequate for replication of indices or statistical modeling included in research reports.
B. Disclosure Items for Qualitative Research
Qualitative research includes focus groups, in-depth interviews, case studies, narrative research, and ethnography, among other approaches. This research generally involves descriptive, unstructured data. We will include the following items in any report of qualitative research or make them available immediately upon release of that report:
1. Who sponsored the research and who conducted it. If different from the sponsor, the original sources of funding will also be disclosed.
2. A definition of the population under study and its geographic location.
3. The instrumentation used (e.g., questionnaires, discussion guides), a description of the data collection strategies employed (e.g., focus groups, semi-structured interviews), and the language(s) used.
4. A description of any relevant stimuli, such as visual or sensory exhibits or show cards.
5. Dates of data collection.
6. The physical location of all data collection activities (e.g., subject home, office/workplace, clinic, focus group facility, street corner).
7. A description of subject eligibility (e.g., age or gender requirements) and the procedures employed to screen and recruit research subjects.
8. The number of research subjects, by data collection strategy.
9. Methods of interviewer and/or coder training, supervision, and monitoring, if interviewers or coders were used.
10. Duration of research participation (e.g., length of interviews, focus group sessions).
11. Any compensation/incentives provided to research subjects.
12. Information regarding whether or not data collection included audio or video recordings.
C. Disclosure Items for Content Analysis
Content analysis includes the systematic analysis of text, images or other content. Content analysis can be qualitative or quantitative and the material to be analyzed can be from any source (e.g., open-ended verbatim responses collected in surveys, newspaper and magazine articles, television and radio broadcasts, politicians’ speeches, tweets, or posts on social networking sites). Content analysis can be performed by human coders and/or via automated coding software. We will include the following items in any content analysis report or make them available immediately upon release of that report:
1. Who sponsored the research and who conducted it. If different from the sponsor, the original sources of funding will also be disclosed.
2. A description of how the content analyzed was collected or obtained. This discussion will state the source(s) used, how much content was analyzed (e.g., number and average length of articles, tweets or blog postings, news broadcasts), dates analyzed, language(s) included, and criteria or decision rules used to include or exclude elements of content.
3. The sampling approach used. If a census of the target population of content was used, that will be explicitly stated.
4. A discussion of any threats or concerns about the validity or quality of the content (e.g., online commentary created by bots, fabricated social media profiles, relevant content missing from the source(s) used) and any steps taken to address them.
5. A description of how the analysis was conducted. This discussion will mention whether coding was done by software or human coders (or both) and whether quantitative or qualitative methods (or both) were used. For analysis using automated coding, this discussion will mention the software and the parameters or decision rules that were used. For analysis using human coding, this discussion will mention the number of coders and any training and instructions provided to them. If a formal coding scheme was used, it will be provided. If no formal coding scheme was used, this will be disclosed.
6. Inter-coder reliability, amount of text analyzed by multiple coders, process for resolving inconsistencies, and any steps taken to increase the level of agreement, if multiple coders were used. If only one coder was used, this will be disclosed.
7. The unit of analysis (e.g., a news article, broadcast, tweet, or blog posting).
8. A description of how the weights were calculated, including the variables used, if weights were used to develop a coding scheme and/or produce final estimates.
9. Other information that may need to be disclosed if the content analysis involves qualitative data collection (see Section III-B) or survey data collection (see Section III-A).
D. If the results reported are based on multiple samples or multiple modes, the preceding items (from Section III-A, III-B, or III-C, as applicable) will be disclosed for each.
E. Reflecting the fundamental goals of transparency and replicability, AAPOR members share the expectation that access to datasets and related documentation will be provided to allow for independent review and verification of research claims upon request. Datasets may be held without release for a period of up to one year after findings are publicly released to allow full opportunity for primary analysis. In order to protect the privacy of individual respondents, such datasets must be de-identified to remove variables that can reasonably be expected to identify a respondent. Those who commission publicly disseminated research have an obligation to disclose the rationale for why eventual public release or access to the datasets is not possible, if that is the case.
F. If any of our work becomes the subject of a formal investigation of an alleged violation of this Code, undertaken with the approval of the AAPOR Executive Council, we will provide additional information on the research study in such detail that a fellow researcher would be able to conduct a professional evaluation of the study.