Skip to Content


Proposed Information Collection; Comment Request; 2015 National Content Test

Document Details

Information about this document as published in the Federal Register.

Published Document

This document has been published in the Federal Register. Use the PDF linked in the document sidebar for the official electronic format.

Start Preamble


U.S. Census Bureau, Commerce.




The Department of Commerce, as part of its continuing effort to reduce paperwork and respondent burden, invites the general public and other Federal agencies to take this opportunity to comment on proposed and/or continuing information collections, as required by the Paperwork Reduction Act of 1995, Public Law 104-13 (44 U.S.C. 3506(c)(2)(A)).


To ensure consideration, written comments must be submitted on or before February 2, 2015.


Direct all written comments to Jennifer Jessup, Departmental Paperwork Clearance Officer, Department of Commerce, Room 6616, 14th and Constitution Avenue NW., Washington, DC 20230 (or via the Internet at

Start Further Info


Requests for additional information or copies of the information collection instrument(s) and instructions should be directed to Erin Love, Census Bureau, HQ-3H154E, Washington, DC 20233; (301) 763-2034 (or via email at

End Further Info End Preamble Start Supplemental Information


I. Abstract

The 2015 National Content Test (NCT) is part of the research and development cycle leading up to the re-engineered 2020 Census. The 2015 NCT will help the Census Bureau achieve one of its Strategic Goals—developing a census that is cost-effective, improves coverage, and reduces operational risk.

The first objective of this test is to evaluate and compare different census content, including race and Hispanic origin, relationship, and within-household coverage. This will be the primary mid-decade opportunity to compare different content strategies prior to making final decisions about the content in the 2020 Census. The test will include a reinterview to further assess the accuracy and reliability of the question alternatives for race, origin, and within-household coverage.

The second objective is to test different contact strategies for optimizing self-response. This includes nine different approaches to encouraging households to respond and, specifically, to respond using the less costly and more efficient Internet response option. These approaches include altering the timing of the first reminder, use of email as a reminder, altering the timing for sending the mail questionnaire, use of a third reminder, and sending a letter in place of a paper questionnaire to non-respondents.

The third objective is to test different options for offering non-English materials. The goal is to provide language support for respondents with limited English proficiency. Options being explored include online Spanish questionnaires, dual-language English and Spanish paper questionnaires and letters, and additional questionnaire options and support in non-English languages.

Regarding the first objective, the classification of racial and ethnic responses to the decennial census by the Census Bureau adheres to the U.S. Office of Management and Budget's (OMB) October 30, 1997 “Revisions to the Standards for the Classification of Federal Data on Race and Ethnicity” (see​omb/​fedreg_​1997standards). There are five minimum categories for data on race: “White,” “Black or African American,” “American Indian or Alaska Native,” “Asian,” and “Native Hawaiian or Other Pacific Islander.” There are two minimum categories for data on ethnicity: “Hispanic or Latino” and “Not Hispanic or Latino.” The OMB standards advise that respondents shall be offered the option of selecting one or more racial designations. The OMB standards also advise that race and ethnicity are two distinct concepts; therefore, Hispanics or Latinos may be any race.

The minimum categories for data on race and ethnicity for Federal statistics, program administrative reporting, and civil rights compliance reporting are defined by OMB as follows:

  • American Indian or Alaska Native—A person having origins in any of the original peoples of North and South America (including Central America), and who maintains tribal affiliation or community attachment.
  • Asian—A person having origins in any of the original peoples of the Far East, Southeast Asia, or the Indian subcontinent including, for example, Cambodia, China, India, Japan, Korea, Malaysia, Pakistan, the Philippine Islands, Thailand, and Vietnam.
  • Black or African American—A person having origins in any of the black racial groups of Africa. Terms such as “Haitian” or “Negro” can be used in addition to “Black or African American.”
  • Hispanic or Latino—A person of Mexican, Puerto Rican, Cuban, South or Central American, or other Spanish culture or origin, regardless of race. The term, “Spanish origin,” can be used in addition to “Hispanic or Latino.”
  • Native Hawaiian or Other Pacific Islander—A person having origins in any of the original peoples of Hawaii, Guam, Samoa, or other Pacific Islands.
  • White—A person having origins in any of the original peoples of Europe, the Middle East, or North Africa.

The 1997 OMB standards state the minimum categories that must be used to collect and present federal data on race and ethnicity. Additionally, the 1997 OMB standards permit the collection of more detailed information on population groups, provided that any additional groups can be aggregated into the minimum standard set of categories. Currently, the Census Bureau collects additional detailed information on Hispanic or Latino groups, American Indian and Alaska Native tribes, Asian groups, and Native Hawaiian and Other Pacific Islander groups.

For example, responses to the race question such as “Navajo Nation,” “Doyon,” and “Mayan” are collected and tabulated in Census Bureau censuses and surveys, and can be aggregated into the total American Indian or Alaska Native population. Detailed responses to the race question such as “Chinese,” “Asian Indian,” and “Vietnamese” are collected and tabulated, and can be aggregated into the total Asian population. Responses to the ethnicity question such as “Mexican,” “Puerto Rican,” and “Cuban” are collected and tabulated in Census Bureau censuses and surveys, and can be aggregated into the total Hispanic or Latino population. Responses to the race question such as “Native Hawaiian,” “Chamorro,” or “Fijian” are collected and tabulated, and can be aggregated into the total Start Printed Page 71378Native Hawaiian or Other Pacific Islander population.

The 2015 NCT will test ways to collect and tabulate detailed information for all groups, including data for White groups, such as German, Irish, and Lebanese, and data for Black groups, such as African American, Jamaican, and Nigerian, which have not been tabulated previously from the question on race. Responses to the race question such as “African American,” “Jamaican,” or “Nigerian” will be collected and tabulated, and can be aggregated to the total Black or African American population. Responses to the race question such as “German,” “Irish,” or “Lebanese” will be collected and tabulated, and can be aggregated into the total White population.

The 2015 NCT will also test a separate “Middle Eastern or North African” category and the collection of detailed groups such as “Lebanese,” “Egyptian,” and “Iranian.” Following the current OMB standards, Middle Eastern and North African responses are classified as “White.”

The results of the 2015 NCT will guide future collection and tabulation of detailed information for all race and ethnicity groups.

Plans for the 2020 Census call for the use of less costly and more efficient web-based response options to collect information, as opposed to a previous predominant reliance on paper-based questionnaires. One benefit of the online response mode is that it allows for more functionality and greater flexibility in designing questions compared to paper, which is constrained by space availability. With the advantage of new technology, the 2015 National Content Test will utilize web-based technology, such as internet, smart phone, tablet, and telephone to improve question designs and optimize reporting of detailed racial and ethnic groups (e.g., Samoan, Iranian, Blackfeet Tribe, Filipino, Jamaican, Puerto Rican, Irish, etc.).

The web-based designs provide much more utility and flexibility for using detailed checkboxes and write-in spaces to elicit and collect data for detailed groups than traditional paper questionnaires, and will help collect data for both the broader OMB categories, as well as detailed responses across all groups.

Components of the Test

A. Race and Origin Content

The Census Bureau conducted an extensive research undertaking as part of the 2010 Census—the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment (AQE) (for details, see​2010census/​news/​press-kits/​aqe/​aqe.html). The 2010 AQE examined alternative strategies for improving the collection of data on a race and Hispanic origin, with four goals in mind:

1. Increasing reporting in the standard race and ethnic categories as defined by the U.S. Office of Management and Budget;

2. Decreasing item non-response for these questions;

3. Increasing the accuracy and reliability of the results for this question; and

4. Eliciting detailed responses for all racial and ethnic communities (e.g., Chinese, Mexican, Jamaican, etc.).

The results of the AQE supported all of these objectives. Additionally, many individuals across communities liked the combined question approach. They believed it presented equity to the different categories. Some of the findings from this research include:

  • Combining race and ethnicity into one question did not change the proportion of people who reported as Hispanics, Blacks, Asians, American Indians and Alaska Natives, or Native Hawaiians and Other Pacific Islanders.
  • The combined question yielded higher response rates.
  • The combined question increased reporting of detailed responses for most groups, but decreased reporting for others.
  • The combined question better reflected self-identity.

The successful strategies from the AQE research have been employed in the design of the Census Bureau's mid-decade research. Four key dimensions of the questions on race and Hispanic origin are being tested in the 2015 NCT. These include question format, response categories, wording of the instructions, and question terminology.

Question Format

The 2015 NCT will evaluate the use of two alternative question format approaches for collecting data on race and ethnicity. One approach uses two separate questions: the first about Hispanic origin and the second about race (“separate questions”). The other approach combines the two items into one question about race and origin (“combined question”). The 2015 mid-decade research will test the approaches with new data collection methods, including internet, telephone, and in-person response.

1. Separate race and origin questions: This is a modified version of the race and Hispanic origin format used in the 2010 Census. Updates since the 2010 Census include added write-in spaces and examples for the “White” and “Black or African Am.” response categories, removal of the term “Negro,” and an instruction to select one or more boxes in the Hispanic origin question.

2. Combined question with checkboxes and write-ins on same screen: This is a modified version of the combined question approaches found to be successful in the 2010 AQE. Checkboxes are provided for the U.S. Office of Management and Budget (OMB) standard categories (per the 1997 Standards for the Classification of Federal Data on Race and Ethnicity) with a corresponding write-in space for each checkbox category. In this version, all write-in spaces are visible at all times. Each response category contains six example origins, which represent the diversity of the geographic definitions of the OMB category. For instance, the “Asian” category examples of Chinese, Filipino, Asian Indian, Vietnamese, Korean, and Japanese represent the six largest detailed Asian groups in the United States, reflecting OMB's definition of Asian (“A person having origins in any of the original peoples of the Far East, Southeast Asia, and the Indian subcontinent.”). Respondents do not have to select an OMB checkbox, but may enter a detailed response in the write-in space without checking a category.

a. Combined question with checkboxes and write-ins on separate screens (Internet-only): In this version, the detailed origin groups are solicited on subsequent screens after the OMB response categories have been selected. On the first screen, the OMB checkbox categories are shown along with their six representative example groups. Once the OMB categories have been selected, one at a time, subsequent screens solicit further detail for each category that was chosen (e.g., Asian), using a write-in space to collect the detailed groups (e.g., Korean and Japanese). The intent is to separate mouse click tasks (checkbox categories) and typing tasks (write-ins) in an attempt to elicit responses that are more detailed. The same version was used as one of three race and origin Internet panels in the 2014 Census Test.

3. Combined question branching with detailed checkbox screens (Internet-only): This version is an alternative method of soliciting detailed origin groups using separate screens, detailed checkboxes, and write-in spaces. On the first screen, the OMB checkbox categories are shown along with their six representative example groups. Once the OMB categories have been selected, one at a time, subsequent screens solicit Start Printed Page 71379further detail for each category, this time using a series of additional checkboxes for the six largest detailed groups (e.g., Chinese, Filipino, Asian, Indian, Vietnamese, Korean, and Japanese) with a write-in space also provided to collect additional groups.

Race Response Categories

The 2015 National Content Test will evaluate the use of the Middle Eastern or North African (MENA) category in the race question. There will be two treatments for testing this dimension:

1. Use of MENA category: This treatment tests the addition of a MENA checkbox category to the race question. The MENA category is placed within the current category lineup, based on estimates of population size, between the categories for Native Hawaiians and Other Pacific Islanders and “Some other race.” With the addition of this new category, the “White” example groups are revised. The Middle Eastern and North African examples of “Lebanese” and “Egyptian” are replaced with the European examples of “Polish” and of “French.” The MENA checkbox category will have the examples of “Lebanese, Iranian, Egyptian, Syrian, Moroccan, Algerian, etc.” All other checkbox categories and write-in spaces remain the same.

2. No separate MENA category: This treatment tests approaches without a separate MENA checkbox category, and represents the current OMB definition of White (“A person having origins in any of the original peoples of Europe, the Middle East, or North Africa.”). Here we will provide examples of Middle Eastern and North African origins (“Lebanese” and “Egyptian”) with European origin groups as part of the “White” racial category.

Wording of the Instructions

1. “Mark [X] one or more boxes”: The current paper version of the instructions on paper states, “Mark [X] one or more boxes AND print your specific origin(s).”

2. “Mark all that apply/You may mark multiple groups”: In this version, the instruction is modified to “Mark all boxes that apply AND print the specific [origin(s)/ethnicities] in the spaces below. Note, you may report more than one group.” Recent qualitative focus groups and cognitive research (e.g., 2010 AQE research; 2013 Census Test research) found that respondents frequently overlook the instruction to “Mark” [X] one or more boxes. The research found that some respondents may have stopped reading the instruction after noticing the visual cue [X] and proceeded directly to do just that—mark a box—overlooking the remainder of the instruction. The new instruction (“Mark all boxes that apply”) is an attempt to improve the clarity of the question and make it more apparent that more than one group may be selected.

Question Terms

1. “Origin” term: The current version of the race and Hispanic origin questions use the terms “race” and/or “origin” to describe the concepts and groups in the question stem, instructions, and examples. For instance, in the combined race and Hispanic origin approach, the question stem is “What is your race or origin?” In addition, prior to each write-in field, respondents are instructed to “Print specific origin(s), for example . . .”

2-3. Alternative terms: Recent qualitative focus groups and qualitative research (e.g., 2010 AQE research; 2013 Census Test research; cognitive pre-testing for 2016 American Community Survey Content Test) found that the term “origin” is confusing or misleading to many respondents, who may think it is asking about where they immigrated from or where they were born. Two alternative options are being explored in cognitive testing and usability research. One approach tests the use of the term “ethnicities” along with “race” (e.g., “Print the specific races(s) and/or ethnicities . . .”). The other approach tests the removal of the terms altogether from the question stem, instructions, and examples. Instead, a general approach asks, “Which categories describe this person?” The exact terminology to be used for the alternative version is pending cognitive testing and usability results later this year, which will inform the wording to be used in the 2015 NCT.

B. Relationship Content

Two versions of the relationship question will be tested. Both versions are the same as those used in a split-sample in the 2014 Census Test, with no changes. The new relationship categories have also been tested in other Census Bureau surveys including the American Housing Survey, American Community Survey, and the Survey of Income and Program Participation (currently used in production). Although research to date has been informative, leading to the development of the revised relationship question, additional quantitative testing is needed. Since the incidence of some household relationships—such as same-sex couples—is relatively low in the general population, the revised question needs to be tested with large, representative samples prior to routinely including them in the 2020 Census questionnaire.

The first version uses the 2010 Census relationship question response options, but in a new order, starting with “husband or wife” and then the “unmarried partner” category. This version also re-introduces the foster child category, which was removed from the 2010 Census form due to space issues.

The second version includes the same basic response options as the 2010 Census version, but modifies/expands the “husband or wife” and “unmarried partner” categories to distinguish between same-sex and opposite-sex relationships.

C. Coverage Content (Internet Only)

The 2012 National Census Test experimented with several methods to improve accurate within-household coverage for Internet respondents. One benefit of the online response mode is that it allows for more functionality and greater flexibility in designing questions compared to paper, which is constrained by space availability. The 2012 test included a coverage follow-up reinterview to evaluate the different Internet design options, but some results were inconclusive. In the 2015 NCT, two designs will be tested to compare different approaches for helping respondents provide a more accurate roster of household residents.

The first approach is the “Rules-Based” approach, and will allow us to see whether the presence of a question asking the number of people in the household along with the residence rule instructions helps respondents create an accurate roster. This is similar to the approach used across all modes in Census 2000 and the 2010 Census, where the respondent was expected to understand our residence rules and apply them to their household. This is followed by a household-level question that probes to determine if any additional people not listed originally should be included for consideration as residents of the household (several types of people and living situations are shown in a bulleted list).

The “Question-Based” approach allows us to ask guided questions to help improve resident information. Respondents are not shown the residence rule instructions and are only asked to create an initial roster of people they consider to be living or staying at their address on Census Day. This is followed by several short household-level questions about types of people and living situations that might apply to Start Printed Page 71380someone in the household that was not listed originally.

D. Optimizing Self Response

The nine proposed contact strategies for optimizing self response (OSR) are summarized as follows:

Internet Push (Control): This is the standard Internet Push strategy used in the most recent series of self response tests, including the 2014 Census Test. This panel will serve as a control panel against which to compare the experimental strategies. There will be nine treatments as part of the OSR test.

Internet Push With Early Postcard: The motivation for this panel is to study the timing of reminders. The hypothesis is that sending the first reminder sooner (closer to the initial Internet push) would provide for a better connection between the two mailings, and could increase response. A side benefit is that this could also reduce the volume of later targeted mailings since responses may be quicker overall.

The motivation for the following sequence of three panels is based on recent American Community Survey (ACS) research, which has found depressed self response rates among certain respondents/areas with lower Internet usage. Testing the delivery of the paper questionnaires at various points in the response process will allow us to have complete response measures under several scenarios for the cost/benefit analysis needed to inform 2020 Census planning. Although these strategies may not make sense for everyone in 2020, using a responsive design and tailoring the contact strategy for certain geographic areas or populations may be beneficial.

  • Internet Push With Early Questionnaire: questionnaire sent at third mailing, one week sooner
  • Internet Push With Even Earlier Questionnaire: questionnaire sent at second mailing, two weeks sooner
  • Internet Choice: questionnaire sent at first mailing, providing a choice of Internet or paper from the beginning

Internet Push With Postcard as Third Reminder: The motivation for this panel is to further encourage self response, after the questionnaire mailing, prior to nonresponse follow-up. Numerous survey research studies have concluded that, while there is a point of diminishing returns, further reminders will inevitably increase self response rates.

Internet Push Postcard: The motivation for this panel is to study the impact of sending a postcard at the first mailing instead of a letter. There are two potential benefits. First is the possible cost savings of printing and mailing a postcard compared to the envelope package (with letter and instruction card). Second is the potential for increased self response because reading a postcard requires less effort by a respondent. In this panel, we send a letter at the third contact (sent to non-respondents only), in place of a postcard, to vary the types of contacts received.

Internet Push With Early Postcard and Second Letter Instead of Mail Questionnaire: The motivation for testing an approach in which we do not send a mail questionnaire is to address the high-level goal of greatly reducing paper responses in the 2020 Census. By testing an approach in which we send an Internet push letter in place of a paper questionnaire at the fourth mailing, we will have a more robust set of response measures for informing cost/benefit analyses.

Internet Push With Postcard and Email as 1st Reminder (same time): The motivation for this panel is to determine if we can take advantage of the email addresses in the supplemental contact frame maintained by the Center for Administrative Records Research and Applications. The hypothesis is that by sending a postcard and email at the same time, we may be able to elicit increased response.

E. Language

In the two mailings that contain a letter for each Optimizing Self response strategy, three different methods will be used to encourage response. In particular, by altering the language support provided in the letter, the goal is to increase response for respondents with limited English proficiency.

The control panel is similar to the 2014 Census Test design, in which the mailing materials are in English with a single Spanish sentence directing respondents to the Web site or the telephone assistance line.

One of the goals of language research is to maximize the number of non-English speakers that receive the same message as English speakers prior to going online to respond. Two panels provide equality between the English and Spanish content in the letter and test whether one method is better at eliciting Spanish responses. The swim-lane design has been used in the past, such as with the bilingual questionnaire in the 2010 Census. The dual-sided letter provides English content on one side and Spanish content on the other side. In addition, because research has shown that Spanish-speaking respondents do not always open the mailings because they may not know that language resource information is provided inside, the outgoing envelope for both panels will include the census test Web site URL and a brief message in both languages.

This test will also explore additional options for non-English speakers to complete the questionnaires.

F. Content Reinterview

A sub-sample of respondents from the 2015 NCT will be selected for a content reinterview, focused on race and origin and within-household coverage, with a goal of assessing accuracy and reliability of the different designs. Reinterviews are conducted with a sub-sample of respondents, by asking more detailed questions on question topics, in order to assess the accuracy of the responses.

II. Method of Collection

The initial mail-out is planned for late August 2015. This contact will explain why we are conducting the mandatory 2015 NCT, assure respondents that their answers are confidential, and inform them of the measures we take to keep their personal information secure. The second mail-out is considered a reminder and is sent to all housing units. All contacts after the second mailing are sent to non-respondents only.

Respondents are encouraged to respond to the 2015 NCT by Internet but may also be able to provide information by phone. Many will also receive a paper questionnaire at some point in the mail-out strategy. The test will be conducted nationally in all 50 U.S. states, the District of Columbia, and Puerto Rico.

III. Data

OMB Control Number: None.

Form Number: TBD.

Type of Review: Regular submission.

Affected Public: Individuals or households.

Estimated Number of Respondents: 1.3 million households. (1.2 million initial response + 100,000 reinterview).

Estimated Time per Response: 10 minutes.

Estimated Total Annual Burden Hours: 216,667.

Estimated Total Annual Cost: There is no cost to respondents except for their time to respond.

Respondent's Obligation: Mandatory.

Legal Authority: Title 13 U.S.C. 141 and 193.

IV. Request for Comments

Comments are invited on: (a) Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have Start Printed Page 71381practical utility; (b) the accuracy of the agency's estimate of the burden (including hours and cost) of the proposed collection of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Comments submitted in response to this notice will be summarized and/or included in the request for OMB approval of this information collection; they also will become a matter of public record.

Start Signature

Dated: November 25, 2014.

Glenna Mickelson,

Management Analyst, Office of the Chief Information Officer.

End Signature End Supplemental Information

[FR Doc. 2014-28247 Filed 12-1-14; 8:45 am]