Skip to Content


Commission Information Collection Activities (FERC-922); Comment Request

Document Details

Information about this document as published in the Federal Register.

Published Document

This document has been published in the Federal Register. Use the PDF linked in the document sidebar for the official electronic format.

Start Preamble


Federal Energy Regulatory Commission, DOE.


Comment request.


In compliance with the requirements of the Paperwork Reduction Act of 1995, 44 U.S.C. 3507(a)(1)(D), the Federal Energy Regulatory Commission (Commission or FERC) is submitting for reinstatement a revised information collection FERC-922, “Performance Metrics for ISOs, RTOs and Regions Outside ISOs and RTOs,” to the Office of Management and Budget (OMB) for review of the information collection requirements. Any interested person may file comments directly with OMB and should address a copy of those comments to the Commission as explained below. The Commission previously issued a Notice in the Federal Register (79 FR 52313, 9/3/2014) requesting public comments. The Commission also issued an errata notice to fix an errant hyperlink in the 60-day notice (8/26/2014). The Commission received seven comments on the FERC-922. The Commission addresses these comments in this notice and in its submittal to OMB.


Comments on the collection of information are due by June 26, 2015.


Comments filed with OMB, identified by the OMB Control No. 1902-0262, should be sent via email to the Office of Information and Regulatory Affairs at: Attention: Federal Energy Regulatory Commission Desk Officer. The Desk Officer may also be reached via telephone at (202) 395-4718.

A copy of the comments should also be sent to the Federal Energy Regulatory Commission, identified by the Docket No. AD14-15-000, by either of the following methods:

Instructions: All submissions must be formatted and filed in accordance with submission guidelines at:​help/​submission-guide.asp. Submissions must be in an acceptable file format, as described at:​docs-filing/​elibrary/​accept-file-formats.asp. The numeric values corresponding to all charts and tables containing metrics must be submitted in an accompanying file, in one of the following formats: Microsoft Office 2003/2007/2010: Excel (.xls or .xlsx), or ASCII Comma Separated Value (.csv). For user assistance contact FERC Online Support by email at, or by phone at: (866) 208-3676 (toll-free), or (202) 502-8659 for TTY.

Docket: Users interested in receiving automatic notification of activity in this docket or in viewing/downloading comments and issuances in this docket may do so at​docs-filing/​docs-filing.asp.

Start Further Info


Ellen Brown may be reached by email at, by telephone at (202) 502-8663, and by fax at (202) 273-0873.

End Further Info End Preamble Start Supplemental Information


Title: FERC-922, Performance Metrics for ISOs, RTOs and Regions Outside of ISOs and RTOs.

OMB Control No.: 1902-0262.

Type of Request: Reinstatement and revision of an information collection.

Abstract: In September 2008, the United States Government Accountability Office (GAO) issued a report recommending that the Chairman of the Commission, among other actions, work with independent system operators (ISOs), regional transmission organizations (RTOs), stakeholders, and other experts to develop standardized measures that track the performance of ISO/RTO operations and markets and report the performance results to Congress and the public annually,[1] while also providing interpretation of (1) what the measures and reported performance communicate about the benefits of ISOs/RTOs and, where appropriate, (2) changes that need to be made to address any performance concerns. The GAO Report also suggested that performance metrics be explored for non-ISOs/RTO regions.

In response to the GAO Report, Commission Staff conducted outreach with ISOs/RTOs and other stakeholders and in October 2010 established metrics Start Printed Page 30231to measure ISO/RTO performance. In April 2011, a report was sent to Congress with an analysis of ISO/RTO performance based on these metrics and a commitment to analyze utilities in non-ISO/RTO regions. After further stakeholder outreach, in August 2014, the Commission Staff issued a “Common Metrics Report,” establishing 30 common metrics that measure performance for ISOs, RTOs and public utilities outside of ISOs/RTOs from 2006-2010.

The Commission is continuing its efforts to collect performance metric information from ISOs, RTOs, and public utilities in non-ISO/RTO regions. This includes the submission of information relating to dispatch reliability, transmission planning, and the marginal cost of energy and resource availability. The information submitted by ISOs, RTOs, and participating public utilities in non-ISO/RTO regions is used to measure the performance of reliability and operations functions in which ISOs, RTOs, and public utilities in non-ISO/RTO regions perform identical activities.

The attached list of metrics will not be published in the Federal Register but will be available as part of this notice in the Commission's eLibrary system under Docket No. AD14-15-000.

Type of Respondents: ISOs, RTOs, and public utilities.

Estimate of Annual Burden:[2] For ISOs, RTOs and public utilities that have submitted performance information previously, their submittals will only include performance information for the 2010-2014 period.[3] For other public utilities that have not submitted performance information previously, their submittals will also provide performance information for the 2010-2014 period.[4] This information is to be filed by October 30, 2015.[5] The estimate of annual burden assumes submittals occur every two years. For this reason, the annual number of responses is “0.5” in the table below.

FERC-922 (AD14-15-000)—Performance Metrics for ISOs, RTOs and Regions Outside ISOs and RTOs 6

Information collection componentNumber of respondentsAnnual number of responses per respondentTotal number of responsesAverage burden hours and cost per response 7Total annual burden hours and total annual costCost per respondent per year ($)
Metrics Data Collection110.55.5229 $18,3661,260 $101,0129,183
Write Performance Analysis110.55.5139 $11,148765 $61,3135,574
Management Review110.55.533 $2,796182 $15,3771,398
Subtotal2,207 $177,70216,155
Collection, writing, and review50.52.5427 $34,4031,068 $86,00817,202
Total163,275 $263,710

Public Comments and FERC's Responses: Comments were filed by the public in response to the FERC-922 Federal Register Notice of Information Collection and Request for Comments and the Commission's responses to those comments are provided below.

Burden Estimate

Edison Electric Institute (EEI) considers the burden estimate to be significantly understated, particularly for “stand-alone utilities” without access to data collection and compilation activities performed by ISO and RTO staff. EEI estimates the response time for stand-alone utilities to be as high as 300-400 hours per utility.

FERC Response: We address EEI's concern by revising the burden estimate. We recognize that certain EEI members have experienced the process of collecting, summarizing, reviewing, and Start Printed Page 30232submitting information as part of this initiative, and therefore might be better positioned to estimate the time and resources involved. In response, we revise the burden estimate to be approximately 400 hours per respondent (401 hours for previous participants and 427 hours for new participants). We believe that the updated burden estimate accounts for the higher response times of certain participants. We also believe that the updated burden estimate accounts for any additional time associated with the instruction to submit the numeric values corresponding to charts and tables in an accompanying file.[10]

Ways To Minimize Information Collection Burden

The ISO/RTO Council (IRC) recommends that data be provided only for the 2010-2014 period. IRC notes that its members have already submitted information through 2010. Southern Company Services, Inc. (Southern) also recommends that only one data collection be required for the 2010-2014 time period. Noting that the Common Metrics Report issued by the Commission in August 2014 provides information for the 2006-2010 period, Southern considers information collection on the 2008-2012 period to be an additional burden and argues that it should be eliminated. Noting that utilities outside of ISOs/RTOs will have to devote considerable resources and expenses to provide data, EEI recommends that the Commission retain the voluntary approach for these utilities and that data collection for ISO and RTO regions only occur when data is readily available and the data collection process can be streamlined. New York Transmission Owners (NYTOs) [11] support continued data collection. NYTOs consider this information to be helpful for analyzing ISO and RTO performance and that the benefits of the information to the Commission and affected parties outweigh any related burdens on respondents. International Transmission Company (ITC) supports the proposed data collection as necessary and not overly burdensome. American Public Power Association (APPA) and American Wind Energy Association (AWEA) also support continued data collection.

FERC Response: We note concerns raised by IRC and Southern over the potential redundancy and additional burden for providing information for the 2008-2012 period. Additionally, we note EEI's concerns with streamlining the collection process. In designing the information collection process, we aim to balance the goal of creating comparable data series across entities with the goals of wide participation and practical submission criteria. Accordingly, all participating entities may submit a single report with information on the 2010-2014 period rather than submitting two reports for the 2008-2012 and 2010-2014 periods. This includes ISOs, RTOs, and public utilities in non-ISO/RTO regions that have submitted performance information previously, and public utilities in non-ISO/RTO regions that have not submitted performance information previously. The reports may be submitted by October 30, 2015. Going forward, Commission Staff will continue to consult with ISOs, RTOs and participating public utilities in the voluntary and collaborative data collection process to address ways to minimize the burden of data collection.

Necessity and Practical Utility of Information Collection

Southern states that developing metrics for bilateral markets is not necessary for the Commission to develop proper standardized measures that track the performance of ISO and RTO operations and markets, which is the goal set for the Commission's performance metrics efforts in a Government Accountability Office (GAO) report.[12] EEI does not consider further data collection to be necessary for the Commission to properly perform its functions. EEI suggests that if the Commission believes data collection is necessary, then the Commission should explain the importance of this data to the Commission's functions and the Commission's intentions for using the data. Southern and EEI also consider the practical usefulness of the information to be limited due to the differences in market structures between utilities outside ISO and RTO markets and ISO and RTO market operators. Southern and EEI state that the usefulness of the information is diminished by errors in the Common Metrics Report, arguing that such errors could have been avoided with review and feedback by participating utilities. Southern and EEI also dispute a statement in the Common Metrics Report that utilities outside of ISOs and RTOs have an incentive to discriminate, and EEI stresses that data voluntarily provided to the Commission should not be used to indicate misconduct or used as record evidence in contested proceedings or in enforcement proceedings against entities providing such data. However, EEI states that utilities will continue to provide data voluntarily to assist the Commission in identifying trends or to highlight areas that could be improved through Commission policy. Similarly, Southern notes its intention to continue to coordinate and work with Commission Staff should the Commission continue with this initiative.

FERC Response: The Commission considers it important to compare the performance of ISOs and RTOs with non-ISO and -RTO regions because large portions of the country, notably the Pacific Northwest and the Southeast, have not engaged in restructuring and remain outside of ISOs/RTOs. GAO and other experts were concerned that the benefits of ISOs and RTOs cannot be assessed in isolation, but are best considered in comparison with non-restructured regions.[13] Furthermore, as the metrics developed by Commission Staff seek to glean information across various categories, the Commission aims to assess whether certain particular features of ISOs and RTOs demonstrate superior performance and/or certain (other) features of non-ISO/RTO regions demonstrate superior performance, with a goal of improving the performance of each type of electricity market.

The practical usefulness of the information is not limited by the differences in market structures between utilities outside ISO and RTO regions and between each ISO and RTO market operator. The metrics common to ISOs and RTOs and public utilities in non-ISO/RTO regions measure the performance of reliability and operations functions in which ISOs and RTOs and public utilities in non-ISO/RTO regions perform identical activities, and therefore the common performance metrics provide useful and meaningful information.

The errors and misstatements cited by Southern and EEI do not diminish the practical usefulness of the information submitted because the public record in Docket Nos. AD12-8-000 and AD14-15-000 includes all the correct Start Printed Page 30233information submitted by Southern. The Common Metrics Report of concern to Southern was intended to evaluate whether the common metrics are measuring the same activities and have the same meaning across the industry.[14] Accordingly, the purpose of the report was not intended to be the primary data source. Nor did the mentioned errors and misstatements,[15] have any impact on the common metrics evaluation.

As for the statement in the Common Metrics Report regarding a utility's incentive to discriminate among users of transmission services, this statement has no bearing on the usefulness or quality of the information collected. Southern's and EEI's comments on the potential use of data in enforcement proceedings are also beyond the scope of this data collection notice and are not reflective of the intention of this data collection which is to measure the performance of reliability and operations functions in which ISOs and RTOs and public utilities outside ISO and RTO markets perform identical activities.

Additional Data Collection

APPA, AWEA, and ITC recommend that additional data be collected and reported in order to further improve the usefulness of the performance metrics. ITC does not consider information on transmission facilities approved for construction for reliability purposes to be meaningful without proper context. Southern and EEI state that the proposed common wholesale price metric for ISOs and RTOs and utilities in non-ISO/RTO regions [16] would not provide relevant or useful information since ISO and RTO markets differ significantly from the bilateral markets in non-ISO/RTO regions.

FERC Response: Commission Staff will discuss additional data collection and metrics of interest to commenters, as well as ways to make the metrics more meaningful, in the ongoing voluntary and collaborative process with ISOs, RTOs, participating utilities in non-ISO/RTO regions, and stakeholders.

Start Signature

Dated: May 20, 2015.

Kimberly D. Bose,


End Signature End Supplemental Information


1.  The 2008 GAO Report also recognized that the extent of the Commission's evaluation of ISO/RTO performance may vary from year to year.

Back to Citation

2.  Burden is defined as the total time, effort, or financial resources expended by persons to generate, maintain, retain, or disclose or provide information to or for a Federal agency. For further explanation of what is included in the information collection burden, see 5 CFR 1320.3 (2014).

Back to Citation

3.  ISOs, RTOs and public utilities who wish to file revisions to previously submitted data (i.e., from periods prior to 2010), may do so.

Back to Citation

4.  Public utilities who have not previously submitted performance information may also voluntarily submit data from the 2008-2009 period along with their 2010-2014 submittals, if they believe that such information would be important to this initiative.

Back to Citation

5.  The Commission will provide public notice prior to the due date for any subsequent collection within the approved information collection period.

Back to Citation

6.  The results in this table have been rounded for display purposes.

7.  The estimates for cost per response are derived using the following formula: Average Burden Hours per Response * $XX per Hour = Average Cost per Response. The hourly cost figure for the metrics data collection and writing the performance analysis is based on the loaded average wage (salary plus benefits) of $80.20/hour for an analyst, attorney, engineer, and economist. The hourly cost figure for the management review is based on the loaded average wage (salary plus benefits) of $84.72/hour for management. Wage and benefits data are from the Bureau of Labor Statistics at​oes/​current/​naics2_​22.htm and​news.release/​ecec.nr0.htm.

8.  Assumes responses from the six RTOs and ISOs and five public utilities that previously submitted data.

9.  Assumes five public utilities that have not previously submitted information will submit data. Assumes that four of these public utilities will submit data for the period covering 2010-2014, and that one public utility will voluntarily provide data for 2008-2009 in addition to 2010-2014. The weighted average wage (salary plus benefits) assumed for new respondents is $80.57 per hour, which reflects the hour-weighted average of the wages assumed for entities that have previously submitted performance information.

Back to Citation

10.  The purpose of the additional instruction is to reduce the potential for error in compiling reports on the information submitted.

Back to Citation

11.  NYTOs are Central Hudson Gas & Electric Corporation, Consolidated Edison Company of New York, Inc., Power Supply Long Island, New York Power Authority, New York State Electric & Gas Corporation, Niagara Mohawk Power Corporation d/b/a National Grid, Orange and Rockland Utilities, Inc., and Rochester Gas and Electric Corporation.

Back to Citation

12.  See U.S. Government Accountability Office, Report to the Committee on Homeland Security and Governmental Affairs, U.S. Senate, Electricity Restructuring: FERC Could Take Additional Steps to Analyze Regional Transmission Organizations' Benefits and Performance (Sept. 2008).

Back to Citation

13.  Id. at 56-57.

Back to Citation

14.  Common Metrics Report at 4.

Back to Citation

15.  I.e., an inaccurate listing of Southern's transmission loading relief data as “No Data” instead of zero and a mischaracterization of Southern's transmission planning process as a SERC planning process instead of a Southeastern Regional Transmission Planning Process (SERTP) planning process.

Back to Citation

16.  See Common Metrics Report at 80.

Back to Citation

[FR Doc. 2015-12701 Filed 5-26-15; 8:45 am]