Federal Energy Regulatory Commission, Department of Energy.
Notice of information collection and request for comments.
In compliance with the requirements of the Paperwork Reduction Act of 1995, the Federal Energy Regulatory Commission (Commission or FERC) staff is soliciting public comment on the reinstatement and revision of the FERC-922, Performance Metrics for ISOs, RTOs, and Regions Outside ISOs and RTOs. The Commission is submitting the information collection to the Office of Management and Budget (OMB) for review. Any interested person may file comments directly with OMB and should address a copy of those comments to the Commission, as explained below.
Comments on the information collection are due March 2, 2020.
Comments filed with OMB, identified by OMB Control No. 1902-0262, should be sent via email to the Office of Information and Regulatory Affairs: email@example.com. Attention: Federal Energy Regulatory Commission Desk Officer. A copy of the comments should also be sent to the Commission, identified by Docket No. AD19-16-000, by either of the following methods:
Instructions: All submissions to the Commission must be formatted and filed in accordance with submission guidelines at: http://www.ferc.gov/help/submission-guide.asp. For user assistance, contact FERC Online Support by email at firstname.lastname@example.org, or by phone at: (866) 208-3676 (toll-free), or (202) 502-8659 for TTY.
Docket: Users interested in receiving automatic notification of activity in this docket or in viewing/downloading comments and issuances in this docket may do so at http://www.ferc.gov/docs-filing/docs-filing.asp.
Start Further Info
FOR FURTHER INFORMATION CONTACT:
Darren Sheets may be reached by email at Darren.Sheets@FERC.gov, or by telephone at (202) 502-8742.
End Further Info
Start Supplemental Information
Title: FERC-922 (Performance Metrics for ISOs, RTOs, and Regions Outside ISOs and RTOs).
OMB Control No.: 1902-0262.
Type of Request: Reinstatement of the FERC-922 information collection, with revisions, as discussed in Docket No. AD19-16-000.
Abstract: On July 10, 2019, the Commission published a notice in the Federal Register in Docket No. AD19-16-000 requesting comment on the proposed information collection. The Commission received eight comments on the proposed reinstatement and revision of the FERC-922. Commission staff addresses these comments in this notice and in its submittal to OMB. In addition to addressing the comments received, Commission staff also has updated the attachments to the notice of information collection to reflect the correction of certain typographical and formatting errors.
In September 2008, the United States Government Accountability Office (GAO) issued a report recommending that the Commission, among other actions, work with Regional Transmission Organizations (RTOs), Independent System Operators (ISOs), stakeholders, and other experts to develop standardized metrics to track the performance of RTO/ISO operations and markets and publicly report those metrics.
In accordance with the 2008 GAO Report, Commission staff developed a set of standardized metrics (the Common Metrics), sought and received OMB approval to collect information on those metrics from RTOs/ISOs, and later non-RTO/ISO utilities, and ultimately issued five public reports (Common Metrics Reports).
In December 2017, the GAO issued a report on the RTOs/ISOs with Start Printed Page 5419centralized capacity markets.
Among other recommendations, the GAO found that the Commission should take steps to improve the quality of the data collected for its Common Metrics Reports, such as implementing improved data quality checks and, where feasible, ensuring that RTOs/ISOs report consistent metrics over time by standardizing definitions. Furthermore, the GAO recommended that the Commission develop and document an approach to regularly identify, assess, and respond to risks that capacity markets face.
In response to the 2017 GAO Report, Commission staff has proposed changes to the Common Metrics information collection. First, Commission staff proposes to improve the data collection process by creating a standardized information collection Input Spreadsheet (i.e., the reporting form) and an updated, more detailed User Guide, which will provide guidance on completing the information collection request, including information about who should respond; the timeline for responses; the metrics being collected, including important definitions and a description of the types of metrics and their structure in the information collection; and how to properly use the reporting form. Also, Commission staff proposes to update the list of Common Metrics to focus on centrally-organized energy markets and capacity markets, which involves adding capacity market metrics.
The update eliminates previously-collected metrics on reliability, RTO/ISO billing controls and customer satisfaction, interconnection and transmission processes, and system lambda. Commission staff proposes eliminating these metrics because they provide limited information, do not significantly help Commission staff or the public draw any conclusions regarding the benefits of an RTO/ISO, and to reduce the reporting burden for respondents. The revised data collection, after additions and deletions, consists of twenty-nine Common Metrics.
In addition to eliminating certain metrics and adding new ones, the Common Metrics are now organized into three groups:
- Group 1 metrics are designed to be collected from all respondents (i.e., RTOs/ISOs and non-RTO/ISO utilities). There are seven Group 1 metrics: Reserve Margins, Average Heat Rates, Fuel Diversity, Capacity Factor by Technology Type, Energy Emergency Alerts (EEA Level 1 or Higher), Performance by Technology Type during EEA Level 1 or Higher, and Resource Availability (Equivalent Forced Outage Rate Demand (EFORd)).
- Group 2 metrics pertain to organized energy markets and, thus, are designed to be collected only from respondents with such energy markets (i.e., all RTOs/ISOs). There are twelve Group 2 metrics: Number and Capacity of Reliability Must-Run Units, Reliability Must-Run Contract Usage, Demand Response Capability, Unit Hours Mitigated, Wholesale Power Costs by Charge Type, Price Cost Markup, Fuel Adjusted Wholesale Energy Price, Energy Market Price Convergence, Congestion Management, Administrative Costs, New Entrant Net Revenues, and Order No. 825 
Shortage Intervals and Reserve Price Impacts.
- Finally, the new Group 3 metrics pertain to organized capacity markets and, thus, are designed to be collected only from respondents with such capacity markets (i.e., all RTOs/ISOs with capacity markets). There are ten Group 3 metrics: Net Cost of New Entry (Net CONE) Value, Resource Deliverability, New Capacity (Entry), Capacity Retirement (Exit), Forecasted Demand, Capacity Market Procurement and Prices, Capacity Obligations and Performance Assessment Events, Capacity Over-Performance, Capacity Under-Performance, and Total Capacity Bonus Payments and Penalties.
A table showing the revised Common Metrics organized by the three groups can be found at the end of this notice.
The updated User Guide for the information collection, as well as the standardized information collection reporting form, are also attached to this notice. These attachments will not be published in the Federal Register but will be available as part of this notice in the Commission's eLibrary system under Docket No. AD19-16-000.
Commission staff has assured itself, by means of internal review, that there is specific, objective support for the burden estimates associated with the information collection requirements.
Type of Respondents: ISOs, RTOs, and non-RTO/ISO utilities.
Estimate of Annual Burden: 
Commission staff expects that respondents will submit information on the Common Metrics every two years. Commission staff is requesting a three-year approval from OMB, so the voluntary information collection would happen in Year 1 and Year 3.
The following table sets forth the estimated annual burden and cost 
for this information collection:
Start Printed Page 5420
Explanation of the Table
The Number of Respondents (1) in the first column varies by Group because all respondents do not provide information on each of the twenty-nine Common Metrics.
Commission staff has estimated the number of respondents for the first three Groups based on the assumption that the six jurisdictional RTOs/ISOs and the five non-RTO/ISO utilities (eleven total respondents) that previously responded to the FERC-922 information collection will provide responses to this revised FERC-922 information collection. Therefore, the estimated number of respondents in Group 1 is eleven, because all respondents can report on the Group 1 metrics. The estimated number of respondents for Group 2 is six because only the jurisdictional RTOs/ISOs with energy markets can respond to the Group 2 metrics. Likewise, the estimated number of respondents in Group 3 is four because only the jurisdictional RTOs/ISOs with capacity markets can respond to the Group 3 metrics. Finally, the table includes a burden estimate for potential new respondents. Since all the jurisdictional RTOs/ISOs previously responded to FERC-922, any potential new respondent would be a non-RTO/ISO utility and, thus, would only submit responses to the Group 1 metrics. The burden estimate for new respondents reflects the fact that a potential new respondent would be submitting for the first time, therefore requiring more hours and cost per new response. Commission staff conservatively estimates that one new non-RTO/ISO utility may respond to this revised FERC-922 information collection.
The second column (2), Number of Responses in Years 1 & 3, is characterized by the number of Balancing Authority Areas (BAAs) each respondent would be reporting on, as the respondent would provide a response to each metric for each of its BAAs. Each RTO/ISO is a single BAA and, therefore, will only provide responses to each metric for one BAA, but non-RTO/ISO utilities may report for multiple BAAs. Therefore, the estimated number of responses for Group 1 (all RTOs/ISOs and non-RTO/ISO utilities) is the number of BAAs in the RTOs/ISOs (i.e., six), plus the number of non-RTO/ISO BAAs (i.e., ten), which equals sixteen total responses. The estimated number of responses for Group 2 (all RTOs/ISOs with energy markets) is the same as the number of respondents (i.e., six), as only the RTOs/ISOs respond and they each have only one BAA. The estimated number of responses for Group 3 (all RTOs/ISOs with capacity markets) is the same as the number of respondents (i.e., four), as only the four RTOs/ISOs with capacity markets can respond and they each have only one BAA. Finally, there is only estimated to be one non-RTO/ISO utility as a potential new respondent, which would only respond to the Group 1 metrics that apply to all respondents.
The Annual Frequency of Filings (3) is 0.67 for all groups. This fraction reflects that there will be two information collections, one each during Year 1 and Year 3 of the three-year OMB authorization period. Therefore, 2/3 or 0.67 is the adjustment to reflect an average yearly value for the burden.
The Total Number of Annual Responses (4) is the product of the second column, Number of Responses in Years 1 and 3 (2), multiplied by the third column, the Annual Frequency of Filings (3). Thus, for the first group of respondents, this value is 16 × 0.67, or 10.72.
The Estimated Burden Hours per Response (5) reflects the total number of estimated burden hours, separated into the three reporting categories (collect, write, review) for each group of respondents. The total estimated burden hours for the first 3 groups of respondents are the same (401 hours) as determined in the previous FERC-922 information collection burden estimates. An increased estimate of the burden hours, 427 hours, is for Potential New Respondents, in recognition of the fact that the burden on a new respondent is likely higher. The number of hours in each reporting category has been adjusted in this collection, as compared to the previous FERC-922 collection burden estimate, to reflect less emphasis on the writing category, as Commission staff has developed a structured data collection tool that will decrease the amount of written text that respondents will provide in the information collection.
Start Printed Page 5421
The Estimated Cost per Response (6) is the product of the following three variables: The Estimated Burden Hours per Response (5) for a category, multiplied by the labor rate (wages plus benefits) for each category (which is not shown in the table), multiplied by the proportion of total hours attributable to a given Group that reports on a category, e.g., the number of metrics in that Group divided by the total number of metrics (also not shown in table). An example in the first row is that for Group 1, Metrics Data Collection category, the $5,391 is the product of 271 hours in column (5) multiplied by the weighted average labor rate for that category ($82.42) multiplied by 0.242 (the ratio of metrics in Group 1, 7, to the total number of metrics, 29 or 7 ÷ 29). This fraction is not displayed in the table.
The Estimated Total Annual Burden Hours (7) is the product of the Total Number of Annual Responses (4) times the Estimated Burden Hours per Response (5). For the first row of the first group of respondents, this value is 2,905 hours (or 10.72 × 271 hours).
Finally, the Estimated Total Annual Cost (8) reflects the total burden to the industry and is calculated by multiplying the Total Number of Annual Responses (4) times the Estimated Cost per Response (6) for each category for all groups and produces an estimated total cost in the last row of the table. The wage rates utilized in this burden estimate have been updated to recent Bureau of Labor Statistics estimates for the same categories as used in the prior burden estimates for the FERC-922 information collection (i.e., Computer Systems Analysts, Lawyers, Electrical Engineers, Economists, and the category Chief Executive) in the Electric Power Generation, Transmission, and Distribution industry. Wage estimates use the hourly mean wage from the Bureau of Labor Statistics data, adjusted upward for the private industry benefits of 29.9 percent, and are an average of those categories.
Public Comments and Commission Staff's Responses: Comments were filed by the public in response to the July 10, 2019 notice published by the Commission in the Federal Register, 84 FR 32,908 (July 10, 2019). Commission staff's responses to those comments are provided below.
General Comments on Reinstatement and Revision of FERC-922 Information Collection
In general, commenters, including APPA, California Independent System Operator Corporation (CAISO), Midcontinent Independent System Operator, Inc. (MISO), the PJM Interconnection, L.L.C. Independent Market Monitor (PJM market monitor), the ISO/RTO Council (IRC), Transmission Access Policy Study Group (TAPS), and the Competitive Transmission Developers,
support Commission staff's efforts to reinstate the FERC-922 information collection and to improve it by adding the Group 3 capacity market metrics, and by providing a new User Guide and Input Spreadsheet. APPA further notes its support of Commission staff's proposal to eliminate the metrics on reliability, RTO/ISO billing controls, interconnection and transmission processes, and system lambda. To further improve the value of the information collection, APPA and the Competitive Transmission Developers comment that the metrics collected should not be limited to information that is already collected and published by RTOs/ISOs. APPA and the Competitive Transmission Developers also comment that Commission staff should increase the quality checks it performs on the data submitted in response to the information collection and undertake critical analysis of the data submitted, including identifying opportunities for comparisons between RTOs/ISOs and non-RTO/ISO utilities. IRC requests a reasonable period to submit information in response to the information collection.
Commission Staff Response: Commission staff believes that staff deliberations, combined with significant public outreach, have resulted in the development of twenty-nine Common Metrics, as well as the associated User Guide and Input Spreadsheet, that address many of the concerns raised by the GAO in the 2017 GAO Report, and that will allow for meaningful evaluations of the performance and reliability of RTOs/ISOs and non-RTO/ISO utilities. Commission staff has not limited the information collection to metrics that are already collected and/or published by the RTOs/ISOs or their market monitors. If and when the information collection is approved by OMB, Commission staff will issue a formal request for information, seeking responses to the information collection within ninety days, which staff believes is a reasonable period of time to respond. Once responses are received, Commission staff intends to undertake additional, improved quality checks on the data, as recommended by GAO.
Comments Requesting Modification of Proposed Metrics and Inclusion of Additional Metrics
The CAISO Department of Market Monitoring (CAISO market monitor) requests the addition of four additional Group 2 metrics regarding the efficiency of congestion revenue rights (CRR) auctions. APPA requests additional Group 2 and Group 3 metrics, including: (1) A metric addressing transmission costs comprehensively; (2) a metric addressing whether existing capacity is over- or under-recovering its costs in the RTO/ISO-operated markets; (3) a metric addressing the concentration of ownership of capacity resources; and (4) a metric regarding the participation and profitability of financial entities in RTO/ISO-operated markets. APPA also recommends that Commission staff retain the RTO/ISO governance metric it proposed deleting from the information collection. The Competitive Transmission Developers recommend inclusion of a transmission metric on constructions costs, comparing initial RTO/ISO cost estimates to actual costs at the time the project went into service, and identifying whether a project was competitive or designated to incumbents. In contrast, IRC does not believe that expansion of the metrics beyond Commission staff's proposal is warranted.
APPA recommends substantive changes to Metrics #13, #16, #18, #22, and #25 on the basis that its proposed changes would increase the accuracy of the metrics, increase comparability, or otherwise add useful data to the information collection. TAPS recommends that sub-part ten of Metric #25 be expanded to include data on generation capacity owned by load serving entities, to allow for greater comparability across markets. The PJM market monitor recommends substantive changes to Metrics #3, #5, #6, #10, #11, #16, #19, #20, and #26, on the basis that its proposed changes would enhance the metrics, better align them with the PJM market monitor's own calculations, or otherwise add useful data to the information collection. The PJM market monitor argues that Metrics #13 and #14 are not useful measures of market performance.
Commission Staff Response: Commission staff agrees with IRC that an expansion of these metrics is not warranted at this time. Some of the Start Printed Page 5422additional metrics recommended by commenters may be calculated by certain RTOs/ISOs or non-RTO/ISO utilities but not by others, thus losing the commonality and comparability of the Common Metrics desired by Commission staff. In many instances, commenters have requested further granularity of specific metrics—either at a sub-RTO/ISO level, further divisibility of the metric, or for information based on individual resources or resource owners. However, Commission staff notes that the Common Metrics collection is aimed at data applicable at the RTO/ISO-level or non-RTO/ISO utility-level based on data that could be calculated using “common” methodologies and not designed for granularity at the individual resource or resource owner level or further split in a manner that loses the commonality for each region.
Commission staff also believes that adding some of the proposed additional metrics, without allowing significant time for further research, outreach, and refinement, would be premature. However, staff commits to continuing to research and discuss additional metrics of interest to commenters in the ongoing voluntary and collaborative process with participating RTOs/ISOs and non-RTO/ISO utilities, and to consider adding additional metrics to the next iteration of this information collection.
Commission staff does not agree with APPA that the customer satisfaction metric staff proposed to eliminate should be retained. Historically, responses to this metric have not provided meaningful data, and therefore the metric has served only to increase the reporting burden on respondents. Staff commits to continuing to research and discuss additional metrics of interest to commenters in the ongoing voluntary and collaborative process, which could include organizational effectiveness.
Commission staff has reviewed the substantive changes recommended by commenters to the proposed metrics and has determined not to make significant modifications to the metrics at this time. Among other considerations, Commission staff believes some of the proposed changes: (1) Would significantly increase the data collection and reporting burden on respondents; (2) would undermine the commonality and comparability of certain metrics across RTOs/ISOs and non-RTO/ISO utilities; and (3) do not support the general purpose of the Common Metrics information collection.
Further, Commission staff believes that certain other refinements would be premature to implement at this time, without additional research, outreach, and refinement. However, Commission staff commits to continue discussing ways to improve the metrics and make them more meaningful in the ongoing voluntary and collaborative process with participating RTOs/ISOs and non-RTO/ISO utilities, and to consider additional refinements to the metrics in the next iteration of this information collection.
Comments Requesting Clarification of Proposed Metrics
Commenters, including CAISO, MISO, the PJM market monitor, and IRC, note that certain respondents may not be able to provide responsive information or data for each metric addressed in the information collection, or may collect data in a manner that deviates from the metric as requested. IRC comments that the wording of Metric #17 implies that the ability of RTOs/ISOs to manage the growth rate of administrative costs will be commensurate with the growth rate of system load—a presumption with which IRC disagrees. APPA and TAPS both recommend certain clarifying edits to the description and calculation of Metric #16. Specifically, with regard to Metric #16, TAPS recommends that Commission staff: (1) Clarify the definition of “congestion revenue”; (2) clarify the definition of “congestion charge”; (3) clarify the information to be submitted in sub-part 5 of Metric #16; and (4) clarify that in reporting congestion revenues returned to load, RTOs should take into account all revenues and charges associated with Financial Transmission Rights (FTRs) and Auction Revenue Rights (ARRs). APPA supports TAPS' comments requesting an improved definition of congestion revenue and congestion charges and also recommends reversing the numerator and denominator of the calculation in sub-part 5 of Metric #16. APPA also recommends that Metric #16 document how the payments for FTRs that are purchased in an auction compare to the revenues paid to the instrument holders. The PJM market monitor recommends that Commission staff clarify: (1) The method of calculating new entrant net revenues in Metric #18; (2) the intent of Metric #19; (3) whether Metric #21 is intended to include aggregate import and exports limits for the RTO/ISO as a whole; and (4) whether Metric #24 should be calculated for each Locational Deliverability Area that price separates in PJM and for PJM as a whole.
Commission Staff Response: Commission staff acknowledges that not all respondents will have responsive information for all of the metrics, and that some respondents may calculate certain data responsive to a metric in a way that deviates from that requested due to administrative and/or structural differences across the different RTOs/ISOs and non-RTO/ISO utilities. Commission staff requests that respondents respond as comprehensively and as close to the form requested as possible and simply note and explain in the “Explanatory Text” field for each metric any deviations or omissions.
Commission staff did not intend for the wording of Metric #17 to imply that administrative costs will always be commensurate with the system load growth; therefore, Commission staff has revised Metric #17 to read:
The ability of RTOs/ISOs to manage the growth rate of administrative costs as the growth rate of system load changes.
Commission staff agrees with APPA and TAPS that enhancing the definitions of congestion charges and congestion revenue in Metric #16 would ensure consistent reporting across RTOs/ISOs, and Commission staff has updated the User Guide and Input Spreadsheet accordingly. Commission staff also agrees that adding a line omitted from the original Input Spreadsheet and reversing the numerator and denominator of sub-part 5 of Metric #16 will improve the metric's clarity, and Commission staff has updated the Input Spreadsheet accordingly. Commission staff does not agree that Metric #16 should examine how payments for FTRs that are purchased in an auction compare to the revenues paid to the instrument holders because the Commission does not generally assess the effectiveness of a market by examining how well specific types of market participants are profiting from participation in the market. Commission staff also does not agree that congestion charges should be reported separately for the day-ahead and balancing markets because only Start Printed Page 5423day-ahead congestion is associated with FTRs.
Commission staff recognizes there are varying methodologies for calculating new entrant net revenues in Metric #18 and requests that respondents explain in the Explanatory Text field any clarifications they wish to provide. The intent of Metric #19 is to measure the impact that shortage events will have on reserve market clearing prices. If respondents would like to provide more granular data or improvements to the methodology, these can be submitted in the Input Spreadsheet and described in the Explanatory Text field provided. The intent of Metric #21 is to measure the maximum importable external capacity into a capacity zone for the purpose of resource deliverability in the capacity auction and should therefore focus on imports by zone. Commission staff confirms Metric #24 should be calculated both by zone and for the RTO as a whole.
Finally, Commission staff notes that this information collection is a voluntary, collaborative process. To the extent respondents have outstanding or additional questions about the twenty-nine Common Metrics, including the relevant definitions and calculations, Commission staff is available to provide guidance.
Further Comments Requested
Comments are invited on: (1) Whether the collection of information is necessary for the proper performance of the functions of the Commission, including whether the information will have practical utility; (2) the accuracy of the agency's estimate of the burden and cost of the collection of information, including the validity of the methodology and assumptions used; (3) ways to enhance the quality, utility, and clarity of the information collection; and (4) ways to minimize the burden of the collection of information on those who are to respond, including the use of automated collection techniques or other forms of information technology.
End Supplemental Information
Dated: January 24, 2020.
Kimberly D. Bose,
[FR Doc. 2020-01692 Filed 1-29-20; 8:45 am]
BILLING CODE 6717-01-P