NAO 216-111: Strategy Execution and Evaluation: NOAA's Performance Management System

Issued 06/11/07; Effective 05/25/12 Reviewed Last: In Process

Attachments

• NAO 216-111
• Clearance Sheet

.SECTION 1.  SCOPE

Strategy Execution and Evaluation (SEE) is established by this Order as NOAA’s performance management system.  It resolves policy and procedure issues in SEE so that it is well understood and engaged in by all relevant stakeholders.  It also authorizes the issuance of related guidance for implementation.

SECTION 2.  PURPOSE

• The purpose of this Order is to codify a process for making well-reasoned, collaborative, transparent decisions that enable NOAA to achieve corporate strategic objectives and annual priorities effectively within estimated funding levels.

• The purpose of SEE is to manage NOAA’s performance toward its Strategic Plan.  Strategy-based performance management is an iterative process of implementation planning, budgeting, execution, evaluation, and the application of evaluation to subsequent planning, budgeting, and execution. 

• SEE is expected to:

• • improve performance management through a system of interdependent, corporate decision-making processes for planning, budgeting, execution and evaluation;

• • • Planning: where NOAA determines what should and can be done within estimated funding levels to accomplish corporate strategic objectives and annual priorities, thereby providing clear direction for subsequent budgeting, execution, and evaluation;

• • • Budgeting:  where NOAA applies the direction from Planning to formulate programmatic and financial proposals, proposes performance targets based on a budget and  justifies  NOAA’s budget request;

• • • Execution: where NOAA allocates resources and executes planned activities to accomplish budgeted performance targets in accordance with corporate strategic objectives and annual priorities, as well as legal, administrative, and policy requirements;

• • • Evaluation:  where NOAA assesses how the agency functions, develops measures of actual outputs and outcomes, and uses those measures to determine how well it performed relative to corporate strategy, fiscal realities, and other factors.

• • strengthen the linkage between strategy and execution by assigning Senior Executives to be a) directly accountable for leading NOAA execution toward  corporate strategic objectives and annual priorities, and b) responsible for associated coordination and integration required for execution;

• • strengthen the linkage across Line and Staff Offices by collaborating to  plan, budget, execute, and evaluate agency activities toward corporate strategic objectives and annual priorities;

• • increase the efficiency of NOAA-wide decision making by planning and executing in the same programmatic units; introducing estimated funding levels in planning; and streamlining decision making processes;

• This policy applies to all NOAA activities.  Specific activities (e.g., facility and fleet construction, operations, and repairs; major systems acquisition; and information technology project acquisition) are subject to additional polices and guidelines.

• SEE must operate simply, clearly, and efficiently to be successful.  SEE must continue to adapt and mature in tandem with changes within and beyond NOAA. 

SECTION 3.  BACKGROUND

SEE is a refinement of the Planning, Programming, Budgeting, and Execution System (PPBES), which NOAA adopted in 2002.  NOAA adopted PPBES to comply with requirements for improved strategic management and performance-based budgeting set forth in the President’s Management Agenda and the Government Performance and Results Act (GPRA).  PPBES was formally adopted with NOAA Administrative Order (NAO) 216-111 (June 2007). NOAA Administrator Lubchenco approved the transition to SEE in a decision memorandum dated September 27, 2010.  SEE is intended to maintain the positive attributes of PPBES while increasing responsiveness to the budgetary, strategic, and performance management requirements of Congress and the Administration.  In particular, SEE differs from PPBES in that it intends to unify the responsibilities for planning, programming and evaluation with those of budgeting and execution.

The Government Performance and Results Modernization Act of 2010, amends GPRA, and places additional conditions on strategic planning, program performance and evaluation for agencies.  The Office of Management and Budget (OMB) Circular A-11 requires agencies to maintain a decision-making process that integrates analysis, planning, evaluation, and budgeting.  OMB Performance Improvement Guidance (June 25, 2010) encourages agencies to use performance information to lead, learn, and improve outcomes; communicate performance coherently and concisely for better results and transparency; and strengthen problem-solving networks, inside and outside government, to improve outcomes and performance management practices.

SECTION 4.  PRINCIPLES

• Fundamental procedural tenants for SEE:

• • SEE is a process to help NOAA be strategic.  SEE is an iterative system of interrelated corporate analyses and decisions to support execution of NOAA’s corporate strategy on an annual to multi-annual schedule pursuant to GPRA-MA requirements and OMB guidance.

• • In SEE, the units we analyze are the units that execute.  Planning and evaluation are conducted using the same programmatic units as budgeting and execution.  Corporate strategic goals and objectives are statements of desired outcomes; they frame how the performance of programmatic units should be managed, but are not programmatic units themselves.

• • To be strategic, we must be able to connect the dots.  That which NOAA plans, budgets, executes, and evaluates are programmatic units’ inputs, activities, outputs, and outcomes; their measurable qualities and quantities; and their causal and interdependent relationships.

• • Senior Executives are accountable for strategy.  It is the responsibility of particular senior executives to lead the agency and coordinate all relevant capabilities toward the accomplishment of corporate strategic objectives.  It is the responsibility of other senior executives to implement and coordinate as needed.

• Fundamental principles embody the values and spirit of this Order to guide its implementation:

• • Alignment and Logic:  Draw clear connections between our mandates, requirements, authorities, organization, strategy, resources, and performance.  Similarly, we must be able to draw cause-and-effect relationships among our inputs, activities, outputs, and outcomes, as well as understand performance at each of these points.

• • Accountability and Coordination:  Be certain which individuals and organizations are accountable for executing which portions of our strategy, and which are accountable for coordinating the planning, monitoring, and evaluating of the activities executed.  Where there is shared accountability, we must enable coordinated work to these ends.

• • Inclusivity and Transparency:  Include multiple perspectives in our corporate decision-making processes, such that leadership and their staff in headquarters offices understand what is really happening “on the front lines,” and so that managers of the programs that perform the mission of the agency can have their voices heard.  Similarly, those beyond headquarters deserve to understand how and why the decisions that affect them are made.

• • Evaluation and Risk Management:  Analyze data from strategic evaluations (progress to plan), program evaluations (all levels), and routine performance monitoring to assess achievement, gaps and identify priorities, and assess risk (changing budgets, inflation, legislative change, technology transition, etc.)

• • Learning and Compliance:  Use performance management to evaluate how NOAA performed toward our strategy, and adjust as needed.  This should be conducted in a way that complies with, but is not limited to, performance management requirements of Congress and OMB.

• • Efficiency and Flexibility:  Minimize duplication of effort, and promote quality reporting products that meet user needs.  Reporting should allow us to quickly understand information and build informational relationships across programs in ways that will add new insight for decision making processes.

SECTION 5.  POLICY
The following are the guidelines for performance management through the SEE process.  Figure 1 (in Section 11, below) depicts the major steps in the multi-annual SEE cycle and related documents. 

• Strategic Planning.  NOAA develops or revises its strategy approximately every 4 years, as directed by the Under Secretary and under relevant laws, which include the Government Performance and Results Act Modernization Act of 2010 that amended the Government Performance and Results Act of 1993.    
  • The Strategic Plan establishes NOAA strategy: a mission and vision for the agency, as well as strategic goals and objectives.  NOAA’s strategy is based on the agency’s statutory and regulatory duties, Administration priorities, Congressional oversight, the demands and concerns of internal and external stakeholders, and assessments of possible developments in NOAA’s internal and external environment over the long-term, such as fiscal realities, trends, and predictions. 
  • Implementation Plans (IPs) are multi-year documents that align specific activities, performance expectations, and resource requirements with corporate strategic objectives, once a new or revised Strategic Plan is in place.  The IP is the central link among planning, execution, and evaluation. It is written once and revised periodically and as necessary to adapt to changing circumstances (performance trends, evaluation findings, legislative or regulatory changes, appropriations, policy changes, etc.).

• Execution Planning.  Planning identifies what should and can be done on a multi-year timescale to achieve NOAA’s strategic objectives based upon a strategic discussion of major policy, performance, managerial, or organizational changes.  Steps in planning are:
  • In the Annual Guidance Memorandum (AGM), focus the agency’s attention on a select number of priorities within the strategic objectives (for the next year and out years) and set resource assumptions for this period;
  • In the IPs and Annual Operating Plans (AOPs), update Line and Staff Office performance expectations and resource requirements over the multi-year timescale, based upon priorities and resource assumptions;
  • In Senior Executive Performance Agreements, set expectations for managerial performance toward strategic objectives and annual priorities, in terms of programmatic results as well as cross-line coordination.
  • In the IPs, develop and assess portfolio options that address priorities within resource assumptions, based upon updated performance expectations and resource requirements, and on a multi-year timescale (the execution year, the following budget preparation year, and five subsequent planning years);
  • In the Corporate Portfolio Analysis (CPA), select a preferred corporate portfolio to direct budget formulation, based upon strategic objectives, annual priorities, and assessment of the portfolio options developed;
  • In the Corporate Portfolio Review (CPR), reassess the corporate portfolio choice, including performance expectations and resource requirements for the next year and out years, based upon Congressional appropriation.
• Budgeting.  Budgeting builds on NOAA’s preferred corporate portfolio established in Planning.  Budgeting includes the detailed resource requests that will be included in NOAA’s annual budget submission that reflects the corporate portfolio decisions, and develops justification documents to support NOAA’s segment of the President’s request.  Budgeting provides a defensible financial plan and is conducted in two steps.
  • Budget Formulation.  This step determines the resource requests that will be included in NOAA’s budget request.  A detailed review of LO/SO budget plans is conducted to ensure ability to execute, alignment with the NOAA Strategic Plan and annual priorities, and the results of multi-year execution planning.  Budget formulation results in NOAA leadership resource allocation decisions and approval of these items that will be included in the NOAA Budget Request Submission to DoC Office of the Secretary, OMB, and Congress.
  • Budget Justification.  The second budgeting step supports the President in meeting the legal requirement to submit a budget to Congress annually.  This step provides required and necessary information to all oversight bodies (DoC, OMB, and Congress) that review the NOAA budget request.
• Execution.  Execution is critical to ensuring that NOAA satisfies assigned statutory and regulatory duties, and delivers the products and services that have been planned, budgeted, and approved though the budget justification process.  Execution includes program performance monitoring, and resource management including people, property, systems and funds. 
  • Performance Monitoring.  This step consists of the responsible LO/SO and program managers ensuring that administrative and programmatic functions are carried out as authorized and appropriated.  Program performance is reviewed using metrics to assess program activities relative to established targets.  The outputs of performance reviews will be used during evaluation.
  • Resource Management.  This step consists of: a) applying appropriate controls to make fiscal resources (e.g. people, property, systems and funds) available to NOAA executing officials, and b) annual financial program closeout activities, performed according to Federal and DOC requirements.
  • Execution Agreements.  To ensure cross-line collaboration toward corporate strategic objectives and annual priorities, formal agreements between operating units may be written and monitored.  These agreements include, but are not limited to, Service Level Agreements (SLAs), funding transfers to support coordinated activities using the existing NOAA policy (06-P0001, signed in 2006), and Cooperative Research and Development Agreements (CRADAs).
• Evaluation.  Evaluation ensures that NOAA learns from execution experience.  Evaluation begins by establishing a program theory, which is a model of how a program causes intended or observed outcomes (Rogers et al., 2000).  Based upon a program theory, NOAA can establish and align performance measures and targets as an empirical means of assessing progress toward strategic objectives.  Evaluation can take one of four forms: evaluation of process, of outcome, of impact, and of benefits relative to costs.  Through evaluation, NOAA can predict, test, and verify causal attributions to see if the program works the way it is intended; identify unknown causes and unanticipated consequences; and make better decisions about whether to continue, halt, or change a program.
  • Program evaluations are individual systematic studies conducted periodically or on an ad hoc basis to assess how well a program (see definition in Section 8) is working to achieve strategic objectives, based on performance and other contextual factors.  Evaluations use measures of program performance, along with other information, to learn the benefits of a program or how to improve it.
  • Performance measurement is the ongoing monitoring and reporting of program accomplishments, particularly progress toward corporate strategic objectives.  Performance measures may address
    • Process: the type or level of program activities conducted
    • Outputs: the direct products and services delivered by a program
    • Outcomes: the results of those products and services
  • Progress-to-Plan (P2P) is a periodic, outcome evaluation of NOAA as a whole to gauge how, and how well the agency has been performing relative to its strategic objectives.  P2P must be performed before any update to the NOAA Strategic Plan, but may occur more frequently if needed.

NOAA’s Line and Staff Offices (LO/SOs) may expand upon this policy with guidance and instructions to meet individual functional and operational requirements. 

SECTION 6.  SUPPLEMENTAL GUIDANCE

• The NOAA Business Operations Manual must be updated regularly to provide detailed descriptions of the processes that support this Order.  Those processes apply to all of NOAA.

• Clear written guidance on SEE deliverables, dates, and deadlines is issued annually by the organizations with primary responsibility for the SEE process.

SECTION 7.  RESPONSIBILITIES

• All NOAA Senior Executives, per OMB Performance Improvement Guidance (June 25, 2010), must set specific outcome-focused goals, measure progress toward meeting the goals, track completion of key milestones, compare progress among peers to identify better practices, look for factors that government can influence and that affect trends, adopt and implementing cogent strategies based on analysis of performance and other relevant data, use performance data to confirm achievement of intended outcomes, make quick adjustments to strategies when they are not working, and report to the public in useful and accessible ways – candidly, coherently, and concisely.

• The Under Secretary schedules periodic reviews of the NOAA Strategic Plan and approves the NOAA Strategic Plan, Annual Guidance Memorandum, Corporate Portfolio Analysis, Corporate Portfolio Review, and Budget Submission.

• Assistant Secretaries and the NOAA Chief Scientist act on behalf of the Under Secretary to lead Line and Staff Office coordination within their functional domains of authority and through their leadership of NOAA’s Councils.

• The Deputy Under Secretary for Operations (DUS-O) must:
  • manage corporate strategy development, planning, budgeting, execution, and evaluation;
  • issue guidance for Annual Operating Plans;
  • designate Assistant Administrators of NOAA Line Offices (LOs), or Directors of NOAA Staff Offices (SOs) as “Champions” that are accountable for supporting the requirements of this policy and leading agency-wide planning, budgeting, execution, and evaluation for corporate strategic objectives;
  • assess the performance of Champions based on the quality of their leadership of agency-wide planning, monitoring, and evaluation for corporate strategic objectives; and
  • assess the performance of all NOAA Senior Executives, including Assistant Administrators of LOs and Directors of SOs, based on their commitment to execute programs under their authority per corporate planning, budgeting, and evaluation decisions, Implementation Plans, Annual Operating Plans, and cross-line execution agreements.
• NOAA Champions are either Assistant Administrators (AAs) of NOAA Line Offices (LOs) or Directors of NOAA Staff Offices (SOs) designated by the DUS-O as accountable for leading agency-wide planning, budgeting, execution, and evaluation for one or more corporate strategic objectives. 

This order establishes championship as a potential responsibility for all AAs and SO Directors, who, as assistants to the Administrator, have a duty to act in the interest of NOAA as a whole, not only their individual offices.  If the AA or the SO Director is a Champion, he or she must:

• • support the requirements of this policy and apply SEE within their functional and organizational areas of responsibility;

• • designate a Strategy Lead and a Budget Lead for their assigned corporate strategic objectives;

• • lead planning, budgeting, execution and evaluation for their assigned corporate strategic objectives, in coordination with other Senior Executives, as appropriate;

• • prepare Implementation Plans and Annual Operating Plans, and execute the programs under their authority according to these plans;

• • monitor execution of funds and ensure programs are properly managed in the interest of corporate strategic objectives assigned to them;

• • report the status of program execution toward corporate strategic objectives assigned to them by the DUS-O on a semi-annual basis;

• • perform program evaluations at the request of the DUS-O, Assistant Administrator for PPI, or NOAA CFO to assess how well the corporate strategic objectives assigned to them are being satisfied;

• • serve as principal coordinators to ensure a consistent NOAA message to external partners and stakeholders (i.e., local, state, regional, national, and international) for the assigned objectives.

• • support NOAA Councils as venues for offices to coordinate decision-making on issues of corporate planning, budgeting, execution, and evaluation; and

• • consult with the DUS-O on the performance of other NOAA Senior Executives  based upon their commitment to execute programs under their authority per corporate planning, budgeting, and evaluation decisions, Implementation Plans, and cross-line execution agreements.

• Strategy Leads must:
  • support the requirements of this policy and apply SEE within their assigned organizational responsibilities;
  • coordinate planning and evaluation for their assigned strategic corporate objectives with Budget Leads, other strategy leads, and other stakeholders; and
  • assist their respective Champion to ensure a consistent NOAA message to external partners and stakeholders for the assigned objectives.
• Budget Leads must:
  • support the requirements of this policy and apply SEE within their assigned organizational responsibilities;
  • lead budgeting activities, coordinating with Chief Financial Officers across LOs and SOs as necessary, to support the corporate strategic objectives for which their designating Senior Executive is accountable, consistent with Department and government-wide Budget guidance.;
  • coordinate with the Strategy Leads to develop Implementation Plans, Annual Operating Plans, and performance measures, and evaluate programs accordingly; and
  • report program performance.
  • Assistant Administrators and Staff Office Directors, must:
  • support the requirements of this policy and apply SEE within their assigned organizational responsibilities.
  • Coordinate with Champions to plan, budget, and evaluate the programs under their authority per NOAA’s corporate strategic objectives and annual priorities.
  • execute programs under their authority per corporate planning, budgeting, and evaluation decisions, Implementation Plans, Annual Operating Plans, and cross-line execution agreements.
• The Assistant Administrator for Program Planning and Integration (PPI) must:
  • manage corporate-level planning and evaluation through SEE, promoting transparency and cross-line dialog where it is needed;
  • lead the development of the Strategic Plan, Annual Guidance Memorandum, Corporate Portfolio Analysis, and Corporate Portfolio Analysis Decision Memorandum;
  • guide development of the Implementation Plans;
  • coordinate NOAA’s corporate performance management with the NOAA Chief Financial Officer (CFO);
  • maintain the Business Operations Manual, which will detail NOAA’s business practices for SEE and other fundamental corporate functions to help NOAA personnel conduct them uniformly;   
  • issue clear, written guidance on planning products and processes as necessary;
  • consult with the DUS-O on the performance of Champions based upon the quality of their leadership of agency-wide planning, budgeting, execution, and evaluation for corporate strategic objectives; and
  • consult with the DUS-O on the performance of  NOAA Senior Executives  based upon their commitment to execute programs under their authority per corporate planning, budgeting, and evaluation decisions, Implementation Plans, Annual Operating Plans, and cross-line execution agreements.
  • conduct evaluations of SEE to improve its effectiveness and reduce its administrative costs

• The NOAA Chief Financial Officer (CFO) must:

• • manage Budgeting during SEE;

• • manage the financial and program close-out elements of Execution;

• • use corporate strategy (per the NOAA Strategic Plan) and annual priorities (per the Annual Guidance Memorandum) as the primary means to frame and focus development and submission of budget and annual performance plans to DoC, OMB and Congress and subsequent defense, reviews, and reports;

• • in coordination with the NOAA Office of Legislative and Intergovernmental Affairs, use corporate strategy (per the NOAA Strategic Plan) and annual priorities (per the Annual Guidance Memorandum) as the primary means to frame and focus interaction with the Legislative Branch during the House and Senate mark-ups of the Congressional submission;

• • use corporate strategy (per the NOAA Strategic Plan) and annual priorities (per the Annual Guidance Memorandum) as the primary means to frame and focus communication to senior leadership of the results of Budgeting, outcomes of the formulation process as well as interactions with DoC, OMB, and Congress during all phases of Budgeting as appropriate;

• • lead the Corporate Portfolio Review, and prepare and communicate the results of the Corporate Portfolio Review after Congressional appropriation;

• • coordinate NOAA’s corporate performance management with the PPI Assistant Administrator; and

• • review and manage strategic performance documents.

• Directors of NOAA offices responsible for stakeholder engagement, including the  Offices of Policy, Legislative and Intergovernmental Affairs, Communications and External Affairs, and Strategic Initiatives and Partnerships, must:

• • use corporate strategy (per the NOAA Strategic Plan) and annual priorities (per the Annual Guidance Memorandum) as the primary means to frame and focus engagement with key stakeholders; and

• • provide feedback to Champions and the Assistant Administrator for PPI on the perspectives of key stakeholders with respect to the development and execution of corporate strategy and annual priorities.

• The NOAA Executive Council (NEC) provides information and counsel to the Under Secretary throughout the SEE process. 

• The NOAA Executive Panel (NEP) provides information and counsel to the DUS-O and the NEC throughout the SEE process. 

• All other NOAA Councils are, by their very nature, bodies intended for multi-line discussion and decision.  As such, they must:

• use corporate strategy as the primary means to frame and focus their proceedings;

• provide a venue for  Champions (or their designates) to coordinate the planning, budgeting, execution and evaluation toward corporate strategic objectives, including coordination needed for Champions to produce associated documentation; and

• advise the NEP and NEC, as appropriate, throughout the SEE process.

SECTION 8.  DEFINITIONS

• Accomplishment – a key outcome or output of NOAA activities that demonstrates progress towards its corporate strategic objectives. 

• Activity – any process through which NOAA uses inputs to generate outputs.  NOAA’s activities represent what NOAA needs to do in order to achieve its corporate strategic objectives. 

• Budgeting – the process through which NOAA applies the direction from Planning to formulate programmatic and financial proposals, proposes performance targets based on a budget and justifies  NOAA’s budget request;

• Champions – either an Assistant Administrators (AAs) of a NOAA Line Office or a Directors of a NOAA Staff Offices who has been designated by the DUS-O as accountable for leading agency-wide planning, budgeting, execution, and evaluation for one or more corporate strategic objectives.

• Council – an advisory body composed of representatives of multiple Line or Staff Offices.  Councils are formally established to provide a venue for coordination and decision-making on cross cutting issues, particularly corporate planning, budgeting, execution, and evaluation. 

• Enterprise – an entity comprised of interdependent resources (e.g., people, processes, organizations, technology, funding) that interact with each other (to, e.g., coordinate functions, share information, allocate funding) and their environment to achieve goals. Enterprise and its boundary are virtual constructs that depend on the make-up, authority, and roles of the participating actors in a community of interest. Enterprises exhibit attributes of a complex adaptive system: they are evolutionary, emergent, adaptive, self-organizing, competitive and cooperative. (Rebovich, 2006)

• Evaluation – the process through which NOAA assesses how the agency functions, develops of measures of actual outputs and outcomes, and uses those measures to determine how well it performed relative to corporate strategy, fiscal realities, and other factors.

• Execution – the process through which NOAA allocates resources and executes planned activities to accomplish budgeted performance targets in accordance with corporate strategic objectives and annual priorities, as well as legal, administrative, and policy requirements;

• Function – Functions are required to execute the mission, consistent with the NOAA Functional Model (approved by the NEC on February 6, 2007 as an accurate and comprehensive high-level depiction of the work of NOAA).  NOAA’s functions are the highest-level categorization of NOAA’s activities and are comprehensive—that is, all activities conducted by NOAA can be traced to a function.  In this manner, all contributors to NOAA’s mission can see how their activities support the plan.  While activities are the particular things that NOAA does, functions are broad categories of these activities.

• Goal – Goals specify the components of NOAA’s vision, translating the vision into a limited number of high-level results that NOAA will seek to achieve.  NOAA’s strategic goals are outcome-oriented—that is, they specify future social, economic, and environmental conditions that the agency is committed to achieving, and how society will benefit from NOAA’s success.  The timeframe for NOAA’s strategic goals is multi-decadal.

• Input – Resources (funds, labor, time, equipment, space, technology etc.) used to produce outputs and outcomes.

• Mission – NOAA’s mission summarizes the agency’s fundamental mandates and responsibilities.  It is a succinct and distinctive statement of what NOAA does.  The mission statement encapsulates the set of statutory requirements that drive NOAA’s functions.  NOAA’s mission is assumed to be stable over the planning period.

• Monitor – A continuing function that uses systematic collection of data on specified indicators to provide scientific and other staff, management, partners, and stakeholders with indications of the extent of progress, achievement of objectives, and use of allocated funds for a program or project (adapted from OECD, 2002).

• Objective – Objectives further describe strategic goals or enterprises by detailing the societal, environmental, or organizational benefits that NOAA seeks to achieve in the five year time frame. Objectives toward goals are outcomes for society and the environment, whereas objectives toward enterprises are outcomes for NOAA to achieve its goals. Objectives should be specific, measureable, attainable, realistic, and time-bound (SMART). 

• Outcome – the result of the products and services delivered by a program.  Outcomes define an event, condition or state of being that is external to the program and that is of direct importance to the intended beneficiaries and/or the public.  For a tornado warning system, outcomes could be the number of lives saved and property damage averted.  Outcomes may be long-term, mid-term, or short-term in nature.  NOAA’s strategic vision, goals, and objectives are all outcomes at different levels of specificity and over different time horizons.

• Output – the direct products and services delivered by an activity. 

• Performance Management – A systematic approach to performance improvement through an ongoing process of establishing strategic performance objectives; measuring performance; collecting, analyzing, reviewing, and reporting performance data; and using that data to drive performance improvement (DOE 2001).  Performance information is used to inform priorities, goals, lead the execution, learn to improve outcomes of agencies performance information is used to communicate results and strengthen problem solving network (OMB 2011). Performance management includes both performance measurement and program evaluations.

• Performance measure – Indicators, statistics, or metrics used to gauge program performance.  It is a structured statement that describes the means by which actual outcomes, outputs, and processes are measured against planned outcomes, outputs and processes.

• Planning – The formal process of establishing missions, goals and objectives (strategic planning) and describing how the goals and objectives are to be achieved by establishing performance expectations and resource requirements (implementation planning).

• Portfolio – A set of investments that yield benefits, have costs and associated risks.  Through management of a portfolio, NOAA can explicitly assess the tradeoffs among competing investment opportunities in terms of their benefits, costs, and risks.

• Priority – Priorities represent choices among a set of options.  Specifically, a priority is an activity, input, output or outcome for which performance has been determined to take precedence over others within the same time period.

• Program –any organized set of activities directed toward a common purpose or goal that an agency undertakes.  The term may describe an agency’s mission, functions, activities, services, projects, and processes, and is defined as an organized set of activities directed toward a common purpose or goal that an entity undertakes or proposes to carry out its responsibilities (Circular A-11, 2010)

• Programmatic Unit: a division of the organization below the Line or Staff Office level (such as an office or a laboratory) that has a manager with the authority, responsibility, and resources for executing particular activities.

• Project – any undertaking of a temporary nature (e.g., research, assessment, prediction, acquisition, or stewardship efforts) designed to create a service, product, system, and/or system upgrade in support of a validated NOAA Mission Requirement.  A NOAA project is intended to address a shortfall within defined budget and schedule constraints and terminates when the service, product, or system achieves full operational capability (FOC), when the capability gap is resolved or, in coordination with other programs, when directed by the project manager or higher authority.

• Program Evaluation – individual studies conducted periodically or on an ad hoc basis to assess how well a program is working.  Program Evaluations are often conducted by experts external to the program, inside or outside the agency, as well as by program managers.  They help managers determine if timely adjustments are needed in program design to improve the rate, or quality, of achievement relative to the committed resources.  There are four main types of evaluation, including outcome evaluation (what the program accomplished), impact evaluation (net effect of the program), process evaluation (extent the program is operating as intended), and cost-benefit/cost-effectiveness evaluation (GAO, 2005; DOE, 2009). 

• Strategic Plan – a plan that identifies what NOAA should produce in the future (i.e., outputs), and why those are important (i.e., outcomes).  Distinguishing between outcomes and outputs gives flexibility to change agency activities while staying true to its overall purpose.

• Strategy – explains what the agency intends to do and why it intends to do it.  It relates an statement of activity and output (e.g., mission, functions or activities) to a statement of outcome (e.g., vision, long-term strategic goals or objectives) to succinctly convey NOAA’s fundamental purpose, direction, and value to society.

• Vision – An envisioned future state of society and the environment that, implicitly, cannot be achieved without NOAA. The vision describes long-term success in terms of the value that NOAA will generate for society—in effect, why the agency exists. The timeframe for NOAA’s vision is multi-decadal.

SECTION 9.  REFERENCES

• Matlock, G., Abreu, M., Goedeke, T., & Sen, A. (2009). An Evaluation of NOAA’s Planning, Programming, Budgeting, and Execution System (PPBES). A report to the Executive Panel of the National Oceanic and Atmospheric Administration (unpublished).
• National Oceanic and Atmospheric Administration, U.S. Department of Commerce. (2010). NOAA's next generation strategic plan. Silver Spring, MD. http://www.ppi.noaa.gov/wp-content/uploads/NOAA_NGSP.pdf
• Organization for Economic Co-operation and Development (2002).  Glossary of Key Terms in Evaluation and Results Based Management.  OECD Publications, Paris, France.  http offsite link:// offsite linkwww offsite link. offsite linkoecd offsite link. offsite linkorg offsite link/ offsite linkdataoecd offsite link/29/21/2754804. offsite linkpdf offsite link
• Rogers, P. J., Petrosino, A., Huebner, T. A., & Hacsi, T. A. (2000). Program theory evaluation: Practice, promise, and problems. New Directions for Evaluation, 2000 (87), 5-13.
• U.S. Congress (1993). Government Performance and Results Act.
• U.S. Congress (2010). Government Performance and Results Modernization Act (GPRA-MA).
• U.S. Department of Energy (2009). Energy Efficiency and Renewable Energy – Glossary.  http://www1.eere.energy.gov/ba/pba/program_evaluation/glossary.html
• U.S. Government Accountability Office (2005). A Glossary of Terms used in the Federal Budget Process. United States Government Accountability Office. http://www.gao.gov/new.items/d05734sp.pdf
• U.S. Office of Management and Budget (2010). Circular No. A-11. Preparation, Submission, and Execution of the Budget.
• U.S. Office of Management and Budget. (2010).  Performance Improvement Guidance.
• U.S. Office of Management and Budget (2011). Economic Perspectives Budget of the US Government. Fiscal Year 2011. http://www.gpoaccess.gov/usbudget/fy11/pdf/spec.pdf

West, W. F., Lindquist, E., Mosher-Howe, K. N. (2009). NOAA’s Resurrection of Program Budgeting: Déjà Vu All Over Again? Public Administration Review, 69 (3), 435-447.

SECTION 10.  FIGURES

Figure 1: The SEE Cycle. This figure depicts the major steps in the multi-annual SEE cycle and the documents associated with each step.  Note: the development and revision of the NOAA Strategic Plan occurs outside of this cycle.  See Section 5 (above) for details.

Figure 2: Governance of SEE.  Champions Lead via IPs and Execution Agreements, collaborating AAs and SO Directors Execute via AOPs, DUS-O enforces via SES Performance Plans, and Councils serve as forums for coordination and collaboration.  See section 7 (above) for details.

SECTION 11.  EFFECT ON OTHER ISSUANCES
 
This Order supersedes NAO 216-111 on the NOAA Planning, Programming, Budgeting, and Execution System.

An electronic copy of this Order will be posted in place of the superseded Order on the NOAA Office of the Chief Administrative Officer website under the NOAA Administrative Issuances Section. 

                                               
________/Signed/____________­­­­­____
Under Secretary of Commerce
for Oceans and Atmosphere

Offices of Primary Interest:

Office of the Chief Financial Officer (OCFO)