Midterm >

Conclusion >

Footnotes >

Project in Excellence and Leadership:
New England Universities' Laboratories Mid-Term Evaluation:
Piloting Superior Environmental Performance in Labs

Part 2

Section 4 Evaluation Approach and Methodology

4.1 Evaluation Purpose

The project reached the midpoint (September 2001) from the formal date that the FPA was signed by participating parties and the project officially entered the implementation stage. The FPA for the New England Labs project is expected to terminate in September 2003, if there is no extension. The goal of this mid-term evaluation is to garner lessons learned from the unique approach of the NEU Labs project and to highlight opportunities to improve the overall environmental performance for the universities for the remainder of the project. In the course of examining this project over the past two years this paper describes the unique environmental management problems that universities face and ways in which this pilot attempts to ameliorate inefficiencies and make environmental gains. An inherent part of telling the story of the experiences of these universities is detailing the status of the universities in achieving environmental compliance and the superior environmental performance goals of this project. Section 8 of this mid-course evaluation describes how the university specific environmental plans were formulated and implemented.

EPA, the universities, C2E2, and the State agencies are all partners in the evaluation and jointly formulated the objectives and the goal of the evaluation. In the course of formulating an evaluation plan the group also laid out a framework for understanding the NEU Labs project with the use of a logic model. A logic model is a graphical representation of the logical relationships between inputs into the program, outputs of the programs, and what the intended outcomes of the program. The logic model for the NEU Labs project is presented in Figure 1(a full-page logic model is in Appendix 1).

An important element to note in the logic model is the unique partnerships between regulators-both federal and state-and the regulated community that are driving the project activities and outputs. For the two years that this project has been in existence, much good work has been done to complete the infrastructure necessary to support the pilot. The logic model helped visually document and outline the various "front-end" pieces of the project to which a project manager at first glance responded by saying, "we have really accomplished a lot!" This evaluation will help lead to a more targeted focus on the outcomes and recommend some tangible solutions for achieving long-term environmental outcomes such as to increase the understanding of environmental regulations and foster environmental awareness among Administrators, faculty, staff and students.

Figure 1: New England Laboratories XL Project Logic

4.2 Evaluation Methodology

A substantial part of the data collected for this evaluation is qualitative in order to gain an understanding of why and how certain aspects of the project are working and why certain long-term environmental objectives may face realistic hurdles in the laboratory setting. A section of this report is dedicated to describing the EMP in detail-the key document used to manage environmental performance at the universities. EPA, with assistance from its contractor, Industrial Economics, Inc. (IEc.) conducted a conference call on the EMP development and implementation phases with the key EHS staff from the three universities and Nexus Environmental Partners on March 5, 2002. EPA also conducted an additional call on EMP development and implementation with the VT DEC on March 7, 2002 as VT DEC is the primary regulatory agency overseeing UVM's EMP.

In an effort to gauge how well the EMP was working in the laboratories, EPA and IEc conducted group discussions based on a standard set of discussion questions at each of the three universities. The universities invited faculty, administrators and students to the group discussions. The format and questions asked of the groups at each school varied depending on how many participants were present. For example, at UMB some students were stopped in their research laboratories and if students were available, IEc asked them specific questions related to training and the EMP. Summaries of the group discussions and the questionnaire are presented in Appendix 2.

As this sector was not inspected on a regular basis by EPA and was not targeted for enforcement by EPA until approximately 10 years ago, a wealth of information does not exist on "typical" laboratory behaviors under the current regulatory scheme. Therefore, data on the three XL universities was collected soon after EMP implementation through announced audits. The audit information is wholly related to RCRA compliance under 40 CFR Part 262 Subpart J. The audits are preliminary and should serve as a baseline data for future audits for this project under the flexible regulatory scheme. Compliance data on non-XL universities was gathered in an effort to compare to the first year audits performed at the participating universities.

Section 9 examines the progress made in meeting the environmental performance indicators and the superior environmental performance requirements outlined in the FPA. The data for this section was compiled by the three universities in an annual progress report and submitted to EPA New England, as required by the FPA, in 2001 and 2002. The reports are available on the Project XL website at: http://www.epa.gov/projectxl/nelabs.

4.3 Evaluation Utilization

This mid-term evaluation should not be interpreted to be a summative evaluation on the merits of regulatory change for laboratories. The NEU Labs program is an innovative approach to laboratory management not only for the universities but also for the Federal and State regulators involved. This project is just one type of pilot for laboratories and is not intended to exist in a vacuum. This evaluation reviews one of a few pilot approaches being tested throughout the country and will eventually feed information into potential future regulatory innovation. Most importantly, it outlines efforts that EPA, the States, and universities can take to move beyond compliance with regulations to improve overall environmental performance. Although there are inherent difficulties in achieving a holistic evaluation based on short-term data, the findings of this study can be supported by prior work done on laboratory management.

The utility of this mid-term evaluation is in identifying the strengths and weaknesses of the NEU Labs program, offering suggestions for continuous improvement, and creating a system of learning within EPA, the States, and the universities on laboratory innovation. The intended users of this evaluation are not only the three XL universities, but also the larger universe of academic institutions all grappling with similar environmental management and regulatory issues. It is for this larger audience that the evaluation provides more background information on EMP development and structure so that pieces of this NEU Labs experiment, if applicable, may be replicated in other institutions. In addition, this evaluation should be useful to various offices of the EPA, as it will inform a national dialogue on the potential for regulatory reform for academic laboratories.

Section 5 Project Design and Implementation

5.1 The Workings of the New England University Laboratories Project

Figure 2: Diagrammatic representation of New England Laboratories project outline.

5.2 Project Implementation

The implementation of the project has been divided into six phases and is structured in the following manner:

Phase 1: Development of baseline assessment:
Each university conducted a baseline assessment of environmental performance, based on representative data. Baseline environmental performance included identification of hazardous chemicals of concern, measurement of laboratory waste reuse and redistribution and hazardous waste generation data from laboratories.

Phase 2: Development of Laboratory EMP:
Each university, working in collaboration with the federal and State agencies, developed EMPs that included policies, procedures, and practices consistent with the Minimum Performance Criteria and the Laboratory EMS.

Phase 3: Review by Project Signatories and Stakeholders:
The written EMPs were provided to EPA, and the applicable State agency, for review and comment to ensure that the requirements of the Laboratory EMS have been met. A copy of each university's EMP was made available to individual stakeholder groups.

Phase 4: Training and Information:
Each university provided laboratory workers with initial training and information on the EMP and continues the training throughout the life of the project.

Phase 5: Project Implementation:
Each university notified EPA and the relevant state agency in writing when it was prepared to implement its approved EMP. Up until such written notification, RCRA regulations (or the equivalent State regulations) applied in full.

Phase 6: Monitoring, Reporting and Evaluation:
Currently, the universities are in Phase 6 of their project implementation plan. Each university is responsible for collecting data and monitoring its environmental performance using the EPIs selected for the project.

Section 6 Anticipated Superior Environmental Performance

As mentioned above, the NEU Labs project seeks to achieve superior environmental performance, beyond that which is achieved under the current RCRA regulatory system, in three areas, which are further described below:

6.1 Setting of Environmental Objectives and Targets and Pollution Prevention

The systematic approach to environmental management sets the stage for better tracking, control, goals setting and pollution prevention.

(a) General Scheme:
The Laboratory EMS is a significant improvement over the current regulatory requirements in that it requires (i) an institutional commitment in the form of a university policy to prevent pollution, (ii) a procedure for conducting an annual survey of hazardous chemicals of concern (HCOC), and (iii) a better system to reduce the potential for hazardous chemicals to accumulate and become wastes. For example, each university's EMP must include or reference:
* A pollution prevention plan8
* Defined procedures for conducting an annual survey of laboratories that potentially store HCOC and that address removal of expired chemicals
* Defined procedures for conducting laboratory decommissioning (i.e., cleanouts)
* Defined procedures for the timely removal of laboratory wastes from the laboratory

(b) Increased Reuse of Laboratory Waste and Laboratory Waste Reduction:
One area targeted for the demonstration of superior environmental performance is in the identification of laboratory hazardous chemicals and reuse of such chemicals. The current regulatory framework does not encourage researchers to identify chemicals as hazardous, nor does it encourage them to identify opportunities for reuse of the chemicals. This project commits the universities to achieve a 10 percent reduction in waste (from baseline values) and to increase reuse or redistribution of chemicals by 20 percent (from the measured baseline) over the life of the project.

(c) Annual Survey of Hazardous Chemicals of Concern:
A tool included in the EMP to help researchers identify hazardous chemicals will be a required annual survey of hazardous chemicals of concern (HCOC) and a risk evaluation survey of the identified HCOC chemicals in the laboratory. Environmental, Health and Safety (EHS) professionals at each university generate the HCOC list based on regulatory concerns, risk concerns, and potential chemical reactions. The criteria at each university includes:
* Chemicals given an expiration date by the manufacturer due to safety considerations (e.g., peroxide forming chemicals, etc.)
* Chemicals which meet the RCRA definitions of reactive or corrosive and have been determined by professional judgment to present a risk to non-lab workers or the environment
* Poison inhalation hazard designation by the U.S. Department of Transportation (covers serious toxics)
* Other chemicals as determined by professional judgment to present a risk to non-lab workers or the environment
* Chemicals may be removed from the HCOC list if there are insufficient quantities to pose a risk

The HCOC list is developed on a university-by-university basis, because the types of hazardous chemicals at a particular university will vary with the type of research work performed there. This project goes beyond the "waste" management regulations prescribed in RCRA by addressing this particular "upstream" issue at its source. It is expected that by providing regular and consistent data on chemicals and chemical storage, the surveys will support university-wide chemical redistribution and/or the timely disposal of hazardous chemicals that are approaching or have exceeded their shelf life. The survey documents whether the HCOC's remaining on the shelf have been assessed for product integrity.

(d) Ongoing Evaluations and Audits:
Evaluations and audits are performed to help assure conformance with the university's EMP. Together with the enhanced environmental awareness training, internal audits/corrective actions provide a way to continually improve the Laboratory EMS and help achieve improved environmental protection.

(e) Compliance with Other Laws and Regulations:
The universities continue to comply with all other Federal, State and local environmental laws and regulations not specifically deferred pursuant to EPA's site specific rule for this project and the legal mechanisms instituted by Vermont and Massachusetts.

(f) Corrective Action for Non-conformance:
Each university's EMP contains corrective action procedures in the event that non-conformances are observed.

6.2 Streamlining the Regulatory Process to Achieve Better Waste Management

The requirement to define and implement laboratory waste management policies and procedures is aimed at the effective management of laboratory wastes at every stage of their handling and disposition, including full compliance with current RCRA requirements once laboratory waste is received at each university's on-site hazardous waste accumulation area. The Minimum Performance Criteria and the procedures for complying with them were written to ensure that enforceable safeguards would be in place. The result of a streamlined regulatory process is to shift the focus to waste at its generation points and to channel time and resources spent by universities on waste pick-up and handling to include pollution prevention and chemical substitution and reuse.

6.3 Environmental Awareness

Under the existing regulatory system, very little attention is paid to employee training, defining policies and procedures for waste management and handling, and enhancing audit programs and pollution prevention strategies as university staff are often focused on managing laboratories for regulatory requirements. With this project, laboratory workers receive enhanced hazardous chemical training with respect to laboratory waste, pollution prevention, and the environmental practices at the university. The training requirements are outlined in the Laboratory EMS. The universities also hope that the training will instill a long-lasting environmental awareness in its students as they graduate and that will carry with them as they pursue different careers.

Section 7 The Laboratory Environmental Management Standard

The Laboratory EMS consists of 4 components:
1. The Minimum Performance Criteria;
2. Requirements for the University's Environmental Management Plan;
3. Organizational responsibilities for the University; and
4. Training and information requirements for laboratory workers.

7.1 Minimum Performance Criteria

The Minimum Performance Criteria (MPC) are specific requirements, pertaining to laboratory activities and wastes that must be met by laboratory workers to assure that the XL regulation is achieving a level of protection exceeding that achieved by traditional RCRA regulations. The minimum performance criteria for all the participating universities are very similar to RCRA satellite accumulation requirements (40 CFR 262,100-106) and include:

* Labeling
* Threshold waste quantities
* Time limits on holding waste
* Maximum waste quantities that can be in a laboratory
* Container management requirements:

* closure
* condition
* compatibility of contents
* inspection requirements
* prohibition of releases

* Emergency response requirements:

* post notification procedures
* appropriateness of equipment
* notification procedures

* Corrective and preventive actions
* Transfer of laboratory wastes
* Training requirements

7.2 Environmental Management Plan

The Environmental Management Plan (EMP) describes the management system that the universities will implement to achieve compliance with the MPC. The EMP describes services provided by Environmental Health and Safety (EHS) staff, such as collection and management of laboratory chemical waste and tracking of regulatory requirements as well as policies of the university to show their environmental commitment. The EMP also includes management tools such as pollution prevention objectives and targets to help upper management assess the success of the University's hazardous waste management program.

The EMP is written to meet the requirements of EPA and State Agency site-specific regulations for the NEU Labs Project. The goal of the EMP is to continuously improve the universities' environmental performance with regard to the management of chemical wastes from its laboratories. The plan outlines the roles and responsibilities of various groups in the campus community in meeting this goal. It describes specific procedures that laboratory workers and supervisors will follow in order to assure that laboratory waste is properly managed in accordance with the Minimum Performance Criteria of the site-specific Project XL rule. Procedures for identifying and taking advantage of opportunities for hazardous waste minimization and pollution prevention are also included. Finally, it describes the training and information mechanisms that will be used to assure that laboratory workers are aware of and comply with the requirements of this plan.

7.3 Organizational Responsibilities

The organizational responsibilities described in the regulation are the administrative steps that must be taken to assure that the EMP is successfully implemented. These steps are included within the EMP.

7.4 Training and Information Requirements

The training and information requirements stipulated in the regulation assure that laboratory workers and visitors throughout the university are aware of the university's hazardous waste management and pollution prevention program.

7.5 Environmental Performance Indicators

Environmental Performance Indicators (EPIs) were not included in the regulation, but are a critical component of the FPA. The EPIs are measures to track each university's success at meeting its environmental goals with respect to laboratory waste management. As part of the FPA, the environmental performance at the universities is measured against specific environmental goals and the EPIs. The EPIs are classified by type-pollution prevention, compliance (e.g. streamlined regulatory requirements) or environmental awareness. To get an accurate picture of performance the EPIs are compared to baseline assessments completed at each university. The baseline assessments include the following factors:

1. A survey of hazardous chemicals of concern and quantity stored on the shelf in those laboratories covered by this project.

2. A measurement of laboratory wastes generated during a defined time period (e.g., over a six month period).

3. An environmental awareness survey of laboratory workers.

4. An evaluation of the amount of all laboratory wastes currently reused or redistributed (note: each university currently estimates this rate (less than one percent) as consistent with data collected in a 1996 survey of 100 academic institutions by the Campus, Safety, Health and Environmental Management Association.

5. A measurement of costs of compliance that includes available information on waste disposal costs.

The environmental goals and indicators presented below, incorporate the superior environmental objectives and corresponding baseline measures into the overall EPI measurement system. In order to measure whether the universities are reaching their stated environmental goals and achieving superior environmental performance above current performance, the results can be compared to the baseline assessments that were conducted in Phase 1.

Section 8 University-Specific EMPs

The EMP approach is based on holistically managing laboratory operations to improve environmental performance. Since this idea is new to this unique sector of university laboratories, the specifics from each university's EMP are presented in Appendix 3. As the EMP is the guiding document and the implementation plan against which success will be measured, it is equally important to understand the EMP components. The descriptions of the EMPs (in Appendix 3) highlight areas of university administration involvement and structure of how the EMP is administered, definitions of laboratory wastes and goals for reducing or reusing chemical wastes, and unique approaches to HCOCs and training.

Each EMP also reflects the scope and institutional culture of each university. These two factors are important considerations in creating an environmental management system (EMS) and can be considered important drivers of creating an EMS. EHS staff at the three XL institutions provided supplemental information on the EMPs and EMP process that is reflected below. The goal of this section is also to document each university's methodology for developing an EMP and process for EMP implementation so that academic institutions of similar size and culture may replicate this EMS approach to laboratory management.

8.1 Boston College

The EMP at Boston College was completed in July 2000. Boston College developed an environmental policy that commits to compliance and was signed by the President of Boston College. Due to a personnel change in the EHS office, there was some adjustment time with the EMP implementation; however, the EMP was slated for implementation in September 2000. The EMP development at Boston College was an internal exercise to create a performance-based document by pulling together existing policies, information and EHS forms. The EHS office had the main responsibility of crafting the document and relied on one of two committees for review and endorsement of the EMP. The Oversight Committee consists of a cross-section of the faculty and a strong Administration presence, which includes the following individuals: facility managers, research directors, legal advisors, risk managers, research deans, the Vice Provost of Finance, and a community affairs representative. As the panel was very inclusive, controversial issues were channeled to the right people to avoid time-consuming delays later in the process. The second committee active in EMP development was the Chemical Hygiene/Environmental Management Committee, which consists of members of the university's scientific community. This committee is an ongoing vital force that motivates those that work within labs to participate in XL-related activities.

EPA-New England also reviewed the Boston College EMP in accordance with the provisions of the FPA. Comments were provided at meetings and by memo on the Boston College Draft EMP in March 2000. Comments on the final EMP were provided by letter in May 2000.

The EHS staff manages and maintains the EMP, including necessary resources-staff and finances-needed to implement and monitor the EMP. There were no additional financial investment costs incurred for daily expenditures above budget for the EMP. Funding activities for the EMP included hiring one person to complete the HCOC survey, and there are costs that EHS had allocated in its budget for circulating and collecting the awareness survey and other EMP development proceedings, and working with the hazardous waste vendor to assist with laboratory audits.

Once it was implemented, Boston College posted its EMP on its website at: http://www.bc.edu/bc_org/fvp/ehs/emp_partone.html.
The EMP is divided into the following six categories:
(1) Administration;
(2) Laboratory Wastes;
(3) Standard Operating Procedures;
(4) Pollution Prevention;
(5) Surveys of Hazardous Chemicals of Concern; and
(6) Information and Training.

8.2 University of Massachusetts Boston

In 1997, UMB developed an environmental, health and safety policy, signed by the Chancellor of the university, that asserts its commitment to environmental health and safety and clarifies responsibilities for all administrators, faculty, staff, students and committees involved in environmental health and safety issues. The UMB plan focuses on the integration of the CHP and the EMP into a CH/EMP. The CH/EMP contained many elements from the CHP (which already had buy-in from the participating laboratories) so there was a level of familiarity with the document as a whole. The plan was written by the EHS Committee and had the endorsement of the whole group. C2E2 also provided input and assisted in the EMP formulation. The school has fewer labs and faculty relative to the other NEU Labs schools, so it was relatively easy for EHS to build on established relationships and trust and to answer questions in person about EMP development. This factor paved the way for EMP implementation. The Administration, including the Administrative and Finance Chancellors, approved the new CH/EMP document and had committed themselves to the process early on. In addition, the Chairs of the Departments were the champions of the EMP and were able to garner support for the EMP with the faculty. This was an important piece of the EMP implementation process as the Chairs influence and direct daily operations and staff in their departments.

EPA-New England also reviewed the UMB CH/EMP in accordance with the provisions of the FPA. Comments were provided at meetings and by memo on the UMB Draft CH/EMP in March 2000. Comments on the final CH/EMP were provided by letter in May 2000.

There were some minor start-up costs related to EMP implementation, but in general there was no need for additional significant resource inputs to implement the project.

The CH/EMP is divided into five main sections:
(1) common elements of the UMB integrated chemical hygiene and environmental management (CH/EM) plans;
(2) developing standard operating procedures;
(3) operational material safety data sheet;
(4) environmental management; and
(5) pollution prevention.

The complete CH/EMP can be accessed at: http://omega.cc.umb.edu/~ehs/ch_emindex.htm.

8.3 University of Vermont

The UVM EMP consists of a coordinated set of policy statements and implementation plans. The work to develop the EMP was conceptually designed before the FPA for the project was signed. The starting point was identifying waste forms that would apply to the EMP. Environmental Safety Facility (ESF) staff initiated EMP development in September 1999 and a draft was sent to the Vermont Department of Environmental Conservation (VT DEC) for review.

VT DEC is the lead agency for RCRA compliance under the RCRA Partnership Agreement with EPA for oversight of the XL project for UVM and therefore was responsible for approving the EMP. Both EPA-New England and VT DEC reviewed the EMP during its development. VT DEC provided formal comments UVM in accordance with the FPA in April 2000 and EPA-New England provided formal comments to UVM in accordance with the FPA in May 2000.

There is a unique requirement in the FPA for UVM that calls for approval of the EMP by the VT DEC. VT DEC approved the EMP in December 2000. Between 1999 and 2000, UVM and VT DEC had a number of informal discussions to clarify certain EMP elements. UVM faced some personnel changes during development of the EMP that lead to delays in finalizing the document. Before the project was implemented ESF met with both the Vice-President of Administration and the Vice Provost for Research to attain buy-in and support for the project. UVM had a development and review committee with the Chemical and Biological Safety Committee to assist with EMP development.

Each section of the UVM EMP includes assignment of roles and responsibilities, a specific procedure to be followed (described by a form and instructions for its use), and records that will be kept, when necessary. UVM has put forth a Policy for Management of Laboratory Waste that has been signed by the University's president and describes the University's commitment to regulatory compliance, waste minimization, risk reduction and continual improvement of the environmental management system. The EMP relies on a partnership between UVM laboratory workers and ESF staff. The EMP is divided into two main sections-the Laboratory Standard Operating Policies and Procedures and Administrative Policies and Procedures.

The Laboratory Standard Operating Policies and Procedures section is divided into five parts and utilizes elements of the CHP to define laboratory SOPs. The Administrative Policies and Procedures section of the EMP has nine parts that address training, HCOC, laboratory audits, decommissioning laboratories, hazardous waste identification, pollution prevention, tracking legal requirements, document control, and annual review of environmental performance. The EMP can be accessed on the Internet at http://esf.uvm.edu/uvmemp/.

Project in Excellence and Leadership: New England Universities' Laboratories Mid-Term Evaluation: Piloting Superior Environmental Performance in Labs