63L-1.001: Purpose and Scope
63L-1.002: Definitions
63L-1.003: Quality Assurance Standards
63L-1.004: Quality Assurance Peer Reviewers
63L-1.005: Conducting Quality Assurance Reviews
63L-1.006: Challenges and Mediation
63L-1.007: Waivers and Alternative Compliance Measures
63L-1.009: Conditional Status
63L-1.010: Failure to Meet Minimum Levels of Performance or Compliance
63L-1.011: Internal Review Board
63L-1.012: Quality Assurance Reporting Requirements
PURPOSE AND EFFECT: Establishing the standards and requirements for the department’s statewide quality assurance system.
SUMMARY: Quality assurance standards are referenced for each type of program, and a process is established by which programs are reviewed. Consequences for noncompliance or failure to meet minimum levels of performance are described, and an internal review process is established.
SUMMARY OF STATEMENT OF ESTIMATED REGULATORY COSTS: No Statement of Estimated Regulatory Cost was prepared.
Any person who wishes to provide information regarding a statement of estimated regulatory costs, or provide a proposal for a lower cost regulatory alternative must do so in writing within 21 days of this notice.
SPECIFIC AUTHORITY: 985.632, 985.64 FS.
LAW IMPLEMENTED: 985.632 FS.
IF REQUESTED WITHIN 21 DAYS OF THE DATE OF THIS NOTICE, A HEARING WILL BE HELD AT THE DATE, TIME AND PLACE SHOWN BELOW:
DATE AND TIME: Monday, March 19, 2007, 10:00 a.m.
PLACE: DJJ Headquarters, 2737 Centerview Dr., Room 108, Tallahassee, Florida
THE PERSON TO BE CONTACTED REGARDING THE PROPOSED RULE IS: John Milla, 2737 Centerview Dr., Ste. 312, Tallahassee, FL 32399-3100, e-mail: john.milla@djj.state.fl.us
THE FULL TEXT OF THE PROPOSED RULE IS:
63L-1.001 Purpose and Scope.
The rule establishes the standards and requirements for the department’s statewide quality assurance system.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.002 Definitions.
(1) Acceptable Performance Rating – A quality assurance rating at the standard and overall performance levels between 70%-79%.
(2) Alternative Compliance Measure – The use of a process, procedure, or practice not outlined in DJJ policy, to accomplish the same objective required in the applicable department policy, as long as the request does not negatively impact staff safety, youth safety, security, treatment, conditions of confinement, public safety, the officer certification process, or another branch of the department.
(3) Assistant Secretary – The person responsible for the management and operation of the respective branch of the department.
(4) Chief Probation Officer – The person responsible for the oversight of community corrections programs in the respective circuit.
(5) Chief of Quality Assurance – The person responsible for the management and operation of the Bureau of Quality Assurance.
(6) Compliance Key Indicator – A key indicator rated based on whether a program’s policies, procedures, and practices comply with department policy, law, or administrative rule. Compliance key indicators are rated “full compliance,” “substantial compliance,” or “non-compliance.”
(7) Conditional Status – The status of any program that fails to meet the minimum levels of performance of any standard in the Quality Assurance process.
(8) Contract – A formal written agreement between the department and an individual or organization for the procurement of services.
(9) Contract Manager – The employee of the department who is responsible for enforcing performance of the contract terms and conditions, and serves as a liaison between the department and the contract provider.
(10) Contract Provider – The entity under contract with the department to provide services to youth.
(11) Exit Conference – A formal meeting at the end of a quality assurance on-site review in which the quality assurance team members provide the preliminary findings of the QA review to the management and staff of the program.
(12) Failure to Meet Minimum Levels of Performance – A quality assurance performance score at the standard or overall performance levels between 0% to 59%.
(13) Full Compliance – The program is predominantly meeting the requirements of the key indicator the vast majority of the time. A slight exception may be noted with either policy, procedure, or practice.
(14) Internal Review Board – The panel empowered to recommend to the Secretary of the department for or against cancellation of a contract for failure to meet the minimum levels of performance for a second consecutive time during a 6-month period.
(15) Juvenile Justice Information System (JJIS) – The Juvenile Justice Information System (JJIS) is a computer application that allows the Department of Juvenile Justice (DJJ) to collect information and generate reports on youth under its custody. Information is processed through assessment centers, detention centers and interaction with juvenile probation officers and commitment managers. This information allows the DJJ to track the location and status of youth assigned to the custody of DJJ.
(16) Key Indicator – The basic level of evaluation in the quality assurance system. Key indicators are used to determine if a program is meeting a standard.
(17) Lead Reviewer – The Bureau of Quality Assurance professional staff member who plans, organizes, directs, and supervises a quality assurance on-site review. The lead reviewer compiles all data from the review and completes the quality assurance program report.
(18) Minimal Performance – A quality assurance rating at the standard and overall performance levels of 60% to 69%.
(19) Monitoring – The acquisition, review, and reporting of information about provider compliance with the terms and conditions of the contract.
(20) Non-compliance – A compliance rating indicating that the program is not meeting the requirements of the key indicator or the policy. Numerous deficiencies were noted in policy, procedure, or practice.
(21) Non-performance – A performance rating indicating the items, elements, or actions necessary to accomplish the indicator are missing or are done so poorly that they do not contribute to the accomplishment of the indicator or the overall standard. The rating will be a 0.
(22) Partial Performance – A rating indicating that a program is not consistently accomplishing policy requirements. Frequent deficiencies in the policy occur or the program is ineffective in implementing the policy. The rating will be a 4 or 5.
(23) Peer Reviewer – A juvenile justice professional at the supervisor/manager level or above who has been certified as a quality assurance peer reviewer and who has been chosen to participate as a quality assurance team member.
(24) Program – Any state operated or contracted general revenue funded prevention or diversion program, residential commitment program, Sheriff’s Training and Respect (STAR) program, juvenile detention center, community corrections program, Day Treatment Program, Children in Need of Services/Families in Need of Services (CINS/FINS) Program, PACE Program, Juvenile Alternative Service Program (JASP), or Intensive Delinquency Diversion Services (IDDS) program operated by or under contract with the department as listed in the JJIS.
(25) Program components – The key programming areas defined for the different program models used to determine a score. The key programming areas defined for the different program models include:
(a) Residential Commitment programs and STAR programs:
1. Management Accountability;
2. Screening, Assessment, and Orientation;
3. Treatment Services;
4. Healthcare Services;
5. Food Services; and
6. Conditional Release if applicable.
(b) Detention:
1. Management Accountability;
2. Screening and Classification;
3. Treatment Services;
4. Healthcare Services; and
5. Food Services.
(c) Children in Need of Service and Families in Need of Services (CINS/FINS):
1. Management Accountability;
2. Centralized Screening and Intake;
3. Temporary Shelter Care;
4. Healthcare Services; and
5. Food Services.
(d) Day Treatment:
1. Management Accountability;
2. Screening, Classification, and Orientation;
3. Treatment Services;
4. Healthcare Services;
5. Food Services; and
6. Conditional Release.
(e) Probation and Community Corrections:
1. Administration: program management and training and staff development;
2. Core Services: detention screening, intake, case management and supervision of committed youth in residential care, probation supervision, and conditional release.
(f) Intensive Delinquency Diversion Services (IDDS):
1. Management, Leadership and Community Relations; and
2. Case Management Service Plans.
(g)
1. Administration;
2. Program Management, Training and Staff Development;
3. Core Services: admissions, social services and treatment planning, and health services;
4. Safety: sanitation and emergency procedures; and
5. Educational Services.
(h) Juvenile Alternative Services Program (JASP):
1. Management, Leadership and Community Relations; and
2. Case Management Diversion Plans.
(26) Program Manager and Supervisor – The program director or corporate official who is responsible for the operation of residential and correctional facilities, detention superintendents who are responsible for the operation of detention centers, program directors or corporate officials who operate or are responsible for the operation of community corrections programs, and juvenile probation officer supervisors who are responsible for the management and supervision of community corrections programs.
(27) Program Monitor – The department employee who provides contract oversight for department-operated and contract provider-operated programs.
(28) Quality Assurance – The process for the statewide assessment of the performance of program operations, management, governance, and service delivery based on an established set of standards.
(29) Quality Assurance Standards – The established set of standards used to evaluate program performance in department operated or contracted programs.
(30) Regional Director – The department employee responsible for the oversight of residential and correctional programs, probation and community corrections, and detention centers in a respective region.
(31) Re-review – A quality assurance review conducted at a program within 6 months when a program has failed to meet the minimum levels of performance set by the department.
(32) Residential Program – Any residential commitment program including STAR programs, Intensive Residential Treatment Programs, Sex Offender Programs, and Serious Habitual Offender Programs.
(33) Safety and Security Monitor – The designated department employee responsible for monitoring safety and security in juvenile justice residential and correctional facilities.
(34) Satisfactory Performance – A quality assurance rating that indicates the program consistently accomplishes all policy requirements in an effective manner. The items, elements, or actions necessary to accomplish the policy are prevailing practice though minor deficiencies may occur occasionally. The rating will be a 6, 7, or 8.
(35) Substantial Compliance – A quality assurance rating that indicates the program is not meeting all the requirements of the key indicator. There were some deficiencies in either policy, procedure, or practice. The rating will be a 5.
(36) Superior performance – A quality assurance rating that indicates the program consistently exceeds all policy requirements with either an innovative approach or an exceptional, program-wide dedication to performance that is readily apparent. There is evidence of very few, if any, exceptions to this. The rating will be a 9 or 10.
(37) Waiver – The authorization by the Secretary of the department to allow a program, facility or service to forego a specific department policy or a portion thereof when implementation of the waiver does not negatively impact staff safety, juvenile safety, security, treatment, conditions of confinement, public safety, the officer certification process, or another branch of the department.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.003 Quality Assurance Standards.
(1) Quality assurance standards shall reflect state and federal laws, administrative rules, and, as incorporated herein, departmental policies, procedures, and manuals relating to each program type. The following quality assurance standards are incorporated by reference and are available at the department’s website (http://www.djj.state.fl.us/QA/ index.html):
(a) Children/Families in Need of Services (CINS/FINS) Programs, revised 12/8/2006.
(b) Community Corrections Programs, revised 11/2004.
(c) Day Treatment Programs, revised 12/8/2006.
(d) Detention Programs, revised 12/8/2006.
(e) Intensive Delinquency Diversion Programs (IDDS), revised 11/1/2006.
(f) Juvenile Alternative Services Programs (JASP), revised 11/1/2006.
(g) Practical and Cultural Education (PACE) Programs, revised 11/2004.
(h) Residential Commitment Programs, revised 9/1/2006.
(2) Quality assurance standards shall incorporate requirements as set forth in Section 985.632(4)(b), F.S.
(3) A program shall comply with all quality assurance standards and key indicators.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.004 Quality Assurance Peer Reviewers.
(1) The following programs shall provide a minimum of one employee from each program in operation to participate as a peer reviewer on at least one on-site quality assurance review during the current QA review cycle, in a judicial circuit other than the circuit in which the program is located.
(a) Day Treatment Programs;
(b) Intensive Delinquency Diversion Programs (IDDS);
(c) Juvenile Alternative Services Programs (JASP);
(d) Practical and Cultural Education (PACE) Programs;
(e) Contracted Community Corrections Programs.
(2) Each CINS/FINS program shall provide a minimum of two employees to participate as a peer reviewer on at least one on-site quality assurance review during the current QA review cycle, in a judicial circuit other than the circuit in which the program is located.
(3) Each residential commitment program (as listed in the JJIS) and detention center shall provide a minimum of one employee each year for every seventy-five beds or slots or fraction thereof in excess of 75 beds or slots to participate as a peer reviewer on at least one on-site quality assurance review during the current QA review cycle, in a judicial circuit other than the circuit in which the program is located.
(4) The Chief Probation Officer in each judicial circuit shall provide the number of employees identified by the department’s quality assurance office as necessary to conduct reviews of community corrections programs. Identified staff will participate on a review in a judicial circuit other than the circuit in which their office is located.
(5) Participation on a quality assurance review shall be at the program’s expense. Travel, lodging, and per diem for participation in the quality assurance peer review certification training shall be at the provider’s expense.
(6) A program shall ensure that all employees who participate on a quality assurance review are program managers or supervisors and possess at least a Bachelor’s degree from an accredited university and have a minimum of 3 years experience working with youth. Exceptions shall be allowed when approved by the regional director or CEO of the organization and the Chief of Quality Assurance when the individual does not have a Bachelor’s degree but has a minimum of 4 years experience in juvenile justice programs and is in a management position.
(7) All employees who participate on a quality assurance review must successfully complete the quality assurance peer review certification training. Twenty hours of training will be awarded to all participants in the quality assurance peer review certification training
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.005 Conducting Quality Assurance Reviews.
(1) Quality assurance reviews shall be unannounced.
(2) All residential commitment programs, detention centers, and residential CINS/FINS programs shall be reviewed each year.
(3) All non-residential programs and supervision programs shall be reviewed every other year. During the off year, the program shall submit a self-report of critical data and activities to the Bureau of Quality Assurance.
(4) A full review shall be conducted any time the department deems the program to be at risk.
(5) Program managers and supervisors shall implement internal systems to ensure all documents requested by the department are available during a quality assurance review.
(6) A program manager/supervisor shall provide files and other documentation requested by a quality assurance team without unnecessary delay. The lead reviewer in consultation with the program director shall determine the amount of time necessary for the program to provide the requested files and other documentation.
(7) The lead reviewer shall have the authority to grant the provider an extension that does not exceed 2 working days of the exit conference.
(8) A lead reviewer shall conduct a daily debriefing with the program director and invited guests to review positive and negative factors and to request additional information as needed.
(9) A lead reviewer shall conduct a pre-exit meeting with the program director and other key program personnel to highlight preliminary findings.
(10) A formal exit conference shall be conducted to explain the preliminary ratings and findings of the review team to the program director and other parties attending the exit conference. The exit conference can be waived upon written request of the program manager/supervisor or at the request of the Chief of Quality Assurance.
(11)All exit conferences shall be audio taped by the quality assurance team leader.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.006 Challenges to Quality Assurance Reports.
(1) A draft report shall be e-mailed to the program director to review for accuracy of findings.
(2) A program director or designee shall have 5 working days to review the report and respond to the department with any challenges related to accuracy, wording, or ratings within the draft report.
(3) Only key indicators rated in the partial performance, non-performance, or non-compliance ranges can be challenged.
(4) The area quality assurance program administrator shall attempt to reach an agreeable resolution of findings that are disputed by the program director.
(5) If the program director is not satisfied with the resolution, the issue will be raised to the Bureau Chief of Quality Assurance who will consult with the Assistant Secretary of the appropriate to arrive at a joint resolution.
(6) Actions that the provider has taken since the quality assurance site visit and/or files and other documentation that were not provided to the lead reviewer within 2 working days of the exit conference, shall not be addressed or considered by the department.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.007 Waivers and Alternative Compliance Measures.
(1) It shall be the responsibility of program managers and supervisors to request a waiver to department policy or to request to utilize an alternative compliance measure when they believe they should not be held to department policy, or have an alternative method to comply with such policy, and the request does not negatively impact staff safety, juvenile safety, security, treatment, conditions of confinement, public safety, the officer certification process, or another branch of the department.
(2) Waivers shall be sent to the regional director and assistant secretary of the respective branch.
(3) A request for a waiver or a request to utilize an alternative compliance measure shall be submitted to the assistant secretary through the regional director and approved by the Secretary prior to the quality assurance on-site review.
(4) A decision to deny a request for a waiver or a request to utilize an alternative compliance measure shall be made by the Assistant Secretary of the branch in which the request applies.
(5) A decision to grant a request for a waiver or a request to utilize an alternative compliance measure shall be made by the Secretary of the department or designee.
(6) The Secretary or designee shall not have the authority to issue a waiver or approve a request to use an alternative compliance measure for any action that is required by
(7) A waiver or approval to use an alternative compliance measure shall be in effect for one year from the signature date on the approval letter unless otherwise specified in writing on the approval.
(8) The department shall have the authority to cancel a Waiver or Alternative Compliance Measure if it is determined that it is not effective or if it has negatively impacted staff safety, juvenile safety, security, treatment, conditions of confinement, public safety; the officer certification process, or another branch of the department.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.008 Conditional Status.
(1) A program shall be placed on Conditional Status when it fails to meet the minimum levels of performance for any quality assurance standard.
(2) The program shall initiate a corrective action plan within thirty calendar days of publication of the quality assurance report to address correction of deficiencies.
(3) Corrective action plans shall include:
(a) Identification of the deficiency documented in the quality assurance report;
(b) Identification of tasks necessary to correct the deficiency;
(c) Identification of persons responsible for completion of the tasks;
(d) Periodic progress reports; and
(e) Target dates for completion.
(4) The contract monitor and/or safety and security monitor shall provide intensive monitoring and technical assistance until program performance is improved.
(5) Contract action shall be taken within six months of the posting of the quality assurance report when the contract monitor finds a contracted program has failed to improve in deficient areas.
(6) If a department-operated program fails to improve, corrective action shall be taken that includes, but is not limited to:
(a) Initiating appropriate disciplinary action against all employees whose conduct or performance is deemed to have materially contributed to the program's failure to meet established minimum thresholds;
(b) Redesigning the program; or
(c) Realigning the program.
(7) A program shall be removed from Conditional Status upon the successful completion of corrective actions and upon the notification of the appropriate Assistant Secretary to the Chief of Quality Assurance.
(8) If specialized training or technical assistance is needed to improve treatment services or a programs behavior management system, the program will be referred to the department’s technical assistance unit.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.009 Failure to Meet Minimum Levels of Performance or Compliance.
(1) A program shall fail to meet the minimum levels of performance or compliance set by the department when the quality assurance performance score is between 0% to 59%.
(2) A re-review shall be conducted at the program within six months of publication of the quality assurance report to determine if corrective actions and program improvements have caused the program to meet the minimum levels or performance or compliance.
(3) If a program fails to meet the minimum standards upon a re-review, the department shall cancel the provider's contract unless the provider has proved there are documented extenuating circumstances that led to the failure to meet the minimum thresholds of performance in accordance with Section 985.632(4)(b), F.S.
(4) When a program fails a second review in a 6-month period, and cancellation of the provider’s contract is being pursued, a provider shall have the right to submit information to the Internal Review Board that documents extenuating circumstances that led to the failure to meet the minimum threshold of performance.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.010 Internal Review Board.
(1) The department shall establish an Internal Review Board chaired by the Deputy Secretary of the department. Membership shall include the Chief of Staff, the Assistant Secretary for the branch of the program reviewed, and the General Counsel.
(2) The Internal Review Board shall meet to determine if there are documented extenuating circumstances that contributed to the program failing to meet the minimum standards of performance following a quality assurance re-review.
(3) The department shall serve the provider with written notice of the proposed meeting by registered or certified mail, return receipt requested.
(4) A provider shall have 10 working days from receipt of the certified letter to present any documented extenuating circumstances, in writing, to the Deputy Secretary.
(5) If a provider fails to respond within 10 working days, the department shall proceed with cancellation of the contract without review.
(6) If the provider submits information stating that there were documented extenuating circumstances that impacted their performance, the Internal Review Board shall review and consider the documentation as well as any other pertinent information.
(7) A provider shall be given an opportunity to present information in person or via conference call.
(8) The Internal Review Board shall consider all information and make a recommendation to the Secretary within 30 working days from the Internal Review Board meeting.
Specific Authority 985.632, 985.64 FS. Law Implemented 985.632 FS. History–New__________.
63L-1.011 Quality Assurance Reporting Requirements.
The department shall submit an annual report to the legislature that includes all information incorporated in Section 985.632(4)(b), F.S., no later than February 1st of each year, as part of the Comprehensive Program Accountability Report.