Mathematica INC.

POLICY AND DATA ANALYSIS IN SUPPORT OF THE OFFICES OF BEHAVIORAL HEALTH, DISABILITY, AND AGING POLICY, AND HUMAN SERVICES POLICY

ORDER #3: ANALYSIS, DATABASES, AND BRIEFINGS

THE PURPOSE OF THIS PROCUREMENT IS TO PROVIDE EVALUATION OF PELL GRANTS FOR ELIGIBLE WORKFORCE PROGRAMS.

ADMINISTRATIVE DATA RESEARCH AND ANALYSIS (ADRA) CONTRACT

STANDARD APPLICATION PROCESS (SAP) PORTAL

INITIATE THE WIC PC 2028 INCLUDING STUDY DEVELOPMENT AND PLANNING, DATA COLLECTION AND ANALYSIS, AND DATA DISSEMINATION AS OUTLINED IN THE OBJECTIVES IN THE WIC PC 2028 AND 2030 IDIQ

LIBRARIES WORKING FOR AMERICA RESEARCH PROJECT IDENTIFIES BARRIERS TO WIOA PARTICIPATION PURSUANT TO SENATE REPORT 118-84 AND REAFFIRMED IN THE FY 2026 APPROPRIATIONS, AND ADVANCES 20 U.S.C. 9121 BY STRENGTHENING PUBLIC LIBRARIES CAPACITY TO SUPPORT

EVALUATION OF CRS PROGRAMS AND SERVICES

THE CORE OBJECTIVE OF THIS PROCUREMENT IS TO INCREASE AGENCY CAPACITY TO DISSEMINATE A GROWING BODY OF EVIDENCE AND EVIDENCE BUILDING RESOURCES TO MULTIPLE AUDIENCES. CRITICAL DISSEMINATION ACTIVITIES INCLUDE CONTENT MANAGEMENT, DISSEMINATION OF RES

DISABILITY ANALYSIS FILE (DAF) CONSTRUCTION: THE CONTRACTOR WILL BE DEVELOPING THE DISABILITY ANALYSIS FILE, MAKING SIGNIFICANT ADJUSTMENTS TO THE FILE STRUCTURE AND PROCESSING TO ENSURE GREATER EFFICIENCIES AND DISCLOSURE PROTECTIONS. THE CONTRACTOR

SUBCONTRACTOR SHALL PERFORM THE FOLLOWING TASKS: TASK 1. PARTICIPATE IN PROJECT MEETINGS, COLLABORATE, AND MEET WITH EXTERNAL ORGANIZATIONS, SUPPORT AND PARTICIPATE IN CONFERENCE CALLS AND THE KICKOFF MEETING WITH AHRQ AND ASSIST IN PREPARING PROGRESS REPORTS. TASK 4. ASSIST IN THE DEVELOPMENT AND IMPLEMENTATION OF THE VISION TO REFINE THE QIS. THIS INCLUDES REVIEWING ENVIRONMENTAL SCANS, PARTICIPATING IN THE REFINEMENT PRIORITIZATION AND VISIONING PROCESS, SYSTEMATICALLY TRACKING TA REQUESTS AND RESPONSES, PLANNING HOW TO MODIFY OR UPDATE SETNAMES AND OTHER CODING TERMINOLOGIES DUE TO CODING UPDATES. UPDATE SETNAMES TO INCORPORATE FISCAL YEAR CODING CHANGES AND REFINEMENTS; CONSTRUCT OR MODIFY SETNAMES SPECIFIC TO THE QIS. REVIEW EXPERT SETNAME-INDICATOR CROSSWALK AND THE MASTER SETNAME-MACRO CROSSWALK. CONTEMPORANEOUSLY UPDATE INDIVIDUAL SETNAME FILES AND SETNAME REFINEMENTS FILES MAINTAINED ON THE BOX PLATFORM THROUGH END OF THE DEVELOPMENT EFFORT FOR BASE YEAR. PROVIDE INPUT INTO THE SETNAME PROCESS AND ALL QUALITY CONTROL CHECKS. REVIEW FINAL SETNAME FILES AND CHANGE LOG PREPARED BY MATHEMATICA BEFORE DELIVERY TO AHRQ. TASK 5. ASSIST IN THE DEVELOPMENT OF REVISED SPECIFICATIONS FOR QI MEASURES THROUGH THE REFINEMENTS TESTING PROCESS. THIS INCLUDES LEADING ANY SETNAME-BASED REFINEMENTS, REVIEWING RESULTS FOR NON-SETNAME REFINEMENT TESTS, AND REVIEWING ANY DOCUMENTATION OR MATERIALS THAT ARE UPDATED TO ACCOUNT FOR IMPLEMENTED REFINEMENTS. TASK 6. PROVIDE AD-HOC SUPPORT AS NEEDED TO CLARIFY QUESTIONS ON RISK ADJUSTMENT MODELS, SIGNAL VARIANCES, AND COMPOSITE WEIGHTS. REVIEW AND COMMENT ON RISK ADJUSTMENT AND EMPIRICAL METHODS REPORTS. TASK 8. REVIEW AND COMMENT ON SCIENTIFIC ACCEPTABILITY REPORTS. TASK 9. ALPHA OR BETA TEST PUBLIC USE SAS SOFTWARE. REVIEW AND COMMENT ON TECHNICAL SPECIFICATIONS, LOG OF UPDATES AND REVISIONS, AND OTHER USER DOCUMENTS. TASK 10. RESPOND TO APPROXIMATELY 50 TECHNICAL ASSISTANCE (TA) REQUESTS IN THE BASE YEAR, REVIEW QUARTERLY TA REPORTS, AND DEVELOP FAQ DOCUMENTS FOR TA RESPONSES. TASK 11. CONDUCT ANNUAL PRIVACY TRAINING AND UPDATE DATA USE AGREEMENTS, AS REQUIRED BY MATHEMATICA€„¢S PROJECT DIRECTOR. TASK 12. SUPPORT THE TESTING OF NEW MEASURES THROUGH EXPERT PANELS, CLINICAL AND CODING REVIEW, AND REVIEW OF ANALYSES. ASSIST IN WRITING SUMMARY REPORT. TASK 13. ADVISE ON THE ENVIRONMENTAL SCAN, LEAD MEASURE VALIDATION, COLLABORATE ON PREVENTABILITY WEIGHTS, RISK ADJUSTMENT, AND BETA RELEASE, AND ENSURE MEASURES' CLINICAL USEFULNESS AND ACCURACY THROUGH DETAILED CHART REVIEWS AND VALIDATION USING DATA SOURCES LIKE THE UNIVERSITY OF CALIFORNIA HEALTH DATA WAREHOUSE.

SUBCONTRACTOR SHALL PERFORM THE FOLLOWING TASKS: TASK 1: PROJECT MANAGEMENT SUBTASK 1.3: ONGOING COMMUNICATIONS. AIR TASK LEADS WILL ATTEND PROJECT TELECONFERENCES WITH THE COR, AS REQUESTED BY THE PROJECT DIRECTOR. €¢ AIR WILL ALSO PREPARE INVOICES TO BE SUBMITTED BY THE 15TH OF EACH MONTH WITH BREAKOUTS BY TEAM MEMBERS FOR HOURS CHARGED BY SUBTASK. INVOICES WILL BE SUBMITTED BY THE 15TH OF THE MONTH AND BE ACCOMPANIED BY A MONTHLY MEMBERS DESCRIBING THE MONTH€„¢S PROJECT ACTIVITIES, ACCOMPLISHMENTS, RISKS OR ISSUES ENCOUNTERED, AND PROGRESS. TASK 2: KNOWLEDGE DEVELOPMENT AND EVALUATION DESIGN OPTIONS SUBTASK 2.1: KNOWLEDGE DEVELOPMENT. TO UNDERSTAND THE ES PROGRAM LANDSCAPE WHILE ALSO GATHERING INSIGHTS NECESSARY TO DEVELOP FEASIBLE AND RIGOROUS IMPACT DESIGN OPTION, THE STUDY TEAM WILL COMBINE DATA FROM MULTIPLE SOURCES TO CREATE AN INITIAL KNOWLEDGE DEVELOPMENT MEMO, WHICH WILL THEN INFORM DESIGN OPTIONS. €¢ AIR WILL: €¢ SUPPORT DOCUMENT REVIEW €¢ CONTRIBUTE TO/REVIEW GUIDES FOR IMPACT FEASIBILITY CALLS AND VISITS €¢ CONDUCT FOUR (4) TWO-PERSON FEASIBILITY PHONE INTERVIEWS WITH STATES (WE ARE ASSUMING A TOTAL OF 20 STATE CALLS TO SELECT 12 FOR THE IMPACT STUDY) SUBTASK 2.2: EVALUATION DESIGN OPTIONS REPORT. €¢ AIR WILL REVIEW AND PROVIDE FEEDBACK ON THE DESIGN OPTIONS MEMO. SUBTASK 2.3: STATE RECRUITMENT AND SELECTION. AFTER THE COR APPROVES THE STATE RECRUITMENT AND SELECTION PLAN, THE STUDY TEAM WILL CONDUCT INITIAL CALLS WITH STATES. AFTER INITIAL CALLS THE TEAM WILL CONDUCT FOLLOW-UP CALLS WITH SITES THAT ARE DEEMED AS WELL SUITED TO THE EVALUATION. IF IT WOULD BE VALUABLE FOR PARTNERSHIP BUILDING, WE WILL CONSIDER CONDUCTING EVALUATION FEASIBILITY AND RECRUITMENT SITE VISITS TO HIGH-PRIORITY LOCATIONS, PRIORITIZING NON-WAIVER STATES SINCE THEY ARE NOT MANDATED TO PARTICIPATE IN THE EVALUATION. €¢ AIR WILL PROVIDE SUPPORT FOR THE RECRUITMENT AND DEVELOPMENT OF MEMORANDA OF UNDERSTANDING WITH FOUR STATES; MATHEMATICA LEADING. TASK 3 IMPACT OUR OUTCOMES STUDY SUBTASK 3.1: IMPACT OR OUTCOME EVALUATION DESIGN REPORT. THE DESIGN REPORT WILL DESCRIBE THE IMPACT STUDY DESIGN, SAMPLE CREATION, SURVEY DATA COLLECTION, ADMINISTRATIVE DATA COLLECTION, ANALYSIS, AND REPORTING. THE DESIGN REPORT WILL DESCRIBE HOW THE IMPACT STUDY WILL BE INTEGRATED WITH THE IMPLEMENTATION STUDY TO DEFINE THE MODELS BEING EVALUATED AND PROVIDE CONTEXT FOR FINDINGS. €¢ AIR WILL CONTRIBUTE TO BRAINSTORMING AND REVIEW OF THE DESIGN REPORT. SUBTASK 3.2. DATA COLLECTION INSTRUMENT DEVELOPMENT AND OFFICE OF MANAGEMENT AND BUDGET (OMB) PAPERWORK REDUCTION ACT (PRA) CLEARANCE FOR IMPACT OR OUTCOME STUDY. THE IMPACT STUDY WILL RELY ON A SURVEY OF LWDBS IN THE SELECTED SITES TO MEASURE ES STAFFING MODELS, SERVICE APPROACHES, AND OTHER PROGRAM FACTORS. THIS SURVEY WILL ALSO BE ESSENTIAL FOR THE IMPLEMENTATION STUDY. €¢ AIR WILL CONTRIBUTE TO BRAINSTORMING DURING THE DEVELOPMENT OF INSTRUMENTS AND REVIEW THEM. SUBTASK 3.3. DATA COLLECTION FOR IMPACT OR OUTCOME STUDY. THE IMPACT ANALYSIS WILL RELY ON SURVEY AND ADMINISTRATIVE DATA SOURCES TO PROVIDE INFORMATION ON (1) CHARACTERISTICS OF AJCS AND THEIR STAFFING MODELS AND (2) INFORMATION ON ES PARTICIPANTS INCLUDING SERVICE RECEIPT AND EMPLOYMENT OUTCOMES. IN ADDITION TO THESE SOURCES, WE WILL USE THE ACS TO MEASURE CHARACTERISTICS OF THE LOCAL AREA AND LABOR MARKET. WE WILL THEREFORE RELY PRIMARILY ON ADMINISTRATIVE DATA SOURCES, SUCH AS THE FOLLOWING: (1) WIPS: (2) STATE-LEVEL REPORTING SYSTEM DATA; (3) THE NDNH; AND (4) AGGREGATE PERFORMANCE REPORTING DATA. €¢ AIR WILL SUPPORT COLLECTION AND CLEANING OF STATE AND LOCAL ADMINISTRATIVE DATA (LEADING IN 4 OF 12 STATES). SUBTASK 3.4. DATA ANALYSIS FOR IMPACT OR OUTCOME STUDY. THE STUDY TEAM WILL BUILD ON THE EVALUATION DESIGN REPORT AND SUBMIT A DETAILED ANALYSIS PLAN TO THE COR. THE ANALYSIS PLAN WILL INCLUDE STEPS TO PREPARE THE DATA FOR ANALYSIS; CLEAR DEFINITIONS OF MEASURES; DATA SOURCES AND DATA COLLECTION STRATEGIES; ANALYTIC METHODS; PLANS FOR ASSESSING AND

TASK 1: PROJECT MANAGEMENT SUBTASK 1.1: KICK-OFF MEETING. DR. PECK WILL PROVIDE INPUT TO AND ATTEND THE EVALUATION€„¢S VIRTUAL KICK-OFF MEETING. SUBTASK 1.2: PROJECT WORK PLAN. DR. PECK WILL PROVIDE INPUT TO AND REVIEW THE WORK PLAN. SUBTASK 1.3: ONGOING COMMUNICATIONS. DR. PECK WILL ATTEND PROJECT TELECONFERENCES WITH THE COR, AS REQUESTED BY THE PROJECT DIRECTOR. €¢ MEF WILL ALSO PREPARE INVOICES TO BE SUBMITTED BY THE 15TH OF EACH MONTH WITH BREAKOUTS BY TEAM MEMBERS FOR HOURS CHARGED BY SUBTASK. INVOICES WILL BE SUBMITTED BY THE 15TH OF THE MONTH AND BE ACCOMPANIED BY A MONTHLY MEMBERS DESCRIBING THE MONTH€„¢S PROJECT ACTIVITIES, ACCOMPLISHMENTS, RISKS OR ISSUES ENCOUNTERED, AND PROGRESS. TASK 2: KNOWLEDGE DEVELOPMENT AND EVALUATION DESIGN OPTIONS SUBTASK 2.1: KNOWLEDGE DEVELOPMENT. TO UNDERSTAND THE ES PROGRAM LANDSCAPE WHILE ALSO GATHERING INSIGHTS NECESSARY TO DEVELOP FEASIBLE AND RIGOROUS IMPACT DESIGN OPTION, THE STUDY TEAM WILL COMBINE DATA FROM MULTIPLE SOURCES TO CREATE AN INITIAL KNOWLEDGE DEVELOPMENT MEMO, WHICH WILL THEN INFORM DESIGN OPTIONS. €¢ MEF WILL: § CONTRIBUTE TO DOCUMENT AND LITERATURE REVIEW. § LEAD DEVELOPMENT OF GUIDES FOR IMPACT/OUTCOME FEASIBILITY CALLS WITH SUPPORT FROM MATHEMATICA AND AIR § CONTRIBUTE TO THE SELECTION OF STATES AND LOCAL BOARDS FOR THE EXPLORATORY CALLS § CONDUCT TEN (10) 2-PERSON FEASIBILITY PHONE INTERVIEWS (ASSUMING 20 CALLS TOTAL TO IDENTIFY 12 STATES) § CONTRIBUTE TO DEVELOPMENT OF KNOWLEDGE DEVELOPMENT MEMO SUBTASK 2.2: EVALUATION DESIGN OPTIONS REPORT. THE REPORT WILL BEGIN WITH A CONCEPTUAL MODEL AND DESCRIBE OPTIONS TO ESTIMATE THE EFFECTS OF ES STAFFING MODELS, SITE-SPECIFIC FACTORS, AND APPROACHES USED TO PROVIDE SERVICES ON PARTICIPANTS€„¢ EMPLOYMENT, EARNINGS, ENROLLMENT IN AND COMPLETION OF TRAINING PROGRAMS, AND CO-ENROLLMENT IN OTHER WIOA PROGRAMS. IT WILL ALSO PROVIDE OPTIONS FOR AN OUTCOMES STUDY THAT CAREFULLY EXAMINES VARIATION IN THESE OUTCOMES. THE REPORT WILL PRESENT THE ADVANTAGES AND DISADVANTAGES OF THE OPTIONS BASED ON CRITERIA AND PRIORITIES DISCUSSED WITH DOL. FOR EACH OPTION, THE REPORT WILL PRESENT THE ASSOCIATED RESEARCH QUESTIONS, EXISTING LITERATURE, PROPOSED RESEARCH DESIGN, REQUIRED DATA AND ANALYSIS, AND REQUIRED TIME AND RESOURCES. THE REPORT WILL CONSIDER A RANGE OF POTENTIAL DESIGN METHODS, FOCUSING ON QEDS WITH DESIGNS THAT EXAMINE VARIATION IN STAFFING MODELS AT THE STATE AND LOCAL AREA LEVEL. IT WILL HIGHLIGHT THE DESIGN OPTIONS THAT ARE MOST APPLICABLE TO DIFFERENT RESEARCH QUESTIONS AND PROVIDE THE MOST RIGOROUS EVIDENCE. WHILE MANY STATISTICAL METHODOLOGIES WILL BE CONSIDERED, THESE METHODS ARE ALL BASED ON A COMPARISON OF PARTICIPANT OUTCOMES IN WAIVER STATES TO PARTICIPANT OUTCOMES IN NON-WAIVER STATES. THESE DESIGNS WOULD DRAW ON DATA FROM WORKFORCE INTEGRATED PERFORMANCE SYSTEM (WIPS) AND NATIONAL DIRECTORY OF NEW HIRES (NDNH) FOR PARTICIPANT OUTCOMES AND THE ACS TO IDENTIFY COMPARISON STATES OR SITES. €¢ MEF WILL LEAD THE EVALUATION DESIGN OPTIONS REPORT. SUBTASK 2.3: STATE RECRUITMENT AND SELECTION. AFTER THE COR APPROVES THE STATE RECRUITMENT AND SELECTION PLAN, THE STUDY TEAM WILL CONDUCT INITIAL CALLS WITH STATES. AFTER INITIAL CALLS THE TEAM WILL CONDUCT FOLLOW-UP CALLS WITH SITES THAT ARE DEEMED AS WELL SUITED TO THE EVALUATION. IF IT WOULD BE VALUABLE FOR PARTNERSHIP BUILDING, WE WILL CONSIDER CONDUCTING EVALUATION FEASIBILITY AND RECRUITMENT SITE VISITS TO HIGH-PRIORITY LOCATIONS, PRIORITIZING NON-WAIVER STATES SINCE THEY ARE NOT MANDATED TO PARTICIPATE IN THE EVALUATION. €¢ MEF WILL CONTRIBUTE TO RECRUITMENT AND DEVELOPMENT OF MOUS WITH STATES; MATHEMATICA LEADING TASK 3 IMPACT OUR OUTCOMES STUDY SUBTASK 3.1: IMPACT OR OUTCOME EVALUATION DESIGN REPORT. THE DESIGN REPORT WILL DESCRIBE THE IMPACT STUDY DESIGN, SAMPLE CREATION, SURVEY DATA COLLECTION, ADMINISTRATIVE DATA COLLECTION, ANALYSIS, AND REPORTING. THE DESIGN REPORT WILL DESCRIBE HOW THE IMPACT STUDY WILL BE INTEGRATED WITH THE IMPLEMENTATION STUDY TO DEFINE THE MODEL

"CONDUCT ALL ACTIVITIES NECESSARY TO PREPARE AND REVISE REPORT FOR PUBLICATION ON THE OPRE WEBSITE, ACTIVITIES NECESSARY FOR LEGAL PROFESSIONALS AND PREPARE FOR PUBLICATION ON THE OPRE WEBSITE, ACTIVITIES NECESSARY TO DEVELOP AND PRESENT AT A NATIONAL WEBINAR FOR LEGAL PROFESSIONALS."