Data-driven solutions
for mission-driven organizations

Process & Outcome Evaluations of a Multifaceted Child Care Improvement Effort

Child Care Solutions is a not-for-profit organization founded in 1975 to provide information and resources to parents, providers, employers, and community leaders in Onondaga and Cayuga Counties (NY) who seek solutions to their child care needs. In partnership with the Early Childhood Alliance, Child Care Solutions has been working tirelessly to assure child care options are not only available but of high quality.

Beginning in 2021, The Child Care Quality Improvement Pilot (CCQI) funded by Onondaga County, provided extra resources to this effort. This year the pilot’s focus is on family child care. Key components include training family care providers to implement an evidenced-based curriculum (The Creative Curriculum), one-on-one coaching in using assessments (a curriculum fidelity tool, the Family Child Care Environment Rating Scale [FCCRS], and the Ages and Stages questionnaire) to inform lesson planning, and communication with parents. There are also stipends for the providers.

As the independent evaluator of the pilot, A&O’s work will have two key components. First, a process evaluation. This is designed to assess CCQI’s progress implementing the project components they’ve planned.

Second, an outcome evaluation will measure progress toward the intended outcomes and describe the activities’ early impacts. Stakeholder feedback and assessment data will be important to that evaluation, and A&O will synthesize data from a variety of sources to formulate suggestions for improvement.

Child Care Solutions of Onondaga and Cayuga Counties (CCS): A Child Care Needs Assessment

In the past, Apter & O’Connor has worked with Child Care Solutions several times to support their needs assessment related to the current state of child care. A&O designed specific inquiries to examine child care supply and demand. The final product illustrated the child care needs unique to these geographic areas and the related work-life issues for both child care providers and the families they serve. The assessments included: a detailed review of national, state and local data; focus groups with child care providers, local government officials, funders and human service experts; surveys for parents and providers; and interviews with human resource professionals.  The rigorous methodology employed for each study provided child care advocates with a powerful tool to design new interventions and educate state and local legislators and funders.

Early Reading First

Early Reading First (ERF) was an early literacy program targeting preschoolers from low income families at high risk of reading failure. Funded by the U.S. Department of Education, each of these projects was a multi-year contract (3-6 years) with features unique to each district. Apter & O’Connor provided rigorous evaluations to support continuous program improvement and to assess both fidelity of implementation and the impact of the ERF interventions. Each scope of work required a detailed evaluation plan, extensive data collection and analysis, and extensive and detailed reporting to satisfy all US DOE GPRA requirements. Our evaluation outcomes enabled the districts to design and sustain changes in professional development practices and classroom instruction, gain a place on the federal What Works Clearinghouse for early childhood best practices, initiate further research on critical factors for student gains, and leverage their data to support applications for additional grant funding. 

City of Syracuse, Department of Neighborhood and Business Development

Apter & O’Connor provides consultant services for the City of Syracuse, Department of Neighborhood and Business Development. Our team works with community based organizations that receive US Department of Housing and Urban Development (HUD) Community Development Block Grant (CDBG) funding.  In these times of shrinking funds, we are assisting CDBG agencies to strengthen their capacity in several areas including: governance and board development, performance outcomes and measurement systems, resource development and funding opportunities, effective collaborations for proposals and service delivery.  Our approach offers individualized technical assistance based upon unique organizational strengths, challenges and needs.   The A&O Team brings all of the needed experience for this project in strategic planning and grant writing as well as expertise as trainers and facilitators.  We are delighted to be collaborating on this project with Mr. Steve Susman, who, as former Executive Director of the Westcott Community Center in Syracuse, brings 37 years of experience in grant writing and reporting, non-profit management and board development.

High School Graduation Initiative

SCSD was awarded a four year Federal grant to tackle high dropout rates and low graduation rates.  The District has contracted with Apter & O’Connor to evaluate the program and document implementation successes and challenges. The High School Graduation Initiative (HSGI) supports any student currently out of school, at-risk of dropping out or not on track for graduation. HSGI adds a strong focus on school attendance with the formation of Attendance Teams to track students early on and assist those with attendance issues through additional supports or alternative programming.  Another important component are the Graduation Initiative Specialists (or Case Mangers) who work directly with students and their families. A&O is conducting a formative and outcome evaluation, utilizing data from the Basic Educational Data System (e-School), standardized test scores, attendance records, and disciplinary referral reports as well as survey/interview with students, counselors and teachers and administrators to examine success of implementation and impact of the interventions. The findings and recommendations will inform further decision making for student supports and improved achievement.

Early Reading First

Early Reading First (ERF) was an early literacy program targeting preschoolers from low income families at high risk of reading failure. Funded by the U.S. Department of Education, each of these projects was a multi-year contract (3-6 years) with features unique to each district. Apter & O’Connor provided rigorous evaluations to support continuous program improvement and to assess both fidelity of implementation and the impact of the ERF interventions. Each scope of work required a detailed evaluation plan, extensive data collection and analysis, and extensive and detailed reporting to satisfy all US DOE GPRA requirements.  Our evaluation outcomes enabled the districts to design and sustain changes in professional development practices and classroom instruction, gain a place on the federal What Works Clearinghouse for early childhood best practices, initiate further research on critical factors for student gains, and leverage their data to support applications for additional grant funding.

Rigorous Inquiries into Trends and Impacts of The Early Childhood Alliance of Onondaga

The Early Childhood Alliance of Onondaga County (ECA) was launched in 2015 to advance a comprehensive, coordinated, well-funded system of high-quality, prenatal-to-age-five early childhood programs and supports. It is a diverse cross-section of community stakeholders with a shared vision that all young children in the county are healthy, thriving, and successful in school and life, and that all families of young children are supported in their parenting. A&O was engaged to examine two of teh ECA’s key areas of effort: funding support and parent engagement.

In 2022, we completed a study of recent early childhood funding levels and trends nationally, statewide, and within Onondaga County. The results serve as an updated baseline for monitoring future growth and to measure ECA’s advocacy impact.

Methods for this inquiry included a document review of past ECA reports, a literature review on early childhood funding levels and trends at state and national levels, interviews with ECA staff, and a spring 2022 funding survey of local anchor systems (nine key early childhood services and supports).

That report, Funding to Support A Strong Early Childhood System 2019–2021, detected an 8% decrease in funding for the early childhood ecosystem from 2019 through 2021 in Onondaga County. Several survey respondents noted that COVID-related challenges, closures, and lockdowns skewed service delivery data and, in some cases, resulted in underutilized services which could explain some of the funding decreases.

The second inquiry, also completed in 2022, examined the Parent Advisory Council (PAC), the primary mechanism for the ECA’s parent engagement work. A group of parents of young children meet regularly to discuss improving the early childhood system, leveraging their own experiences to provide the ECA with input about its initiatives.

The ECA funded this study to learn how the PAC was functioning, what impact the members have on ECA planning and decision-making, and if the parent participants perceive any value or benefit for themselves. ECA was also interested in gaining new knowledge and ideas from other parent engagement professionals and endeavors statewide and nationally to help inform their overall parent engagement strategy.

A thorough report, Parents in Partnership: An Initiative of the Early Childhood Alliance of Onondaga County, offered valuable insights to the Alliance and strong recommendations and references to strengthen their early efforts.

The Gifford Foundation: ADVANS Consultation

Apter & O’Connor provides consultation and assistance to a wide array of not-for-profit agencies to increase their ability to respond and thrive in challenging fiscal climates. Our capacity-building work with health, educational and human service agencies is taking several forms, depending on the needs and “life-cycle” stage of the organization. As part of this renewed emphasis on capacity building and organizational development, A&O President, Cynthia O’Connor, has served as a consultant for ADVANS (Advancing and Developing the Value & Assets of Non-Profits in Syracuse), an initiative developed and funded by the Gifford Foundation and the Ready or Not initiative supported by the Health Foundation of Western and Central New York (HFWCNY).

Cynthia was certified by Dr. Susan Kenny Stevens, Gifford’s lead consultant for the project and the author of the award-winning book Nonprofit Lifecycles Stage–Based Wisdom for Nonprofit Capacity. This organizational development framework assumes predictable needs, challenges, and strengths dependent on an agency’s life cycle — from start up to maturity — and works intensively with agency leadership teams to assess and fortify their infrastructure across management, governance, financial, and administrative systems, as well as programs and services.

Cynthia has had the privilege of three 2-year engagements through the Gifford ADVANS program and a fourth for the HFWCNY’s Ready or Not initiative.

21st Century Community Learning Centers (21st CCLC) and the Extended Day/School Violence Prevention (ED/SVP) programs

A&O conducts evaluations for several 21st Century Community Learning Centers (21st CCLC) and the Extended Day/School Violence Prevention (ED/SVP) programs. These programs are federally, and state funded to support tutoring, academic enrichment and a broad array of youth development opportunities before and after school to complement the school’s regular academic program. It features partnerships with school districts and community-based organizations to programs target students from high-poverty neighborhoods and attending low-performing schools.

A&O’s evaluation activities include systematic observations at all sites, interviews with staff and administrators, surveys of all stakeholder (students, staff and parents) and extensive analysis of student academic data. A&O has worked with large multi-site school districts as well as smaller community agency sites, including The Syracuse City School District, Watervliet Central School District, The YMCA of Central New York, and the Center for Community Alternatives.

Program Directors consistently note the value of these rigorous evaluations and the ongoing feedback and technical assistance for program improvement that the A&O team provides. These independent evaluations have provided the needed evidence and critical information to convince legislators at all levels of government to continue funding these after school programs.

21st Century Community Learning Centers (21st CCLC) and the Extended Day/School Violence Prevention (ED/SVP) programs

A&O conducts evaluations for several 21st Century Community Learning Centers (21st CCLC) and the Extended Day/School Violence Prevention (ED/SVP) programs. These programs are federally and state funded to support tutoring, academic enrichment and a broad array of youth development opportunities before and after school to complement the school’s regular academic program. It features partnerships with school districts and community-based organizations to programs target students from high-poverty neighborhoods and attending low-performing schools.

A&O’s evaluation activities include systematic observations at all sites, interviews with staff and administrators, surveys of all stakeholder (students, staff and parents) and extensive analysis of student academic data. A&O has worked with large multi-site school districts as well as smaller community agency sites, including The Syracuse City School District, Watervlieit Central School District, The YMCA of Central New York, and the Center for Community Alternatives.

Program Directors consistently note the value of these rigorous evaluations and the ongoing feedback and technical assistance for program improvement that the A&O team provides. These independent evaluations have provided the needed evidence and critical information to convince legislators at all levels of government to continue funding these after school programs.

A&O Senior Associate Vicki Quigley has received high praise for her work with The Erie Canal Museum, Canal Landing in Chittenango, and The Skaneateles Historical Society, all located in Central New York.

One client told us, “This is one invoice we are happy to pay. I don't have enough words to thank Vicki for all her help and good work. We could not have achieved all we have without her.” 

Many arts organizations, struggling post-pandemic, are seeing the benefit of revisiting and, in some cases, re-envisioning their mission and activities. To the tough and rewarding strategic planning process, Vicki brings her experience in not-for-profit and museum management. Her legal knowledge (she holds a J.D. from Syracuse University College of Law, where she specialized in issues of charitable corporations) is an additional benefit to our clients. 

Apter and O’Connor assisted the One Love Foundation staff in Yonkers, NY in completing a comprehensive assessment to better understand the educational needs to prevent sexual and dating violence among young people. A team of four at A&O began by helping the staff at One Love to prioritize data elements and indicators and identify relevant secondary data sources. Our team then worked to review, analyze and interpret all of this existing data. We used local measures to compare with regional, state, or national benchmarks to provide context for the findings. Finally, we provided guidance and technical assistance to the One Love staff to develop a logic model and evaluation plan, strengthen survey tools and plan for future focus groups and interviews.   

21st Century Community Learning Centers (21st CCLC) and the Extended Day/School Violence Prevention (ED/SVP) programs

A&O conducts evaluations for several 21st Century Community Learning Centers (21st CCLC) and the Extended Day/School Violence Prevention (ED/SVP) programs. These programs are federally, joand state funded to support tutoring, academic enrichment and a broad array of youth development opportunities before and after school to complement the school’s regular academic program. It features partnerships with school districts and community-based organizations to programs target students from high-poverty neighborhoods and attending low-performing schools.

A&O’s evaluation activities include systematic observations at all sites, interviews with staff and administrators, surveys of all stakeholder (students, staff and parents) and extensive analysis of student academic data. A&O has worked with large multi-site school districts as well as smaller community agency sites, including The Syracuse City School District, Watervliet Central School District, The YMCA of Central New York, and the Center for Community Alternatives.

Program Directors consistently note the value of these rigorous evaluations and the ongoing feedback and technical assistance for program improvement that the A&O team provides. These independent evaluations have provided the needed evidence and critical information to convince legislators at all levels of government to continue funding these after school programs.

Health Foundation of Western and Central New York: Central New York Neighborhood Initiative — Helping Seniors to Age in Place

The Health Foundation of Western and Central New York in partnership with the Gifford Foundation funded a Central New York Neighborhood Action Initiative (CNYNAI) to create and test evidence-based practices that will enable seniors to remain in their homes and own communities as they grow older rather than in nursing homes or other forms of institutionalized care. To do this, the Foundation provided seven neighborhood-based organizations with resources to enable them to test best practice approaches designed to help people continue to live in their neighborhoods and to share knowledge, ideas and lessons learned with each other.

Apter & O’Connor served as the evaluator for this exciting initiative.  In addition, we facilitated “learning communities” for the seven selected organizations.  We were able to articulate the unique characteristics of each project that may have had the most significant influence or impact on creating conditions and supports that allow elderly residents to age in their own homes and neighborhoods.   Each of the NAI funded agencies successfully served their target population of low-income seniors within their identified neighborhoods and received positive reviews and accolades from their participants. All found the Learning Sessions a valuable opportunity to network, share ideas and resources and support each other.  Our final report, still in use, offered ideas for lessons learned by the Foundations as well as the seven agencies in serving this often challenging population in unique and creative ways.

St. Joseph Hospital and Health Center, Syracuse NY: The Children’s Health Innovation Project (CHIP)

CHIP was a funded by the National Institute for Mental Health as a system of care grant intended to improve mental health services and outcomes for children and youth. Apter & O’Connor provided a process evaluation, technical assistance for planning that included practitioners, administrators and parents, and a focused inquiry involving a series of interviews with primary care physicians and mental health professionals.

Jowonio School

The Jowonio School in Syracuse, NY is an internationally recognized model for inclusive education of young children with a wide range of abilities. A&O facilitated a diverse group of stakeholders including parents, staff, early childhood professionals, program supporters, funders and board members to develop a concise logic model to articulated their many program facets. From there, A&O provided guidance and technical assistance to Jowonio staff to build their capacity for systematic tracking, measurement and reporting of the school’s impact on children, families and staff. Armed with clear knowledge of their data collection needs and methods for collecting useful data they were able to secure funding to develop the technology for a performance management system which they implemented and continue to use today. 

Addison Central School District: Innovative Approaches to Literacy (IAL): Read To Learn Project

The Addison Central School District is comprised of rural communities in New York State’s southern Tier. A&O has a long-standing relationship with the District including successful evaluation work on their multi-year federally funded Early Reading First project and, the Innovative Approaches to Literacy Read to Learn Projects. A&O has also assisted them secure additional funding to support these and other related projects. Each of these initiatives was designed to promote early literacy among families, motivate students to read and increase student achievement through improved classroom instruction, the use of school libraries, book distribution to families and high-quality family literacy activities. We used both quantitative and qualitative analysis with observations, interviews, surveys and student performance data to determine progress toward goals and ultimate impact of the project. The evaluation demonstrated that the professional development activities have enhanced teacher’s skills for literacy instruction and the new library collections and supports have strengthened classroom reading instruction. Addison used their evaluation results to successfully apply for continued federal funding and was one of 46 school districts in the county to be refunded for a 2nd cycle. This enabled the District to expand its scope to serve children from age 3 through Grade 6.

Early Reading First

Early Reading First (ERF) was an early literacy program targeting preschoolers from low income families at high risk of reading failure. Funded by the U.S. Department of Education, each of these projects was a multi-year contract (3-6 years) with features unique to each district. Apter & O’Connor provided rigorous evaluations to support continuous program improvement and to assess both fidelity of implementation and the impact of the ERF interventions. Each scope of work required a detailed evaluation plan, extensive data collection and analysis, and extensive and detailed reporting to satisfy all US DOE GPRA requirements. Our evaluation outcomes enabled the districts to design and sustain changes in professional development practices and classroom instruction, gain a place on the federal What Works Clearinghouse for early childhood best practices, initiate further research on critical factors for student gains, and leverage their data to support applications for additional grant funding.

Huntington Family Centers

The Huntington Family Centers Brighton Tower’s Service Coordination Program needed an independent quality assurance review on a tight timeline. Funded by the U.S. Department of Housing and Urban Development, the program provides residents in this 600 unit subsidized senior housing complex with the personal assistance they need to access resources and services.

The review examined the 12-month period ending June 30, 2014. A&O worked with residents, apartment building staff and Huntington to answer the three questions all funders and providers need answered: 1) Did we implement with fidelity? 2) Did we do it with quality? 3) Did our services have an impact on our constituents? We conducted interviews, held group discussions with residents, and used questionnaires to collect anonymous feedback. Sixty residents participated (approximately 10% of total residents, and 25% of the residents served in 2013). We wanted to know about their satisfaction and utilization of the services offered as well as barriers the residents faced in accessing them. We also gathered ideas for improvement for the Service Coordination Program.

In addition to providing the deliverable of a high quality independent program evaluation for the client, our process of communicating with residents about the Service Coordination Program increased their understanding of this valuable resource.