CS Outreach Program Evaluation Network (CS OPEN)
History and Overview
With a desire to support the development of effective and equitable CS opportunities for girls and the acknowledgment that many nonprofits lack the evaluative capacity needed to ensure high quality programming, the Google K-12 Outreach team and the National Girls Collaborative Project (NGCP) developed the CS Outreach Program Evaluation Network (CS OPEN). CS OPEN was developed to provide support to develop the evaluative capacity of network grantees. It was announced at a July 9, 2015 event co-hosted by the Center for Gender Equity in Science and Technology and the White House Council on Women and Girls. CS OPEN is a project designed to improve opportunities and empower underserved girls through CS education by boosting select NGC programs' knowledge on exemplary evaluation practices in studying CS education. For the 2015-2016 pilot, Google--in partnership with Haynie Research and Evaluation--provided expertise and professional development to promote evaluation of CS education initiatives within the NGC network. CS OPEN Cohort I Grantees are listed below. The overall goal of the pilot was not only to enhance the programs participating, but also to inform the field of CS education.
The CS OPEN network was designed to meet the wide range of needs through professional development opportunities and individualized support. The ECB opportunities provided for CS OPEN Grantees consisted of the following:
- The CS OPEN network community of practice
- Grantee involvement in their own evaluation process, including writing an evaluation plan, developing instruments, collecting data, analyzing data, and reporting
- Monthly videoconference meetings including topic-based training
- A web repository of online evaluation resources
- Individualized support and mentoring to help address specific ECB needs
During the 2015-2016 time period, the twelve CS OPEN projects served a total of 3,183 participants, of which 1,649 were girls. The median number of female participants is 89; half of the CS OPEN projects served girls only. Project evaluations were conducted by each of the grantees. Each of the project evaluations included an evaluation plan, evaluation questions, methodology, a data collection plan, analysis work, and a final report to the CS OPEN team.
CS OPEN 2015-2016
“It was helpful to learn I wasn’t alone – there are a lot of people all over the United States doing this stuff. It was encouraging to me. The ideas and plans that everyone was doing. Different ideas, but at the end we all have the same goal.”—Regarding Monthly Community Meetings
“We were able to consult not only with local evaluators, but also with the CS-Open team and other awardees. Due to the similarities of several CS-Open projects, we were able to get ideas and receive feedback as part of our monthly meetings.” – Regarding Collaboration with other CS OPEN Grantees
CS OPEN enabled five of the project evaluations to happen and the expansion of six existing evaluation efforts. Grantees were asked through survey, interview, and final report to describe their growth in evaluative capacity, as well as the ways they attribute this growth to participating in the CS OPEN initiative.
- Actively streamline evaluation tools across all programs
- Created and developed better evaluation tools and instruments with more relatable context
- Empowering all staff
- Surveying schedule
- Move beyond basic surveys
- Qualitative evaluation approaches
- Framework and support needed to shape the department
- Creating milestones and measures
- Transcribing data
Broader Evaluation Findings
When analyzed collectively, the broader evaluation findings on engaging girls in CS education were as follows:
- Tap girls’ natural enthusiasm at young ages. Girls are interested and want to be engaged in CS.
- Consider best environments. Girls do well in all-girl settings. Physical activity can be integrated.
- Offer hands-on, exploratory activities. Use hands-on activities, not lectures. Make activities engaging and relevant.
- Provide support and encouragement. Provide help that is always available.
- Use longer delivery formats. More contact time can support stronger outcomes.
- Provide role models, a vision for future career. Linking CS to the real world is key.
CS OPEN Cohort I Grantees