assessment

Educational Talent Search Project

Student Affairs Division

The Educational Talent Search Project is dedicated to the service of at least 900 eligible youth from the six westernmost counties of North Carolina and the Cherokee Indian Reservation. By following the carefully detailed intervention plan defined in our approved grant application, we will provide focused guidance programs and innovative educational opportunities that assist our participants in completing secondary school and matriculating to the post secondary educational programs of their choosing.

Student Affairs Mission: In support of the university, the division of Student Affairs is committed to the personal, social, and intellectual development of students as they live and learn within a diverse community. Programs, services, outreach activities, and facilities enhance the ability of current and prospective students to apply learning outcomes as they develop independent thought, take advantage of community involvement, and understanding global responsibility.

University Mission: Teaching and learning constitute the central mission of Western Carolina University. The University seeks to create a community of scholarship in which the activities of its members are consistent with the highest standards of knowledge and practice in their disciplines.

92 % of non-senior secondary school participants served during each project period will be promoted to the next grade level at the end of each academic school year.

(1) Appropriate to objectives: Progression to the next grade level will be verified for each student with school records that will be documented in each student’s file within 90 days of the start of the next school year and reported in the Annual Performance Report.

(2) (i) Formative: The Project will count the number of participants who have been promoted to the next grade level in school at the end of each budget period. That number will also be shown as a percentage of our enrollment. This retention and promotion will be documented in each student’s file with a printout from the target school detailing the student and their current grade. This is an ongoing process which uses monthly checks to determine if we are making progress toward achievement of this objective. It will be completed within 90 days of the next school year. All outcomes will be reported to the Department in the Annual Performance Report.

(2) (ii) Summative: At the end of the five year project period, the Project will calculate all previous yearly percentages to determine the overall percentage for participants who have been promoted to the next grade level over the entire project period. All outcomes will be reported to the Department in the Annual Performance Report.

(3) Unanticipated outcomes: All unanticipated outcomes will be reported to the Department in the Annual Performance Report.

92% of high school seniors (and their equivalents in alternative education programs) will graduate from secondary school or receive a certificate of high school equivalency during each project period.

(2) (i) Formative: The Project will count the number of college ready participants who have graduated from high school or received a certificate of high school equivalency at the end of each budget period. That number will also be shown as a percentage of our college ready students. This graduation or certification will be documented in each student’s file with a printout from the target school detailing the student’s status. This is an ongoing process, which uses monthly checks to determine if we are making progress toward achievement of this objective. It will be completed within 90 days of the end of the budget period. All outcomes will be reported to the Department in the Annual Performance Report.

(2) (ii) Summative: At the end of the five year project period, the Project will calculate all previous yearly percentages to determine the overall percentage for participants who have graduated from high school or received a certificate of high school equivalency during the course of the entire project period. All outcomes will be reported to the Department in the Annual Performance Report.

(3) Unanticipated outcomes: All unanticipated outcomes will be reported to the Department in the Annual Performance Report.

70% of “college ready” project participants will apply for financial aid during each project period.

(1) Appropriate to objectives: Data collected from the student sessions as well as follow-ups will show proof that the student applied for financial aid. This documentation will be placed in the student’s file, documented in our database and reported on the Annual Performance Report within 90 days after the end of the budget period. Copies of financial aid applications will be used for documentation whenever possible.

(2) (i) Formative: The Project will count the number of college ready participants who apply for financial aid during each during each budget period. That number will also be shown as a percentage of our college ready students. This documentation will be placed in each student’s file to show the student’s status in applying for financial aid within 90 days of the end of the budget period. All outcomes will be reported to the Department in the Annual Performance Report.

(2) (ii) Summative: At the end of the five year project period, the Project will calculate all previous yearly percentages to determine the overall percentage for participants who have applied for financial aid. All outcomes will be reported to the Department in the Annual Performance Report.

(3) Unanticipated outcomes: All unanticipated outcomes will be reported to the Department in the Annual Performance Report.

85 % of “college ready” project participants will apply for postsecondary school admission during each project period.

(1) Appropriate to objectives: Data collected from the student sessions as well as follow-ups will show proof that the student applied for postsecondary school admission. This documentation will be placed in the student’s file, documented in our database and reported on the Annual Performance Report within 90 days after the end of the budget period. Copies of college applications will be used for documentation whenever possible.

(2) (i) Formative: The Project will count the number of college ready participants who apply for postsecondary school admission during each during each budget period. That number will also be shown as a percentage of our college ready students. This documentation will be placed in each student’s file to show the student’s status in applying for postsecondary school admission within 90 days of the end of the budget period. All outcomes will be reported to the Department in the Annual Performance Report.

(2) (ii) Summative: At the end of the five year project period, the Project will calculate all previous yearly percentages to determine the overall percentage for participants who have applied for postsecondary school admission. All outcomes will be reported to the Department in the Annual Performance Report.

(3) Unanticipated outcomes: All unanticipated outcomes will be reported to the Department in the Annual Performance Report.

72% of “college ready” participants will enroll in a program of postsecondary education during each project period (or during the next fall term).

(1) Appropriate to objectives: Data collected from the National Student Clearinghouse of enrollment in post secondary education as well as that obtained by phone and follow-up interviews will be placed in the student’s file, documented in our database and reported on the Annual Performance Report within 90 days after the end of the budget period.

(2) (i) Formative: Talent Search staff will use phone follow-ups, and cross reference the National Student Clearinghouse database (to which our University subscribes) to identify all seniors who enrolled in postsecondary programs within 90 days after our budget period. From this we will collect the name and type of institution into which the participant enrolled. We will report the numbers and percentage of college ready students enrolled in postsecondary placements on the Annual Performance Report.

(2) (ii) Summative: At project period end, we will calculate the total number of college ready participants served and tally the number and percentage who actually enrolled in postsecondary programs throughout the five-year project period. The project will report both these numbers and percentages in the Annual Performance Report.

(3) Unanticipated outcomes: All unanticipated outcomes will be reported in the Annual Performance Report.

The Director of Assessment for the Office of University Planning at Western Carolina University will serve as the project evaluator.
(1) Types of data will be collected:
Demographic and contact data as well as program feedback will be recorded and tracked on an ongoing basis.

A. Quantitative and demographic data
1) Computer Database: Talent Search staff will maintain accurate and up-to-date data files on each enrolled Talent Search participant regarding the following: demographic data, eligibility, service received, persistence and promotion in school, needs assessments, post-secondary plans, receipt of financial aid services, and verification of enrollment in post-secondary education program.
This data will be used to compile accurate demographic and quantitative achievement objectives for the Annual Performance Report, and it is also readily available for immediate analysis from our College Access software database.
2) Hard File: Written documentation for each student will include the following information in each participant’s file: project application, need assessments, family financial statement, career plans, release for accessing academic records, potential for postsecondary education, and demographic data.

B. Surveys from participants, parents and school personnel
1) Student Evaluations: Talent Search Staff will conduct written surveys with participants, especially all seniors with the “Senior Exit Survey” at the end of each budget period. The survey addresses participant satisfaction with and perceived effectiveness of project services and project staff and provides valuable program feedback. These surveys are currently developed and on file in the Project office. Student surveys are conducted each spring. An analysis of results is reviewed and submitted with possible project changes to the Assistant Vice Chancellor of Student Affairs and posted to the university Assessment website.
2) School Personnel Evaluation: At the end of each budget period, Talent Search staff will conduct written surveys with school personnel (teachers and/or counselors). The results will help us remain effective facilitators and allow us to obtain information regarding unanticipated outcomes resulting from our services.
3) “Debriefing”: Students are encouraged to give feedback and reflect on their learning experience in small group after each and every activity provided by Talent Search. Based on the feedback from these sessions we know that we are addressing many of the CAS Student Learning Outcomes such as intellectual growth, effective communication, enhanced self-esteem, realistic self-appraisal, clarified values, career choice, leadership development, healthy behavior, meaningful interpersonal relationships, interdependence, social responsibility, appreciating diversity, spiritual awareness and personal and educational goals.
4) Parent Evaluations: After each parent program parents will be asked to complete an evaluation of that program and of the Project. Since most parent programs are completed by the spring semester this information will be available each summer. Feedback will be used to enhance our parent night programs.
5) Student Letters: We receive many letters and e-mails each year reflecting on lessons learned during our programs. These are filed in the Talent Search office.
(2) When various types of data will be collected;

A. Compilation and reporting of program performance:
Data collection will be an ongoing process with final results filed electronically following Department of Education guidelines within 90 days after the end of the previous budget period.
1) The Annual Performance Report: At the end of each school year we will compile all demographic and quantitative data pertaining to services delivered, as well as qualitative reports from files for the Annual Performance Report to the Department of Education. A detailed profile of the project’s performance and services will be measured against our objectives. Any unanticipated project outcomes will also be reported. This process is aided by use of Student Access software database. Data will also be tallied from participant and school personnel surveys used to assess the progression toward, or actual achievement of, project objectives and any unanticipated project outcomes. This report will be filed electronically following Department of Education guidelines within 90 days after the end of the previous budget period.
2) Summative achievement: At the end of the project period, the Talent Search Director will use the Annual Performance Reports to compile, assess and report the project’s total summative achievement and any unanticipated outcomes.
3) Student Affairs Annual Report: The Talent Search director will also file an annual report with the Student Affairs Division of Western Carolina University that demonstrates the Project’s contribution to the university’s objectives and student outcomes.
(3) Methods used:
Table 31
Student Evaluations
Parent Evaluations

“Debriefing”
Other feedback

(4) Instruments developed and when used:
Satisfaction, effectiveness and the quality of the ETS Project programs, services and facilities will be measured using the following:
Table 32
Summary of Assessment Measures Program/Service Assessment Timeline
o Document review and analysis
o Observation
o Surveys
o Debriefing, Interviews—individual and focus group
o Student Letters
o Tracking of data collected o Low and High Element Challenge courses: Fall semesters
o Outdoor Classroom: Spring semester
o FAFSA nights: January/February
o College and Cultural Trips: Ongoing
o Project Assessment: Ongoing
o Summer Programs: Summer

(5) How the data will be analyzed:

Information collected from the measures above will be summarized, documented, and discussed with possible program changes at our weekly staff meetings. In addition we will have two working retreats during the year to hammer out changes in our programs based on the results of our evaluations. It is not anticipated that there would be any changes in scope; however, the Department’s approval would be secured before any such changes in scope would be made.
(6) When reports and outcomes will be available:
All results will be completed within 90 days of the end of the budget period. All outcomes will be reported to the Department in the Annual Performance Report.
(7) How we will use the information collected through evaluation to monitor progress of the funded project and to provide accountability information, both about success at the initial site, and effective strategies for replication in other settings:
Evaluative information will be applied to all programs as they relate to the project’s objectives. Some Project procedures may be changed as a result of assessment. Permission from the Department of Education will be properly obtained before any changes in scope are implemented.

All of the above will be conducted annually and findings will be measured and tallied in a summative report.