Terms of Reference/Request for quotations

Consultancy for Mid-Term Evaluation of BMZ-funded Project in Ukraine

Project Title: “Strengthening the resilience of vulnerable households and war affected communities through community-driven recovery in Buchnky Rajon”    

Submission for proposals to AWO International until 18 June 2026 COB to [email protected]. 

1.              Introduction

The project is being implemented by AWO International e.V. and its Ukrainian partner the Charitable organization “Charitable Fund “Right to Protection” (R2P).

AWO International from Germany is the Workers’ Welfare (Arbeiterwohlfahrt) Association for Development Cooperation and Humanitarian Action. The overall goal of AWO International’s work is to contribute to the improvement of peoples’ living conditions in a sustainable manner and to enhance their own initiative. We support people/communities to develop their own life perspectives, to gain access to necessary resources and services to improve the social participation of disadvantaged social groups such as children and adolescents, women, migrants, indigenous people and elderly or sick people. AWO International has been working in Ukraine since 2022 and implementing several projects funded by different donors, e.g. Aktion Deutschland Hilft (ADH / German Relief Coalition), BMZ and GFFO in cooperation with Ukrainian partner NGOs in Kyivska, Lvivska, Chernihivska, Sumska, Zhytomyrska, Mykolaivska and Khersonska oblasts.

AWO International invites consultants to conduct a Mid-Term Evaluation based on project proposal, logframe, budget and interim report.

This Terms of Reference outlines the scope and responsibilities for the Mid-Term Evaluation (MTE). The evaluation aims to assess the project’s overall performance and its contribution to increasing income security and improving the well-being of individuals and communities in the project region.

This Terms of Reference (ToR) outlines the scope, objectives, and responsibilities for an external Mid-Term Evaluation (MTE) of the above-mentioned project. AWO International undertakes this evaluation to assess progress to date, evaluate the effectiveness and efficiency of project implementation, and derive lessons to improve delivery in the remaining project period. The project is funded by BMZ with co-funding from ADH and is being evaluated mid-way through its implementation. The MTE will be carried out by an independent consultant/team and will employ a hybrid approach (remote and in-person methods). It will result in actionable recommendations for course correction and enhanced impact in the project’s second phase. This ToR is intended for public announcement to solicit proposals from qualified evaluators, and all evaluation deliverables and communications will be in English.

2.             Background

Location: Kyivska oblast, Buchansky Rajon; focus on Makariv, Bucha, Irpin.

Project period: 01 September 2024 – 31 December 2027.

Target groups: 5,370 direct and 30,000 indirect beneficiaries.

Overall objective / outcome (as stated in draft ToR): improving income security, wellbeing and resilience of vulnerable people in Buchanski Rajon.

Main components to be covered by the evaluation:

-       Livelihood & economic recovery: retraining/upskilling with companies (internships/on-the-job learning) for at least 600 people; business start-up training/advisory and start-up support; networking meetings and business clubs.

-       Social cohesion: public cohesion events; information workshops on support services; CSO capacity strengthening trainings.

-       Participation in reconstruction & hubs: public discussions of reconstruction plans, recommendations and fundraising strategies; renovation and equipping of hubs; strengthening hub-operating CSOs and sustainability mechanisms.

-       Knowledge transfer/advocacy: exchange formats (online / oblast-level) to share good practices on participatory reconstruction (as described in proposal).

3.              Evaluation Objectives and Scope

Evaluation Objectives: The MTE will assess progress to date, quality and early outcomes, and produce actionable recommendations for course correction and improved impact in the remaining project period.

Specifically, the evaluation will:

•           Measure progress against the project’s intended results (outcomes and outputs) as stated in the project logframe. This includes assessing the extent to which planned outputs have been delivered and whether the project is on track to achieve its outcome of improved well-being and resilience for the target population.

•           Assess key evaluation criteria – namely the project’s relevance, effectiveness, efficiency, coherence, potential impact, and prospects for sustainability – in the context of the operating environment at mid-term.

•           Identify good practices, innovations, and lessons learned so far. This will highlight what is working well, unexpected successes, or approaches that could be replicated or scaled up in the remaining implementation period or in future projects.

•           Assess the project’s management and partnership approach, including the efficiency and effectiveness of the organizational structure and consortium arrangements. Given the project involves local partners and coordination with authorities, the evaluation will consider how well these arrangements are functioning and contributing to results.

•           Provide actionable recommendations for course correction or enhancement. Based on findings, the evaluator will propose practical suggestions to improve project performance, address any shortcomings, and adapt to any changes in context for the remaining project duration. These recommendations should be specific, feasible, and oriented toward maximizing the project’s impact and sustainability.

Scope of the Evaluation: The evaluation will cover the first half of the project’s implementation (Sep 2024 – May 2026) across all components and all target locations. It will include both operational and contextual aspects, examining not only what the project has achieved so far, but also how it has been implemented. Key parameters of the scope include:

•           Geographic scope: All project sites in the two target Oblast. This encompasses community-level interventions (e.g. repaired water systems in villages, distributed supplies to households) as well as institutional sites (e.g. social protection centers that received support). The evaluator is expected to visit the  projectsites in person; otherwise, remote data collection will be used with the assistance of staff.

•           Thematic scope: The evaluation will consider the full range of project objectives as outlined in the project design. It will evaluate each of the major result areas and how they interrelate. Cross-cutting issues such as gender, disability inclusion, and conflict sensitivity will be examined within each thematic area.

•           Temporal scope: Activities and results from project start up to the time of evaluation. The evaluation is not expected to assess activities prior to project inception. However, it will take into account baseline data and the context prior to project launch (from the needs analysis) to judge changes over time. It will also consider plans for the second half of the project insofar as assessing sustainability and forward-looking relevance.

•           Stakeholder scope: The evaluation will gather input from a broad range of stakeholders, including project beneficiaries (community members, IDPs, etc.), local partner organizations, local authorities (e.g. water utility companies, social services, health departments), project field staff, and the donor representation if applicable. The consultant must ensure that voices of vulnerable groups (women, elderly, persons with disabilities) are included, using appropriate methods and safeguards.

4. Evaluation Questions (Scope of Work)

To meet the above objectives, the evaluation will be guided by a set of key questions organized according to the OECD/DAC evaluation criteria. The questions will help structure the inquiry and ensure all critical aspects of performance are examined. The evaluator is expected to refine these questions in the inception phase, but at minimum, the evaluation should answer the following:

Relevance – Examining the extent to which the project design and objectives are suited to the priorities and needs of the target population and context:

•           To what extent was the project design appropriate for the needs of the target populations? Are the activities well-aligned with the most urgent needs and local socio-cultural conditions?

•           How well did the intervention align with local and national priorities, including those of affected communities, local authorities, and coordination frameworks?

Effectiveness – Assessing the extent to which the project is achieving its intended outputs and outcomes:

•           To what extent have the planned outputs and interim outcomes been achieved so far? This includes quantitative progress (e.g. number of established hubs, beneficiaries reached with aid) and qualitative results (e.g. improved well-being or coping capacity among beneficiaries). Is the project on track to meet its targets by project end?

•           Were there any unanticipated positive or negative outcomes to date? For example, did the project generate any unintended benefits (such as improved community cohesion or spill-over effects to non-target areas) or unintended harms (such as tensions over grants distribution or environmental impacts)? How has the project team addressed any unexpected issues?

Efficiency – Evaluating how well resources (funds, staff, time) are being used to achieve results:

•           Were inputs (funds, expertise, time) used efficiently to achieve the outputs? Given the challenging context, assess whether the project has managed to deliver services in a cost-effective way (e.g. leveraging local materials, avoiding delays, minimizing overhead costs).

•           Were the management and coordination structures of the project fit for purpose? This looks at whether the implementation arrangements – including the project team’s composition, decision-making processes, and coordination with partners – support timely and efficient delivery. Could the project be managed with fewer resources or in a simpler way without compromising quality?

Coherence – Considering how well the project coordinates with and complements other interventions in the same context:

•           How did the intervention complement, coordinate with, or duplicate other humanitarian or recovery efforts by local, national, or international actors? Determine if there were overlaps or gaps – for instance, is the project working in synergy with other NGOs or UN agencies in the area (avoiding duplication), and is it filling critical gaps in assistance?

•           How aligned is the project with relevant policies and strategies (e.g. Government of Ukraine humanitarian policies, local authorities’ plans, cluster coordination strategies)? Does the project design take into account existing frameworks, standards, and guidelines (such as Clusters` strategies, MHPSS guidelines, etc.), ensuring a coherent approach with wider efforts?

Impact (prospective) – Though it is early to measure long-term impact at mid-term, the evaluation should identify any emerging changes and the potential impact trajectory:

•           What observable changes have occurred in the target communities or institutions as a result of the project? Consider both intended changes and any unintended changes (either positive or negative) that communities report.

•           How has the intervention contributed to increasing income security and improving the well-being of individuals and communities in the project region? For example, are communities better prepared to handle the economical crisis or future shocks due to project activities? Although full impact cannot be measured at mid-term, the evaluator should discuss the likely longer-term effects if current progress continues.

Sustainability – Assessing the likelihood that the benefits of the project will continue after external support ends, and the measures in place to support this:

•           Are the project’s benefits and services likely to continue beyond the project’s funding period? Evaluate whether there are exit or handover strategies for each component (e.g., are established hubs maintained by local authorities or communities? Will psychosocial support groups continue with local counselors?).

•           What systems or capacities have been established to support sustainability of outcomes – so called social structures? This includes looking at whether local institutions and communities have been sufficiently involved and trained to take ownership. Identify any risks to sustainability (such as ongoing conflict or resource constraints) and how the project is mitigating them.

These questions provide a framework for the evaluation. The evaluator may add sub-questions or refine wording in the Inception Report, but any significant deviation from these key lines of inquiry should be agreed upon with AWO Int. in advance. Each evaluation question should be answered with evidence and analysis in the evaluation report, leading to conclusions and recommendations.

5. Methodology

The evaluation methodology should be rigorous, mixed-method, and participatory, ensuring that findings are credible and useful. Bidders should propose a detailed methodology in their technical proposal, but the approach is expected to include the following elements:

•           Desk Review of Documents: A thorough review of project documents and data will be conducted. This includes the project proposal and logframe, baseline or needs assessment reports, monitoring reports (quantitative output tracking, qualitative monitoring logs), prior evaluation or situational analysis documents, and any relevant external research or coordination meeting minutes. The desk review will provide context and help refine evaluation instruments.

•           Key Informant Interviews (KIIs): Semi-structured interviews will be held with key stakeholders. At a minimum, this will involve project management, representatives of national partner organizations, local government officials involved in social services, and community leaders or committee members in the target areas. Interviews may be conducted remotely (e.g. via Zoom/phone) or in-person during site visits.

•           Focus Group Discussions (FGDs): The evaluator will conduct FGDs with project beneficiaries, segmented by stakeholder group.For instance, an FGD with community members who took part in development of local strategies, or with staff of a local hubs that benefitted from repairs, can generate qualitative insights on project effectiveness and relevance. Participatory techniques should be used.  

•           Field Observations: During field travel to Bychansky Rajon, the evaluator should visit a sample of project sites (e.g. established hubs, a center where counseling is provided, households that received business trainings and start-up grants) to observe the outputs firsthand. Direct observation will be used to verify reported achievements (for example, whether household income has increased or whether a business grant has helped to start a new business) and to assess the quality of infrastructure works and appropriateness of materials provided.

The evaluation must adhere to principles of rigor and impartiality. The evaluator is expected to use appropriate sampling strategies for selecting interview and FGD participants (ensuring representation of different locations and sub-groups). The sample size and selection rationale should be explained in the inception report. Likewise, the evaluator should describe the analytical methods (e.g. coding of qualitative data, statistical analysis if any quantitative data are used) to be employed.

Given the hybrid nature of this evaluation, the methodology should leverage both remote and on-site approaches: Some interviews may be conducted remotely, while critical on-ground data collection on site will be done through visits in person. AWO International will assist with access, introductions, and adherence to security protocols, but the evaluator bears responsibility for their own travel logistics and safety precautions.

Prior to the start of data collection, the evaluator will prepare an Inception Report detailing the refined methodology, data collection tools (interview guides, survey questionnaires, etc.), sampling plan, and a work plan. The Inception Report will also identify any limitations foreseen (e.g. possible non-response, access issues) and propose mitigation strategies. Only upon approval of the Inception Report will the evaluator proceed to full data collection in the field.

Throughout the evaluation, the approach must be inclusive and conflict-sensitive. This means ensuring the participation of women and marginalized groups, providing translation to local language if needed, obtaining informed consent from all participants, and being mindful of not exacerbating any tensions or trauma. The evaluator should incorporate gender and inclusion lenses in the methodology.

In summary, the evaluation methodology is expected to be comprehensive, transparent, and robust, yielding findings that stakeholders recognize as valid. Bidders should demonstrate in their proposal how their chosen methods will effectively answer the evaluation questions while adhering to these principles.

AWO International will provide available project data and documents to support the evaluation, but the consultant is expected to be self-sufficient in conducting the study. All data collected by the evaluator will remain confidential and should be stored securely. The evaluator must also be prepared to present preliminary findings to project stakeholders for validation before finalizing the analysis (see Deliverables section).

6. Deliverables and Timeline

The consultant is expected to deliver the following outputs, in English and Ukrainian, according to the tentative schedule below. The timeline can be adjusted in the inception phase by mutual agreement, but any changes to key dates must ensure the evaluation’s utility for informing the remainder of the project:

•           Inception Report – A detailed report (max 6 pages excluding annexes) outlining the evaluation design, including refined questions, methodology, sampling plan, data collection tools, and a work plan with a schedule. The inception report should also include a brief literature review/context update and any revisions to the theory of change or evaluation matrix. An annotated table of contents for the final report may be provided.

•           Data Collection– Upon approval of the inception report, the evaluator will conduct data collection over an estimated 2-3week period. This includes remote interviews (before and/or after project travel) and visits to project sites in Buchansky Rajon.

•           Debriefing Presentation – A presentation (PowerPoint or similar) of preliminary findings, delivered to AWO International and R2P). This should occur shortly after data collection is completed. The debrief will summarize initial findings, conclusions, and tentative recommendations, allowing for validation and feedback from the project team.

•           Draft Evaluation Report – A comprehensive draft report documenting the evaluation process, findings, conclusions, and recommendations. The draft should be written in a clear and accessible manner. It must include at least: an Executive Summary (2-3 pages), Background and Context, Methodology (incl. limitations), Findings organized by evaluation criteria/questions, Conclusions, Lessons Learned, and Recommendations. Annexes should contain the ToR, data collection tools, list of people interviewed, and any other relevant technical detail (e.g. data tables). The draft will be reviewed by AWO International, who will provide feedback and comments.

•           Final Evaluation Report – The final report (max ~30 pages main text) incorporating feedback on the draft. The final version should address or acknowledge all comments received and be of publishable quality. It should include a standalone Executive Summary summarizing key findings and recommendations.

The report should include:

-       Executive summary (max. 3 pages)

-       Background and context

-       Methodology

-       Findings by evaluation criterion

-       Conclusions and lessons learned

-       Actionable recommendations

-       Annexes (tools, ToR, data tables, etc.)

All written deliverables are to be submitted in English and Ukrainian, in MS Word format and PDF for final versions. The evaluator is responsible for editing and quality control of language. The final report will be the property of AWO International. Appropriate credit will be given to the evaluator in any publications. The evaluator may not publish the report or any data without prior permission from AWO International.

Proposed Timeline: (the following timeline is indicative and will be finalized with the consultant)

-       Proposal submission deadline: 18th June 2026

-       Contract award: 26th June 2026

-       Inception phase: Online Kickoff Workshop on 30th June; inception report due on 6th July 2026

-       Data collection (remote + project sites): 7th to 31st July 2026 (including minimum 9 working days at the project sites)

-       Preliminary findings debrief: middle of August 2026

-       Draft report submission:17th August 2026

-       Participation and presentation of findings at offline one-day workshop in Kyiv (end of August 2026)

-       Final report submission: 11th September 2026

 7. Consultant Qualifications

AWO International invites proposals from individual consultants or teams with demonstrated expertise in local developments program evaluation. The lead evaluator (or evaluation team collectively) should meet the following minimum qualifications:

•           Professional Evaluation Experience: At least 5 years of experience in conducting evaluations of humanitarian assistance and local socio-economical development projects or similar relevant experience. Familiarity with evaluation of capacity-building or organizational development components will be an asset (due to the institutional strengthening aspect).

•           Technical Knowledge: Strong substantive knowledge of humanitarian principles and quality standards. Specific expertise in local socio-economical development projects in conflict settings is required. The evaluator should also be well-versed in cross-cutting issues such as gender equality, protection, and inclusion of vulnerable groups.

•           Regional/Country Experience: Prior working experience in Ukraine is mandatory. This includes familiarity with the ongoing conflict dynamics, cultural norms, and the operating environment for NGOs in Ukraine. Ability to speak or understand Ukrainian is mandatory.

•           Methodological Skills: Proven capacity to design and implement evaluations using mixed methods (qualitative and quantitative). The consultant should be skilled in designing interview guides, leading focus groups sensitively, and analyzing data. Experience with participatory evaluation approaches and remote data collection tools is important. The proposal should highlight evaluations of a similar nature that the consultant has successfully completed.

•           Donor and Quality Standards: Familiarity with BMZ  is an asset, as the project is BMZ-funded. Likewise, knowledge of OECD/DAC evaluation criteria is expected.

•           Strong Analytical and Writing Skills: The evaluator must have the ability to analyze complex information, draw out key findings, and write clear, concise, and insightful reports in English. A sample of a recent evaluation report authored by the applicant will be required. Excellent command of written and spoken English is mandatory.

•           Independence and Ethics: The consultant must be independent of the project (no prior involvement or conflict of interest) and adhere to ethical standards. If a team is proposed, team members’ roles and contributions should be clearly defined, and the team leader must have demonstrated leadership in managing evaluations.

•          

•           Availability: The team must be available to start by the proposed timeline and commit the necessary time to complete the evaluation within the schedule. Given the mid-term timing, any other commitments must not interfere with delivering the outputs on time.

9. Payment Terms

Payments will be made against invoices to a bank account with the following modalities:

●     30% upon signing the contract

●     70% upon approval of the final mid term evaluation report including annexes.

10. Application

Submission until 18^th June COB, 2026 to [email protected].

Interested persons/firms shall send their technical and financial proposal (indicating the full gross amount of their compatible consultancy fee in EUR as a lump sum figure covering a) fees (provide break down of daily rate per consultant), b) travel expenditures for the project visits and c) other costs, if reasonably justified). The technical proposal may not exceed six pages (Arial, 10) and must cover the methodological proposal and clear descriptions of (1) technical, methodological, organizational and other capacities of relevance; (2) composition of the expert team including description and division of roles and tasks for all team members; (3) statement of independence for all involved consultants. The curriculum vitae shall be annexed.

Proposals will be assessed taking into account the technical proposal (50%) and financial offer (50%).

AWO International encourages female consultants, female headed firms, veterans and members of marginalised communities to apply, because AWO International supports a working environment of diversity based on justice and mutual respect.