Publishing in the social sciences and its representation in research evaluation and funding systems

The paper describes the full range of publishing and its purposes in the social sciences – from scholarly publishing via professional communication to societal interaction in public media – and how it is represented in five different contexts for research evaluation and funding. The five contexts are: applications for external project funding, applications for positions or promotions, indicator-based institutional funding systems, summative organizational evaluation systems, and formative organizational evaluation systems. The chapter provides a critical discussion of how publications from the social sciences may be filtered out or placed in predetermined hierarchies in these evaluation and funding contexts, and also of how the evaluation and funding procedures can be improved to appropriately represent social science research and publishing.

Building Evaluation Capacity in Youth-Serving Organizations Through Evaluation Advisory Boards

Youth-serving organizations seek effective and cost-efficient solutions to build evidence and advance their impact. Some common challenges include choosing data systems or assessments, budgeting and planning for 3rd-party studies, and refining measurement and outcomes when programs expand or change. Evaluation advisory boards (EABs) are a low-cost solution to add evaluation capacity and can be mutually beneficial to both youth-serving organizations and evaluation experts. Previous research suggests that EABs may encourage meaningful use of data, support internal evaluators, and/or facilitate difficult conversations among stakeholders. However, there are very few examples of successful EABs in practice. This paper shares the perspectives of EAB members and organizational evaluation leaders from a large national after-school program, After-School All-Stars (ASAS), including (a) a description of the benefits of EABs, (b) how EABs may be especially helpful with the context of the COVID-19 pandemic, and (c) examples of youth-serving organizations’ EABs. The experiences and lessons learned by ASAS and its EAB are generalizable to other non-profit youth development programs. Recommendations for structuring EABs based on organizational goals are provided.

Structures and functions of complex evaluation systems: comparison of six Central and Eastern European countries

Evaluation practice is vital for the accountability and learning of administrations implementing complex policies. This article explores the relationships between the structures of the evaluation systems and their functions. The findings are based on a comparative analysis of six national systems executing evaluation of the European Union Cohesion Policy. The study identifies three types of evaluation system structure: centralized with a single evaluation unit, decentralized with a coordinating body and decentralized without a coordinating body. These systems differ in terms of the thematic focus of evaluations and the targeted users. Decentralized systems focus on internal users of knowledge and produce mostly operational studies; their primary function is inward-oriented learning about smooth programme implementation. Centralized systems fulfil a more strategic function, recognizing the external audience and external accountability for effects. Points for practitioners Practitioners who design multi-organizational evaluation systems should bear in mind that their structure and functions are interrelated. If both accountability and learning are desired, the evaluation system needs at least a minimum degree of decentralization on the one hand and the presence of an active and independent coordination body on the other.

Understanding Evaluation Policy and Organizational Capacity for Evaluation: An Interview Study

Evaluation policy has been identified as an important means of shaping and influencing organizational evaluation practice, yet, to date, little empirical research has been conducted to deepen our understanding of this relationship. The purpose of this study was to illuminate evaluation policy’s role in leveraging organizational capacity to do and use evaluation. We interviewed 18 published evaluation scholars and practitioners from North America and Europe about this topic. A thematic analysis of findings underscores the importance of context, policy attributes, enablers, and organizational benefits. Based on the findings, we developed an ecological conceptual framework to guide thinking about the role of evaluation policy in capacity building. We discuss these findings in terms of practical implications of understanding context, redressing the imbalance between learning and accountability purposes of evaluation, and organizational leadership, and we conclude with some implications for research.

Toward an Evidence-Based Approach to Building Evaluation Capacity

Organizations are required to evaluate their programs for both learning and accountability purposes, which has increased the need to build their internal evaluation capacity. A remaining challenge is access to tools that lead to valid evidence supporting internal capacity development. The authors share practical insights from the development and use of the Evaluation Capacity Needs Assessment tool and framework and implications for using its data to make concrete decisions within Canadian contexts. The article refers to validity evidence generated from factor analyses and structural equation modelling and describes how applying the framework can be used to identify individual and organizational evaluation capac­ity strengths and gaps, concluding with practice considerations and future directions for this work.

Metodologia de avaliação organizacional de universidades corporativas

Purpose – In the available literature, we can find research on how to conduct an assessment of an organization’s Corporate Education, but not of the Corporate University. Therefore, in our research we propose a methodology to evaluate the organizational management of Corporate Universities.Design/methodology/approach – This research is descriptive and uses the multicriteria analysis method as basis. Specifically, we use the Multi-Attribute Global Inference of Quality (Magiq) methodology for the definition of weights, to measure the importance of factors and sub-factors that determine the organizational evaluation of Corporate Universities.Findings – The resulting methodology contemplates six factors and 18 sub-factors. The six factors in descending order of weights are: Strategic Management, Organizational Learning, Process, Performance, Institutional and Market.Research limitations/implications – The fact that our research is based on the knowledge of Brazilian experts, it infers the use of this methodology only for the case Brazil, which can be seen as a limitation of this research.Practical implications – The leaders of Corporate Universities will henceforth be able to mirror and look for greater performance to achieve their goals.Originality/value – Our research intends to contribute to reduce the gap either in theory and practice of how to evaluate Corporate Universities. From there, we hope to make possible to compare them, to create rankings and to establish awards, since the existing literature provides criteria on how to evaluate Corporate Education, but not Corporate Universities.

Governance of knowledge and innovation in the Ibero-American agri-food system

Aim of study: Governance and the knowledge and innovation system (KIS) are interrelated concepts. Knowledge management best practices are linked to KIS performance. This article explores the governance of the leading research, development, and innovation institutes in Ibero-American agriculture, food, and agro-industry sector. The paper reports mapping of the governance of 20 agricultural research institutes.Area of study: Latin America and the Caribbean, Spain and Portugal.Material and methods: In total, 51 strategic objectives for effective governance were identified. Self-evaluation by the National Agricultural Research Institutes (NARIs) was validated at a workshop backed by FONTAGRO, a cooperation mechanism amongst Latin American and the Caribbean countries, Portugal and Spain, and the Ibero-American Network of NARIs.Main results: As a strength, the key dimension of NARIs appears to be coordination and cooperation. This result was acknowledged in the internal and external evaluations and supports previous research on the relevance of innovation networks in Latin America. By contrast, as a challenge, the key dimension appears to be demand articulation, followed closely by capacity building. Most of the institutes are also well-positioned to develop deeper ties with social and environmental challenges.Research highlights: In the medium and long term, NARIs should make efforts to improve the processes of organizational evaluation and learning, demand articulation, and strategic direction of the institutions. Improvement in management processes, in addition to best practices social responsibility and gender equality, appear to be short-term priorities.

Implementasi Business Intelligence Pada Analisis Perkembangan Hasil Pertanian Provinsi Sumatera Selatan

In the process of data collection of agricultural food products and horticulture consists of several data covering the time, area, area of land, crop yields, and types of plants stored in the form of files with large amounts of data. The large amount of data can cause errors in compiling information for the process of analysis and evaluation for the agriculture department. The analysis process can be faster and more precise by implementing Business intelligence. Business intelligence is a concept and method for managing and analyzing data and facts into more effective information to improve the quality of organizational evaluation and decision-making processes. One form of information that can be provided from the application of business intelligence is a visual dashboard that can help display information that makes it easier to analyze the development of agricultural products efficiently and interactively. Processing data for analysis in this study using the Pentaho application. By implementing Business Intelligence, it is expected to be able to assist the Department of Food Agriculture and Horticulture in South Sumatra Province in analyzing the development of agricultural products

Caractéristiques organisationnelles qui influencent le renforcement des capacités en évaluation chez les organismes communautaires du Québec : une recension des écrits

Few studies have conceptualized evaluation capacity for community organizations (COs), and even fewer in the specific context of COs in Québec. The objective of this article, therefore, is twofold: 1) to identify barriers to, and opportunities for, building evaluation capacity among OCs and 2) to verify the extent to which an existing conceptual framework of organizational evaluation capacity established by Isabelle Bourgeois and J. Bradley Cousins could be useful in its current form for understanding the evaluation capacity needs of COs in Québec. We first conducted a review of the scientific literature to identify the barriers and opportunities for evaluation capacity building in OCs, and analyzed the results of the literature review against the conceptual framework. Our results point to the fact that Bourgeois and Cousins’ framework is appropriate for understanding the key factors influencing the capacity to do and to use evaluations among OCs. Additional efforts would, however, be necessary to improve the framework’s compatibility with the practices and values of COs in Québec.Les capacités en évaluation sont peu étudiées et encore moins conceptualisées en fonction du contexte particulier des organismes communautaires (OC) du Québec. Ainsi, l’objectif de cet article est double : 1) identifier chez les OC les barrières et les éléments qui facilitent le renforcement des capacités en évaluation et 2) vérifier dans quelle mesure un cadre conceptuel établi par Isabelle Bourgeois et J. Bradley Cousins pourrait s’avérer propice dans sa forme actuelle pour l’analyse des capacités en évaluation du milieu communautaire québécois. Nous avons d’abord identifié les barrières et les éléments qui facilitent le renforcement des capacités en évaluation des OC à partir d’une recherche documentaire d’articles scientifiques pertinents pour ensuite effectuer une analyse à partir du cadre conceptuel choisi. Nos résultats démontrent que le cadre conceptuel de Bourgeois et Cousins est pertinent pour répertorier les facteurs clés influençant la capacité à effectuer et à utiliser l’évaluation chez les OC. Des efforts supplémentaires seraient toutefois nécessaires pour améliorer la compatibilité du cadre conceptuel avec les pratiques et valeurs du milieu communautaire québécois.

Herramienta para medir la gestión de la Responsabilidad Social Empresarial en empresas constructoras

la presente investigación responde a la interrogante de que si es posible generar una herramienta para medir la gestión de la Responsabilidad Social Empresarial (RSE) en las empresas constructoras. La metodología aplicada es de tipo cualitativo, tomando como base conceptual para la realización de la herramienta El Libro Verde. Fomentar un marco europeo para la responsabilidad social de las empresas, de la comisión de las comunidades europeas y la herramienta brindada por el CERES 2010. Como resultado, se expone una herramienta validada para analizar la gestión de RSE en empresas constructoras, a base de los ejes: gobernabilidad, relaciones laborales, medio ambiente, proveedores, clientes, comunidades y políticas públicas; con la finalidad de que organizaciones de este sector empresarial puedan evaluar e identificar las principales falencias en cuanto a su gestión de la RSE y, con ello, establecer estrategias y facilitar la toma de decisiones. Palabras clave: Constructora, Evaluación organizacional, Gestión, RSE, Stakeholders. The present investigation responds to the question of whether it is possible to create a tool to measure the management of corporate social responsibility (CSR) of construction companies. The methodology applied is of a qualitative nature as a conceptual basis for the realization of the tool is El Libro Verde, Fomentar un marco europeo para la responsabilidad social de las empresas of The Commission of the European Communities and the Tool Provided by CERES 2010. The result is a validated tool for the construction organizations based on the axes: governance, labor relations, environment, suppliers, customers, communities and public policies. Giving opportunity to the organizations to this kind of organization to evaluate and identify the main shortcomings in terms of their management of CSR and with this, strategies are established to facilitate decision-making.Key words: Construction, Management, Organizational Evaluation, CSR, Stakeholders