The Evolution of Topic Modeling

Topic models have been applied to everything from books to newspapers to social media posts in an effort to identify the most prevalent themes of a text corpus. We provide an in-depth analysis of unsupervised topic models from their inception to today. We trace the origins of different types of contemporary topic models, beginning in the 1990s, and we compare their proposed algorithms, as well as their different evaluation approaches. Throughout, we also describe settings in which topic models have worked well and areas where new research is needed, setting the stage for the next generation of topic models.

Transformational Change for People and the Planet: Evaluating Environment and Development – Introduction

The world is facing multiple crises as manifested in runaway climate change, a global pandemic, loss of ecosystems and biological species, and rapidly growing inequality. These are all closely interlinked as recognized in the 2030 Agenda for Sustainable Development. Addressing them will require broad transformational change that encompasses the economy, institutions, and how we interact with the natural environment. This chapter introduces the book that is intended to highlight how evaluation can contribute to such transformations. The chapter first reviews the state of development evaluation. It then briefly introduces the state of the global environment before discussing the implications of this context for evaluation, and how evaluation as a profession and practice must change in order to respond to the challenges of sustainability. The chapter ends by explaining the flow of the book in its four parts that focus on: transformational change, drivers of sustainability, climate change mitigation and adaptation, and evaluation approaches.

Pathway to the Transformative Policy of Agenda 2030: Evaluation of Finland’s Sustainable Development Policy

Finland has been the first country in the world to conduct a comprehensive evaluation of the national implementation the Agenda 2030. The purpose of the evaluation was to support efficient implementation of the agenda by producing information on the nation’s sustainability work for all administrative branches. The evaluation results are used for coherence in the policies and long-term sustainable development activities. The evaluation produced concrete recommendations on future directions for sustainable development policy. It also proposed future evaluation approaches.In this chapter, the authors present the evaluation approach and discuss the key results and their usage. They identify the essential elements of the utility of the evaluation in contributing to national progress of sustainable development policy. The Agenda 2030 evaluation approach was developmentally oriented and conducted in a very participatory manner. The authors reflect on the evaluative lessons learned and future options. They encourage emphasis on learning throughout the evaluation process even more in policy-level evaluations, and special attention to usefulness of the evaluation results already in evaluation design. Designing inclusive evaluation processes is a crucial precondition for evidence-informed learning and decision making in promoting transformative policy in the country context.

Evaluation in Impact Investing

The emergence of impact investing over the past decade has been accompanied by an increased interest of practitioners and scholars in the impact evaluation topic, one of the twofold pillars of the such an innovative financial approach. To contribute to the international debate, this study adopts a qualitative approach by obtaining results from a systematic literature review of extant research. This is useful to 1) identify the current existing impact evaluation approaches adopted in the field and 2) derive an empirical analysis in the impact investing sector with a focus on impact measurement in social impact bonds. The study opens interesting insights into recognizing the potential for the whole impact investing field, deriving both from theory and evidence of impact evaluation practices.

Query Processing of Geosocial Data in Location-Based Social Networks

The increasing use of social media and the recent advances in geo-positioning technologies have produced a great amount of geosocial data, consisting of spatial, textual, and social information, to be managed and queried. In this paper, we focus on the issue of query processing by providing a systematic literature review of geosocial data representations, query processing methods, and evaluation approaches published over the last two decades (2000–2020). The result of our analysis shows the categories of geosocial queries proposed by the surveyed studies, the query primitives and the kind of access method used to retrieve the result of the queries, the common evaluation metrics and datasets used to evaluate the performance of the query processing methods, and the main open challenges that should be faced in the near future. Due to the ongoing interest in this research topic, the results of this survey are valuable to many researchers and practitioners by gaining an in-depth understanding of the geosocial querying process and its applications and possible future perspectives.

The Research and Development Survey (RANDS) during COVID-19

The National Center for Health Statistics’ (NCHS) Research and Development Survey (RANDS) is a series of commercial panel surveys collected for methodological research purposes. In response to the COVID-19 pandemic, NCHS expanded the use of RANDS to rapidly monitor aspects of the public health emergency. The RANDS during COVID-19 survey was designed to include COVID-19 related health outcome and cognitive probe questions. Rounds 1 and 2 were fielded June 9–July 6, 2020 and August 3–20, 2020 using the AmeriSpeak® Panel. Existing and new approaches were used to: 1) evaluate question interpretation and performance to improve future COVID-19 data collections and 2) to produce a set of experimental estimates for public release using weights which were calibrated to NCHS’ National Health Interview Survey (NHIS) to adjust for potential bias in the panel. Through the expansion of the RANDS platform and ongoing methodological research, NCHS reported timely information about COVID-19 in the United States and demonstrated the use of recruited panels for reporting national health statistics. This report describes the use of RANDS for reporting on the pandemic and the associated methodological survey design decisions including the adaptation of question evaluation approaches and calibration of panel weights.

Investigating the constraints for managing the protection of historic buildings in remote areas of Saudi Arabia: a DEMATEL modelling approach

PurposeThe paper is aimed at identifying, analysing and prioritizing the critical constraints affecting efficient management of historic buildings in remote areas of Saudi Arabia.Design/methodology/approachThematic content analysis and expert-based evaluation approaches were used to identify and evaluate the constraints. The DEMATEL technique was deployed to define the interrelationship complexities between the constraints and evaluate the impact of these interrelationships to ascertain the influential constraints.FindingsThe results identify “lack of clearly defined roles for the multiple government agencies” as the most influential constraint for managing historic sites and buildings in remote areas of Saudi Arabia.Research limitations/implicationsThe contextual interrelationship between the constraints is due to the experts' perceptions, which may be biased due to their proficiencies and professional backgrounds. Since the evidence on which the findings of this paper are established is predominantly from experiences related to historic sites and buildings in remote areas of Saudi Arabia, the results of this paper may not be entirely applicable elsewhere.Practical implicationsThe paper provides invaluable methodology that can support practitioners and policymakers to establish sustainable strategies that can enhance the management and protection of historic buildings in remote areas of Saudi Arabia.Originality/valueThis study noticeably contributes to knowledge by providing comprehensive understanding of the major impediments to the successful management of historic buildings in remote areas of Saudi Arabia, which can assist in mitigating the potential impacts of these constraints and to advocate for the achievement of efficient management and protection of the historic sites and buildings.

TOWARDS A QUALITATIVE-QUANTITATIVE DECISION-MAKING AID FOR FATIGUE LIFE EVALUATION OF NAVAL HIGH SPEED LIGHT CRAFT

The Life of Type (LOT) of a naval High Speed Light Craft (HSLC) can be limited by its structural fatigue life. The fatigue life of a ship is influenced by many factors, such as geometry, fabrication quality, the long-term load distribution, and analytical techniques. The complex dependencies between these factors and the fatigue life cause uncertainty in predictions. A lack of understanding of uncertainty can adversely affect the management of LOT risks, resulting in the reduction of availability of the ship and costly repairs. Therefore, improved understanding of the benefits and limitations of different fatigue life evaluation approaches informs the management of risks relating to the ship’s LOT. This paper presents the first phase of work in a comparative analysis of different fatigue life evaluation approaches for naval HSLC. The present work involves a holistic data review, codification of the data to reveal key themes, and individual expert comparative analysis of the different approaches. The next phase of the study is also described.

Adaptasi Ekonomi Rumah Tangga di Desa Sukowiryo Kabupaten Jember melalui Budidaya Tanaman Sayuran secara Vertikultur

The paddy-paddy-palawija or paddy-paddy-tobacco cropping pattern is generally cultivated by most of members Hidayah Tani group in Sukowiryo Village, Jelbuk district, Jember. Meanwhile vegetables are rarely cultivated in there. Vegetable cultivation in yard around the house using verticulture is one of the household economic adaptation efforts during pandemic for increasing income. The aim of this community service are provide knowledge and insight about alternative uses of yard around the house for vegetable cultivation using verticulture, improve family welfares, and environmentally sustainable agriculture implementation. The method used in this program are preparation, socialization, training, and evaluation approaches. The results of this program showed that the community was enthusiastic in vegetable cultivation using verticulture, because its efficient built on limited yard, easy and simple to built in, could increase household income if its commercialized, supporting family food security, easy to get healty food, verticulture building is easy to move.

Development and validation pathways of artificial intelligence tools evaluated in randomised clinical trials

ObjectiveGiven the complexities of testing the translational capability of new artificial intelligence (AI) tools, we aimed to map the pathways of training/validation/testing in development process and external validation of AI tools evaluated in dedicated randomised controlled trials (AI-RCTs).MethodsWe searched for peer-reviewed protocols and completed AI-RCTs evaluating the clinical effectiveness of AI tools and identified development and validation studies of AI tools. We collected detailed information, and evaluated patterns of development and external validation of AI tools.ResultsWe found 23 AI-RCTs evaluating the clinical impact of 18 unique AI tools (2009–2021). Standard-of-care interventions were used in the control arms in all but one AI-RCT. Investigators did not provide access to the software code of the AI tool in any of the studies. Considering the primary outcome, the results were in favour of the AI intervention in 82% of the completed AI-RCTs (14 out of 17). We identified significant variation in the patterns of development, external validation and clinical evaluation approaches among different AI tools. A published development study was found only for 10 of the 18 AI tools. Median time from the publication of a development study to the respective AI-RCT was 1.4 years (IQR 0.2–2.2).ConclusionsWe found significant variation in the patterns of development and validation for AI tools before their evaluation in dedicated AI-RCTs. Published peer-reviewed protocols and completed AI-RCTs were also heterogeneous in design and reporting. Upcoming guidelines providing guidance for the development and clinical translation process aim to improve these aspects.