Using Pig as a data preparation language for large-scale mining software repositories studies: An experience report

2012 ◽  
Vol 85 (10) ◽  
pp. 2195-2204 ◽  
Author(s):  
Weiyi Shang ◽  
Bram Adams ◽  
Ahmed E. Hassan
Keyword(s):  
Large Scale ◽  
Mining Software Repositories ◽  
Experience Report ◽  
Data Preparation ◽  
Software Repositories

An Experience Report on Producing Verifiable Builds for Large-Scale Commercial Systems

2021 ◽  
pp. 1-1
Author(s):  
Young Shi ◽  
Mingzhi Wen ◽  
Filipe Roseiro Cogo ◽  
Boyuan Chen ◽  
Zhen Ming Jack Jiang
Keyword(s):  
Large Scale ◽  
Experience Report

scholarly journals GrimoireLab: A toolset for software development analytics

2021 ◽  
Vol 7 ◽  
pp. e601
Author(s):  
Santiago Dueñas ◽  
Valerio Cosentino ◽  
Jesus M. Gonzalez-Barahona ◽  
Alvaro del Castillo San Felix ◽  
Daniel Izquierdo-Cortazar ◽  
...  
Keyword(s):  
Software Development ◽  
Preliminary Analysis ◽  
Data Retrieval ◽  
Mining Software Repositories ◽  
Data Sources ◽  
Data Sets ◽  
Community Based ◽  
Academic Environments ◽  
Software Repositories ◽  
Main Components

Background After many years of research on software repositories, the knowledge for building mature, reusable tools that perform data retrieval, storage and basic analytics is readily available. However, there is still room to improvement in the area of reusable tools implementing this knowledge. Goal To produce a reusable toolset supporting the most common tasks when retrieving, curating and visualizing data from software repositories, allowing for the easy reproduction of data sets ready for more complex analytics, and sparing the researcher or the analyst of most of the tasks that can be automated. Method Use our experience in building tools in this domain to identify a collection of scenarios where a reusable toolset would be convenient, and the main components of such a toolset. Then build those components, and refine them incrementally using the feedback from their use in both commercial, community-based, and academic environments. Results GrimoireLab, an efficient toolset composed of five main components, supporting about 30 different kinds of data sources related to software development. It has been tested in many environments, for performing different kinds of studies, and providing different kinds of services. It features a common API for accessing the retrieved data, facilities for relating items from different data sources, semi-structured storage for easing later analysis and reproduction, and basic facilities for visualization, preliminary analysis and drill-down in the data. It is also modular, making it easy to support new kinds of data sources and analysis. Conclusions We present a mature toolset, widely tested in the field, that can help to improve the situation in the area of reusable tools for mining software repositories. We show some scenarios where it has already been used. We expect it will help to reduce the effort for doing studies or providing services in this area, leading to advances in reproducibility and comparison of results.

scholarly journals Guest Editorial: Mining software repositories

2017 ◽  
Vol 22 (3) ◽  
pp. 1143-1145 ◽  
Author(s):  
Romain Robbes ◽  
Yasutaka Kamei ◽  
Martin Pinzger
Keyword(s):  
Guest Editorial ◽  
Mining Software Repositories ◽  
Software Repositories
Sign in / Sign up

Export Citation Format

Share Document