Swiftlink: a distributed, parallelised and scriptable linkage engine architecture for the 21st century

IntroductionMany currently extant generic tools for performing bulk linkage were developed last century and while they have accreted features to improve their utility they have failed to fully exploit modern computing architectures. Custom highly-parallelised and distributed solutions exist but there is a need for a modern generalised linkage toolkit. Objectives and ApproachThe aim of this project has been to design, from the ground up, a modern flexible, i.e. scriptable, linkage suite which also takes advantage of contemporary computing architectures to address issues such as distribution of computation, parallelisation of computation and cloud-based computation. Rather than being a monolithic linkage tool or a programming suite, a domain specific language has been developed to specifically describe linkage tasks. Linkage tasks written in this language are then 'executed' or 'compiled and run' to perform pair-wise calculations on data elements. Since linkage tasks are generally bespoke, scriptability has been an important consideration. ResultsDeveloping a domain specific linkage language has enabled problem specification to be more descriptive and flexible than a monolithic linkage application. The shift in focus from a programming language to a linkage language has enabled a corresponding shift in focus to linkage-related effort (such as blocking and comparison strategies) away from distracting 'glue' code which relates not to linkage tasks under consideration but rather to the distracting bookkeeping aspects of programme execution. The same linkage task may be compiled against different back ends and languages, e.g. FEBRL (python), swift, Amazon lambda (go). The architecture has enabled otherwise idle computing resource to be utilised as well as cloud-based computing facilities for increased throughput and performance. The architecture of the linkage system will be shown with examples. Conclusion/ImplicationsContemporary advances in computing sciences can and must be leveraged in modern linkage tools. By providing a custom scriptable linkage language, tasks may be more clearly specified in a manner more flexible than monolithic linkage applications and by uncoupling linkage specification from execution, linkage may be performed optimally across multiple machines and resources.

Embedding an interpreted language using higher-order functions and types

Using an embedded, interpreted language to control a complicated application can have significant software-engineering benefits. But existing interpreters are designed for embedding into C code. To embed an interpreter into a different language requires an API suited to that language. This paper presents Lua-ML, a new API that is suited to languages that provide higher-order functions and types. The API exploits higher-order functions and types to reduce the amount of glue code needed to use an embedded interpreter. Where embedding in C requires a special-purpose “glue function” for every function to be embedded, embedding in Lua-ML requires only a description of each function's type. Lua-ML also makes it easy to define a Lua function whose behavior depends on the number and types of its arguments.

Separation of Concerns in Model-Based Development of Distributed Real-Time Systems

Software is increasingly being used to enable new features in systems in multiple domains. These domains include automotive, avionics, telecomunication, and industrial automation. Because the user of these systems is not aware of the presence of the software, this type of software is known as embedded software. More importantly, such a software, and the whole system in general, must satisfy not only logical functional requirements but also parafunctional (a.k.a. nonfunctional) properties such as timeliness, security, and reliability. Traditional development languages and tools provide powerful abstractions such as functions, classes, and objects to build a functional structure that reduces complexity and enables software reuse. However, the software elements responsible for the parafunctional behaviors are frequently scattered across the functional structure. This scattering prevents the easy identification of these elements and their independent manipulation/reuse to achieve a specific parafunctional behavior. As a result, the complexity of parafunctional behaviors cannot be reduced and even worse, the construction of those behaviors can corrupt the functional structure of the software. In this chapter, we propose a model-based framework for designing embedded real-time systems to enable a decomposition structure that reduces the complexity of both functional and parafunctional aspects of the software. This decomposition enables the separation of the functional and parafunctional aspects of the system into semantic dimensions (e.g., event-flow, timing, deployment, fault-tolerant) that can be represented, manipulated, and modified independent of one another from an end-user point of view. The realizations of these dimensions, however, do interact on the target platform since they consume common resources and impose constraints. These interactions can be captured during model construction and resource demands mediated during platform deployment. The use of semantic dimensions results in three significant benefits. First of all, it preserves the independence of the functional structure from parafunctional behaviors. Secondly, it enables the user to manipulate different parafunctional concerns (e.g., timeliness, reliability) independent of one another. Lastly, it enables the reuse of compositions along any dimension from other systems. The second core abstraction in our modeling approach is an entity called a coupler. A coupler expresses a particular relationship between two or more components, and can also be used recursively. Couplers enable the hierarchical decomposition of functional as well as parafunctional aspects. Aided by semantic dimensions and multiple coupler types, our framework enables the auto-generation of glue code to produce a fully deployable system. Our framework can also construct a detailed timing and resource model. This model in turn is used to optimize the usage of a given hardware configuration, or synthesize a configuration to suit a given software model. Our framework is implemented in a tool (de Niz, Bhatia & Rajkumar 2006) called SysWeaver that had been used to generate glue code and analyze the timing behavior of avionics, automotive, and software-radio pilot systems.

A role-based component architecture for computer assisted interventions: illustration for electromagnetic tracking and robotized motion rejection in flexible endoscopy

This paper presents an original role-based software architecture facilitating the flexible composition, configuration and collaboration of separated components in the field of computer assisted interventions. Roles, which can be seen as methods dynamically attached to objects, are embedded in components, to limit build level dependencies and improve flexibility. An appropriate component definition and composition language is proposed to declare softwares, without any specific initialization or glue code, this remaining a challenging issue in component oriented programming. The potential of this architecture is illustrated for a software coupling electromagnetic tracking with a robotized system dedicated to the physiological motion rejection in flexible endoscopy. This software consists in several independent components with are combined at runtime thanks to a concise XML-based declaration.

Managing Scientific Software Complexity with Bocca and CCA

In high-performance scientific software development, the emphasis is often on short time to first solution. Even when the development of new components mostly reuses existing components or libraries and only small amounts of new code must be created, dealing with the component glue code and software build processes to obtain complete applications is still tedious and error-prone. Component-based software meant to reduce complexity at the application level increases complexity to the extent that the user must learn and remember the interfaces and conventions of the component model itself. To address these needs, we introduce Bocca, the first tool to enable application developers to perform rapid component prototyping while maintaining robust software-engineering practices suitable to HPC environments. Bocca provides project management and a comprehensive build environment for creating and managing applications composed of Common Component Architecture components. Of critical importance for high-performance computing (HPC) applications, Bocca is designed to operate in a language-agnostic way, simultaneously handling components written in any of the languages commonly used in scientific applications: C, C++, Fortran, Python and Java. Bocca automates the tasks related to the component glue code, freeing the user to focus on the scientific aspects of the application. Bocca embraces the philosophy pioneered by Ruby on Rails for web applications: start with something that works, and evolve it to the user's purpose.