scholarly journals ARCH-COMP18 Repeatability Evaluation Report

10.29007/n9t3 ◽  
2018 ◽  
Author(s):  
Taylor T. Johnson
Keyword(s):  
Hybrid Systems ◽  
International Competition ◽  
Benchmark Problems ◽  
Evaluation Report ◽  
Standard Practice

This report presents the results of the repeatability evaluation for the 2nd International Competition on Verifying Continuous and Hybrid Systems (ARCH-COMP'18). The competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2018. In its second edition, twenty-five tools submitted artifacts for the repeatability evaluation and applied to solve benchmark problems for seven competition categories. The repeatability results represent a snapshot of the current landscape of tools and the types of benchmarks for which they are particularly suited and for which others may repeat their analyses. Due to the diversity of problems in verification of continuous and hybrid systems, as well as basing on standard practice in repeatability evaluations, we evaluate the tools with pass and/or failing being repeatable.

scholarly journals ARCH-COMP19 Repeatability Evaluation Report

10.29007/wbl3 ◽  
2019 ◽  
Author(s):  
Taylor T. Johnson
Keyword(s):  
Internet Of Things ◽  
Virtual Environments ◽  
Hybrid Systems ◽  
Cyber Physical Systems ◽  
International Competition ◽  
Benchmark Problems ◽  
Evaluation Report ◽  
Standard Practice ◽  
Physical Systems

This report presents the results of the repeatability evaluation for the 3rd International Competition on Verifying Continuous and Hybrid Systems (ARCH-COMP'19). The competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2019, affiliated with the Cyber-Physical Systems and Internet of Things (CPS-IoT Week'19). In its third edition, twenty-five tools submitted artifacts through a Git repository for the repeatability evaluation, applied to solve benchmark problems for eight competition categories. The majority of participants adhered to new requirements for this year's repeatability evaluation, namely to submit scripts to automatically install and execute tools in containerized virtual environments (specifically Dockerfiles to execute within Docker). The repeatability results represent a snapshot of the current landscape of tools and the types of benchmarks for which they are particularly suited and for which others may repeat their analyses. Due to the diversity of problems in verification of continuous and hybrid systems, as well as basing on standard practice in repeatability evaluations, we evaluate the tools with pass and/or failing being repeatable.

scholarly journals ARCH-COMP17 Repeatability Evaluation Report

10.29007/7hvk ◽  
2018 ◽  
Author(s):  
Taylor T. Johnson
Keyword(s):  
Formal Verification ◽  
Hybrid Systems ◽  
Benchmark Problems ◽  
Safety Verification ◽  
Evaluation Report ◽  
Standard Practice ◽  
Friendly Competition

This report presents the results of the repeatability evaluation for a friendly competition for formal verification of continuous and hybrid systems. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2017. In its first edition, thirteen tools have been applied to solve benchmark problems for the six competition categories, of which, ten tools were evaluated and passed the repeatability evaluation. The repeatability results represent a snapshot of the current landscape of tools and the types of benchmarks for which they are particularly suited and for which others may repeat their analyses. Due to the diversity of problems in verification of continuous and hybrid systems, as well as basing on standard practice in repeatability evaluations, we evaluate the tools with pass and/or failing being repeatable. These re- sults probably provide the most complete assessment of tools for the safety verification of continuous and hybrid systems up to this date.

scholarly journals ARCH-COMP20 Repeatability Evaluation Report

10.29007/8dp4 ◽  
2020 ◽  
Author(s):  
Taylor T Johnson
Keyword(s):  
Performance Evaluation ◽  
Virtual Environments ◽  
Hybrid Systems ◽  
International Competition ◽  
Benchmark Problems ◽  
Fourth Edition ◽  
World Congress ◽  
Evaluation Report ◽  
Execution Platform ◽  
Evaluation Information

This report presents the results of the repeatability evaluation for the 4th International Competition on Verifying Continuous and Hybrid Systems (ARCH-COMP’20). The competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2020, affiliated with the IFAC World Congress. In its fourth edition, twenty-eight tools submitted artifacts through a Git repository for the repeatability evaluation, applied to solve benchmark problems for seven competition categories. The majority of participants adhered to the requirements for this year’s repeatability evaluation, namely to submit scripts to automatically install and execute tools in containerized virtual environments (specifically Dockerfiles to execute within Docker), and several categories used performance evaluation information from a common execution platform. The repeatability results represent a snapshot of the current landscape of tools and the types of benchmarks for which they are particularly suited and for which others may repeat their analyses. Due to the diversity of problems in verification of continuous and hybrid systems, as well as basing on standard practice in repeatability evaluations, we evaluate the tools with pass and/or failing being repeatable.

scholarly journals ARCH-COMP18 Category Report: Continuous and Hybrid Systems with Linear Continuous Dynamics

10.29007/73mb ◽  
2018 ◽  
Author(s):  
Matthias Althoff ◽  
Stanley Bak ◽  
Xin Chen ◽  
Chuchu Fan ◽  
Marcelo Forets ◽  
...  
Keyword(s):  
Formal Verification ◽  
Hybrid Systems ◽  
Benchmark Problems ◽  
Safety Verification ◽  
Alphabetical Order ◽  
Friendly Competition

This report presents the results of a friendly competition for formal verification of continuous and hybrid systems with linear continuous dynamics. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2018. In its second edition, 9 tools have been applied to solve six different benchmark problems in the category for linear continuous dynamics (in alphabetical order): CORA, CORA/SX, C2E2, Flow*, HyDRA, Hylaa, Hylaa-Continuous, JuliaReach, SpaceEx, and XSpeed. This report is a snapshot of the current landscape of tools and the types of benchmarks they are particularly suited for. Due to the diversity of problems, we are not ranking tools, yet the presented results probably provide the most complete assessment of tools for the safety verification of continuous and hybrid systems with linear continuous dynamics up to this date.

scholarly journals ARCH-COMP18 Category Report: Hybrid Systems with Piecewise Constant Dynamics

10.29007/p11g ◽  
2018 ◽  
Author(s):  
Goran Frehse ◽  
Alessandro Abate ◽  
Dieky Adzkiya ◽  
Lei Bu ◽  
Mirco Giacobbe ◽  
...  
Keyword(s):  
Formal Verification ◽  
Hybrid Systems ◽  
Benchmark Problems ◽  
Safety Verification ◽  
Railway Network ◽  
Piecewise Constant ◽  
Friendly Competition

This report presents results of a friendly competition for formal verification of continuous and hybrid systems with piecewise constant dynamics. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2018. In this second edition, five tools have been applied to solve five different benchmark problems in the category for piecewise constant dynamics: BACH, Lyse, PHAVer, PHAVer-lite, and VeriSiMPL. Compared to last year, a new tool has participated (PHAVer-lite) and a benchmark has been made more complex (Dutch Railway Network). The result is a snapshot of the current landscape of tools and the types of benchmarks they are particularly suited for. Due to the diversity of problems, we are not ranking tools, yet the presented results probably provide the most complete assessment of tools for the safety verification of continuous and hybrid systems with piecewise constant dynamics up to this date.

scholarly journals ARCH-COMP20 Category Report: Continuous and Hybrid Systems with Nonlinear Dynamics

10.29007/zkf6 ◽  
2020 ◽  
Author(s):  
Luca Geretti ◽  
Julien Alexandre Dit Sandretto ◽  
Matthias Althoff ◽  
Luis Benet ◽  
Alexandre Chapoutot ◽  
...  
Keyword(s):  
Nonlinear Dynamics ◽  
Formal Verification ◽  
Hybrid Systems ◽  
Reachability Analysis ◽  
Current Status ◽  
Benchmark Problems ◽  
Hybrid Dynamics ◽  
Friendly Competition

We present the results of a friendly competition for formal verification of continuous and hybrid systems with nonlinear continuous dynamics. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2020. This year, 6 tools Ariadne, CORA, DynIbex, Flow*, Isabelle/HOL, and JuliaReach (in alphabetic order) participated. These tools are applied to solve reachability analysis problems on six benchmark problems, two of them featuring hybrid dynamics. We do not rank the tools based on the results, but show the current status and discover the potential advantages of different tools.

scholarly journals ARCH-COMP17 Category Report: Bounded Model Checking of Hybrid Systems with Piecewise Constant Dynamics

10.29007/rvk6 ◽  
2018 ◽  
Author(s):  
Lei Bu ◽  
Rajarshi Ray ◽  
Stefan Schupp
Keyword(s):  
Model Checking ◽  
Formal Verification ◽  
Hybrid Systems ◽  
Bounded Model Checking ◽  
Benchmark Problems ◽  
Future Event ◽  
Alphabetical Order ◽  
Piecewise Constant ◽  
The Future ◽  
Friendly Competition

This report presents results of a friendly competition for formal verification of continuous and hybrid systems with linear continuous dynamics. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2017. In its first edition, three tools have been applied to solve three different benchmark problems in the category ofbounded model checking of hybrid systems with piecewise constant dynamics (in alphabetical order): BACH, HyDRA, and XSpeed. The result is a snapshot of the current landscape of tools and the types of benchmarks they are particularly suited for. Due to the diversity of problems, we are not ranking tools and we also welcome more tools to join in this friendly competition in the future event.

scholarly journals ARCH-COMP19 Category Report: Hybrid Systems with Piecewise Constant Dynamics

10.29007/rjwn ◽  
2019 ◽  
Author(s):  
Goran Frehse ◽  
Alessandro Abate ◽  
Dieky Adzkiya ◽  
Anna Becchi ◽  
Lei Bu ◽  
...  
Keyword(s):  
Formal Verification ◽  
Hybrid Systems ◽  
Benchmark Problems ◽  
Safety Verification ◽  
Piecewise Constant ◽  
Friendly Competition

This report presents the results of a friendly competition for formal verification of continuous and hybrid systems with piecewise constant dynamics. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2019. In this third edition, six tools have been applied to solve five different benchmark problems in the category for piecewise constant dynamics: BACH, Lyse, Hy- COMP, PHAVer/SX, PHAVerLite, and VeriSiMPL. Compared to last year, a new tool has participated (HyCOMP) and PHAVerLite has replaced PHAVer-lite. The result is a snap- shot of the current landscape of tools and the types of benchmarks they are particularly suited for. Due to the diversity of problems, we are not ranking tools, yet the presented results probably provide the most complete assessment of tools for the safety verification of continuous and hybrid systems with piecewise constant dynamics up to this date.

scholarly journals ARCH-COMP19 Category Report: Bounded Model Checking of Hybrid Systems with Piecewise Constant Dynamics

10.29007/g965 ◽  
2019 ◽  
Author(s):  
Lei Bu ◽  
Rajarshi Ray ◽  
Stefan Schupp
Keyword(s):  
Model Checking ◽  
Formal Verification ◽  
Hybrid Systems ◽  
Bounded Model Checking ◽  
Optimization Techniques ◽  
Benchmark Problems ◽  
Future Event ◽  
Alphabetical Order ◽  
Piecewise Constant ◽  
Friendly Competition

This report presents results of a friendly competition for formal verification of continuous and hybrid systems with linear continuous dynamics. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2019. In its third edition, three tools have been applied to solve three different benchmark problems in the category ofbounded model checking of hybrid systems with piecewise constant dynamics (in alphabetical order): BACH, HyDRA, and XSpeed. Compare to last year, HyDRA is equipped with new optimization techniques and the performance is improved accordingly. This report is a snapshot of the current landscape of tools and the types of benchmarks they are particularly suited for. Due to the diversity of problems, we are not ranking tools and we also welcome more tools to join in this friendly competition in the future event.

scholarly journals ARCH-COMP18 Category Report: Bounded Model Checking of Hybrid Systems with Piecewise Constant Dynamics

10.29007/q5tq ◽  
2018 ◽  
Author(s):  
Lei Bu ◽  
Rajarshi Ray ◽  
Stefan Schupp
Keyword(s):  
Model Checking ◽  
Formal Verification ◽  
Hybrid Systems ◽  
Bounded Model Checking ◽  
Benchmark Problems ◽  
Future Event ◽  
Alphabetical Order ◽  
Piecewise Constant ◽  
The Future ◽  
Friendly Competition

This report presents results of a friendly competition for formal verification of contin- uous and hybrid systems with linear continuous dynamics. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2018. In its second edition, three tools have been applied to solve three differ- ent benchmark problems in the category ofbounded model checking of hybrid systems with piecewise constant dynamics (in alphabetical order): BACH, HyDRA, and XSpeed. This report is a snapshot of the current landscape of tools and the types of benchmarks they are particularly suited for. Due to the diversity of problems, we are not ranking tools and we also welcome more tools to join in this friendly competition in the future event.

Sign in / Sign up

Export Citation Format

Share Document