From mess to mass: a methodology for calculating storm event pollutant loads with their uncertainties, from continuous raw data time series

2011 ◽  
Vol 63 (3) ◽  
pp. 369-376 ◽  
Author(s):  
M. Métadier ◽  
J. -L. Bertrand-Krajewski
Keyword(s):  
Time Series ◽  
Oxygen Demand ◽  
Software Tool ◽  
Large Data ◽  
Sensor Calibration ◽  
Storm Event ◽  
Storm Events ◽  
Time Step ◽  
Raw Data ◽  
Sewer Systems

With the increasing implementation of continuous monitoring of both discharge and water quality in sewer systems, large data bases are now available. In order to manage large amounts of data and calculate various variables and indicators of interest it is necessary to apply automated methods for data processing. This paper deals with the processing of short time step turbidity time series to estimate TSS (Total Suspended Solids) and COD (Chemical Oxygen Demand) event loads in sewer systems during storm events and their associated uncertainties. The following steps are described: (i) sensor calibration, (ii) estimation of data uncertainties, (iii) correction of raw data, (iv) data pre-validation tests, (v) final validation, and (vi) calculation of TSS and COD event loads and estimation of their uncertainties. These steps have been implemented in an integrated software tool. Examples of results are given for a set of 33 storm events monitored in a stormwater separate sewer system.

scholarly journals Assessing dry weather flow contribution in TSS and COD storm events loads in combined sewer systems

2011 ◽  
Vol 63 (12) ◽  
pp. 2983-2991 ◽  
Author(s):  
M. Métadier ◽  
J. L. Bertrand-Krajewski
Keyword(s):  
Time Series ◽  
Oxygen Demand ◽  
Seasonal Effect ◽  
Storm Events ◽  
Combined System ◽  
Time Step ◽  
Urban Catchment ◽  
Sewer Systems ◽  
Continuous Discharge

Continuous high resolution long term turbidity measurements along with continuous discharge measurements are now recognised as an appropriate technique for the estimation of in sewer total suspended solids (TSS) and Chemical Oxygen Demand (COD) loads during storm events. In the combined system of the Ecully urban catchment (Lyon, France), this technique is implemented since 2003, with more than 200 storm events monitored. This paper presents a method for the estimation of the dry weather (DW) contribution to measured total TSS and COD event loads with special attention devoted to uncertainties assessment. The method accounts for the dynamics of both discharge and turbidity time series at two minutes time step. The study is based on 180 DW days monitored in 2007–2008. Three distinct classes of DW days were evidenced. Variability analysis and quantification showed that no seasonal effect and no trend over the year were detectable. The law of propagation of uncertainties is applicable for uncertainties estimation. The method has then been applied to all measured storm events. This study confirms the interest of long term continuous discharge and turbidity time series in sewer systems, especially in the perspective of wet weather quality modelling.

scholarly journals Managing and Utilizing Big Data in Atmospheric Monitoring Systems for Underground Coal Mines

2021 ◽  
Vol 5 (1) ◽  
pp. 78
Author(s):  
Juan Diaz ◽  
Zach Agioutantis ◽  
Dionissios T. Hristopulos ◽  
Steven Schafrik
Keyword(s):  
Time Series ◽  
Real Time ◽  
Monitoring And Evaluation ◽  
Barometric Pressure ◽  
Nonlinear Dependence ◽  
Sensor Calibration ◽  
Monitoring Systems ◽  
Transient Effects ◽  
Time Step ◽  
Atmospheric Monitoring

Underground coal mining Atmospheric Monitoring Systems (AMS) have been implemented for real-time or near real-time monitoring and evaluation of the mine atmosphere and related parameters such as gas concentration (e.g., CH4, CO, O2), fan performance (e.g., power, speed), barometric pressure, ambient temperature, humidity, etc. Depending on the sampling frequency, AMS can collect and manage a tremendous amount of data, which mine operators typically consult for everyday operations as well as long-term planning and more effective management of ventilation systems. The raw data collected by AMS need considerable pre-processing and filtering before they can be used for analysis. This paper discusses different challenges related to filtering raw AMS data in order to identify and remove values due to sensor breakdowns, sensor calibration periods, transient values due to operational considerations, etc., as well as to homogenize time series for different variables. The statistical challenges involve the removal of faulty values and outliers (due to systematic problems) and transient effects, gap-filling (by means of interpolation methods), and homogenization (setting a common time reference and time step) of the respective time series. The objective is to derive representative and synchronous time series values that can subsequently be used to estimate summary statistics of AMS and to infer correlations or nonlinear dependence between different data streams. Identification and modeling of statistical dependencies can be further exploited to develop predictive equations based on time series models.

Modelling of overflow emissions in flanders

1998 ◽  
Vol 38 (10) ◽  
pp. 41-48 ◽  
Author(s):  
G. Vaes ◽  
J. Berlamont
Keyword(s):  
Time Series ◽  
Storm Events ◽  
Short Time Series ◽  
Hydrodynamic Simulations ◽  
Sewer Systems ◽  
Linear Behaviour ◽  
Long Time ◽  
Short Time ◽  
Different Characteristics

Ideally, for emission calculations long term hydrodynamic simulations should be performed, but this requires long calculation times. Simplifications are consequently necessary. Due to the non-linear behaviour of sewer systems, hydrodynamic simulations using single storm events often will not lead to a good probability estimation of the overflow emissions. Simplified models using long time simulations give better results if they are well calibrated. To increase the accuracy hydrodynamic simulations with short time series can be used. The short time series are selected from the long time historical rainfall series using a simplified model. To test the accuracy of these three methods, hydrodynamic long term simulations were performed for several (small) sewer systems with different characteristics to compare with.

Simulation of nonpoint source pollutant loadings from urban area during rainfall: an application of a physically-based distributed model

1998 ◽  
Vol 38 (10) ◽  
pp. 199-206 ◽  
Author(s):  
Zhang Haiping ◽  
Kiyoshi Yamada
Keyword(s):  
Urban Area ◽  
Overland Flow ◽  
Drainage System ◽  
Distributed Model ◽  
Storm Event ◽  
Urban Watershed ◽  
Storm Events ◽  
Runoff Generation ◽  
Time Step ◽  
Physically Based

A physically-based, distributed model, PROUW, is applied to a small urban watershed in Japan with an area of 66.18 ha. The model includes a description of evapotranspiration, percolation, runoff generation, overland flow routing, pollutant accumulation in dry weathers and washoff during storm events, overland pollutant routing, and flow and pollutant routing in drainage system. The finite difference schematization of the urban watershed provides a representation of the spatial pattern of topography, land-use, soil types and meteorological inputs. The watershed is divided into 7500 grids of 10m × 10m and the runoff rate and pollutant loadings are simulated with a time step of 5 sec. The data for the storm event of April 28, 1995 is used for model calibration. Simulated hydrograph and pollutographs of the storm event of April 18, 1995 are compared with the observed data. Results show a reasonable degree of fit, indicating that the model provides a reasonable interpretation of the overall runoff and pollutant generation processes in the urban area. The results also suggest that the model should be improved further by incorporating new reliable equations for pollutant washoff estimation.

Influence of local calibration on the quality of online wet weather discharge monitoring: feedback from five international case studies

2014 ◽  
Vol 71 (1) ◽  
pp. 45-51 ◽  
Author(s):  
Nicolas Caradot ◽  
Hauke Sonnenberg ◽  
Pascale Rouault ◽  
Günter Gruber ◽  
Thomas Hofer ◽  
...  
Keyword(s):  
Online Monitoring ◽  
Oxygen Demand ◽  
Storm Events ◽  
Local Calibration ◽  
Combined Sewer Overflows ◽  
Measured Concentration ◽  
Representative Sampling ◽  
Sewer Systems ◽  
Wet Weather

This paper reports about experiences gathered from five online monitoring campaigns in the sewer systems of Berlin (Germany), Graz (Austria), Lyon (France) and Bogota (Colombia) using ultraviolet–visible (UV–VIS) spectrometers and turbidimeters. Online probes are useful for the measurement of highly dynamic processes, e.g. combined sewer overflows (CSO), storm events, and river impacts. The influence of local calibration on the quality of online chemical oxygen demand (COD) measurements of wet weather discharges has been assessed. Results underline the need to establish local calibration functions for both UV–VIS spectrometers and turbidimeters. It is suggested that practitioners calibrate locally their probes using at least 15–20 samples. However, these samples should be collected over several events and cover most of the natural variability of the measured concentration. For this reason, the use of automatic peristaltic samplers in parallel to online monitoring is recommended with short representative sampling campaigns during wet weather discharges. Using reliable calibration functions, COD loads of CSO and storm events can be estimated with a relative uncertainty of approximately 20%. If no local calibration is established, concentrations and loads are estimated with a high error rate, questioning the reliability and meaning of the online measurement. Similar results have been obtained for total suspended solids measurements.

scholarly journals Evaluating Simple Methodology for Piezoelectric Level Sensors Protection

Proceedings ◽  
2019 ◽  
Vol 48 (1) ◽  
pp. 30
Author(s):  
Luis Hamilton Pospissil Garbossa ◽  
Argeu Vanz ◽  
Matias Guilherme Boll ◽  
Hamilton Justino Vieira
Keyword(s):  
Water Level ◽  
Flood Control ◽  
Low Cost ◽  
Solid Particles ◽  
Storm Events ◽  
Sewer Systems ◽  
Continuous Monitor ◽  
Level Monitoring ◽  
Water Level Monitoring

The increasing frequency of extreme storm events has implications for the operation of sewer systems, storm water, flood control monitoring and tide level variations. Accurate and continuous monitor water level monitoring is demanded in different environments. Piezoelectric sensors are widely used for water level monitoring and work submerged in waters subject to the presence of solid particles, biological fouling and saltwater oxidation. This work aimed to develop a simple, low-cost methodology to protect sensors over long-term deployment. The results show that simple actions, costing less than 2 EUR, can protect and extend the lifecycle of equipment worth over 2000 EUR, ensuring continuous monitoring and maintaining quality measurements.

scholarly journals Critical news reading with Twitter? Exploring data-mining practices and their impact on societal discourse

2017 ◽  
Vol 42 (2) ◽  
Author(s):  
Mathias Verbeke ◽  
Bettina Berendt ◽  
Leen d’Haenens ◽  
Michaël Opgenhaffen
Keyword(s):  
Media Studies ◽  
Software Tool ◽  
Large Data ◽  
Science Research ◽  
Language Modeling ◽  
Methodological Innovation ◽  
News Sources ◽  
Digital World ◽  
Interesting Connection ◽  
User Content

AbstractThis article shows that the collaboration between social science and computer science scholars proves fruitful in enhancing conceptual and methodological innovation in research appropriate for the digital world. It presents arguments for ways in which a multi-disciplinary approach can strengthen media studies and nnovatively advance both research breadth and depth. To illustrate this interesting connection of both disciplines, we present the example analysis of large data from Twitter and discuss this analysis in a communication science research environment. We propose TwiNeR, a software tool that analyzes tweet content using an advanced language modeling approach for classifying tweets into five prototypical messages referring to ‘activities’ related to news and news sources in the Twitter network (i.e., source-fed article, user-fed article, content spread by user, other source content, other user content).

A Novel Solution to Compute Stress Time Series in Nonlinear Hydro-Structure Simulations

2015 ◽  
Author(s):  
Fabien Bigot ◽  
François-Xavier Sireta ◽  
Eric Baudin ◽  
Quentin Derbanne ◽  
Etienne Tiphine ◽  
...  
Keyword(s):  
Time Series ◽  
Frequency Domain ◽  
Time Domain ◽  
Hot Spot ◽  
Structural Response ◽  
Container Ship ◽  
Time Step ◽  
Computation Cost ◽  
Hull Girder ◽  
Non Linear

Ship transport is growing up rapidly, leading to ships size increase, and particularly for container ships. The last generation of Container Ship is now called Ultra Large Container Ship (ULCS). Due to their increasing sizes they are more flexible and more prone to wave induced vibrations of their hull girder: springing and whipping. The subsequent increase of the structure fatigue damage needs to be evaluated at the design stage, thus pushing the development of hydro-elastic simulation models. Spectral fatigue analysis including the first order springing can be done at a reasonable computational cost since the coupling between the sea-keeping and the Finite Element Method (FEM) structural analysis is performed in frequency domain. On the opposite, the simulation of non-linear phenomena (Non linear springing, whipping) has to be done in time domain, which dramatically increases the computation cost. In the context of ULCS, because of hull girder torsion and structural discontinuities, the hot spot stress time series that are required for fatigue analysis cannot be simply obtained from the hull girder loads in way of the detail. On the other hand, the computation cost to perform a FEM analysis at each time step is too high, so alternative solutions are necessary. In this paper a new solution is proposed, that is derived from a method for the efficient conversion of full scale strain measurements into internal loads. In this context, the process is reversed so that the stresses in the structural details are derived from the internal loads computed by the sea-keeping program. First, a base of distortion modes is built using a structural model of the ship. An original method to build this base using the structural response to wave loading is proposed. Then a conversion matrix is used to project the computed internal loads values on the distortion modes base, and the hot spot stresses are obtained by recombination of their modal values. The Moore-Penrose pseudo-inverse is used to minimize the error. In a first step, the conversion procedure is established and validated using the frequency domain hydro-structure model of a ULCS. Then the method is applied to a non-linear time domain simulation for which the structural response has actually been computed at each time step in order to have a reference stress signal, in order to prove its efficiency.

NONLINEAR GATED EXPERTS FOR TIME SERIES: DISCOVERING REGIMES AND AVOIDING OVERFITTING

1995 ◽  
Vol 06 (04) ◽  
pp. 373-399 ◽  
Author(s):  
ANDREAS S. WEIGEND ◽  
MORGAN MANGEAS ◽  
ASHOK N. SRIVASTAVA
Keyword(s):  
Time Series ◽  
Real World ◽  
Markov Models ◽  
Multilayer Perceptrons ◽  
Time Step ◽  
Local Complexity ◽  
Previous State ◽  
Segmentation Task ◽  
Gating Network ◽  
Update Rules

In the analysis and prediction of real-world systems, two of the key problems are nonstationarity (often in the form of switching between regimes), and overfitting (particularly serious for noisy processes). This article addresses these problems using gated experts, consisting of a (nonlinear) gating network, and several (also nonlinear) competing experts. Each expert learns to predict the conditional mean, and each expert adapts its width to match the noise level in its regime. The gating network learns to predict the probability of each expert, given the input. This article focuses on the case where the gating network bases its decision on information from the inputs. This can be contrasted to hidden Markov models where the decision is based on the previous state(s) (i.e. on the output of the gating network at the previous time step), as well as to averaging over several predictors. In contrast, gated experts soft-partition the input space, only learning to model their region. This article discusses the underlying statistical assumptions, derives the weight update rules, and compares the performance of gated experts to standard methods on three time series: (1) a computer-generated series, obtained by randomly switching between two nonlinear processes; (2) a time series from the Santa Fe Time Series Competition (the light intensity of a laser in chaotic state); and (3) the daily electricity demand of France, a real-world multivariate problem with structure on several time scales. The main results are: (1) the gating network correctly discovers the different regimes of the process; (2) the widths associated with each expert are important for the segmentation task (and they can be used to characterize the sub-processes); and (3) there is less overfitting compared to single networks (homogeneous multilayer perceptrons), since the experts learn to match their variances to the (local) noise levels. This can be viewed as matching the local complexity of the model to the local complexity of the data.

scholarly journals The effects of the big storm events in the first half of 2015 on the radiation belts observed by EPT/PROBA-V

2016 ◽  
Vol 34 (1) ◽  
pp. 75-84 ◽  
Author(s):  
V. Pierrard ◽  
G. Lopez Rosson
Keyword(s):  
Charged Particle ◽  
High Resolution ◽  
Geomagnetic Storm ◽  
Energetic Particle ◽  
High Energy ◽  
Radiation Belts ◽  
Radiation Environment ◽  
Storm Event ◽  
Storm Events ◽  
Particle Radiation

Abstract. With the energetic particle telescope (EPT) performing with direct electron and proton discrimination on board the ESA satellite PROBA-V, we analyze the high-resolution measurements of the charged particle radiation environment at an altitude of 820 km for the year 2015. On 17 March 2015, a big geomagnetic storm event injected unusual fluxes up to low radial distances in the radiation belts. EPT electron measurements show a deep dropout at L > 4 starting during the main phase of the storm, associated to the penetration of high energy fluxes at L < 2 completely filling the slot region. After 10 days, the formation of a new slot around L = 2.8 for electrons of 500–600 keV separates the outer belt from the belt extending at other longitudes than the South Atlantic Anomaly. Two other major events appeared in January and June 2015, again with injections of electrons in the inner belt, contrary to what was observed in 2013 and 2014. These observations open many perspectives to better understand the source and loss mechanisms, and particularly concerning the formation of three belts.

Sign in / Sign up

Export Citation Format

Share Document