No events listed.
STSMs 2016/2017
The action has funded 6 STSMs over the coming months!
Successful meeting on Project Risk & Asset Management Uncertainty Assessment
October 2016: The Action held a workshop on Project Risk & Asset Management Uncertainty hosted by colleagues at TU Delft
Expert Judgement Workshop, 26th August 2016
An expert judgement workshop is being held at the University of Strathclyde on Friday 26th August!
Posted on October 21 2014
Kick Off Conference - Experts in Uncertainty
The First meeting of the Action will be at the University of Strathclyde, Glasgow on 2-4 April 2014. Speakers include Roger Cooke, Willy Aspinall, Abby Colson and Mark Burgman.
Programme:
Expert Judgment Network: Bridging the Gap Between Scientific Uncertainty and Evidence-Based Decision Making
Open meeting - University of Strathclyde, Glasgow, 2-4 April 2014
The programme for 2 April is designed for decision makers, policy makers and advisors, and will give an overview of Structured Expert Judgement methods and applications with leading international figures.
Professor Tim Bedford (University of Strathclyde, Glasgow):
Structured Expert Judgement
Governments and Businesses have to make decisions in the face of risks and uncertainties, but time pressures too often hamper the collection of large-scale datasets that might help reduce those uncertainties. Structured Expert Judgement aims to provide a proper evidence base that enables decision makers to obtain reasonable uncertainty assessments in a timely and defensible way. The best methods of Structured Export Judgement ensure that expert data is collected in a way that minimizes potential biases and, crucially, tests expert performance.
Decision makers can and should demand that expert judgement procedures provide for the most unbiased possible forms of judgement. Unfortunately events such as the prosecution of experts in Italy in relation to the Aquilla earthquakes illustrate that there are many conflicting priorities which can negatively affect the potential for future unbiased judgement. In this talk we will take a broad look at the challenges and demands for uncertainty assessment that make this COST Action so important.
Professor Mark Burgman (Australian Centre of Excellence in Risk Analysis, University of Melbourne):
The intelligence game
This presentation will outline the rules of the game designed by IARPA in the United States to evaluate expert predictions of geopolitical events. It will describe the full results from the first two years of operation. It will outline preliminary results from 2014, the final year of the game, including the performance of alternative approaches to weighting. In 2010, the US Intelligence Advanced Research Projects Activity (IARPA) announced a 4-year forecasting “tournament”. Five collaborative research teams attempted to outperform a baseline opinion pool in predicting hundreds of geopolitical events. Biosecurity specialist Mark Burgman led one of the teams, eliciting forecasts from Delphi-style groups in the US and Australia. Probabilities for events like "Will Australia formally transfer uranium to India by 1 June 2012?" are elicited and combined using measures of expert performance. The Delphi teams perform well compared to a host of alternative approaches, and have the advantage of operational simplicity. More generally, the project has emphasized the need to develop and test tools for elicitation that disregard conventional views on an expert’s status. Even this ambitious project explored a tiny part of the expert judgment space, namely, binary predictions for geopolitics assessed using a Brier score. It underscores how much work there is to do, and how urgent and critically important it is.
Professor Willy Aspinall (University of Bristol, Aspinall & Associates):
Uncertainty Erupting
When a volcano becomes restless, public officials must take life and death decisions in the face of massive uncertainties. Geophysicist Willy Aspinall is internationally renowned for corralling the, often conflicting, opinions of individual volcanologists in order to provide to decision-makers in the volcanic hot seat real time assessments of the evolving probabilities for catastrophic eruption. Over the years his approach has yielded probabilistic forecasting returns of which stock brokers can only dream, and Willy will share some of his more colourful experiences and show how his results stack up over the long term Montserrat Volcano Observatory. Also, he will share practice insights from facilitating elicitations for a multitudinous range of diverse scientific, medical, engineering, security and safety problems.
Professor Roger Cooke (Resources for the Future, Washington DC and University of Strathclyde, Glasgow):
Quantifying Uncertainty on thin Ice
Risk analysis is about improbable and unwelcome events. Data is always in short supply and uncertainty quantification is the order of the day. Mathematician risk analyst Roger Cooke has a distinguished career of taking big uncertainties to the mat. He has recently been engaged in quantifying uncertainty in ice sheet dynamics in the face of global warming. This problem is not only the major wild card in global climate change, it forces analysts to go beyond the off-the-shelf methods. Roger was lead author in the 5th assessment of the Intergovernmental Panel on Climate Change for the chapter on Risk and Uncertainty.
Abigail Colson (Center for Disease Dynamics, Economics and Policy, Washington DC):
Bang for Benevolent Bucks
Worthy goals are easy to formulate but hard to verify. Even philanthropic policy makers want to know the bang for their bucks. How much good have we accomplished? What would have happened if we hadn't been there? The Robert Wood Johnson Foundation recently contracted for the development of innovative measures of policy effectiveness, and Abby Colson was able to apply the principles of structured expert judgment to score policy effectiveness. There are many pointers for wider application.
Professor David Rios Insua (Rey Juan Carlos University, Madrid):
Some problems in uncertainty modelling and foundational issues
I shall outline some possible problems of interest concerning uncertainty modelling and foundational issues in relation with expert judgement. These might include: elicitation for main distributions, risk matrices, proper discretisations, multivariate extreme models, scoring rules, expert dependencies, preference modelling, adversarial uncertainty, sensitivity analysis and technology. My objective is to stimulate discussion and brainstorm about problems of interest for WG3 in coordination with other WGs.
Dr Anca Hanea (Delft University of Technology):
Eliciting dependence: The why, what and how
Most elicitation methods are focused on making assessments of the uncertainty of single quantities. However, risk models usually have more than one uncertain input, and we must consider how to represent uncertainty about multiple quantities jointly. The simplest option is to assume independence, but this is often a faulty assumption, especially for tail events. Nevertheless, to formulate a structured process for eliciting dependence, one has to first answer questions of the following sort:
Professor John Quigley (University of Strathclyde, Glasgow):
Multiple Experts Making Multiple Assessments: A Case Concerning Infrastructure Assessment
Testing engineering structures to assess reliability can be challenging and expensive, hence motivating the design of novel test and sampling strategies. For example, restricted access may limit the possible number of samples; safety considerations may constrain the location from which samples can be gathered and the types of engineering tests that can be undertaken. While test and statistical sampling strategies can be devised, a mechanism is required to select between alternative proposals to ensure that representative and informative test samples will be obtained. However it is challenging to support decisions between alternative testing strategies due to the lack of objective data and the reliance on expert judgement, which may be vague and potentially in conflict.
This research involves developing a process to elicit and structure expert engineering judgement to be represented as a Bayesian Network (BN), which provides a model to support comparison of alternative test and sampling strategies. A BN is capable of modelling the interaction between multiple factors and, where the influences of these factors are uncertain, measuring these uncertainties probabilistically through a prior distribution. The decision support provided by the BN is achieved by comparing the anticipated test results obtained through the elicitation of the prior distribution with pre-posterior assessments.
The case study motivating our research will be presented. We discuss how our method has been used to support a comparison of alternative test and sampling strategies being considered to assess the state of the anchorages of the Forth Road Bridge, a suspension bridge near Edinburgh. The Forth Road Bridge has more than 11,600 steel wires grouped into 37 strands within each main cable. The cables are secured to the north and south shores of the river by pairs of anchorages, the key components of which are buried deep in rock with no direct means of access. Five groups of test were considered to assess the state of the anchorages. The BN constructed is based on the judgements of multiple engineering experts, and captures the nine variables representing the state of the strands and the inter-relationships between them.
This talk will be used to highlight several challenges in the process of eliciting expert judgment and be used to motivate the key objective for Working Group 1 of the COST project.
For more information regarding our programme please contact: Dr. Kevin Wilson, University of Strathclyde- kevin.j.wilson@strath.ac.uk
For information about getting to Strathclyde, please see below:
Getting to Strathclyde University
If you are looking for a possible accommodation during this time, there are many options for hotels within a short walk from the conference venue and bookings can be made directly on the hotels’ own websites. A few options are listed below:
Premier Inn Glasgow City Centre
Millenium Hotel Glasgow
Mercure Glasgow City
Copyright 2015-2025 COST | Terms Of Use | Login