|
|
|
|
|
|
|
|
|
|
A number of conferences attendees will present a poster at the conference. The poster presentations will be in the form of a poster walk. That means that the audience walks along the exhibited posters accompanied by a guide. Each of the posters presenters then has a couple minutes to verbally explain what the poster is about. The listeners then have the opportunity to ask question during two extra minutes.
The list of all poster presentations is given below.
|
|
|
|
|
|
As the pace of experimentation has increased, experimental designs have had to adapt to meet the demands of experimentalists, their goals, and the spaces they explore. This pace has increased from runs/year to runs/day and now to runs/minute in high throughput experimentation. The emphasis is now on the discovery of low-probability, high-value occurrences (hits) by searching extensive experimental spaces. These have the issues of:
- High-dimensional experimental spaces (many system constituents and experimental parameters);
- Synergies, or beneficial nonlinear interactions between system constituents;
- Unpredictable behavior, including both temporal unpredictability (i.e., chaos) and the inability to derive experimental results from basic chemical and physical laws
ProtoLife Srl has developed Predictive Design Technology™ (PDT) to deal with these issues. It has the advantage of proprietary prediction algorithms that efficiently search vast experimental spaces.
After tests with simulations, PDT has been successfully tested in complex experimental spaces involving drug combinations, catalysts, and drug formulations. In all cases it has located the optima using a small fraction of the runs needed in alternative methods of experimental design.
|
|
|
|
|
|
Many complex, real world phenomena are difficult to study directly using controlled experiments. Instead, the use of computer simulations has become commonplace as a cost effective alternative. However, regardless of Moore’s law, performing high fidelity simulations still requires a great investment of time and money. Surrogate modeling (meta-modeling) has become indispensable as an alternative solution for relieving this burden. Surrogates are compact and cheap to evaluate, and have been proven very useful for tasks such as optimization, design space exploration and sensitivity analysis. Consequently, there is great interest in techniques that facilitate the construction and evaluation of such approximation models, while minimizing the computational cost and maximizing model accuracy. We present a novel, unified approach to surrogate modeling, placing a strong focus on adaptivity, self-tuning and robustness in order to maximize efficiency and make our algorithms and tools easily accessible to other scientists in computational science and engineering.
|
|
|
|
|
|
|
The mathematics for analyzing complex industrial problems is very mature, with methods
& software available to address even incompletely understood situations; yet the
application of DOE’s lag far behind the analytic capability.
Using a taxonomy of problem solving, conditions for a successful DOE can be listed,
with guidance to better achieve those successful conditions in a project development.
Motivated individuals, reasonably knowledgeable in the applicable technology, addressing
problems with definable responses (dependent variables) and numbers of controllable
potential factors (independent variables), are most likely to achieve positive
resolutions with DOE’s. In fact, under these conditions a DOE is likely to be the most
economically effective means for developing technically and financially beneficial information.
|
|
|
|
|
|
Supersaturated designs examine more than N – 1 factors in N experiments, and as a consequence individual factor effect estimation becomes problematic. A new method, called the Fixing Effects and Adding Rows (FEAR) method, is proposed to estimate the effects in supersaturated designs more accurately. method is based on the idea that too few experiments are executed to estimate the examined factor effects properly, and that effect sparsity occurs when several factors are examined. Therefore zero effect rows are added to the design matrix, followed by consecutively fixing the largest estimated effects.
The FEAR method is compared with Multiple Linear Regression (MLR) methods, as backward elimination, forward selection, and stepwise regression, and with the alternative ridge regression, all methods proposed earlier to interpret supersaturated design results. A fully simulated, a partially simulated and an experimental data set were used for the evaluation of the methods.
It was found that the FEAR method performs better than the MLR and ridge regression methods, since the significant main effects are more accurately estimated, and because fewer effects are incorrectly considered significant (false positives) or non-significant (false negatives).
|
|
|
|
|
|
In a classical conjoint choice experiment, respondents choose one profile from each choice set that has to be evaluated. However, in real life the respondent does not always make a choice: often he/she does not prefer any of the alternatives offered. Therefore, including a no-choice option in a choice set makes a conjoint choice experiment more realistic. In the literature three different models are used to analyze the results of a conjoint choice experiment with a no-choice option: the no-choice multinomial logit model, the extended no-choice multinomial logit model and the nested no-choice multinomial logit model. We develop optimal designs for the two most appealing of these models using the D-optimality criterion and the modified Fedorov algorithm and compare these optimal designs with a reference design, which is constructed ignoring the no-choice option, in terms of estimation and prediction accuracy. We conclude that taking into account the no-choice option when designing a no-choice experiment only has a marginal effect on the estimation and prediction accuracy as long as the model used for estimation matches the data generating model.
|
|
|
|
|
|
Random effects or mixed logit models are often used to model differences in consumer preferences. Data from choice experiments are needed to estimate the mean vector and the variances of the multivariate heterogeneity distribution involved. In this paper, an efficient algorithm is proposed to construct semi-Bayesian D-optimal mixed logit designs that take into account the uncertainty about the mean vector of the distribution. These designs are compared to locally D-optimal mixed logit designs (see Sándor and Wedel, 2002), Bayesian and locally D-optimal designs for the multinomial logit model (see Kessels et al., 2006a; Huber and Zwerina, 1996) and to nearly orthogonal designs (see Sawtooth (CBC)) for a wide range of parameter values. It is found that the semi-Bayesian mixed logit designs outperform the competing designs not only in terms of estimation efficiency but also in terms of prediction accuracy. In particular, it is shown that assuming large prior values for the variance parameters for constructing semi-Bayesian mixed logit designs is most robust against the misspecification of the prior mean vector. In addition, the semi-Bayesian mixed logit designs are compared to the fully Bayesian mixed logit designs which take also into account the uncertainty about the variances in the heterogeneity distribution and which can only be constructed using prohibitively large computing power. The differences in estimation and prediction accuracy turn out to be rather small in most cases which indicates that the semi-Bayesian approach is currently the most appropriate one if one needs to estimate mixed logit models.
|
|
|
|
|
|
|
Understanding consumer preferences is a critical first step in developing a successful product or service. Companies that can accurately predict customer preferences have a competitive advantage in launching innovative products or services leading to an increase in customer base. Performing conjoint experiments is a powerful way to predict people's choices for prospective goods. In the conjoint experiments we consider, respondents rate a set of goods on a scale. These goods are presented as profiles or alternatives of combinations of different component attributes. The usefulness of the predictions resulting from the analysis of the experimental data depends on the profiles and the number of raters. The assignment of the profiles to the raters also plays a key role. An optimal design maximizes the amount of information and determines the optimal number of raters given the total number of profiles. In the conjoint design literature, Kessels, Goos and Vandebroek (2007) produced design results for main-effects models including categorical attributes only. Designs for models with continuous attributes and two-way interaction effects have not been extensively studied. However, recognizing the true nature of attributes in combination with identifying and estimating interactions can yield more efficient designs. In the poster, we show how to construct optimal conjoint designs in the presence of interactions and continuous attributes and compare their distinctive features with some well-known combinatorial designs.
|
|
|
|
|
|
Space filling designs are important for deterministic computer experiments. However, each single experiment is often time consuming and has many input parameters. Furthermore the underlying function is mostly highly nonlinear. Hence the points at which the experiment is conducted have to be chosen carefully in order to gain a good knowledge about the unknown function. There exist many design criteria, which are to be optimized in order to get a good space filling design. Nearly all of them require algorithmic minimization. In contrast to this a method is developed here, which does not need algorithmic optimization. Therefore a mesh of nearly regular simplices is constructed and the vertices of the simplices are used as design points. This yields good space filling designs according to the maximin criteria.
|
|
|
|
|
|
Unreplicated two level fractional factorials are a common type of experimental design used in the early stages of industrial experimentation to identify probable active treatment factors. One of the most common methods used in detecting active effects is the normal plot of effects; ANOVA methods cannot be easily used due to the lack of degrees of freedom available for estimation of variance of experimental error. Fully Bayesian methods are rarely used in multifactor experiments, however there is virtually always some prior knowledge about the sizes of effects and so using this
in a Bayesian data analysis seems natural. The analysis of this type of experiment can be impacted in a disastrous way in the presence of outliers. Outliers are an unavoidable circumstance, the possibility that they may occur is always present and so by applying different prior model distributions of the data, an analysis that is more robust to outliers is sought. The use of fairly vague priors are explored as well as more informative priors. A thorough study of this shows that some improvement on standard analyses is made and heavy-tailed family of distributions are seen to be more robust. Smaller fractions and more saturated designs are also explored to assess robustness to outliers when complex aliasing is involved.
|
|
|
|
|
|
Classical design of experiments is a two-stage process in which first the experiment is designed and then the results are analyzed and processed. In sequential design, the goal is to improve efficiency by transforming this method into an iterative process, in which data acquired from previous iterations is used to guide future sampling. The design is thus refined with each iteration to achieve improved accuracy. A novel sequential design method is introduced, which incorporates both an error-based measure using gradient estimations of the objective function and a density-based measure using a voronoi tesselation. This method is a trade-off between error-based methods which are highly efficient but not robust and density-based methods which are robust but ignore function behaviour.
|
|
|
|
|
|
|
A good working knowledge of DOE (Design of Experiments) is essential for both industrial statisticians and engineers. It is therefore essential that courses in statistics pay sufficient attention to this topic. However, a distinctive feature of DOE is that it is pro-active, unlike many other statistical techniques. Hence, this requires a teaching approach that forces students to actively think about several aspects of setting up an experimental design, without steering the student too much. In order to create such a teaching environment to be used in statistics courses at various departments of Eindhoven University of Technology, a web based tool called StatLab has been developed, which can freely be used through www.win.tue.nl/statlab.
StatLab adapts itself to the student, who is being led through one of several possible case studies and assignment. In for instance a screening experiment the student needs to create a two-level factorial design in order to determine the significant factors in a (generally chemical) process. Decisions have to be taken then concerning design size, blocks, high and low factor values, aliasing structure, centre points, replication and randomisation. Realistic experimental data are simulated then for further analysis and model estimation. After identifying the significant effects, the student can switch to the optimisation phase using response surface methods (RSM). In this part optimal values of the significant effects are searched for, using the method of steepest ascent.
The current version of StatLab supports two level factorial designs (both with and without noise factors), Taguchi inner/outer arrays, Plackett-Burman designs and Response Surface designs. Additional features of StatLab are that it is multilingual, the program flow is determined by the choices of the student, an automatic grading system is included that sends a commented email of the student's actions to the teacher, automatic generation of available designs is implemented using Franklin and Bailey's algorithm [1] and a graphical visualisation of fitted response surfaces is provided.
Students generally consider using StatLab as a stimulating teaching environment. Furthermore, StatLab can be considered as a useful addition to existing teaching tools like the well-known helicopter experiment or the experiments described in e.g. [2], [3] and [4].
[1] Franklin M.F. and Bailey R.A. (1977). Selection of Defining Contrasts and Confounded Effects in Two-level Experiments. Applied Statistics 26, 1, 321-326.
[2] Anderson-Cook C.M. (1998). Designing a First Experiment: A Project for Design of Experiment Courses, American Statistician 52, 4, 333-342.
[3] Kopas D.A. and McAllister P.R. (1992). Process Improvement Exercises for the Chemical Industry. American Statistician 46, 1, 34-41.
[4] http://ucs.kuleuven.be/env2exp/index.html
|
|
|
|
|
|
|
Simulation-optimisation aims to identify the setting of the input parameters (input variables, factors) of the simulated system that leads to the optimal system performance. In practice, however, some of these parameters cannot be perfectly controlled – due to measurement errors or other implementation issues, and the inherent uncertainty caused by fluctuations in the environment (e.g., demand). Consequently, the classic optimal solution may turn out to be sub-optimal or infeasible. The solution is robust optimisation (RO), which derives solutions that are relatively insensitive to perturbations caused by the so-called noise factors.
Several methods have been proposed for achieving RO in simulation; see Beyer and Sendhoff (2007). Our approach assumes Discrete-Event Simulation, and combines the Taguchian view and the Kriging metamodeling method. At least two metamodels are considered, namely one for the expected (mean) main performance function and one for its variance caused by the noise factors. During the iterative RO process, we need to carefully consider the updating and validation of these metamodels. Our simplest RO solves a constrained stochastic optimisation problem, namely minimizing (or maximizing) the expected main objective function, such that the variance does not exceed a user-defined threshold. Due to the discrete nature of the decision variables, our RO applies Nonlinear Integer Mathematical Programming to the Kriging metamodels; see Kleijnen et al. (2008) and also Biles et al. (2007).
Beyer, H.G. and B. Sendhoff, “Robust Optimization – A Comprehensive Survey”. Computer Methods in Applied Mechanics and Engineering, Vol. 196, No. 33-34, pp. 3190-3218 (2007)
Biles, W.E., J.P.C. Kleijnen, W.C.M. van Beers, and I. van Nieuwenhuyse, “Kriging Metamodeling in Constrained Simulation Optimization: an Explorative Study”. Proceedings of the 2007 Winter Simulation Conference, S. G. Henderson, B. Biller, M.-H. Hsieh, J. Shortle, J. D. Tew, and R. R. Barton (eds.), pp. 355-362 (2007)
Kleijnen, J.P.C., W.C.M. van Beers, and I. van Nieuwenhuyse, “Constrained optimization in simulation: novel approach”, Poster, International Conference on Design of Industrial Experiments, Antwerp, January 14th - 15 th, 2008.
|
|
|
|
|
|
In practice, simulation models have multiple outputs. In this research, one output is selected as the objective to be minimized, while the other outputs should not violate prespecified target values. The simulation outputs are assumed to be random (but the approach may also be adapted for deterministic simulation). Moreover, the simulation inputs must meet prespecified constraints, including the constraint that the inputs be integer. The resulting constrained optimization problem is solved through (i) experimental design (namely a space-filling design adapted for the input constraints) specifying the simulation input, (ii) Kriging metamodeling analyzing the simulation input/output data, and (iii) integer nonlinear programming applied to the Kriging model. The resulting heuristic is applied to a call-center case-study, and compared with a popular heuristic, namely OptQuest.
|
|
|
|
|
|
Second-order response surface designs which can efficiently estimate the terms of a second-order polynomial model are used in many areas of application. A rich class of second-order response surface designs with three levels of each factor called subset designs was introduced by Gilmour (2006). These designs can be specifically applied to the experiments, for example on biotechnological processes, where run-to-run variation is typically high. The designs consist of the combination of subsets of the 3q design. In this poster the subset designs are studied for missing data as the situation may occur that an observation is missing or there exists an outlier that really needs to be removed from the data to make the results more reliable. The robustness of subset designs to missing data is discussed and the designs are rebuilt to make them more robust to a missing design point. The results are illustrated with an example.
|
|
|
|
|
|
Discrete event simulation is often used to estimate performance metrics of interest in manufacturing settings. Simulation allows for virtual factory experimentation and the ability to see how changes in factory loadings and parameters affect factory performance. Often manufacturing simulations are used to generate graphical representations of how changes in the inputs change response variables of interest. One popular graph is known as the cycle time – throughput (CT – TH) graph. This CT – TH curve characterizes the average time a product spends in the factory from start to finish as a function of how much product is being loaded. This paper establishes a new form of the metamodel used to characterize this curve and details the use of D-optimality to decide which input points need to be simulated.
|
|
|
|
|
|
When an experiment is performed one run at a time, a time trend is often present among the observations. The trend may be due to instrumental drift, the fatigue of the experimenter or the ageing or warming up of materials. This practical issue has received some attention in the literature. At present, an exchange algorithm is the most often utilized method for constructing trend-resistant run orders for experimental designs. Nevertheless, this method suffers from criticism. First of all, the exchange move is not the only action one can take to optimize the experimental design’s run order. Secondly, the calculation time required to find trend-resistant run orders is prohibitively large. We propose a variable neighbourhood search algorithm to deal with these two issues. This method is shown to outperform the exchange algorithm when it comes to finding optimal run orders.
|
|
|
|
|
|
The problem of model discrimination arises when several rival models are proposed to describe one and the same process. To identify the best one, it may be necessary to collect new information about the system, and thus additional experiments have to be performed. Since experiments are typically time and money consuming, it is of course desirable to minimize the experimental effort. In literature, methods have been proposed to design experiments that allow to discriminate among the rival models. In essence, these methods look for that experiment that maximizes the difference between the model predictions, hereby taking the uncertainty on the predicted outcomes of the experiment into account. This work presents a new method in which the information that will be gathered from a newly designed experiment is accounted for. This should not only lead to a better estimate of the uncertainty on the model predictions, but should also allow one to design experiments in which part of the data gathered serves to decrease the model prediction uncertainty in favor of model discrimination.
|
|
|
|
|
|
|
When planning an experiment for comparing the effects of several treatments it is well understood that the experimental analyst should select the best possible design for the experimental situation under investigation. This often means choosing a design which is optimal or near-optimal according to one or more criteria. However, there are experimental situations where there is a real possibility that observations may be lost, due to a variety of causes, during the course of the experiment, so that the properties of the design are changed. In such circumstances it is reasonable to expect the experimental analyst to guard against a poor eventual design. In particular, a design which is optimal at the beginning of an experiment may become disconnected because of the loss of one or more observations.
Our purpose is to extend the work of Godolphin (2004) and suggest a measure of vulnerability of a planned design to become disconnected through observation loss. This measure is given in terms of the minimal size of the observation sets required to yield a disconnected eventual design and the total number of these observation sets of the minimal size. It is evident from this work that high efficiency, in the sense of near-optimality, does not necessarily imply minimal or even low vulnerability to observation loss. This is illustrated by reference to the class of non-balanced D(v,b,k) incomplete block designs of Shah and Das (1992), where v=6, b=7 and k=3. It is found that the E-optimal design of this size which is recommended by these authors is more vulnerable than many competing designs.
Godolphin, J.D. (2004). Simple pilot procedures for the avoidance of disconnected experimental designs. Applied Statistics 53, 133-147.
Shah, K.R. and Das, A. (1992). Binary designs are not always the best. Canadian J. Statistics 20, 347-351.
|
|
|
|
|
|
Network simulation is a tool which is widely used in engineering, for example when commissioning computer networks, or investigating problems in existing ones. Evaluating performance of networks becomes harder as the underlying systems and the simulations which represent them become more complex, so it is important to develop techniques to select inputs (experimental points at which to perform the simulation) which will provide the most accurate information about the system whilst reducing the time taken by the experiment. The poster provides some background on the statistical theory of design of experiments and on the statistics of queueing theory, and outlines work done so far to find effective designs for experiments on data networks, and the interesting statistical challenges this raises.
|
|
|
|
|
|
|
|
|