Y. Kim. M, M, A, A, Schnabel Schnabel (eds.), (eds.), Emerging Emerging Experience Experience inin Y. Ikeda, Ikeda, C. C. M. M. Herr, D. Holzer, S. Kaijima, M. J. Kim. Past, Present and Future of Digital Architecture, Proceedings of the the 20th 20th International International Conference Conference of ofthe the Association for forComputer-Aided Computer-AidedArchitectural ArchitecturalDesign Design Research in Asia CAADRIA 000–000. © Research in Asia CAADRIA 2015,2015, 603–612. © 2015, 2015, The Association for Computer-Aided Architectural Design Research in Asia (CAADRIA), Hong The Association for Computer-Aided Architectural Design Research in Asia (CAADRIA), Hong Kong Kong PERFORMANCE-BASED PARAMETRIC DESIGN A framework for building envelope design THIAN-SIONG CHOO1 and PATRICK JANSSEN2 1,2 Department of Architecture, National University of Singapore 1 choothiansiong@yahoo.co.uk; 2 patrick@janssen.name Abstract. Existing performance-based design exploration methods typically suffer from a lack of real-time feedback and a lack of actionable feedback. This paper proposes a hybrid design exploration method that overcomes these issues by combining parametric modelling, surrogate modelling, and evolutionary algorithms. The proposed method is structured as a mixed-initiative approach, in which parametric modelling is the key to creating a synergistic relationship between the architect and the computational system. Surrogate-based techniques will address the issue of real-time feedback, the evolutionary exploration techniques will address the issue of actionable feedback. As a first stage in developing the PEX method, this paper reports on two experiments conducted to identify an appropriate surrogate modelling technique that is efficient and robust. Keywords. Performance-based design, parametric modelling, surrogate modelling, evolutionary algorithms 1. Introduction This paper proposes a performance-based parametric design exploration method to enhance the design workflow of the architect. In general, the design exploration process of many architects includes two cyclic interlinked loops of divergent and convergent design exploration (Cross, 2008; Janssen et al., 2011), as shown in Figure 1 (left). The divergent design exploration loop embodies the idea of developing alternative design concepts that are appropriate to the design scenario. The convergent design exploration loop embodies the idea of repeatedly exploring design variants based on the same underlying design concept. 604 2 T. S. CHOO AND P. JANSSEN For a parametric exploration process, design concepts are translated by design modelling techniques into digital design models. The diagram in Figure 1 (left) can be modified to explicitly represent the design modelling techniques, as shown in Figure 1 (right). Two main types of design modelling techniques are identified: direct modelling and parametric modelling. In the divergent design exploration loop, the architect uses direct modelling techniques to analyse different design concepts. Once the architect is satisfied with the selected concept, they proceed to the convergent design exploration loop. The design concept is then translated into a parametric model. This model can be used to explore the design space by iteratively generating design variants. Performance-based design exploration techniques allow the exploration process to be guided by key performance criteria relevant to the design problem. Figure 1: The two cyclic interlinked loops of the divergent-convergent design exploration process. Left: The non-computational process using traditional media. Right: The computational process using direct and parametric modelling. This paper proposes a hybrid design exploration method that combines parametric modelling, surrogate-modelling and evolutionary algorithms. Section 2 gives a brief overview of alternative performance-based design exploration techniques. Section 3 describes the proposed method. Section 4 describes a set of experiments that aim to identify an appropriate surrogate modelling technique to be used within the proposed method. Finally, the conclusions section summarises the findings. 2. Performance-based design exploration techniques For the convergent design exploration loop, four types of exploration techniques that could be used for performance-based design are identified: knowledge-based exploration, simulation-based exploration, surrogate-based exploration, and population-based exploration. Knowledge-based exploration techniques include expert system-based exploration techniques and the fuzzy logic-based exploration techniques. These exploration techniques emulate the human expert. They consist of a knowledge base which has domain-specific knowledge and an inference en- PERFORMANCE-BASED PARAMETRIC DESIGN 605 3 gine that uses the embedded knowledge to determine the solution for a userspecified problem. Simulation-based exploration techniques use performance simulation models to evaluate key performance criteria relevant to the design problem. Simulation models are constructed from first principles and aim to imitate real-world process or system over time. Surrogate-based exploration techniques use approximation models to replace accurate but time consuming performance simulations. Surrogate models are typically constructed using a data-driven, bottom-up approach and aim to emulate the behaviour of complex simulations while being computationally cheap to evaluate (Gorissen, 2010). Population-based exploration techniques use search algorithms to explore complex design spaces with varying multiple conflicting performance criteria. They include evolutionary algorithms, ant colony algorithms and particle swarm algorithms (Engelbrecht, 2007). 2.1. ISSUES The application of these performance-based design exploration techniques are hindered by two key issues: a) the lack of real-time feedback and b) the lack of actionable feedback. Real-time feedback refers to an immediate feedback on design performance. The feedback should help the architect to evaluate the effect that design changes are having on the performance of the design. The speed of the feedback should be fast enough in order to ensure that it does not hinder the fluid design process. For example, if the architect is interactively changing the size of a window, the feedback should inform the architect on the effect those changes are having on daylight levels. Actionable feedback refers to feedback that helps the architect decide on an appropriate course of action. The feedback should help the architect to decide how the design can be modified in order to improve the performance of the design (Huang et al., 2008). For example, if the amount of daylight on the desk is too low, then the feedback should suggest how improved daylighting might be achieved. A summary of the issues addressed by the different performance-based design exploration techniques is shown in Table 1. Real-time feedback for knowledge-based techniques and for surrogate-based techniques have a tick with an asterisk because an additional process is required. For knowledgebased techniques, the knowledge base needs to be developed. For surrogatebased techniques, the surrogate model needs to be trained. The time required 606 4 T. S. CHOO AND P. JANSSEN to develop a knowledge base or train a surrogate model depends on the complexity of the design problem. Table 1: Summary of performance-based design exploration techniques Performance-based design Real-time exploration techniques Feedback Actionable Feedback Knowledge-based 9* 9 Simulation-based x x Surrogate-based 9* x Population-based x 9 3. Proposed PEX Method To overcome the two key issues of a lack of real-time feedback and actionable feedback, a hybrid design exploration method is proposed. It combines parametric modelling, surrogate modelling, and evolutionary algorithms. The proposed method is structured as a mixed-initiative approach, in which parametric modelling is the key to creating a synergistic relationship between the architect and the computational system. Surrogate-based techniques will address the issue of real-time feedback and the evolutionary exploration techniques will address the issue of actionable feedback. The proposed method is referred to as the performance-based exploration (PEX) method. One of the main challenges to achieving real-time feedback is the long processing time of computationally expensive simulation models. Surrogate modelling offers a solution. For each computationally expensive simulation, a surrogate can be created. However, in order to be able to create such surrogate models, the variability of possible design scenarios needs to be restricted. The PEX method proposes to create surrogate models that are specific to the site context for a particular design project and to the parametric model developed by the architect. The parametric model inherently limits the range of possible design variants that can be generated, thereby making the creation of the appropriate surrogate models more feasible. 3.1. PEX EXPLORATION LOOP In the PEX method, the surrogate model is created and applied during the convergent design exploration loop, as shown in Figure 2. The loop consists of four computational steps: 1) create a performance-based parametric model, 2) replace simulation models with surrogate models, 3) generate, evaluate, and select design variants, 4) consider modifying the parametric model. PERFORMANCE-BASED PARAMETRIC DESIGN 607 5 x In step 1, a performance-based parametric model for design exploration is created for the selected design concept. This model will incorporate simulation models to evaluate key performance criteria. x In step 2, surrogate models are automatically generated for each of the simulation models that are computationally expensive. The parametric model is then updated to use these surrogate models that are fast to execute. x In step 3, the architect uses the surrogate-based parametric model to generate, evaluate, and select different design variants. These variants are generated by the same parametric model and therefore share the same underlying design concept. x After numerous iterative cycles of exploring different design variants, in step 4 the results are analysed and possible modifications to the parametric model are considered. Such modification will constitute a change to the design concept. Figure 2: Proposed PEX method Steps 3 forms an important iterative sub-loop in the exploration process. This sub-loop uses the parametric model from step 2, where the computationally expensive simulations have been replaced by fast surrogate models that can be executed in real-time. The sub-loop can either be performed manually by the architect or can be semi-automated using evolutionary algorithms. The manual mode addresses the issue of real-time feedback, while the semi-automated mode addresses the issue of actionable feedback. In the manual mode, the architect uses the parametric model to interactively explore design variants, with performance being evaluated using the surrogate models. In the semi-automated mode, the architect is provided with partial 608 6 T. S. CHOO AND P. JANSSEN maps of the design space being explored, which can be used to support decision making. These partial maps are created using evolutionary algorithms. The proposed PEX method requires a computational support system that we refer to as the PEX system. Three key components for this system are the surrogate modelling component in step 2, the evolutionary algorithm component in step 3, and the design analysis component in step 4. In this paper, we focus on the surrogate modelling component. The first step in developing such a component is identify an appropriate surrogate modelling technique that is robust and efficient. The following section describes a set of experiments that compare a number of surrogate modelling techniques. 4. Selecting a Surrogate Modelling Technique Two experiments were developed to compare selected surrogate modelling techniques for the PEX method. The comparison of the different surrogate modelling techniques is based on how efficient and robust they are. A surrogate modelling technique is considered more efficient than another if it can be trained faster with the least number of samples. It is considered robust if the model can be trained with a RMSE (Root Mean Squared Error) of equal or less than 10% for both experiments. A target accuracy of 10% RMSE is within the range recommended by Forrester et al. (2008). Four surrogate modelling techniques were selected based on a literature review (Jin et al., 2001; Villa-viallaneix et al., 2012; Gorissen, 2010; Forrester et al., 2008). They are: a) Radial Basis Function (RBF), b) Kriging (KG), c) Support Vector Machine (SVM) and d) Artificial Neural Network (ANN). They were selected based on their ability to model non-linear relationships between the design variables and the system response. A detailed explanation of these surrogate modelling techniques is beyond the scope of this paper. Ryberg et al’s (2012) summary is recommended for further reading. Two experiments were then conducted to compare the four selected surrogate modelling techniques. Experiment 1 consists of 2 design variables and experiment 2 consists of 6 design variables. The increase in design variables aims to investigate whether all four types of surrogate modelling techniques are efficient and robust enough to handle larger numbers of design variables. The performance output measure used is daylight autonomy which is simulated with DAYSIM (Reinhart, 2010). Daylight autonomy is the percentage of working hours where there is a minimum level of illuminance with the room for the entire year or a defined period of time. DAYSIM is a Radiancebased daylight analysis software that simulates annual daylight performance for building design (Reinhart, 2010; Ward and Shakespeare, 1998). PERFORMANCE-BASED PARAMETRIC DESIGN 609 7 For these two experiments the DAYSIM simulation software is coupled with Houdini (Sidefx, 2014) and Dexen (Janssen et al., 2012) to generate a dataset of daylight autonomy calculations for 10,000 design variants. The surrogate models are then trained separately with SUMO Toolbox (Gorissen et al., 2010) with input from the sample dataset created. Houdini is a procedural modelling software (Sidefx, 2014). A custom node within Houdini which links to the DAYSIM is created. Dexen allow simulations to run on multiple cores and computers in parallel (Janssen et al., 2012). This dramatically speeds up the execution time required to generation 10,000 design variants. SUMO Toolbox (Gorissen et al., 2010) is a flexible global surrogate modelling software with adaptive sampling capability (Gorissen et al., 2010). 4.1. EXPERIMENT 1 The parametric model used for this experiment represents a typical office room with a width and depth of 4m and a height of 3m. The model consists of two design variables, as shown in Figure 3. The first design variable is the height of spandrel 1 varies from 0 to 1.5m along the top edge of the façade represented by variable 1 in Figure 3. Similarly, the height of spandrel 2 varies from 0 to 1.5m along the floor edge of the facade, is represented by variable 2 in Figure 3. The sensor to measure daylight autonomy is set at the centre of the room, 0.85m from the floor. The wall is assigned a reflectance of 50%, the ceiling with a reflectance of 80%; the floor is assigned a reflectance of 20%. The glass of the window is assigned with a visible light transmittance of 88.4%. Daylight autonomy is calculated at a point 0.85m above the floor at the centre of the room, as shown in Figure 3. Figure 3: Office space as test case for experiment 1. Each surrogate model is trained by iteratively adding a new sample until it reaches a target accuracy of 10% or below. Each sample in the dataset 610 8 T. S. CHOO AND P. JANSSEN consists of the two design variables used for generating the design variant and the daylight autonomy score. 4.2. EXPERIMENT 2 The parametric model used for this experiment represents a typical office space with a façade facing north. The dimensions of the space are 4m deep, 16m wide, and 3m high. The model has 6 design variables as shown in Figure 2. The façade can vary from 1 bay to 4 bays. Each bay has a window that varies in width and height. The position of the windows also shifts in a vertical and horizontal direction from the centre of the façade. If there are multiple bays, variables 1 to 5 (see Figure 4 (right)) are applied similarly to all of them. Figure 4: Office space as test case for experiment 2. Left: The north facing facade varies from 1 bay to 4 bays of windows and has horizontal shading. Right: Detailed parametric model of a single bay. The surface properties are similar to those used in experiment 1. There is one sensor in front of each window, at the centre of each bay of the office space, at 0.85m from the floor. If there is more than one window or bay, then the daylight autonomy will be calculated as an average value. 4.3. DISCUSSION For experiment 1, all surrogate modelling techniques were able to reach the target accuracy (see Table 2). KG was most efficient, it used 24 samples and completed the run in 1 sec. This was followed by ANN which took 4 sec with 24 samples. SVM took 1 min 2 sec with 36 samples. Coming in last is RBF which took 1 min 5 sec and used 40 samples. For experiment 2, ANN is the only one that manages to reach the target accuracy without stagnating (see Table 3). It is the only one able to fit the design space of 6 design variables with an accuracy of less than 10% as recommended by Forrester et al. (2008). It completed its training within 3 min with 94 samples. PERFORMANCE-BASED PARAMETRIC DESIGN 611 9 From the two experiments, we could see that not all surrogate modelling techniques are efficient and robust. Design problems vary and the relationship between the design variables and the performance measure is not known prior to the creation of the surrogate model. Hence, an efficient and robust surrogate modelling technique like ANN may be required for a range of design problems. Previous research has found that KG, RBF and SVM can be time consuming to train and may become intractable when the number of design variables increases (Jin et al., 2001; Villa-viallaneix et al., 2012). This maybe a reason why KG, RBF and SVM stagnate for more than 2 days with more than 1000 samples. More experiments need to be conducted to determine if ANN is efficient and robust for a wider range of building design problems. Table 2: Comparison of four surrogate modelling techniques used to fit the daylight autonomy simulation with target accuracy of 10% RMSE for experiment 1. Model Types RMSE (%) No of Samples Used Execution Time (H:M:S) RBF 9.85 40 00:01:05 KG 6.05 24 00:00:01 SVM 9.58 36 00:01:02 ANN 7.29 24 00:00:04 Table 3: Comparison of four surrogate modelling techniques used to fit the daylight autonomy simulation with target accuracy of 10% RMSE for experiment 2. Model Types RMSE (%) No of Samples Used Execution Time (H:M:S) RBF - Stagnated with >1000 > 2 days KG - Stagnated with >1000 > 2 days SVM - Stagnated with >1000 > 2 days ANN 9.16 94 00:03:00 5. Conclusion A PEX method has been proposed for addressing the two key issues in a design exploration process highlighted in Section 2. It is a hybrid method which combines parametric modelling, surrogate modelling, and evolutionary algorithms. The PEX method is structured as a mixed-initiative approach, in which parametric modelling is the key to creating a synergistic relationship between the architect and the computational system. The surrogatebased exploration technique will address the issue of real-time feedback, while the evolutionary technique will address the issue of actionable feedback for a multi-objective design problem. As highlighted in Section 3, the 612 10 T. S. CHOO AND P. JANSSEN surrogate model can be used during the convergent design exploration cycle of PEX method. The surrogate models replace the computationally expensive simulation models, thereby allowing the architect to use the parametric model to interactively explore design variants. The PEX method requires a computational support system. A key component of this PEX systems is a surrogate modelling component. In order to identify an efficient and robust surrogate modelling technique, two experiments were conduct. These experiments showed ANN to be the most efficient and robust among the other types of surrogate modelling techniques. In the next stage of the research, the automation of the surrogate modelling processes with performance-based simulation processes will be implemented as part of the development for the PEX system. References Cross, N.: 2008, Engineering Design Methods: Strategies for Product Design, Wiley. Davis, D.: 2013, Modelled on Software Engineering: Flexible Parametric Models in the Practice of Architecture, PhD thesis from the Royal Melbourne Institute of Technology. Engelbrecht, A.P.: 2007, Computational Intelligence: An Introduction, Wiley. Forrester, A., Sobester, A. and Keane, A.: 2008, Engineering Design via Surrogate Modelling, Wiley. Gorissen, D., Crombecq, K., Couckuyt, I., Dhaene, T. and Demeester, P.: 2010, A Surrogate Modeling and Adaptive Sampling Toolbox for Computer Based Design, Journal of Machine Learning Research, 11, 2051−2055. Horvitz, E.: 1999, Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 159-166. Huang, Y., Lam, K.P. and Dobbs, G.: 2008, A Scalable Lighting Simulation Tool for Integrated Building Design, IBPSA-USA National Conference 2008, 30 July - 1 August, Berkeley, California, 206-213. Janssen, P. and Kaushik, V.: 2012, Iterative Refinement Through Simulation – exploring trade-offs between speed and accuracy, ECAADE 2012, Prague, Czech Republic, 555-564. Jin, R., Chen, W. and Simpson, T.: 2001, Comparative Studies of Metamodelling Techniques Under Multiple Modelling Criteria, Structural and Multidisciplinary Optimization, 23(1), 1-13. Villa-Vialaneix, N., Fallador, M., Ratto, M. and Leip, A.: 2012, A Comparison of Eight Metamodeling Techniques for the Simulation of N20 fluxes and N leaching from Corn Crops, Environmental Modelling & Software, 34, 51-66. Reinhart, C.: 2010, Tutorial on the Use of DAYSIM Simulations for Sustainable Design, Harvard University, Cambridge. Ryberg, A. B., Domeij Bäckryd, R., and Nilsson, L.: 2012, Metamodel-based multidisciplinary design optimization for automotive applications, Technical Report, Linköping University, Sweden. Ward, G. L. and Shakespeare, R. A.: 1998, Rendering with Shakespeare – The Art and Science of Lighting Visualization, Booksurge LLC, Charleston. Sidefx: 2014. Houdini 3D Animation Tools, available from http://www.sidefx.com/, accessed on 12 April 2014. Woodbury, R. F. and Burrow, A. L.: 2006, Whither design space?, AIE EDAM: Artificial Intelligence for Engineering Design, Analysis, and Manufacturing 20(02), 63-82.