Journal of Computing and Information Science in Engineering

Design Synthesis Through a Markov Decision Process and Reinforcement Learning Framework

Share This Post


This article presents a framework that mathematically models optimal design synthesis as a Markov Decision Process (MDP) that is solved with reinforcement learning. In this context, the states correspond to specific design configurations, the actions correspond to the available alterations modeled after generative design grammars, and the immediate rewards are constructed to be related to the improvement in the altered configuration’s performance with respect to the design objective. Since in the context of optimal design synthesis the immediate rewards are in general not known at the onset of the process, reinforcement learning is employed to efficiently solve the MDP. The goal of the reinforcement learning agent is to maximize the cumulative rewards and hence synthesize the best performing or optimal design. The framework is demonstrated for the optimization of planar trusses with binary cross-sectional areas, and its utility is investigated with four numerical examples, each with a unique combination of domain, constraint, and external force(s) considering both linear-elastic and elastic-plastic material behaviors. The design solutions obtained with the framework are also compared with other methods in order to demonstrate its efficiency and accuracy.


Visit the ASME Digital Collection archives for JCISE

More To Explore


Special Issue on Geometric Data Processing and Analysis for Advanced Manufacturing

Geometric information, such as three-dimensional (3D) shapes and network topologies, has been increasingly explored in manufacturing research. For example, characterizing geometric information in 3D-printed parts, in-situ or ex-situ, opens opportunities for defect detection, quality improvement, and product customization. However, geometric data mining remains critically challenging. Geometric information is embedded in complex data structures, such as 3D point clouds, graphs, meshes, voxels, high-dimensional images, and tensors, which possess challenges for analysis due to their high-dimensionality, high-volume, unstructured, multimodality characteristics. Additional challenges stem from compromised data quality (e.g., noisy and incomplete data), the need for registration, etc.


2023 Reviewer’s Recognition

The Editor and Editorial Board of the Journal of Computing and Information Science in Engineering would like to thank all of the reviewers for volunteering their expertise and time reviewing manuscripts in 2023. Serving as reviewers for the journal is a critical service necessary to maintain the quality of our publication and to provide the authors with a valuable peer review of their work.