Sparse Bayesian reinforcement learning
| dc.contributor.author | Lee, Minwoo, author | |
| dc.contributor.author | Anderson, Charles W., advisor | |
| dc.contributor.author | Ben-Hur, Asa, committee member | |
| dc.contributor.author | Kirby, Michael, committee member | |
| dc.contributor.author | Young, Peter, committee member | |
| dc.date.accessioned | 2017-09-14T16:04:58Z | |
| dc.date.available | 2017-09-14T16:04:58Z | |
| dc.date.issued | 2017 | |
| dc.description | Zip file contains supplementary video. | |
| dc.description.abstract | This dissertation presents knowledge acquisition and retention methods for efficient and robust learning. We propose a framework for learning and memorizing, and we examine how we can use the memory for efficient machine learning. Temporal difference (TD) learning is a core part of reinforcement learning, and it requires function approximation. However, with function approximation, the most popular TD methods such as TD(λ), SARSA, and Q-learning lose stability and diverge especially when the complexity of the problem grows and the sampling distribution is biased. The biased samples cause function approximators such as neural networks to respond quickly to the new data by losing what was previously learned. Systematically selecting a most significant experience, our proposed approach gradually stores the snapshot memory. The memorized snapshots prevent forgetting important samples and increase learning stability. Our sparse Bayesian learning model maintains the sparse snapshot memory for efficiency in computation and memory. The Bayesian model extends and improves TD learning by utilizing the state information in hyperparameters for smart decision of action selection and filtering insignificant experience to maintain sparsity of snapshots for efficiency. The obtained memory can be used to further improve learning. First, the placement of the snapshot memories with a radial basis function kernel located at peaks of the value function approximation surface leads to an efficient way to search a continuous action space for practical application with fine motor control. Second, the memory is a knowledge representation for transfer learning. Transfer learning is a paradigm for knowledge generalization of machine learning and reinforcement learning. Transfer learning shortens the time for machine learning training by using the knowledge gained from similar tasks. The dissertation examines a practice approach that transfers the snapshots from non-goal-directive random movements to goal-driven reinforcement learning tasks. Experiments are described that demonstrate the stability and efficiency of learning in 1) traditional benchmark problems and 2) the octopus arm control problem without limiting or discretizing the action space. | |
| dc.format.medium | born digital | |
| dc.format.medium | doctoral dissertations | |
| dc.format.medium | ZIP | |
| dc.format.medium | MP4 | |
| dc.identifier | Lee_colostate_0053A_14302.pdf | |
| dc.identifier.uri | https://hdl.handle.net/10217/183935 | |
| dc.language | English | |
| dc.language.iso | eng | |
| dc.publisher | Colorado State University. Libraries | |
| dc.relation.ispartof | 2000-2019 | |
| dc.rights | Copyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright. | |
| dc.subject | continuous action space | |
| dc.subject | practice | |
| dc.subject | sparse learning | |
| dc.subject | knowledge retention | |
| dc.subject | Bayesian learning | |
| dc.subject | reinforcement learning | |
| dc.title | Sparse Bayesian reinforcement learning | |
| dc.type | Text | |
| dcterms.rights.dpla | This Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). | |
| thesis.degree.discipline | Computer Science | |
| thesis.degree.grantor | Colorado State University | |
| thesis.degree.level | Doctoral | |
| thesis.degree.name | Doctor of Philosophy (Ph.D.) |