Ben Krause, PhD student

I am a machine learning PhD student at the University of Edinburgh. My supervisors are Steve Renals and Iain Murray. My research is focused on auto-regressive sequence modelling with recurrent neural networks. I was also a member of the the University of Edinburgh's team to build a social bot for the Amazon Alexa prize.


Basic info

Email: ben.krause@.ed.ac.uk

Office: INF-305

Github: https://github.com/benkrause

Multiplicative LSTM

Multiplicative long short-term memory (mLSTM) is recurrent neural network architecture designed primarily for sequence modelling. mLSTM combines a multiplicative RNN and an LSTM. mLSTM was used by Open AI for sentiment analysis, and by Tilde for neural machine translation. mLSTM was published as a workshop paper at ICLR 2017, and the most recent version is given below.

Krause, B., Lu, L., Murray, I., & Renals, S. (2016). Multiplicative LSTM for sequence modelling. arXiv preprint arXiv:1609.07959.

Dynamic Evaluation

Dynamic evaluation is a method for gradient based adaptation to sequence history that can exploit re-occurring sequential patterns. I explored and developed dynamic evaluation methodology to improve the state-of-the-art at several commonly benchmarked character and word-level language modelling tasks.

Krause, B., Kahembwe, E., Murray, I., & Renals, S. (2017). Dynamic Evaluation of Neural Sequence Models. arXiv preprint arXiv:1709.07432.

Conversational AI

During my time working on the Amazon Alexa prize, I developed data driven methods for building open domain conversation agents that combined retrieval and generative approaches, and contributed to the development of a new data collection technique called self-dialogues. Our conversation corpus collected from Amazon Mechanical Turk will be publicly available soon.

Krause, B., Damonte, M., Dobre, M., ... & Webber, B. (2017). Edina: Building an Open Domain Socialbot with Self-dialogues. arXiv preprint arXiv:1709.09816.

Optimization

My master's thesis and my early PhD work explored Hessian-free optimization in LSTMs

Krause, B. (2015). Optimizing and Contrasting Recurrent Neural Network Architectures. arXiv preprint arXiv:1510.04953.

Krause, B., Lu, L., Murray, I., & Renals, S. (2015). On the Efficiency of Recurrent Neural Network Optimization Algorithms. NIPS Workshop on Optimization for Machine Learning, Montreal, Canada, 2015.

Neuroimaging

I also have past neuroimaging research studying the role of neurotransmitters in brain blood flow and schizophrenia

Krause, B. W., Wijtenburg, S. A., Holcomb, H. H., Kochunov, P., Wang, D. J., Hong, L. E., & Rowland, L. M. (2014). Anterior cingulate GABA levels predict whole-brain cerebral blood flow. Neuroscience letters, 561, 188-191.

Rowland, L. M., Krause, B. W., Wijtenburg, S. A., McMahon, R. P., Chiappelli, J., Nugent, K. L., ... & Hong, L. E. (2016). Medial frontal GABA is lower in older schizophrenia: a MEGA-PRESS with macromolecule suppression study. Molecular psychiatry, 21(2), 198-204.

Korenic, S. A., Nisonger, S. J., Krause, B. W., Wijtenburg, S. A., Hong, L. E., & Rowland, L. M. (2016). Effectiveness of fast mapping to promote learning in schizophrenia. Schizophrenia Research: Cognition, 4, 24-31.