Stepwise Noise Elimination for Better Motivational and Advisory Texts Classification
Patrycja Swieczkowska, Rafal Rzepka, and Kenji Araki
Language Media Laboratory, Division of Media and Network Technologies, Graduate School of Information Science and Technology, Hokkaido University
Nishi 9, Kita 14, Kita-ku, Sapporo, Hokkaido 060-0814, Japan
There is little research into designing artificial motivational agents. The end-goal of our studies is therefore to create a dialogue system that would motivate users to do their everyday tasks using natural language. In this paper, we present a method of distinguishing texts containing motivational advice from regular texts to sort out noise in training data for our dialogue system. We implemented a novel method of chaining two shallow networks together by utilizing the output results of the first network to determine the input for the second one. We achieved F-score of 0.94 and 0.97 with our proposed method. The contributions of this paper are threefold: first, we successfully identified 14 hand-crafted features that make a text motivational/advisory. Secondly, we were able to create a classifying algorithm that distinguishes motivational/advisory texts from regular ones. Finally, our proposed method can be applied to other text classification tasks.
-  R. M. Badubi, “Theories of motivation and their application in organizations: A risk analysis,” Int. J. of Innovation and Economic Development, Vol.3, No.3, pp. 43-50, 2017.
-  B. Gerhart and M. Fang, “Pay, intrinsic motivation, extrinsic motivation, performance, and creativity in the workplace: Revisiting long-held beliefs,” Annual Review of Organizational Psychology and Organizational Behavior, Vol.2, pp. 489-521, 2015.
-  S. Kultalahti and R. L. Viitala, “Sufficient challenges and a weekend ahead–Generation Y describing motivation at work,” J. of Organizational Change Management, Vol.27, No.4, pp. 569-582, 2014.
-  A. Elmelid, A. Stickley, F. Lindblad, M. Schwab-Stone, C. C. Henrich, and V. Ruchkin, “Depressive symptoms, anxiety and academic motivation in youth: Do schools and families make a difference?,” J. of Adolescence, Vol.45, pp. 174-182, 2015.
-  D. Litalien, F. Guay, and A. J. S. Morin, “Motivation for PhD studies: Scale development and validation,” Learning and Individual Differences, Vol.41, pp. 1-13, 2015.
-  L. M. Fussner, K. J. Mancini, and A. M. Luebbe, “Depression and approach motivation: Differential relations to monetary, social, and food reward,” J. of Psychopathology and Behavioral Assessment, Vol.40, No.1, pp. 117-129, 2018.
-  R. Hershenberg, “Activating happiness: A jump-start guide to overcoming low motivation, depression, or just feeling stuck,” New Harbinger Publications, 2017.
-  HealthDirect, “Motivation: How to get started and staying motivated,” 2018, https://www.healthdirect.gov.au/motivation-how-to-get-started-and-staying-motivated [accessed October 23, 2018]
-  R. Nussbaum, “18 ways to motivate yourself that actually work,” 2017, Greatist, https://greatist.com/grow/motivation-tips-that-work [accessed October 23, 2018]
-  Reddit, “Get Disciplined,” 2018, https://www.reddit.com/r/getdisciplined/ [accessed October 2, 2018]
-  Reddit, “Today I Learned,” 2018, https://www.reddit.com/r/todayilearned/ [accessed May 28, 2018]
-  Reddit, “API Documentation,” 2018, https://www.reddit.com/dev/api/ [accessed November 27, 2018]
-  E. Cambria, S. Poria, D. Hazarika, and K. Kwok, “SenticNet 5: discovering conceptual primitives for sentiment analysis by means of context embeddings,” Proc. of 32nd Association for the Advancement of Artificial Intelligence (AAAI) Conf. on Artificial Intelligence, pp. 1795-1802, 2018.
-  S. Deshpande, G. K. Palshikar, and G. Athiappan, “An unsupervised approach to sentence classification,” Proc. of Int. Conf. on Management of Data COMAD, pp. 88-99, 2010.
-  Princeton University, “About WordNet,” 2010, https://wordnet.princeton.edu [accessed May 15, 2018]
-  G. Zhu and C. A. Iglesias, “Computing semantic similarity of concepts in knowledge graphs,” IEEE Trans. on Knowledge and Data Engineering, Vol.29, No.1, pp. 72-85, 2017.
-  S. Bird, E. Klein, and E. Loper, “Natural Language Toolkit,” 2018, https://www.nltk.org/ [accessed May 15, 2018]
-  P. Swieczkowska, R. Rzepka, and K. Araki, “Analyzing motivating texts for modelling human-like motivation techniques in emotionally intelligent dialogue systems,” Proc. of Biologically Inspired Cognitive Architectures Meeting, pp. 355-360, 2018.
-  D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” Proc. of the 3rd Int. Conf. for Learning Representations, 2014.
-  X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” Proc. of the 13th Int. Conf. on Artificial Intelligence and Statistics, pp. 249-256, 2010.
-  T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” Proc. of the 26th Int. Conf. on Neural Information Processing Systems, pp. 3111-3119, 2013.
-  Wikimedia Foundation, “Wikipedia dump,” 2018, https://dumps.wikimedia.org/backup-index.html [accessed February 8, 2018]
-  S. Arora, Y. Liang, and T. Ma, “A simple but tough-to-beat baseline for sentence embeddings,” 2017, https://openreview.net/pdf?id=SyK00v5xx [accessed April, 15 2017]
-  Nealrs, “Contractions.py,” 2014, https://gist.github.com/nealrs/96342d8231b75cf4bb82 [accessed November 27, 2018]
-  Mammothb, “Symspellpy,” 2018, https://github.com/mammothb/symspellpy [accessed November 27, 2018]
-  Z. Callejas and D. Griol, “An affective utility model of user motivation for counselling dialogue systems,” Proc. of Int. Workshop on Future and Emerging Trends in Language Technology, pp. 86-97, 2016.
-  B. Weiner, “An attribution theory of motivation and emotion,” Psychological Review, Vol.92, No.4, pp. 548-573, 1985.
-  M. Tielman, M. Neerincx, and W.-P. Brinkman, “Generating situation-based motivational feedback in a PTSD E-health system,” Proc. of Int. Conf. on Intelligent Virtual Agents, pp. 437-440, 2017.
-  H. A. He, S. Greenberg, and E. M. Huang, “One size does not fit all: Applying the transtheoretical model to energy feedback technology design,” Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, pp. 927-936, 2010.
-  E. C. Grigore, A. Pereira, and B. Scassellati, “Modeling motivational states in adaptive robot companions,” Proc. of Association for the Advancement of Artificial Intelligence (AAAI) Fall Symp., pp. 77-80, 2015.
-  B. J. Fogg, “A behavior model for persuasive design,” Proc. of the 4th Int. Conf. on Persuasive Technology, Article No.40, 2009.
-  H. Oinas-Kukkonen, “A foundation for the study of behavior change support systems,” Personal and Ubiquitous Computing, Vol.17, No.6, pp. 1223-1235, 2013.
-  L. Süssenbach, N. Riether, S. Schneider, I. Berger, F. Kummert, I. Lütkebohle, and K. Pitsch, “A robot as fitness companion: Towards an interactive action-based motivation model,” Proc. of the 23rd IEEE Int. Symp. on Robot and Human Interactive Communication, pp. 286-293, 2014.
-  S. Schneider and F. Kummert, “Motivational effects of acknowledging feedback from a socially assistive robot,” Proc. of Int. Conf. on Social Robotics, pp. 870-879, 2016.
-  M. Kaptein, B. de Ruyter, P. Markopoulos, and E. Aarts, “Adaptive persuasive systems: A study of tailored persuasive text messages to reduce snacking,” ACM Trans. on Interactive Intelligent Systems (TiiS), Vol.2, No.2, Article No.10, 2012.
-  M. Kaptein, P. Markopoulos, B. de Ruyter, and E. Aarts, “Can you be persuaded? Individual differences in susceptibility to persuasion,” Proc. of IFIP Conf. on Human-Computer Interaction, pp. 115-118, 2009.
-  S.-B. Cho and J. H. Kim, “Combining multiple neural networks by fuzzy integral for robust classification,” IEEE Trans. on Systems, Man, and Cybernetics, Vol.25, No.2, pp. 380-384, 1995.
-  A. K. Jerebko, J. D. Malley, M. Franaszek, and R. M. Summers, “Multiple neural network classification scheme for detection of colonic polyps in CT colonography data sets,” Academic Radiology, Vol.10, No.2, pp. 154-160, 2003.
-  K. Tsuda, H. Shin, and B. Schölkopf, “Fast protein classification with multiple networks,” Bioinformatics, Vol.21, No.Suppl_2, pp. ii59-ii65, 2005.
-  I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Proc. of Advances in Neural Information Processing Systems, pp. 2672-2680, 2014.
-  G. Koch, R. Zemel, and R. Salakhutdinov, “Siamese neural networks for one-shot image recognition,” Proc. of the 32nd Int. Conf. on Machine Learning (ICML), Deep Learning Workshop, Vol.2, 2015.
-  G. Gkioxari, A. Toshev, and N. Jaitly, “Chained predictions using convolutional neural networks,” Proc. of European Conf. on Computer Vision, pp. 728-743, 2016.