Technology

History of a Breakthrough

In the early 1990s, Steve Gallant co-created a new technology in computer analysis of unstructured text: fully distributed vector representations. Unlike rules-based Natural Language Processing (NLP) which traces its roots to the 1950s, vector representations offered an ideal way to represent text to leverage the power of machine learning algorithms for creating powerful predictive models.

But distributed vector representations, including modern versions like GloVe vectors, have a weakness: they are based on “a bag of words.” Thus, much of the information in the text is lost – negation, parse structure, etc. This limits the amount of information captured in the vector, and therefore limits the accuracy of any model using them.  This issue interested Professor Gallant for almost 20 years.

Then, in 2013, he had a breakthrough, finding a good way to include arbitrary structure in the same vector as the words. This is the NoNLP™ technology patented by Textician.  Machine learning on NoNLP vectors accesses a much deeper set of information in the vector, leading to powerful models.

Legacy Applications

With NoNLP

Selected Publications On Text & Machine Learning
  • Gallant, S. I. & Culliton, P. (2016). Positional Binding with Distributed Representations. ICIVC 2016, Portsmouth, England, August 3-5, 2016 (http://www.textician.com/rch-content/uploads/Positional-Binding-ICIVC16.PUBLISHED.pdf)
  • Gallant, S. I. & Okaywe, T. W. (2013) Representing Objects, Relations, and Sequences. Neural Computation 25, 2038-2078. (abstract/order) (late draft)
  • Caid WR, Dumais ST and Gallant SI. Learned vector-space models for document retrieval. Information Processing and Management, Vol. 31, No. 3, pp. 419-429, 1995. (pdf)
  • Gallant, S. I. Neural Network Learning and Expert Systems. M.I.T. Press. 380 Pages, March 1993 (ISBN 0-262-07145-2).
  • Gallant, S. I. Context Vectors: A Step Toward a “Grand Unified Representation”. Wermter, Stefan and Sun, Ron (Eds.) Hybrid Neural Systems. Springer Berlin Heidelberg: Lecture Notes in Computer Science Volume 1778, 2000, pp 204-210. (pdf-scanned)
  • Gallant, S. I., W. R. Caid, et al. Feedback and Mixing Experiments With MatchPlus. Harman, D. (ed), The Second Text REtrieval Conference (TREC-2), NIST publication SP 500-215, Washington, DC., Nov. 4–6, 1992, pg. 101-104. (pdf-scanned) (text)
  • Gallant, S. I., Caid, W. R., Carleton, J., Hecht-Nielsen, R., Pu Qing, K., & Sudbeck, D. HNC’s MatchPlus System. Harman, D. (ed), The First Text REtrieval Conference (TREC-1), NIST publication SP 500-207, Washington, DC., Nov. 4–6, 1992, pg. 107-111. (pdf-scanned)
  • Gallant, S. I. A Practical Approach for Representing Context And for Performing Word Sense Disambiguation Using Neural Networks. Neural Computation, Vol. 3, No. 3, 1991, 293-309. (abstract/order) Conference version: IJCNN-91, Seattle, Washington, July 8-12, 1991. (pdf-scanned)
  • Gallant, S. I., & Smith, D. (1987). Random cells: An idea whose time has come and gone. . .and come again? In Proceedings of the IEEE International Conference on Neural Networks (Vol. 2, pp. 671-678). Piscataway, NJ: IEEE. (pdf-scanned)
  • Gallant, S. I. and King, D. J. Experiments with Sequential Associative Memories. Cognitive Science Society Annual Conference, Montreal, August 17-19, 1988, 40-47. (pdf-scanned)
  • Gallant, S. I. Words and weights: what the network’s parameters tell the network’s programmers. International Symposium on Integrating Knowledge and Neural Heuristics, Pensacola Beach FL., May 9-10, 1994, pg. 21-24. (pdf-scanned)
  • Gallant, S. I. Three Constructive Algorithms for Network Learning. Eighth Annual Conference of the Cognitive Science Society, Amherst, Ma., Aug. 15-17, 1986, 652-660. (pdf-scanned)

  • Culliton, P, Levinson, M, Ehresman, A, Wherry, J, Steingrub, J, Gallant, S. (2017). Predicting Severe Sepsis Using Text from the Electronic Health Record. Neural Information Processing Systems (NIPS) Workshop on Machine Learning For Health. http://arxiv.org/abs/1711.11536

  • S. Gallant, P. Culliton, M. Levinson, A. Ehresman, J. Wherry, J. Steingrub (2018). Predicting Severe Sepsis from the Electronic Health Record Using Machine Learning. American Thoracic Society (ATS) 2018 Conference (abstract & poster), May 18-23, 2018, San Diego.