The work is described in the journal , in a paper by Rumen Dangovski and Li Jing, both MIT graduate students; Marin Soljačić, a professor of physics at MIT; Preslav Nakov, a senior scientist at the Qatar Computing Research Institute, HBKU; and Mićo Tatalović, a former Knight Science Journalism fellow at MIT and a former editor at magazine.
From AI for physics to natural language The work came about as a result of an unrelated project, which involved developing new artificial intelligence approaches based on neural networks, aimed at tackling certain thorny problems in physics.
This infection, termed "baylisascariasis," kills mice, has endangered the allegheny woodrat and has caused disease like blindness or severe consequences.
This infection, termed "baylisascariasis," kills mice, has endangered the allegheny woodrat.
Such systems are widely used for pattern recognition, such as learning to identify objects depicted in photos.
But neural networks in general have difficulty correlating information from a long string of data, such as is required in interpreting a research paper.Various tricks have been used to improve this capability, including techniques known as long short-term memory (LSTM) and gated recurrent units (GRU), but these still fall well short of what's needed for real natural-language processing, the researchers say.The team came up with an alternative system, which instead of being based on the multiplication of matrices, as most conventional neural networks are, is based on vectors rotating in a multidimensional space.The team also had help from the Science Daily website, whose articles were used in training some of the AI models in this research. If your browser does not accept cookies, you cannot view this site.We noticed that hey, if we use that, it could actually help with this or that particular AI algorithm." This approach could be useful in a variety of specific kinds of tasks, he says, but not all."We can't say this is useful for all of AI, but there are instances where we can use an insight from physics to improve on a given AI algorithm." Neural networks in general are an attempt to mimic the way humans learn certain new things: The computer examines many different examples and "learns" what the key underlying patterns are.Tatalović was at the time exploring AI in science journalism as his Knight fellowship project."And so we tried a few natural language processing tasks on it," Soljačić says.Each subsequent word swings this vector in some direction, represented in a theoretical space that can ultimately have thousands of dimensions.At the end of the process, the final vector or set of vectors is translated back into its corresponding string of words.