23 October 2015 – Earlier this week I attended the IEEE International Conference on Data Science and Advanced Analytics which, as its web sites notes, aims to provide a premier forum that brings together researchers, industry practitioners, as well as potential users of big data, for discussion and exchange of ideas on the latest theoretical developments in data science as well as on the best practices for a wide range of applications.
It is a rather intense, comprehensive event with 30+ long presentations and 50+ short presentations plus special sessions and exhibitions. It covers trends and controversies in data science, Big Data and distributed technologies/intelligent agents, emotion and sentiment in intelligent systems, social data analysis, geo-spatial data analytics, etc. There was even a module on “machine learning” techniques and their impact upon the legal practice. And a very cool session on how machine learning algorithms automatically identify spam, trained through carefully validated training sets of data – a topic near and dear to our legal industry predictive coding readers.
Of special note was a new paper presented by MIT’s Computer Science and Artificial Intelligence Laboratory that detailed the evolution of the university’s Data Science Machine – a sort-of AI system that is adept at spotting trends and patterns in large chunks of data.
The MIT development may allow artificial intelligence to ease the shortage of data scientists who build models that explain and predict patterns in the ocean of “Big Data” representing today’s world. The MIT computer software has proved capable of building better predictive models than the majority of human researchers it competed against in several recent data science contests.
Until now, well-paid data scientists have relied on their human intuition to create and test computer models that can explain and predict patterns in data. But MIT’s “Data Science Machine” software represents a fully automated process capable of building such predictive computer models by identifying relevant features in raw data. Such a tool could make human data scientists even more effective by allowing them to build and test such predictive models in far less time. But it might also help more individuals and companies harness the power of Big Data without the aid of trained data scientists.
MIT ran the machine as a ringer in three human data science tournaments and had considerable success. Out of 906 human teams in the competition to find patterns in data fields, the computer system beat 615 of them. The Data Science Machine managed to get within 87 and 96 per cent of the accurate answers submitted by human competitors. But, crucially, the Data Science Machine managed to do the job much faster than its fleshy competitors – human teams took weeks to divine patterns from the data while the computer took a maximum of 12 hours.
The competitive success of the Data Science Machine suggests it has a role alongside data scientists. Currently, data scientists are very involved in the feature generation and selection processes. The MIT results showed that the Data Science Machine can automatically create features of value and figure out how to use those features in creating a model. Although humans beat the Data Science Machine for all data sets, the machine’s success-to-effort ratio suggests there is a place for it in data science.
Quoting from one of the presentation sheets:
The Data Science Machine is able to derive predictive models from raw data automatically. To achieve this automation, we first propose and develop the Deep Feature Synthesis algorithm for automatically generating features for relational datasets. The algorithm follows relationships in the data to a base field, and then sequentially applies mathematical functions along that path to create the final feature.
Second, we implement a generalizable machine learning pipeline and tune it using a novel Gaussian Copula process based approach. We entered the Data Science Machine in 3 data science competitions that featured 906 other data science teams. The approach beats 615 teams in these data science competitions. In 2 of the 3 competitions we beat a majority of competitors, and in the third, we achieved 94% of the best competitor’s score.
In the best case, with an ongoing competition, we beat 85.6% of the teams and achieved 95.7% of the top submissions score.
To read the full paper click here.
If some of this sounds a bit familiar, yes, you (kind of) heard it before, so let’s travel in “The Way Back Machine” to 2006. It concerns digital neural nets.
Digital neural nets were invented in the 1950s, but it took decades for computer scientists to learn how to tame the astronomically huge combinatorial relationships between a million—or 100 million—neurons. The key was to organize neural nets into stacked layers. In 2006, Geoff Hinton, then at the University of Toronto, made a key tweak to this method, which he dubbed “deep learning.” He was able to mathematically optimize results from each layer so that the learning accumulated faster as it proceeded up the stack of layers. Deep-learning algorithms accelerated enormously a few years later when they were ported to GPUs. The code of deep learning alone is insufficient to generate complex logical thinking, but it is an essential component of all current AIs, including IBM’s Watson, Google’s search engine, and Facebook’s algorithms. And now, MIT’s Data Science Machine.
Concluding note: and speaking of rides in “The Way Back Machine”, during the conference someone noted the work of L. Thorne McCarty (a fellow sailor I soon discovered) who was among the first (maybe the first?) to do significant research and write about artificial intelligence and legal reasoning back in the mid-1970s. It was news to me.