AutoAI for Text - overview
Scaling AI technologies for NLP and text data to a large variety of users.
Why AutoAI for Text
This technology fills a gap in the area of AutoML technologies by addressing the needs of the many users who have text data and want to quickly build advanced NLP models. AutoAI for Text is a framework that provides both the components and the tools that facilitate the process of NLP model development for text data, via automation, ease-of-use and explainability.
We target users at different levels of skills or expertise, with a particular focus on 1) data scientists who want to quickly get a head start in their text processing projects, and 2) advanced model developers who will want to optimize and fine-tune their models before deployed in production. The goal of AutoAI for Text is to make such users more efficient by reducing model development time and effort while at the same time improving the quality of their models through the use of automatic optimization.
AutoAI for Text starts from the premise that a large space of components is already available, including many types of features (e.g., TFIDF, embeddings), machine learning algorithms (both classical ML and deep learning or DL), as well as pre-existing models (e.g., from customers, from the development teams). However, combining them into feasible pipelines that perform well over a given text dataset and task is a daunting optimization problem, better left for automated algorithms. This is especially challenging in practice when one often has additional constraints that go beyond performance (or accuracy). These constraints may include: the need for explainability, training time budget, inference time constraints, resource availability (e.g., GPUs), and more.
What AutoAI for Text Provides
AutoAI for Text is a platform that aims to solve the above challenge. This platform combines LALE, a scikit-learn framework for defining and combining types of features and models (including both classical ML and DL models), with specialized operators for deep learning style of backpropagation and batching, and with an optimization layer that performs neural architecture search (NAS), as well as combined algorithm selection and hyperparameter tuning (CASH).
The overall architectural vision of AutoAI for Text is captured conceptually by the following diagram, containing its core elements. At the heart of the system are the three blocks for: 1) data preprocessing, 2) feature engineering and 3) pipeline selection / model tuning, each with a set of capabilities that can be combined, tuned and explored, ultimately leading to the generation of the top pipelines. The data quality estimation and data augmentation blocks offer additional capabilities to understand the characteristics of the dataset and, respectively, to handle class skew in the input data. One of the important blocks (explainability) focuses on presentation of the resulting pipelines and interaction with the human. Explanations can arrive either at the model-level (e.g., when directly intepretable models in the form of rules have been selected) or at the instance-level (i.e., by inspecting the behavior of the model on particular examples).