Which AI am I ? [Azure AI Applied Services : Part 1]

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.

This article is intended for someone completely new to the Azure and wanting to explore the AI suite of services provided by Azure

Overview

This is the introductory blog providing the basic knowledge of AI and its various branches. This article will be followed by bifercation of various Cognitive Services with the help of flow charts and graphs helping you better understand the use case and which service to opt for under which scenario

What is AI (Artificial Intelligence) ?

John McCarthy one of the "founding fathers" of artificial intelligence coined the term in 1955 as “ the science and engineering of making intelligent machines”. In simple terms the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.

What is ML (Machine Learning) ?

In, 1959 Arthur Samuel defined machine learning as a “Field of study that gives computers the ability to learn without being explicitly programmed”. Machine learning is considered a subset of AI. Machine learning focuses on training machines to analyse and learn from data the way humans do. It’s important to note that although all machine learning is AI, not all AI is machine learning.

Machine learning enables people to perform such tasks as predicting the future, classifying things in a meaningful way, grouping relative information together in clusters and making the best rational decision in a given context

What are Artificial Neural Networks?

Human brains have millions of neurons, which are cells that receive, process, and transmit electric and chemical signals. Neurons connect to other neurons and transmit information between them using chemicals, whereas information inside the neuron itself is electrically processed. Artificial neural networks are based on these biological analogies and their components.

Neural networks resemble nothing more than a series of multiplications and summations through algorithms. These algorithms help recognize patterns, classify & cluster information. Frank Rosenblatt at the Cornell Aeronautical Laboratory created the first example of a neuron of this kind in 1957, the perceptron, a computer program with human-like thought processes, primarily designed for image recognition.

These algorithms are extraordinarily effective at solving complex problems such as image and sound recognition or machine language translation; using specialized hardware, they can execute prediction computations quickly.

What is Deep Learning?

Artificial neural networks (ANNs) are comprised of a node layers, containing an input layer, one or more hidden layers, and an output layer. Each node, or artificial neuron, connects to another and has an associated weight and threshold. If the output of any individual node is above the specified threshold value, that node is activated, sending data to the next layer of the network. Otherwise, no data is passed along to the next layer of the network .

The “deep” in deep learning is referring to the depth of layers in a neural network. A neural network that consists of more than three layers—which would be inclusive of the inputs and the output—can be considered a deep learning algorithm. Deep learning models take in information from multiple data sources and analyze that data in real time, without the need for human intervention. In deep learning, graphics processing units (GPUs) are optimized for training models because they can process multiple computations simultaneously. Deep learning is what drives many artificial intelligence (AI) technologies that can improve automation and analytical tasks. Deep learning is also useful for applications like image recognition, where the AI needs to find the edges of a shape before it can identify what is in the image.

Azure provides an entire suite of AI enabled services. As a beginner it can get confusing as to the choice of these services to meet your application needs. This article is written with the intent to make this task easy for someone exploring these services

Azure Machine Learning Services

Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. It helps with tools to accelerate and automate their day-to-day workflows. These are intended if you wish to run your training script in the cloud or build a model from scratch. Essentially when experimenting with data, algorithms, and models, development is iterative. Here a project lifecycle would look as follows

1) Azure Machine Learning Studio

The Azure Machine Learning studio is a graphical user interface for a project workspace.

A workspace organizes a project and allows for collaboration for many users all working toward a common objective. Users in a workspace can easily share the results of their runs from experimentation in the studio user interface or use versioned assets for jobs like environments and storage reference. Azure Machine Learning studio is a web portal in Azure Machine Learning that contains low-code and no-code options for project authoring and asset management.

Azure Machine Learning includes several resources and assets to enable you to perform your machine learning tasks. These resources and assets are needed to run any job.

Resources: setup or infrastructural resources needed to run a machine learning workflow. Resources include:

Assets: created using Azure ML commands or as part of a training/scoring run. Assets are versioned and can be registered in the Azure ML workspace. They include:

Azure Machine Learning supports the use of both CPU, and GPU virtual machines when creating compute instances or clusters.

2) Azure Data Science Virtual Machine

The Azure Data Science Virtual Machine (DSVM) is a virtual machine image pre-loaded with data science & machine learning tools. Use this VM to build intelligent applications for advanced analytics. Its essentially useful short term experimentation and preferred for deep learning algorithms based on GPU’s. The comparison table available here will also help in terms deciding between Azure ML compute or spinning up DSVM. You can also find the available list of tools here

3) Azure Databricks

Databricks Machine Learning is an integrated end-to-end machine learning environment incorporating managed services for experiment tracking, model training, feature development and management, and feature and model serving. Databricks provides Databricks Runtime for Machine Learning which automates the creation of a cluster optimized for machine learning. Databricks Runtime ML clusters include the most popular machine learning libraries.

With Databricks Machine Learning, you can:

Train models either manually or with AutoML.
Track training parameters and models using experiments with MLflow tracking.
Create feature tables and access them for model training and inference.
Share, manage, and serve models using Model Registry.

4) Azure Cognitive Services

Azure Cognitive Services are cloud-based artificial intelligence (AI) services that help you build cognitive intelligence into your applications. You can think of them as Azure Cognitive Services as task-specific AI with built-in business logic, programming, orchestration and customization for ready-to-deploy AI solutions. They are available as REST APIs, client library SDKs, and user interfaces.

Based on their application and the kind of functionality (type of data they operate with and what they do with the data) they are broadly categorized as follows

1) Vision API's - Analysing images/text through pre-programmed algorithm to provide meaningful insights

2) Speech API's - Assist in spoken language transformations

3) Language API's - Understand conversations and unstructured text

4) Decision API's - Make smarter decisions faster

The Vision and Decision API's include search functionality of a solution to create knowledge base for further application like bot framework

You can follow the chart below to make the choice of service to start with when experimenting with ML offerings in Azure

Azure Open Datasets

Given all of the above services there is another service which should be considered before developing machine learning models. Azure Open Datasets are curated public datasets that you can use to add scenario-specific features to machine learning solutions for more accurate models. Open Datasets are in the cloud on Microsoft Azure and are integrated into Azure Machine Learning and readily available to Azure Databricks and Machine Learning Studio (classic). Curated open public datasets in Azure Open Datasets are optimized for consumption in machine learning workflows. Datasets include public-domain data for weather, census, holidays, public safety, and location that help you train machine learning models and enrich predictive solutions.

Now that you are familiar with the Azure Machine Learning services let's dig a little deeper if you want to leverage pre-build (predict/analyze/classify/identify/etc) models for specific datasets based on their structure.

Azure Cognitive Services provides a variety of API's that can be used for specific purpose. If you are certain that you want to use Azure for specific task without the overhead of maintaining the underlying infrastructure for compute go ahead with these services. The flow chart below will help you understand the choice of API services available to you.

Step 1) The first choice you need to make is whether you are using Cognitive services to gain insights from Docs/Texts/Images or does it have anything to do with Spoken Language ( this could also be in the form of Docs/Texts/Images)

Step 2) Once the first decision is made you go deeper into your requirements are you trying to observe & assess docs/text/documents (Vision API's) or do you want to observe and take some decision on the display (Decision API's). In case your decision had something to do with spoken langagues was it anything to do with Speech-To-Text, Text-To-Speech, Speech-To-Speech (Speech API's) or perform some kind of language analysis/Text Assessment/ Text-To-Text (Language API's)