Category Knowing Vertex AI feature store

Before going any further, it is imperative to understand the basic concept of the web. The idea behind HTTP 1.X is that a client sends an HTTP request to a server, and then the server responds to that client. That can sound trivial if you have web development experience. However, it is one of the most important web programming concepts, irrespective of whether you are building web APIs, websites, or complex cloud applications.Let’s reduce an HTTP request lifetime to the following:

1. The communication starts.
2. The client sends a request to the server.
3. The server receives the request.
4. The server does something with the request, like executing code/logic.
5. The server responds to the client.
6. The communication ends.

After that cycle, the server is no longer aware of the client. Moreover, if the client sends another request, the server is unaware that it responded to a request earlier for that same client because HTTP is stateless.There are mechanisms for creating a sense of persistence between requests for the server to be “aware” of its clients. The most well-known of these is cookies.If we dig deeper, an HTTP request comprises a header and an optional body. Then, requests are sent using a specific method. The most common HTTP methods are GET and POST. On top of those, extensively used by web APIs, we can add PUT, DELETE, and PATCH to that list.Although not every HTTP method accepts a body, can respond with a body, or should be idempotent, here is a quick reference table:

Method	Request has body	Response has body	Idempotent
GET	No*	Yes	Yes
POST	Yes	Yes	No
PUT	Yes	No	Yes
PATCH	Yes	Yes	No
DELETE	May	May	Yes

* Sending a body with a GET request is not forbidden by the HTTP specifications, but the semantics of such a request are not defined either. It is best to avoid sending GET requests with a body.

An idempotent request is a request that always yields the same result, whether it is sent once or multiple times. For example, sending the same POST request multiple times should create multiple similar entities, while sending the same DELETE request multiple times should delete a single entity. The status code of an idempotent request may vary, but the server state should remain the same. We explore those concepts in more depth in Chapter 4, Model-View-Controller.Here is an example of a GET request:

GET http: //www.forevolve.com/ HTTP/1.1
Host: www.forevolve.com
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,fr-CA;q=0.8,fr;q=0.7
Cookie: …

The HTTP header comprises a list of key/value pairs representing metadata that a client wants to send to the server. In this case, I queried my blog using the GET method and Google Chrome attached some additional information to the request. I replaced the Cookie header’s value with … because it can be pretty large and that information is irrelevant to this sample. Nonetheless, cookies are passed back and forth like any other HTTP header.

Anti-patterns and code smells are bad architectural practices or tips about possible bad design. Learning about best practices is as important as learning about bad ones, which is where we start. The book highlights multiple anti-patterns and code smells to help you get started. Next, we briefly explore the first few.

Anti-patterns

An anti-pattern is the opposite of a design pattern: it is a proven flawed technique that will most likely cause you trouble and cost you time and money (and probably give you headaches).An anti-pattern is a pattern that seems a good idea and seems to be the solution you were looking for, but it causes more harm than good. Some anti-patterns started as legitimate design patterns and were labelled anti-patterns later. Sometimes, it is a matter of opinion, and sometimes the classification can be influenced by the programming language or technologies.Let’s look at an example next. We will explore some other anti-patterns throughout the book.

Anti-pattern – God Class

A God class is a class that handles too many things. Typically, this class serves as a central entity which many other classes inherit or use within the application it is the class that knows and manages everything in the system; it is the class. On the other hand, it is also the class that nobody wants to update, which breaks the application every time somebody touches it: it is an evil class!The best way to fix this is to segregate responsibilities and allocate them to multiple classes rather than concentrating them in a single class. We look at how to split responsibilities throughout the book, which helps create more robust software.If you have a personal project with a God class at its core, start by reading the book and then try to apply the principles and patterns you learn to divide that class into multiple smaller classes that interact together. Try to organize those new classes into cohesive units, modules, or assemblies.To help fix God classes, we dive into architectural principles in Chapter 3, Architectural Principles, opening the way to concepts such as responsibility segregation.

Code smells

A code smell is an indicator of a possible problem. It points to areas of your design that could benefit from a redesign. By “code smell,” we mean “code that stinks” or “code that does not smell right.”It is important to note that a code smell only indicates the possibility of a problem; it does not mean a problem exists. Code smells are usually good indicators, so it is worth analyzing your software’s “smelly” parts.An excellent example is when a method requires many comments to explain its logic. That often means that the code could be split into smaller methods with proper names, leading to more readable code and allowing you to get rid of those pesky comments.Another note about comments is that they don’t evolve, so what often happens is that the code described by a comment changes, but the comment remains the same. That leaves a false or obsolete description of a block of code that can lead a developer astray.The same is also true with method names. Sometimes, the method’s name and body tell a different story, leading to the same issues. Nevertheless, this happens less often than orphan or obsolete comments since programmers tend to read and write code better than spoken language comments. Nonetheless, keep that in mind when reading, writing, or reviewing code.

Since you just purchased a book about design patterns, I guess you have some idea of what design patterns are, but let’s make sure that we are on the same page.Abstract definition: A design pattern is a proven technique that we can use to solve a specific problem.In this book, we apply different patterns to solve various problems and leverage some open-source tools to go further, faster! Abstract definitions make people sound smart, but understanding concepts requires more practice, and there is no better way to learn than by experimenting with something, and design patterns are no different.If that definition does not make sense to you yet, don’t worry. You should have enough information by the end of the book to correlate the multiple practical examples and explanations with that definition, making it crystal clear.I like to compare programming to playing with LEGO® because what you have to do is very similar: put small pieces together to create something bigger. Therefore, if you lack imagination or skills, possibly because you are too young, your castle might not look as good as someone with more experience. With that analogy in mind, a design pattern is a plan to assemble a solution that fits one or more scenarios, like the tower of a castle. Once you designed a single tower, you can build multiple by following the same steps. Design patterns act as that tower plan and give you the tools to assemble reliable pieces to improve your masterpiece (program).However, instead of snapping LEGO® blocks together, you nest code blocks and interweave objects in a virtual environment!Before going into more detail, well-thought-out applications of design patterns should improve your application designs. That is true whether designing a small component or a whole system. However, be careful: throwing patterns into the mix just to use them can lead to the opposite result: over-engineering. Instead, aim to write the least amount of readable code that solves your issue or automates your process.As we have briefly mentioned, design patterns apply to different software engineering levels, and in this book, we start small and grow to a cloud-scale! We follow a smooth learning curve, starting with simpler patterns and code samples that bend good practices to focus on the patterns—finally ending with more advanced topics and good practices.Of course, some subjects are overviews more than deep dives, like automated testing, because no one can fit it all in a single book. Nonetheless, I’ve done my best to give you as much information about architecture-related subjects as possible to ensure the proper foundations are in place for you to get as much as possible out of the more advanced topics, and I sincerely hope you’ll find this book a helpful and enjoyable read.Let’s start with the opposite of design patterns because it is essential to identify wrong ways of doing things to avoid making those mistakes or to correct them when you see them. Of course, knowing the right way to overcome specific problems using design patterns is also crucial.

Give your feedback straight to the author himself and chat to other early readers on our Discord server (find the “architecting-aspnet-core-apps-3e” channel under EARLY ACCESS SUBSCRIPTION).

https://packt.link/EarlyAccess

The goal of this book is not to create yet another design pattern book; instead, the chapters are organized according to scale and topic, allowing you to start small with a solid foundation and build slowly upon it, just like you would build a program.Instead of a guide covering a few ways of applying a design pattern, we will explore the thought processes behind the systems we are designing from a software engineer’s point of view.This is not a magic recipe book; from experience, there is no magical recipe when designing software; there are only your logic, knowledge, experience, and analytical skills. Let’s define “experience” as your past successes and failures. And don’t worry, you will fail during your career, but don’t get discouraged by it. The faster you fail, the faster you can recover and learn, leading to successful products. Many techniques covered in this book should help you achieve success. Everyone has failed and made mistakes; you aren’t the first and certainly won’t be the last. To paraphrase a well-known saying by Roosevelt: the people that never fail are the ones who never do anything.At a high level:

• This book explores basic patterns, unit testing, architectural principles, and some ASP.NET Core mechanisms.
• Then, we move up to the component scale, exploring patterns oriented toward small chunks of software and individual units.
• After that, we move to application-scale patterns and techniques, exploring ways to structure an application.
• Some subjects covered throughout the book could have a book of their own, so after this book, you should have plenty of ideas about where to continue your journey into software architecture.

Here are a few pointers about this book that are worth mentioning:

• The chapters are organized to start with small-scale patterns and then progress to higher-level ones, making the learning curve easier.
• Instead of giving you a recipe, the book focuses on the thinking behind things and shows the evolution of some techniques to help you understand why the shift happened.
• Many use cases combine more than one design pattern to illustrate alternate usage so you can understand and use the patterns efficiently. This also shows that design patterns are not beasts to tame but tools to use, manipulate, and bend to your will.
• As in real life, no textbook solution can solve all our problems; real problems are always more complicated than what’s explained in textbooks. In this book, I aim to show you how to mix and match patterns to think “architecture” instead of giving you step-by-step instructions to reproduce.

The rest of the introduction chapter introduces the concepts we explore throughout the book, including refreshers on a few notions. We also touch on .NET, its tooling, and some technical requirements.In this chapter, we cover the following topics:

• What is a design pattern?
• Anti-patterns and code smell.
• Understanding the web – request/response.
• Getting started with .NET.

Following are some of the limitations of Explainable AI in Vertex AI:

• Each attribution merely displays how much the attribute influenced the forecast for that case. A single attribution may not represent model behavior. Aggregate attributions is preferred over a dataset to understand approximate model behavior.
• Model and data determine attributions. They can only show the model’s data patterns, not any underlying linkages. The target’s association with a feature does not depend on its strong attribution. The attribution indicates if the model predicts using the characteristic.
• Attributions alone cannot determine quality of the model; it is recommended to consider assessment of the training data and evaluation metrics of the model.
• Integrated gradients method works well for the differentiable models (where derivative of all the operations can be calculated in TensorFlow graph). Shapley method is used for the Non-differentiable models (non-differentiable operations in the TensorFlow network, such as rounding operations and decoding).

Conclusion

In this book, we started by understanding the cloud platform, a few important components of the cloud, and the advantages of the cloud platforms. We started working development of the machine learning models through Vertex AI AutoML for tabular, text, and image data, we deployed the trained models onto the endpoints for the online predictions. Even before entering into the complexity of the custom model building, we worked to understand how to leverage pre-build models of the platform to obtain predictions. For the custom models, we utilized a workbench for the code development for the model training and utilized docker images to submit the training jobs, also worked on the hyperparameter tuning to further enhance the model performance using Vizier. We worked on the pipeline components of the platform to train the model and evaluate and deploy the model for online predictions using both Kubeflow and TFX. We worked on creating a centralized repository for the features using the feature store of the Vertex AI. This is the last chapter of the book, where we learned about the explainable AI, need of it. We trained the AutoML classification model for image and tabular data for the explanations and obtained the explanations using the Python code. GCP is adding lot of new components and features to enhance its capability, check the platform (documentation of the platform) regularly to keep yourself updated.

Questions

1. Why explainable AI is important?
2. What are the different types of explanations supported by Vertex AI?
3. What is the difference between example based and feature based examples?

In case of the image data, users had to configure explainable AI during the training and deployment phase, whereas, in case of tabular data, explainable AI needs to be configured only during the deployment phase (AutoML will enable the explainable AI by default during the training phase for the tabular data). Follow the steps mentioned in Chapter 2, Introduction to Vertex AI & AutoML Tabular for the tabular dataset creation and tabular AutoML model training. Follow the below mentioned steps for the model deployment of the trained model.

Step 1: Trained model in the model registry

Trained model will be listed in the model registry as shown in the following figure:

Figure 10.17: Model registry (tabular classification model)

1. tabular_classification trained using AutoML. Click on the model and the version of the same.

Step 2: Deploy to end point

Once the model is selected (along with the version) users will get option to evaluate the model, deploy and test the model, and so on. Follow the steps mentioned below to deploy the model:

Figure 10.18: Trained tabular classification model

1. Select DEPLOY AND TEST tab.
2. Click DEPLOY TO ENDPOINT.

Step 3: Endpoint definition

Follow the steps mentioned below to define the endpoint:

Figure 10.19: Endpoint definition tabular classification model

1. Provide Endpoint name.
2. Click CONTINUE.

Step 4: Model settings

Follow the steps mentioned below to configure the model settings and to enable the explain ability options:

Figure 10.20: Model settings (enabling explain ability)

1. Set the Traffic split to 100.
2. Set the Minimum number of compute nodes to 1.
3. Set the Maximum number of compute nodes to 1.
4. Select n1-standard-8 in Machine type.
5. Enable the Explainability options.
6. Click EDIT.

Step 5: Set the Explainability options

You can set the Explainability options by following the steps shown in the following figure:

Figure 10.21: Sampled Shapley path count

1. Select Sampled Shapley method.
2. Set the Path count to 7 (randomly chosen).
3. Click DONE.

Step 6: Model monitoring

Follow the below mentioned steps to disable the model monitoring (since it is not needed for the explanations):

Figure 10.22: Model monitoring

1. Disable Model monitoring options.
2. Click DEPLOY.

Once the model is deployed successfully, open the Jupyter lab from the workbench created and enter the Python code given in the following steps:
Step 1: Install the required packages
Type the following Python code to install the required packages:
!pip install tensorflow
!pip install pip install google-cloud-aiplatform==1.12.1

Step 2: Kernel restart
Type following commands in the next cell, to restart the kernel: (Users can restart kernel from the GUI as well):
import os
import IPython
if not os.getenv(“”):
Ipython.Application.instance().kernel.do_shutdown(True)

Step 3: Importing required packages
Once the kernel is restarted, run the following lines of codes to import the packages:
import base64
import tensorflow as tf
import google.cloud.aiplatform as gcai
import explainable_ai_sdk
import io
import matplotlib.image as mpimg
import matplotlib.pyplot as plt

Step 4: Input for prediction and explanation
Choose any image from the training set (stored in the cloud storage for the prediction) and provide the full path of the image chosen in the following code. Run the cell to read the image and covert the image to the required format:
img_input = tf.io.read_file(“gs://AutoML_image_data_exai/Kayak/adventure-clear-water-exercise-1836601.jpg”)
b64str = base64.b64encode(img_input.numpy()).decode(“utf-8”)
instances_image = [{“content”: b64str}]

Step 5: Selection of the endpoint select
Run the following lines of code to select the endpoint where the model is deployed. In this method, we are using the display name of the endpoint (instead of the endpoint ID). Image_ex is the endpoint name where the model is deployed. Full path of the endpoint (along with the endpoint ID) will be displayed in the output:
endpoint = gcai.Endpoint(gcai.Endpoint.list(
filter=f’display_name={“image_ex”}’,
order_by=’update_time’)[-1].gca_resource.name)
print(endpoint)

Step 6: Image prediction
Run the following lines of code to get the prediction from the deployed model:
prediction = endpoint.predict(instances=instances_image)
print(prediction)
Prediction results will be displayed as shown in the following figure which contains display names and the probability of the classes:

Figure 10.15: Image classification prediction result
Note: Since we are running this code using the Vertex AI workbench, we are not using service account for authentication.
Step 7: Explanations
Run the following lines of codes to get the explanations for the input image:
response = endpoint.explain(instances=instances_image)

for explanation in response.explanations:
attributions = dict(explanation.attributions[0].feature_attributions)
image_ex = io.BytesIO(base64.b64decode(attributions[“image”][“b64_jpeg”]))
plt.imshow(mpimg.imread(image_ex, format=”JPG”), interpolation=”nearest”)
plt.show()

The output of the explanations is shown in the figure. Highlighted areas in green indicates areas/pixels which played important role for the prediction of the image:

Figure 10.16: Image classification model explanation

In this exercise, we will try to understand how explainable AI can be used to understand model prediction using image data and the tabular data with the help of AutoML of Vertex AI. The data used for AutoML tables and images will be used for this exercise as well. (Refer chapters Introduction to Vertex AI and AutoML Tabular and AutoML Image, text and pre-built models).

AutoML_image_data_exai bucket is created under us-centra1 (single region).

Three folders containing image data (Cise_ships, Ferry_boat and Kayak) is uploaded and CSV file (class_labels.csv) is created as shown in Chapter 3, AutoML Image, text and pre-built models (refer Figure 3.1 and 3.2 for csv creation). heart_2020_train_data.csv is uploaded to the same folder which will be used for AutoML tables. Figure 10.2 shows the data uploaded to the cloud storage:

Figure 10.2: Data uploaded to the cloud storage

Model training for image data

The initial steps for dataset creation for the image data will have no changes. Refer Image dataset creation of Chapter 3, AutoML Image, text and pre-built models and follow the below mentioned steps for AutoML model training.

Step 1: Train new model

Newly created dataset will be listed under the dataset of the Vertex AI. Follow the below steps to initiate the model training as shown in the following screenshot:

Figure 10.3: Image dataset created on Vertex AI

1. Click on Datasets section of Vertex AI (open the newly created image dataset).
2. Click TRAIN NEW MODEL.

Step 2: Training method selection

Training method step does not have difference as mentioned in Chapter 3, AutoML Image, text and pre-built models. Follow these steps to set the training method:

Figure 10.4: Training method selection

1. Select AutoML.
2. Select Cloud (we will deploy the model to get predictions and explanations).
3. Click on CONTINUE.

Step 3: Model details

Follow the steps mentioned below to set the model details:

Figure 10.5: Image classification model details

1. Select Train new model.
2. Provide a Name for the model.
3. Provide Description for the model.
4. Under Data split select Randomly assigned.
5. Click CONTINUE.

Step 4: Training options

Follow the below mentioned steps for training options:

Figure 10.6: Image classification training options

1. Select Default training method.
2. Click CONTINUE (we do not have to enable incremental training for the explainable AI).

Each approach for attributing features is based on Shapley values, which is an algorithm derived from cooperative game theory that gives credit for a given result to each participant in a game. When this concept is applied to models for machine learning, it indicates that each model feature is dealt with as if it were a “player” in the game. The Vertex Explainable AI provides a certain amount of credit to each individual characteristic based on its weight in the overall forecast:

• Sampled Shapley method: The sampled Shapley technique offers an estimate of the actual Shapley values via the use of sampling. Tabular models created using AutoML make use of the sampled Shapley approach to determine the relevance of features. For these models, which are meta-ensembles of tree and neural network structures, the Sampled Shapley method performs quite well.
• Integrated gradients method: Along an integral route, the integrated gradients approach calculates the gradient of the prediction output with respect to the characteristics of the input. This is done to get an accurate result. Calculations of the gradients are performed at various time intervals along a scaling parameter. Utilizing the Gaussian quadrature rule allows for the calculation of the size of each interval. (When dealing with picture data, think of this scaling option as a “slider” that sets all the image’s pixels to a black value.) The integration of the gradients is done as follows:
  • An approximation of the integral may be found by using a weighted average.
  • Calculations are performed to get the element-wise product of the original input and the averaged gradients.
• XRAI method: To discover which parts of a picture that contribute the most to a certain prediction of class, the XRAI approach uses a combination of the integrated gradients method and some extra phases.
• Pixel-level attribution: XRAI can do pixel-level attribution for the picture that is sent into it. In this stage of the process, XRAI makes use of the integrated gradients approach, applying it to both, a black and a white baseline.
• Over segmentation: XRAI generates a tiny patch over the picture by over segmentation , which is done independently of pixel-level attribution. In order to construct the picture segments, XRAI takes advantage of Felzenswalb’s graph-based technique.
• Region selection: XRAI compiles the pixel-level attribution included inside each segment to calculate the attribution density of that segment. XRAI assigns a ranking to each segment based on these values, and then it arranges the segments from most positive to least positive. This identifies which parts of the picture contribute the most strongly to a certain class prediction, as well as which parts of the image are most prominent.

All types of models are supported for feature-based explanations. Classification models are supported for AutoML images and classification, and regression models are supported for AutoML tabular models.

Introduction

This last chapter of the book covers explainable AI. We will start with understanding what is explainable AI, its need, how explainable AI works on Vertex AI (for image and tabular data) and how to get the explanations from the deployed model.

Structure

In this chapter, we will discuss the following topics:

• What is Explainable AI
• Need of Explainable AI
• XAI on Vertex AI
• Data for Explainable AI exercise
• Model training for image data
• Image classification model deployment
• Explanations for image classification
• Tabular classification model deployment
• Explanations for tabular data
• Deletion of resources
• Limitations of Explainable AI

Objectives

By the end of this chapter, you will have a good idea about explainable AI and will know how to get the explanations from the deployed model in Vertex AI.

What is Explainable AI

Explainable AI (XAI) is a subfield of Artificial Intelligence (AI) that focuses on developing methods and strategies for using AI in a way that makes the outcomes of the solution understandable to human specialists. The mission of XAI is to ensure that AI systems be open and honest about not just the function they perform but also the purpose they serve. Interpretability is the broader umbrella under AI which includes explainable AI as one of its subcategories. Users can grasp what a model is learning, the additional information it must provide, and the reasoning behind its judgments concerning the problem that exists in the real world that we are seeking to solve, thanks to the model’s interpretability.

Explainable AI is one of the core ideas that define trust in AI systems (along with accountability, reproducibility, lack of machine bias, and resiliency). The aim and ambition shared by data scientists and machine learning technologists is the development of AI that is explainable.