Module2- Getting started with cognitive services

 

Provisioning Cognitive Services Resources

Azure cognitive services include a wide range of AI capabilities that you can use in your applications. To use any of the cognitive services, you need to create appropriate resources in an Azure subscription to define an endpoint where the service can be consumed, provide access keys for authenticated access, and to manage billing for your application's usage of the service.

Options for Azure resources

For many of the available cognitive services, you can choose between the following provisioning options:

Multi-service resource

You can provision a Cognitive Services resource that supports multiple different cognitive services. For example, you could create a single resource that enables you to use the Text Analytics, Computer Vision, Speech, and other services.

This approach enables you to manage a single set of access credentials to consume multiple services at a single endpoint, and with a single point of billing for usage of all services.

Single-service resource

Each cognitive service can be provisioned individually, for example by creating discrete Text Analytics and Computer Vision resources in your Azure subscription.

This approach enables you to use separate endpoints for each service (for example to provision them in different geographical regions) and to manage access credentials for each service independently. It also enables you to manage billing separately for each service.

Single-service resources generally offer a free tier (with usage restrictions), making them a good choice to try out a service before using it in a production application.

Training and prediction resources

While most cognitive services can be used through a single Azure resource, some offer (or require) separate resources for model training and prediction. This enables you to manage billing for training custom models separately from model consumption by applications, and in most cases enables you to use a dedicated service-specific resource to train a model, but a generic Cognitive Services resource to make the model available to applications for inferencing.

Endpoints, Keys, and Locations

When you provision a cognitive service resource in your Azure subscription, you are defining an endpoint through which the service can be consumed by an application.

To consume the service through the endpoint, applications require the following information:

• The endpoint URI. This is the HTTP address at which the REST interface for the service can be accessed. Most cognitive services software development kits (SDKs) use the endpoint URI to initiate a connection to the endpoint.

• An subscription key. Access to the endpoint is restricted based on an subscription key. Client applications must provide a valid key to consume the service. When you provision a cognitive services resource, two keys are created - applications can use either key. You can also regenerate the keys as required to control access to your resource.

• The resource location. When you provision a resource in Azure, you generally assign it to a location, which determines the Azure data center in which the resource is defined. While most SDKs use the endpoint URI to connect to the service, some require the location.

Cognitive Services REST APIs

Cognitive services provide REST application programming interfaces (APIs) that client applications can use to consume services. In most cases, service functions can be called by submitting data in JSON format over an HTTP request (which may be a POST, PUT, or GET request depending on the specific function being called). The results of the function are returned to the client as an HTTP response, often with JSON contents that encapsulate the output data from the function.

The use of REST interfaces with an HTTP endpoint means that any programming language or tool capable of submitting and receiving JSON over HTTP can be used to consume cognitive services. This includes common programming languages such as Microsoft C#, Python, and JavaScript; as well as utilities such as Postman and cURL, which can be useful for testing.'

Cognitive Services SDKs

While you can develop an application that uses cognitive services using REST interfaces, it is generally easier to build more complex solutions using native libraries for the programming language in which you are developing the application.

Microsoft provides SDKs for common programming languages that abstract the REST interfaces for most cognitive services. SDK availability varies by individual cognitive services, but in the case of most services there is an SDK for languages such as:

• Microsoft C# (.NET Core)

• Python

• JavaScript (Node.js)

• Go

• Java

Each SDK includes packages that you can install in order to use service-specific libraries in your code, and online documentation to help you determine the appropriate classes, methods, and parameters used to work with the service.

Note: Throughout this course, we'll generally use the JSON messages used by the REST interface to describe cognitive services operations. Even if you plan to use SDKs rather than build applications with the REST interfaces, familiarity with the underlying JSON structures will help you understand the object models used by the SDKs for different programming languages.

Considerations for Cognitive Services Security

Access to cognitive services resources is restricted through the use of subscription keys. Management of access to these keys is a primary consideration for security.

Regenerating keys

You should regenerate keys regularly to protect against the risk of keys being shared with or accessed by unauthorized users. You can regenerate keys by using the visual interface in the Azure portal, or by using the az cognitiveservices account keys regenerate Azure command line interface (CLI) command.

Each cognitive service is provided with two keys, enabling you to regenerate keys without service interruption. To accomplish this:

1. Configure all production applications to use key 2.

2. Regenerate key 1

3. Switch all production applications to use the newly regenerated key 1.

4. Regenerate key 2.

Protecting keys with Azure Key Vault

Azure Key Vault is an Azure service in which you can securely store secrets (such as passwords and keys). Access to the key vault is granted to security principals, which you can think of a user identities that are authenticated using Azure Active Directory (Azure AD). Administrators can assign a security principal to an application (in which case it is known as a service principal) to define a managed identity for the application. The application can then use this identity to access the key vault and retrieve a secret to which it has access. Controlling access to the secret in this way minimizes the risk of it being compromised by being hard-coded in an application or saved in a configuration file.

You can store the subscription keys for a cognitive services resource in Azure Key Vault, and assign a managed identity to client applications that need to use the service. The applications can then retrieve the key as needed from the key vault, without risk of exposing it to unauthorized users.

You can develop applications that consume cognitive services by using a key for authentication. However, this means that the application code must be able to obtain the key. One option is to store the key in an environment variable or a configuration file where the application is deployed, but this approach leaves the key vulnerable to unauthorized access. A better approach when developing applications on Azure is to store the key securely in Azure Key Vault, and provide access to the key through a managed identity (in other words, a user account used by the application itself).