Microsoft AI-102 - Designing and Implementing a Microsoft Azure AI Solution

Box 1: " categories " : [ " Locations " , " Persons " , " Organizations " ],

Locations, Persons, Organizations are in the outputs.

Scenario: Contoso plans to develop a searchable knowledgebase of all the intellectual property

Note: The categories parameter is an array of categories that should be extracted. Possible category types: " Person " , " Location " , " Organization " , " Quantity " , " Datetime " , " URL " , " Email " . If no category is provided, all types are returned.

Box 2: { " name " : " entities " }

The include wikis, so should include entities in the outputs.

Note: entities is an array of complex types that contains rich information about the entities extracted from text, with the following fields

name (the actual entity name. This represents a " normalized " form)

wikipediaId

wikipediaLanguage

wikipediaUrl (a link to Wikipedia page for the entity)

etc.

Within a search service, synonym maps are a global resource that associate equivalent terms, expanding the scope of a query without the user having to actually provide the term. For example, assuming " dog " , " canine " , and " puppy " are mapped synonyms, a query on " canine " will match on a document containing " dog " .

Create synonyms: A synonym map is an asset that can be created once and used by many indexes.

Scenario: When the response confidence score is low, ensure that the chatbot can provide other response options to the customers.

When no good match is found by the ranker, the confidence score of 0.0 or " None " is returned and the default response is " No good match found in the KB " . You can override this default response in the bot or application code calling the endpoint. Alternately, you can also set the override response in Azure and this changes the default for all knowledge bases deployed in a particular QnA Maker service.

Choosing Ranker type: By default, QnA Maker searches through questions and answers. If you want to search through questions only, to generate an answer, use the RankerType=QuestionOnly in the POST body of the GenerateAnswer request.

The requirements state:

All AI solution responses must have a confidence score ≥ 70%.

If the response confidence score is < 70%, the response must be improved by human input.

Members of the Consultant-Bookkeeper group must be able to process financial documents, which includes performing manual review when the AI confidence is below threshold.

Looking at the provided JSON:

" fields " : {

" ReceiptType " : {

" type " : " string " ,

" valueString " : " Itemized " ,

" confidence " : 0.672

},

" MerchantName " : {

" type " : " string " ,

" valueString " : " Tailwind " ,

" confidence " : 0.913

}

}

ReceiptType.confidence = 0.672 → this is below 0.7.

MerchantName.confidence = 0.913 → this is above 0.7.

Therefore, the correct condition for triggering manual review is:

documentResults.fields. < field > .confidence < 0.7

Now, evaluating the options:

A. documentResults.docType == " prebuilt:receipt "

Always true for receipts, does not check confidence. Not correct.

B. documentResults.fields.*.confidence < 0.7

This is the correct general expression: trigger manual review whenever any field confidence is below 0.7.

C. documentResults.fields.ReceiptType.confidence > 0.7

This would bypass manual review when ReceiptType has high confidence. The requirement is to trigger review when confidence < 0.7, so this is the opposite.

D. documentResults.fields.MerchantName.confidence < 0.7

This only checks one field (MerchantName). In the JSON, MerchantName has confidence 0.913 ( > 0.7), so this condition would not trigger, but ReceiptType clearly needs review. Too narrow.

Correct Answer: B. documentResults.fields.*.confidence < 0.7

Microsoft References

Azure AI Document Intelligence – Confidence scores

Human-in-the-loop for Document Intelligence

C: Analyze Receipt - Get Analyze Receipt Result.

Query the status and retrieve the result of an Analyze Receipt operation.

Request URL: https://{endpoint}/formrecognizer/v2.0-preview/prebuilt/receipt/analyzeResults/{resultId}

E: POST {Endpoint}/vision/v3.2/read/analyze

Use this interface to get the result of a Read operation, employing the state-of-the-art Optical Character Recognition (OCR) algorithms optimized for text-heavy documents.

Scenario: Contoso plans to develop a document processing workflow to extract information automatically from PDFs and images of financial documents

The document processing solution must be able to process standardized financial documents that have the following characteristics:

- Contain fewer than 20 pages.

- Be formatted as PDF or JPEG files.

- Have a distinct standard for each office.

*The document processing solution must be able to extract tables and text from the financial documents.

The document processing solution must be able to extract information from receipt images.

To let an Azure AI Agent access publicly available, recent web content, you add the Bing grounding tool. In the SDK, you first define a ToolConnectionList with the Bing connection ID (here, " bingConnectionId " ), then construct a BingGroundingToolDefinition(connectionList) so the agent can call Bing Search with recency filters (e.g., last 90 days) when answering questions.

When creating the agent, tool definitions are passed via the tools: parameter of CreateAgentAsync(...) as a List < ToolDefinition > . The toolResources parameter is used when you must also supply resource descriptors (for example, an Azure AI Search index configuration), which is not needed here because the Bing grounding tool already references the connection through the ToolConnectionList .

Therefore:

Use BingGroundingToolDefinition to ground the agent on current public web data.

Provide that tool in tools: new List < ToolDefinition > { grounding } when calling CreateAgentAsync(...) .

Microsoft Azure AI References (titles only)

Azure AI Agent Service – Bing web grounding tool (tool definitions and connections)

Azure AI Agent Service SDK – Agent creation ( CreateAgentAsync parameters: tools vs toolResources )

Azure AI Agent Service – Tool connections and ToolConnectionList usage

Parameter type: stream

Visual features list: VisualFeatureTypes.Description

Result to use: results.Description.Captions[0]

To generate accessible alt text , you should use the caption produced by Azure Computer Vision’s Description feature (it produces a human-readable sentence with a confidence score). Therefore:

The image input should be a stream , because you’re uploading images (not just passing URLs) during product creation and AnalyzeImage…Async supports image streams .

Request the Description feature in VisualFeatureTypes so the service returns results.Description.Captions.

Use results.Description.Captions[0] and return the caption text if its confidence is high enough (e.g., > 0.5) to meet the accessibility requirement that all images must have relevant alt text .

Other features (Tags, Objects, Brands) are useful for enrichment but do not directly return natural-language captions suitable for alt text.

Microsoft Azure AI Solution References

Computer Vision (Image Analysis) – Description/Captions and features: Microsoft Docs, Image Analysis – VisualFeatureTypes.Description returns description.captions.

https://learn.microsoft.com/azure/ai-services/computer-vision/concept-image-analysis

SDK usage (analyze image from a stream): Microsoft Docs, Analyze an image by using the Computer Vision client library .

https://learn.microsoft.com/azure/ai-services/computer-vision/how-to/call-analyze-image?tabs=version-3-2#analyze-an-image-from-a-stream

Comprehensive Detailed Explanation

The requirement is to build an Adaptive Card for the chatbot that supports multilingual product display and meets accessibility requirements.

Product Name Display

The product JSON stores localized names like " name " : { " en " : " ... " , " es " : " ... " , " pt " : " ... " } .

To dynamically render based on the current language, you must use name[language].

This ensures the product name shown in the card adapts automatically to the user’s preferred language.

Stock Level Warning

Business rules require warnings when stock is low or out of stock.

This means you display the stock warning only when stockLevel is not ' OK ' .

Therefore, the correct condition is:

" $when " : " ${stockLevel != ' OK ' } "

Image Alt Text for Accessibility

Accessibility requirements specify that all images must have alt text in English, Spanish, and Portuguese.

In the product JSON, " alttext " is also localized:

" altText " : { " en " : " Bicycle " , " es " : " Bicicleta " , " pt " : " Bicicleta " }

To render the correct localized alt text dynamically, use image.altText[language].

Correct Selections:

First blank: name[language]

Second blank: " $when " : " ${stockLevel != ' OK ' } "

Third blank: image.altText[language]

Microsoft References

Adaptive Cards Templating

Adaptive Card $when property

Multilingual data in Azure Cognitive Search and Adaptive Cards

Scenario: Support autocompletion and autosuggestion based on all product name variants.

A: Call a suggester-enabled query, in the form of a Suggestion request or Autocomplete request, using an API. API usage is illustrated in the following call to the Autocomplete REST API.

POST /indexes/myxboxgames/docs/autocomplete?search & api-version=2020-06-30

{

" search " : " minecraf " ,

" suggesterName " : " sg "

}

B: In Azure Cognitive Search, typeahead or " search-as-you-type " is enabled through a suggester. A suggester provides a list of fields that undergo additional tokenization, generating prefix sequences to support matches on partial terms. For example, a suggester that includes a City field with a value for " Seattle " will have prefix combinations of " sea " , " seat " , " seatt " , and " seattl " to support typeahead.

F. Use the default standard Lucene analyzer ( " analyzer " : null) or a language analyzer (for example, " analyzer " : " en.Microsoft " ) on the field.