Part 5 Conceptual Complete

Part 5: The Intelligent Layer — Azure AI & Machine Learning

CloudVault wants to add intelligent features to their platform. Marcus explores Azure AI Services (Vision, Language, Speech), Azure OpenAI for LLMs, and Azure Machine Learning. We integrate AI capabilities into a Java backend application and understand when to use pre-built APIs vs. custom models.

March 6, 2025 5 min read

Part 5: The Intelligent Layer — Azure AI & Machine Learning

Adding Intelligence to CloudVault

The AI Opportunity

Week six. The product team approaches Marcus with an exciting idea: “What if we could analyze customer documents automatically? Extract information, detect fraud, summarize transactions?”

Marcus sees the opportunity. Azure offers multiple AI services, from pre-built APIs to custom machine learning models. The question is: which approach fits CloudVault’s needs?

“Let’s start with what we can do quickly,” Marcus suggests. “Then we’ll explore custom models if needed.”

Azure AI Services vs. Custom ML

Azure offers two paths to AI:

Approach	When to Use	Effort	Cost
Pre-built APIs	Common tasks (vision, language, speech)	Low	Low-Medium
Custom ML Models	Specialized tasks, competitive advantage	High	Medium-High

Most teams start with pre-built APIs, then move to custom models only when needed.

Vision API

The Vision API analyzes images: detects objects, reads text (OCR), analyzes faces, etc.

Use case: CloudVault wants to extract information from uploaded documents (invoices, receipts).

Setting Up Vision API

az cognitiveservices account create \
  --resource-group rg-cloudvault-prod \
  --name vision-cloudvault \
  --kind ComputerVision \
  --sku S1 \
  --location eastus

Using Vision API from Java

Add dependencies:

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-vision-imageanalysis</artifactId>
    <version>1.0.0</version>
</dependency>

Configure:

@Configuration
public class VisionConfig {

    @Bean
    public ImageAnalysisClient imageAnalysisClient() {
        return new ImageAnalysisClientBuilder()
            .endpoint("https://vision-cloudvault.cognitiveservices.azure.com/")
            .credential(new AzureKeyCredential(System.getenv("VISION_API_KEY")))
            .buildClient();
    }
}

Use in your service:

@Service
public class DocumentAnalysisService {

    @Autowired
    private ImageAnalysisClient imageAnalysisClient;

    public DocumentAnalysisResult analyzeDocument(String imageUrl) {
        ImageAnalysisOptions options = new ImageAnalysisOptions()
            .setLanguage("en")
            .setFeatures(Arrays.asList(
                ImageAnalysisFeature.CAPTION,
                ImageAnalysisFeature.READ,
                ImageAnalysisFeature.OBJECTS
            ));

        ImageAnalysisResult result = imageAnalysisClient.analyze(imageUrl, options);

        DocumentAnalysisResult analysisResult = new DocumentAnalysisResult();

        // Extract text (OCR)
        if (result.getRead() != null) {
            StringBuilder extractedText = new StringBuilder();
            for (ReadResult readResult : result.getRead().getBlocks()) {
                for (LineResult line : readResult.getLines()) {
                    extractedText.append(line.getText()).append("\n");
                }
            }
            analysisResult.setExtractedText(extractedText.toString());
        }

        return analysisResult;
    }
}

Language API

The Language API performs NLP tasks: sentiment analysis, entity recognition, key phrase extraction, etc.

Use case: CloudVault wants to analyze customer feedback and detect sentiment.

Using Language API from Java

Add dependencies:

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-textanalytics</artifactId>
    <version>5.3.0</version>
</dependency>

Use in your service:

@Service
public class FeedbackAnalysisService {

    @Autowired
    private TextAnalyticsClient textAnalyticsClient;

    public FeedbackAnalysis analyzeFeedback(String feedback) {
        // Sentiment analysis
        DocumentSentiment sentiment = textAnalyticsClient.analyzeSentiment(feedback, "en");

        // Entity recognition
        RecognizeEntitiesResult entities = textAnalyticsClient.recognizeEntities(feedback, "en");

        // Key phrases
        ExtractKeyPhrasesResult keyPhrases = textAnalyticsClient.extractKeyPhrases(feedback, "en");

        FeedbackAnalysis analysis = new FeedbackAnalysis();
        analysis.setSentiment(sentiment.getSentiment().toString());
        analysis.setConfidenceScores(sentiment.getConfidenceScores());
        analysis.setKeyPhrases(keyPhrases.getKeyPhrases());

        return analysis;
    }
}

Azure OpenAI Service

Azure OpenAI provides access to GPT-4, GPT-3.5-Turbo, and DALL-E models. It’s more powerful than pre-built APIs but requires careful prompt engineering.

Use case: CloudVault wants to generate personalized financial advice based on customer data.

Setting Up Azure OpenAI

az cognitiveservices account create \
  --resource-group rg-cloudvault-prod \
  --name openai-cloudvault \
  --kind OpenAI \
  --sku S0 \
  --location eastus

az cognitiveservices account deployment create \
  --resource-group rg-cloudvault-prod \
  --name openai-cloudvault \
  --deployment-id gpt-4 \
  --model-name gpt-4 \
  --model-version "0125-preview"

Using Azure OpenAI from Java

Add dependencies:

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-openai</artifactId>
    <version>1.0.0</version>
</dependency>

Use in your service:

@Service
public class FinancialAdviceService {

    @Autowired
    private OpenAIClient openAIClient;

    public String generateAdvice(String customerProfile, BigDecimal balance) {
        String prompt = String.format(
            "You are a financial advisor. Based on the following customer profile, " +
            "provide personalized financial advice:\n\n" +
            "Customer: %s\n" +
            "Balance: $%s\n\n" +
            "Provide 3 specific, actionable recommendations.",
            customerProfile,
            balance
        );

        List<ChatCompletionRequestMessage> messages = new ArrayList<>();
        messages.add(new ChatCompletionRequestSystemMessage(
            "You are a helpful financial advisor for a fintech platform."
        ));
        messages.add(new ChatCompletionRequestUserMessage(prompt));

        ChatCompletions completions = openAIClient.getChatCompletions(
            "gpt-4",
            new ChatCompletionOptions(messages)
                .setTemperature(0.7)
                .setMaxTokens(500)
        );

        return completions.getChoices().get(0).getMessage().getContent();
    }
}

Azure Machine Learning

For specialized tasks where pre-built models don’t work, use Azure Machine Learning to train custom models.

Example: CloudVault wants to detect fraudulent transactions with 99% accuracy.

Training a Custom Model

from azureml.core import Workspace, Experiment, ScriptRunConfig
import joblib

ws = Workspace.from_config()
experiment = Experiment(ws, "fraud-detection")

config = ScriptRunConfig(
    source_directory=".",
    script="train_model.py",
    compute_target="ml-cluster"
)

run = experiment.submit(config)
run.wait_for_completion(show_output=True)

run.register_model(
    model_name="fraud-detector",
    model_path="outputs/model.pkl"
)

Marcus’s AI Strategy

After exploring the options, Marcus decides:

Use Vision API for document analysis (quick win, pre-built)
Use Language API for sentiment analysis (straightforward, pre-built)
Use Azure OpenAI for financial advice (powerful, minimal training needed)
Defer custom ML until they have specific use cases that pre-built models can’t handle

“This approach gets us AI capabilities quickly without the overhead of training models,” Marcus explains. “We can always build custom models later if needed.”

Cost Considerations

Vision API: ~$1-10 per 1,000 API calls
Language API: ~$1-10 per 1,000 API calls
Azure OpenAI: ~$0.03 per 1,000 tokens (varies by model)
Azure ML: ~$0.50-2.00 per compute hour

Start with pre-built APIs. They’re cheaper and faster to implement.

Key Takeaways

Pre-built APIs (Vision, Language, Speech) are fast and cost-effective for common tasks
Azure OpenAI provides LLM capabilities without training
Azure ML is for specialized tasks requiring custom models
Start simple, add complexity only when needed
Understand token costs — they add up with LLMs

What’s Next?

Marcus has added intelligence to CloudVault. Now he needs to automate everything: deployment, monitoring, and operations. In the final chapter, we’ll explore DevOps on Azure and establish production-ready practices.

The intelligence is in place. Now let’s automate the climb.

This is Part 5 of a 6-part series: “The Azure Ascent: A Backend Engineer’s Journey to Cloud Mastery.”

Series Progress 7/7 parts

0 The Azure Ascent: A Backend Engineer's Journey to Cloud Mastery Complete 1 Part 1: The Summit Awaits — Azure Fundamentals & Core Concepts Complete 2 Part 2: Building the Engine — Azure Compute Services Complete 3 Part 3: Storing the Treasures — Azure Data & Storage Services Complete 4 Part 4: Connecting the Dots — Azure Networking & Security Complete 5 Part 5: The Intelligent Layer — Azure AI & Machine Learning Complete 6 Part 6: Automating the Climb — Azure DevOps & Deployment Complete