Alasdair's Blog - work

Returning structured data from an LLM via LangChain

2025-09-22T17:00:00+01:00

In my last article, we walked through how to turn our simple langchain -> ollama model into a simple chat bot over an API using with in memory history and seperate users, so each user gets their chat history with the bot. Another very useful case for llm's is categorisation and forming a structured json response from user input, say you have a triage system, where you want to extract key data and return it in a structured format for another service to consume. Let's stick with the same app structure as before, we'll add in a new service class since we want this llm instance to be configured seperated without history. We'll call our new file 'llm_service_structured_data' - very clear what it's going to do!

 ┣ app
 ┃  ┣ services
 ┃  ┃ ┣ llm_service_structured_data.py
 ┃  ┃ ┣ llm_service.py
 ┃  ┣ __init__.py
 ┃  ┣ llmroutes.py
 ┃  ┣ inMemoryHistory.py
 ┃  ┗ models.py
 ┣ config.py
 ┗ run.py

There are a few differences from before, we want to use a 'RunnablePassthrough' class instead of RunnableWithMessageHistory and we want to define our prompt template to strictly return json data in a parameterised way. We'll start with our imports

from langchain_core.output_parsers import JsonOutputParser
from langchain_ollama import OllamaLLM
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough

Then lets sort our init for our new class, in the init we'll define our prompt template, tell it the format we expect back for our json and create a chain to call via a method.

class LLMServiceJson:
    def __init__(self):
        self.model = OllamaLLM(model="gpt-oss:20B")
        self.prompt = ChatPromptTemplate.from_template("""
            You are a service that extracts customer info.
            Always respond with ONLY valid JSON in this format:

            {{
              "customerName": string,
              "issueType": string,
              "priority": "low" | "medium" | "high"
            }}

            User message: {message}
            """)

        self.parser = JsonOutputParser()

        self.chain  = (
            {"message": RunnablePassthrough()}  # passes user input
            | self.prompt                            # formats prompt
            | self.model                             # sends to LLM
            | self.parser                            # enforces JSON parsing
        )

And thats our class setup, now we just need a simple method, lets call it classify which will process user messages and return the json structure from the prompt. Very simple, takes in self, a message from the calling object and tells the caller to expect a string back.

    def classify(self, message: str) -> str:
        """Process a chat message with persistent memory."""
        response = self.chain.invoke(message)
        return response

Let's add a new route in our llmroutes.py file, it'll just do some validation on the message and pass it through.

@llm_bp.route("/enquiries", methods=["POST"])
def enquiries():
    data = request.get_json()
    message = data.get("message", "")

    if not message:
        return jsonify({"error": "Message is required"}), 400

    response = llm_structured_data.classify( message)

    return jsonify({"response": response})

And now that's us ready to run and test our api! If I send in a message about my car being broken down roadside, I get a response like this...

curl -X POST http://localhost:8083/llm/enquiries \
     -H "Content-Type: application/json" \
     -d '{"message": "Hi, my name is Alasdair, my car has broken down and I need help at the roadside "}'
{
  "response": {
    "customerName": "Alasdair",
    "issueType": "Roadside assistance for broken down car",
    "priority": "high"
  }
}

Which is great, thats a correctly formatted Json response that we can integrate into another service nicely. We can test it with other types of problem too,

curl -X POST http://localhost:8083/llm/enquiries \
     -H "Content-Type: application/json" \
     -d '{"message": "Hi, my name is Alasdair, my tap is leaking slowly, can I book a plumber for next week? "}'
{
  "response": {
    "customerName": "Alasdair",
    "issueType": "tap leak",
    "priority": "medium"
  }
}

Great, it works across a few different types of enquiries that need to be classified!

A simple chatbot using Ollama running on a Flask API

2025-09-21T21:00:00+01:00

Following on from my last article, where we defined the simplest possible chatbot accessed via the terminal and using langchain to talk to the model, the natural next step is to think about how to build an API around the langchain code and expose it for a nice and user friendly front end or another integrating system to connect to the llm. Given we're running in python along with it's great library and ecosystem support, it makes a lot of sense to do the API in a python library, normally I lean slightly more towards Django Rest Framework (drf) but it's a 'batteries included' type of framework so lets go with a simple flask api which lets us expose the llm service quickly.

Lets think about the architecture of what our API app will be, something like the below is probably a good layout

 ┣ app
 ┃  ┣ services
 ┃  ┃ ┣ llm_service.py
 ┃  ┣ __init__.py
 ┃  ┣ llmroutes.py
 ┃  ┣ inMemoryHistory.py
 ┃  ┗ models.py
 ┣ config.py
 ┗ run.py

Lets start with our init.py for our app, we'll create it inside the app folder and define the basics. The url_prefix here, is sort of doing a similar job to defining your urls.py in a django setup, the blueprint stores the mapping of the actual endpoint, this would be the equivalent of the global urls.py that gets each django's app's wiring imported.

from flask import Flask
from .llmroutes import llm_bp
from config import Config

def create_app():
    app = Flask(__name__)

    # Initialize app
    db.init_app(app)

    # Register blueprints
    app.register_blueprint(llm_bp, url_prefix="/llm")

    return app

So we'll call this from the run.py file that sits just outside our app folder,

from app import create_app

app = create_app()

if __name__ == "__main__":
    app.run(host='0.0.0.0', port=8082, debug=True)

Really simple but a nice little touch here is setting the host to 0.0.0.0, it allows other machines on our local network to connect to the machine thats running the app, really good for testing. Next step is to setup the routes, routes are analogous to controllers in an ASP.Net core project or an APIView in Django rest framework. We declare our blueprint and set it up with the url routes and whatever verb action we want to allow against each function. In this case, given it's an API, we want to at least have some semblance of pretending to have a user system in place, so what we'll do is take the user_id string in and create a seperate conversation chain for each user. It won't have a full login system but it's more than good enough for a proof of concept.

from flask import Blueprint, request, jsonify

from app.services.llm_service import LLMService

llm_bp = Blueprint("llm", __name__)
llm_service = LLMService()

@llm_bp.route("/chat", methods=["POST"])
def chat():
    data = request.get_json()
    user_id = data.get("user_id")
    message = data.get("message", "")

    # Do some basic validation on our input - do we have it?
    if not user_id:
        return jsonify({"error": "User ID is required"}), 400
    if not message:
        return jsonify({"error": "Message is required"}), 400

    response = llm_service.chat(user_id, message)

    return jsonify({"response": response, "user_id": user_id})

The next step is to look at our LLMService class that you can see I've used, this is really just the modernisation of the ConversationChain class from the last article into it's own wrapper service and using the newer RunnableWithMessageHistory class instead as ConversationChain has been deprecated. We'll start by doing our imports and our init method (similar idea as a constructor for those of you more familiar with c#)

from langchain_ollama import OllamaLLM

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from .InMemoryHistory import InMemoryHistory
from langchain_core.chat_history import BaseChatMessageHistory
from app.services.memory_service import DatabaseConversationMemory

class LLMService:
    def __init__(self):
        self.llm = OllamaLLM(model="gpt-oss:20B")
        self.prompt = ChatPromptTemplate.from_messages([
        ("system", "You are a helpful assistant, specialising as a programming assistant."),
        MessagesPlaceholder(variable_name="history"),
        ("human", "{input}"),
        ])

        self.chain = self.prompt | OllamaLLM(model="gpt-oss:20B")
        # Remove in-memory storage - we'll create chains on-demand with persistent memory
        self.store = {}

        # Create a RunnableWithMessageHistory instance
        self.runnable_with_history = RunnableWithMessageHistory(
            self.chain,
            self.get_by_session_id,
            input_messages_key="input",
            history_messages_key="history",
        )

As you can see above, we've defined our template pulled in all the imports we need and defined a chain to pass into the RunnableWithMessageHistory class. There is a step which I'll go over after this function, we need to define a small in memory class to hold the history, however for now, you can see we're using it to get a session_id and mapping that to user_id coming from the api's route. The actual usage is very similar to the ConversationChain class, call an invoke method on the class and pass back to the response.

    def get_by_session_id(self, session_id: str) -> BaseChatMessageHistory:
        if session_id not in self.store:
            self.store[session_id] = InMemoryHistory()
        return self.store[session_id]

    def chat(self, user_id: str, message: str) -> str:
        """Process a chat message with persistent memory."""
        response = self.runnable_with_history.invoke(
            {"input": message},
            config={"configurable": {"session_id": user_id}}
        )
        return response

This was an example class to use to store history in memory, for a production level system, this would be built out properly to store in some persistent data storage.

from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.messages import BaseMessage
from pydantic import BaseModel, Field

class InMemoryHistory(BaseChatMessageHistory, BaseModel):
    """In memory implementation of chat message history."""

    messages: list[BaseMessage] = Field(default_factory=list)

    def add_messages(self, messages: list[BaseMessage]) -> None:
        """Add a list of messages to the store"""
        self.messages.extend(messages)

    def clear(self) -> None:
        self.messages = []

At this point, there's nothing to do except activate your virtual environment, set your working directory in your terminal to wherever you've stored the run.py and execute like...

python run.py

That should be your API up and running and ready to act as an API chatbot!

You can call it using curl or PostMan like this

curl -X POST http://localhost:8083/llm/chat \
     -H "Content-Type: application/json" \
     -d '{"user_id": "alasdair", "message": "Hi, c# or python - choose! "}'

The response I received was...

  "response": "Sure thing! \ud83c\udfaf  \nWhich one is \u201cbest\u201d really depends on **what you want to build** and a few other factors. Here\u2019s a quick rundown so you can decide:\n\n| Factor | C# | Python |\n|--------|----|--------|\n| **Primary domain** | .NET ecosystem (Windows desktop, enterprise apps, Unity 3D, Xamarin/MAUI mobile, Azure services, high\u2011performance back\u2011ends) | Web (Flask, Django, FastAPI), data science/ML (NumPy, pandas, scikit\u2011learn, TensorFlow), scripting, automation, and a huge amount of general\u2011purpose libraries |\n| **Performance** | Generally faster (compiled, JIT, strong typing) | Slower (interpreted), but often fast enough for I/O\u2011bound or prototyping workloads |\n| **Learning curve** | Moderate \u2013 you\u2019ll need to understand the CLR, LINQ, async/await, etc. | Low \u2013 dynamic typing, readable syntax, huge amount of learning resources |\n| **Tooling** | Excellent IDE (Visual Studio, Rider), 
  great debugging, built\u2011in support for dependency injection, strong type checking | Good tooling (PyCharm, VS\u202fCode, Jupyter notebooks), but debugging can feel less powerful for large projects |\n| **Cross\u2011platform** | .NET 6/7+ is truly cross\u2011platform (Windows, macOS, Linux). | Native cross\u2011platform, 
  but binary distribution can be trickier |\n| **Community & Ecosystem** | Large corporate support, extensive library catalog, NuGet, strong game\u2011dev community via Unity | Massive open\u2011source community, Python Package Index (PyPI), huge scientific & ML ecosystem |\n| **Future prospects** | Growing with .NET Core/5+ and Azure cloud services | Dominant in AI/ML, data science, and remains popular for scripting and web apps |\n| **Typical Use Cases** | Windows desktop apps, enterprise web services, game development, mobile (Xamarin/MAUI) | Web APIs, data pipelines, ML/AI, automation, rapid prototyping |\n\n### Quick decision checklist\n\n| Question | Likely C# | Likely Python |\n|----------|-----------|---------------|\n| You\u2019re building a Windows\u2011only desktop app or a game in Unity? | \u2705 | \u274c |\n| You\u2019re targeting Azure, need strong typing, or want to use a statically typed language? | \u2705 | \u274c |\n| 
  You need to spin up a data\u2011science pipeline or prototype an ML model quickly? | \u274c | \u2705 |\n| You\u2019re doing web scraping, automation scripts, or a small web API? | \u274c | \u2705 |\n| 
  You prefer a language that compiles to a single binary and offers very fast execution? | \u2705 | \u274c |\n| You\u2019re comfortable with dynamic typing and want a gentle learning curve? | \u274c | \u2705 |\n\n---\n\n## Bottom line\n\n- **Go with C#** if you\u2019re targeting the .NET ecosystem (Windows, Azure, Unity, MAUI) or need strong typing & performance.\n- **Go with Python** if you\u2019re in data science, want quick prototyping, or want a language with a huge collection of scientific/ML libraries.\n\nIf you can share a bit more about what you\u2019re planning (web app, game, automation, data pipeline, etc.), I can fine\u2011tune the recommendation!",
  "user_id": "alasdair"
}

Pretty thorough answer!

A simple terminal chatbot using Ollama

2025-09-21T14:00:00+01:00

I'm a fan of using local resources and being in control of your resources as much as possible, so naturally when looking at AI (and llm's in particular) I've gravitated towards use of things and services like Ollama and ExoLabs, so thought it would be a good idea to write up a simple way to wire up a simple program which use's the gpt-oss:20B model as a simple terminal chatbot. While you can start ollama directly and chat with it, this is the start of setting it up as a service which can be consumed by other applications and we have the ability to give the instance a prompt template to customise how we want our llm service to behave.

We'll start by creating our virtual env, I use virtualenv wrapper on my mac so it's a simple

mkvirtualenv llmexp

Use this to activate your virtualenv

workon llmexp

Then we'll want to use pip to get the two langchain packages we want,

 pip install langchain langchain-ollama

From here, our next step is creating a python file, since it's a simple service, we'll just go with main.py - we can wrap it in a service behind a flask api in a later article. We'll start with adding our imports...

from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory
from langchain_ollama import OllamaLLM
from langchain.prompts import PromptTemplate

This gives us a memory buffer for our chatbot, a prompt template and the OllamaLLM libaries themselves to hook up.

We'll then get our llm instance, do our template and prompt config (for a more complex example, should stick this in some config files outside the main code.)

llm = OllamaLLM(model="gpt-oss:20B")
template = """You are a helpful assistant, specialising as a programming assistant."""
prompt = PromptTemplate.from_template(template)

We'll then setup our ConversationChain object, which is what we're using really - it's a langchain service designed to make chatbots quite simple to use, it's still fundamentally the same idea as using any llm, a prompt goes to the llm and gives you a response, but it's setup to accept a memory buffer as well.

memory = ConversationBufferMemory()
conversation = ConversationChain(llm=llm, memory=memory)
conversation.prompt = prompt

You can see from the above, that we pass our llm (we're using gpt-oss:20B) and our memory object into the ConversationChain then setting a prompt for it to use.

From this point, we're setup for our simplest possible chatbot with memory in langchain, so lets get a simple loop to keep it running and accept user input to pass to it.

print("Chat starting!")

while True:
    user_input = input("You: ")
    if user_input.lower() == "quit":
        break
    response = conversation.invoke(user_input)
    print("Chatbot:", response["response"])

Using this, we'll have a simple loop on the terminal where it's clear that you're talking to an llm. You can ask it any question you'd like however the bot has a template prompt which will sway it's behaviour towards being a helpful assistant with programming tasks!

For my next article, I'll write up about putting our llm behind a service which can be used by either web front end or integrated and plugged into another service.

Azure Language Services with C#

2025-09-16T20:00:00+01:00

In another article I wrote about

deploying Azure resources through bicep
creating your project in Azure Language Services via the Text Authoring API
Uploading your training data
Using the Text Authoring API to train your model
Then finally deploy it, ready to be consumed by your applications!

So we're ready to integrate with the language service, it has some quirks though, the only method available to get label classification result takes a batch of messages and returns a ClassifyDocumentOperation, which contains a ClassifyDocumentResultCollection.

        var batchInput = new List<string> { message };

        ClassifyDocumentOperation operation = await _textAnalyticsClient.SingleLabelClassifyAsync(
            WaitUntil.Completed,
            batchInput,
            _languageSettings.projectName,
            _languageSettings.deploymentName);

This is fine but for a lot of use cases you may have an api calling this and only really want a single message classified at a time, and you may not want your larger function to deal with extracting the only result from the collection result. So we'll move it into a simple Classify function which calls the deployed classification model and deals with extracting the single result.

Call it through code

    private async Task<ClassifyDocumentResult> ClassifyAsync(string message)
    {
        var batchInput = new List<string> { message };

        ClassifyDocumentOperation operation = await _textAnalyticsClient.SingleLabelClassifyAsync(
            WaitUntil.Completed,
            batchInput,
            _languageSettings.projectName,
            _languageSettings.deploymentName);

        await foreach (var documentsInPage in operation.Value)
        {
            var result = documentsInPage.FirstOrDefault();
            if (result != null && !result.HasError)
            {
                return result;
            }
            else if (result?.HasError == true)
            {
                _logger.LogError("Error back from Azure Language Service enquiry");
                _logger.LogError(result.Error.Message);
            }
        }

        return null;
    }

You may notice in here, I have

_textAnalyticsClient 
& 
_languageSettings

LanguageSettings is a small settings class which allows us to get our config at startup and add it to the DI and TextAnalyticsClient is an Azure package which allows us to make calls to the Azure Language Service.

    public class LanguageSettings
    {
        public string url { get; set; }

        public string projectName { get; set; }

        public string deploymentName { get; set; }
    }

Then in our program.cs (or startup.cs if you're still using that.)

var languageSettings = services.AddSettings<LanguageSettings>(builder.Configuration);

var client = new TextAnalyticsClient(new Uri(languageSettings.url), new DefaultAzureCredential());
services.AddSingleton(client);
services.AddSingleton(languageSettings);

In our consuming class, we can pass them through into the constructor like so below.

using Azure.AI.TextAnalytics;


public class MyClassificationService : IMyClassificationService
{

    private readonly TextAnalyticsClient _textAnalyticsClient;
    private readonly LanguageSettings _languageSettings;

    public MyClassificationService(TextAnalyticsClient textAnalyticsClient, LanguageSettings langSettings)
    {
        _textAnalyticsClient = textAnalyticsClient;
        _languageSettings = langSettings;
    }

}

The next step will be to have a method within MyClassificationService which is public can be used by other classes.

    var result = await ClassifyAsync(dto.Message);
    if (result != null)
    {
        var classification = result.ClassificationCategories.FirstOrDefault();
        if (!string.IsNullOrEmpty(classification.Category))
        {
            myReturnDto.ClassificationConfidenceScore = classification.ConfidenceScore;
            myReturnDto.Classification = classification.Category;
        }
    }

We can also do a language detection on our mesage using the same textAnalyticsClient, this can be very useful if you have users with multiple languages accessing your services, however it should be noted you need to have training data on multiple languages if you expect your classifier above to be accurate.

    try
    {
        Response<DetectedLanguage> response = _textAnalyticsClient.DetectLanguage(dto.Message);

        DetectedLanguage language = response.Value;
        Console.WriteLine($"Detected language {language.Name} with confidence score {language.ConfidenceScore}.");

        myReturnDto.ClassifiedLanguage = language.Name;
        myReturnDto.LanguageClassificationConfidenceScore = language.ConfidenceScore;
    }
    catch (RequestFailedException exception)
    {
        Console.WriteLine($"Error Code: {exception.ErrorCode}");
        Console.WriteLine($"Message: {exception.Message}");
    }

And at that point, thats us able to call Azure's Language Service against the model we've trained and get a classification back for our message request!

Azure Language Services

2025-09-15T20:00:00+01:00

Create resources in bicep

Before we make any calls to the Azure language service, we need to define our resources, you can of course do this through the UI but ideally these things will be done idempotently through pipelines so you have a repeatable result and can be content with how that will run regardless of which environment in your release phase you're releasing to. I'm a fan of bicep however I'm sure the same result could be achieved with HashiCorp's Terraform.

In bicep we'll start by defining our storage account which will contain the training data that our language service runs on, a fairly ordinary blob storage account with a container for training data, the only thing of note is we need to define some cors rules explicitly to allow access.

resource languageServiceStorage 'Microsoft.Storage/storageAccounts@2023-01-01' = {
  name: '${environmentName}langstorage'
  location: environmentSettings.location
  tags: tags
  identity: {
    type: 'SystemAssigned'
  }
  sku: {
    name: 'Standard_LRS'
  }
  kind: 'StorageV2'
  properties: {
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    accessTier: 'Hot'
  }

    resource blobStorage 'blobServices@2023-05-01' = {
    name: 'default'
    properties: {cors: {
      corsRules: [
        {
          allowedOrigins: ['https://language.cognitive.azure.com']
          allowedMethods: [
            'GET'
            'PUT'
            'POST'
            'HEAD'
            'OPTIONS'
          ]
          maxAgeInSeconds: 3600
          exposedHeaders: ['*']
          allowedHeaders: ['*']
        }
      ]
    }}

    resource trainingdata 'containers@2023-05-01' = {
      name: 'trainingdata'
      properties: {
        publicAccess: 'None'
      }
    }
  }
}

Once we've defined our storage account - lets define our language service, using a custom domain is recommended, from memory there is some issues in obtaining access tokens if interacting with a user managed identity if you use the regional endpoints. The 'userOwnedStorage' entry is what is used to link your language service to your storage account.

resource langStudio 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
  name: '${environmentName}languageservice'
  location: environmentSettings.location
  tags: tags
  identity: {
    type: 'SystemAssigned'
  }
  kind: 'TextAnalytics'
  sku: {
    name: 'S'
  }
  properties: {
    publicNetworkAccess: 'Enabled'
    customSubDomainName: '${environmentName}languageservice'
    userOwnedStorage: [
      {
        resourceId: languageServiceStorage.id
      }
    ]
  }
}

Our last step for now, is to give our language service the correct role based access to the storage account.

resource langToStorageRbac 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(languageServiceStorage.id, 'StorageBlobDataContributor', langStudio.id)
  scope: languageServiceStorage
  properties: {
     roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions',
      'ba92f5b4-2d11-453d-a403-e96b0029c9fe' // Storage Blob Data Contributor
    )
    principalId: langStudio.identity.principalId
    principalType: 'ServicePrincipal'
  }
}

That should be us all setup to move on to create our project for actually doing whatever classification we want to do. It's worth noting as this point, I throughly recommend if you're in the Azure infrastructure ecosystem to use managed identities your app resources, if you are doing this, it's worth remembering that you'll need to add rbac for your app service to connect to the language studio. And you'll also need need to have the cognitiveServicesLanguageOwner assigned to whatever identity you're using to call the text authoring api to create your project/import and train/deploy your model.

Powershell script to create project in Azure Language Services

To create a project, we call the azure text authoring api with a patch verb and provide a json body with some metadata around what the project is. If we set it to multilingual, we can use the language detection api later, to check what language azure thinks our input is, we thankfully don't have to train to detect the language however, if you do expect to be running a result on this, it's a good idea to have training data for multiple languages.

    $projectName = 'myproject'

    $projectUri = "$endpoint/language/authoring/analyze-text/projects/${projectName}?api-version=2022-05-01"

    $projectBody = @{
        projectName = $projectName
        description = "Relevant or not"
        language = 'en'
        multilingual = "True"
        projectKind = "CustomSingleLabelClassification"
        storageInputContainerName = 'trainingdata'
    } | ConvertTo-Json -Depth 5

    $response = Invoke-RestMethod -Method Patch -Uri $projectUri -Headers $headers -Body $projectBody

Copy your training data to storage account linked to Azure's language service

I like to use azcopy for this, you can copy to your storage account by obtaining a sas token automate it that way.

If you've pre labeled it - call upload endpoint with json body detailing file name + label

This is a bit of a tricky part, you'll want to have a json body that looks something like the below, it'll be the metadata for your project plus an array object giving the label that each file in your training data has.

{
  "projectFileVersion": "2022-05-01",
  "stringIndexType": "Utf16CodeUnit",
  "metadata": {
    "projectName": "myproject",
    "storageInputContainerName": "trainingdata",
    "projectKind": "CustomSingleLabelClassification",
    "description": "Relevant or not",
    "language": "en",
    "multilingual": true,
    "settings": {}
  },
  "assets": {
    "projectKind": "CustomSingleLabelClassification",
    "classes": [
      {
        "category": "relevant"
      },
      {
        "category": "not"
      }
    ],

The next entry will be a long array pairing your documents in your training folder with the labels you want for each file. I'd suggest creating this programatically, it'll be more than a little tedious otherwise.

    "documents": [
      {
        "location": "001.txt",
        "language": "en",
        "class": {
          "category": "relevant"
        }
      },
      {
        "location": "002.txt",
        "language": "en",
        "class": {
          "category": "not"
        }
      },
      .......
    ]
  }

}

Train your model

Fairly simple here, just want to make a rest call to the text authoring api, you'll want to hit the endpoint that does the training. Lets start by getting a bearer token to put in our call, I'm assuming you're using a managed identity to connect, otherwise you can obtain a key from the resource itself to connect but as always, managed identities avoid you having to copy keys/secrets around.

    $token = (ConvertFrom-SecureString (Get-AzAccessToken -ResourceUrl https://cognitiveservices.azure.com -AsSecureString).Token -AsPlainText)

Build up a headers object to send with your call...

    $headers = @{
        "Authorization" = "Bearer $token"
        "Content-Type" = "application/json"
    }

Now lets build up the body to send, you can see here that we can define a split for how the training job will use our test data to train vs test it's outcome. For my jobs, I've had roughly 1000 samples in my training data so an 80/20 split is pretty good.

    $modelLabel = "mybetamodel"
    $deploymentName = "deploymentprime"

    # kick off training job
    Write-Host "`nStarting training..."
    $trainUri = "$endpoint/language/authoring/analyze-text/projects/$projectName/:train?api-version=2022-05-01"
    $trainBody = @{ 
        modelLabel = $modelLabel
        trainingConfigVersion = "latest"
        evaluationOptions = @{
            kind = "percentage"
            testingSplitPercentage = 20
            trainingSplitPercentage = 80
        } } | ConvertTo-Json

And finally, make the call itself to kick off the training job.

    Invoke-RestMethod -Method Post -Uri $trainUri -Headers $headers -Body $trainBody

The training job can take quite some time to actually run, you can check for the job completing by polling the training endpoint.

Start by getting the latest training job id

    $joburi = "$endpoint/language/authoring/analyze-text/projects/$projectName/train/jobs?api-version=2022-05-01"
    $jobresponse = Invoke-RestMethod -Method Get -Uri $joburi -Headers $headers

    $latestJob = $jobResponse.value | Sort-Object -Property createdDateTime -Descending | Select-Object -First 1
    $latestJobId = $latestJob.jobId

    Write-Host "Latest training job ID: $latestJobId"
    Write-Host "Initial status: $($latestJob.status)"

Then we can plug it into the job status endpoint like so and poll the endpoint until the job returns a status of succeeded. The job will take a while so I'd recommend a fairly high polling interval of at least 3 minutes.

    $jobStatusUri = "$endpoint/language/authoring/analyze-text/projects/$projectName/train/jobs/$($latestJobId)?api-version=2022-05-01"

    while ($true) {
        $statusResponse = Invoke-RestMethod -Method Get -Uri $jobStatusUri -Headers $headers
        $status = $statusResponse.status

        Write-Host "Training job status: $status - Checked at $(Get-Date -Format "HH:mm:ss")"

        if ($status -eq "succeeded") {
            Write-Host "Training job succeeded."
            break
        }
        elseif ($status -eq "running") {
            Start-Sleep -Seconds 180
        }
        else {
            throw "Training job failed or entered unexpected status: $status"
        }
    }

Now that our training job has succeeded, we can call the deployment endpoint to get our model deployed and ready for our applications to use.

Deploy model

This one is actually fairly simple, which makes a nice change! Just a simple json body with the name of your model label to be send to your project which is identified in the route of the url we hit.

   Write-Host "`nDeploying model..."
   $deployUri = "$endpoint/language/authoring/analyze-text/projects/$projectName/deployments/$($deploymentName)?api-version=2022-05-01"
   $deployBody = @{
       trainedModelLabel = $modelLabel
    } | ConvertTo-Json


    $response = Invoke-RestMethod -Method Put -Uri $deployUri -Headers $headers -Body $deployBody
    Write-Host "Deployment request succeeded."

After this, you're ready to use your trained language classifier!

Deploying a virtual network in Azure using bicep

2025-09-12T22:00:00+01:00

Deploying Virtual Networks via Bicep

To secure an app, it's often necessary (or even just preferable) to have your application and all it's associated services talk to each other in private, rather than public, especially if it's opening up unnecessray security risks for api endpoints or services which just simply shouldn't be accessible. Within Azure, the way to do this is to create a virtual network where you can then define whether a service is publically available or is only available on the virtual network. There's a few steps involved to do this,

Define some variables for our Virtual network
Create the Virtual network
Define the Subnets
Update your services to be on the virtual network
Think about private endpoints

Lets start with some variables for our virtual network, for the sake of this example, we'll hard code some subnets. We'll assume we've got an api to connect. For real use you would probably want to have these passed in at runtime from a hosting profile. I make a point here of tags, as tagging your infrastruture in azure can become essential fairly quickly for managing your estate.

param tags object = {
  owner: 'alasdair'
  environment: 'dev'
  updatedDate: utcNow('yyyy-MM-dd HH:mm')
}

param environmentSettings object = {
  location: 'UKSouth'
  vnetName: 'myvnet'
  vnetAddressSpace: '10.1.0.0/16'
  appServicesSubnetAddressSpace: '10.1.1.0/24'
}

Now that we've got our variables, lets create our virtual network,

resource vnet 'Microsoft.Network/virtualNetworks@2024-05-01' = {
  name: environmentSettings.vnetName
  location: environmentSettings.location
  tags: tags

  properties: {
    addressSpace: {
      addressPrefixes: [ 
        environmentSettings.vnetAddressSpace
      ]
    }

  }
}

That takes care of creating our most basic virtual network, but you'll notice that we still need to setup some subnets within our virtual network for use with various types of app services. For the subnets, within properties object, lets add a subnets array object with for appServices, you can add these as you need to connect different types of resource, I'm working under the assumption that appServices is an api, maybe you'd have a subnet for private links, one for web sites, maybe one for cosmos instances.

    subnets: [
      {
        name: 'AppServices'
        properties: {
          addressPrefix: environmentSettings.appServicesSubnetAddressSpace
          delegations: [
            {
              id: resourceId('Microsoft.Network/virtualNetworks/subnets', environmentSettings.vnetName, 'AppServices')
              name: 'delegation'
              properties: {
              serviceName: 'Microsoft.Web/serverfarms'
              }
              type: 'Microsoft.Network/virtualNetworks/subnets/delegations'
            }
          ]
        }
      }
    ]

That takes care of defining your virtual network but to actually use it, we'd want to assign it to our app service like so...

resource apiAppService 'Microsoft.Web/sites@2024-04-01' = {
  name: 'myawesomeapi'
  location: environmentSettings.location
  tags: tags
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    ...
    virtualNetworkSubnetId: vnet::appServicesSubnet.id
    ....
  }
}

And that should take care of setting up a basic a virtual network and attaching your api to it! In my next post, I'll look at private endpoints, so we can expose endpoints only through the virtual network.

Managed Identity with SQL Server & Entity Framework

2024-08-04T23:00:00+01:00

It's a good idea if possible to move away from credentials in strings where possible, luckily using system assigned managed identity with Azure makes this pretty easy to do with just a small list of things to sort. I'm going to assume you have an existing dotnet application which is connecting to a SQL Server instance hosted within Azure and you've already setup a directory Admin for that server. You shouldn't need any RBAC(role based access control) setup added. We need to work through the following list to get us setup with a system assigned managed identity connection to our sql server instance.

Marking our resource to use system assigned identity in our bicep
Adapting the connection string to use 'Azure Managed Identity'
Creating a DB user for our app
Small bit of setup within our program.cs

I like Bicep for defining resources within Azure, it's repeatable, trusted and frankly far more readable than ARM templates (which is what Bicep gets compiled into before running anyway) I find it's more similar to a scripting language than reams of JSON, it even allows for creating reusable modules so I'm going to give my example for deploying an API resource as a bicep snippet.

resource api 'Microsoft.Web/sites@2022-03-01' = {
  name: apiName
  location: appHostingProfile.location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    serverFarmId: serverFarmId
    httpsOnly: true
    siteConfig: {
      alwaysOn: true
      ftpsState: 'FtpsOnly'
      minTlsVersion: '1.2'
      connectionStrings: [
        {
          name: 'MyConnection'
          connectionString: 'Server=tcp:myserver.database.windows.net,1433;Initial Catalog=mydatabase;Encrypt=True;Authentication="Active Directory Managed Identity";'
          type: 'SQLAzure'
        }
      ]
    }
  }
}

The section

  identity: {
    type: 'SystemAssigned'
  }

Is essential here as it tells Azure that we want a system assigned identity, it lives with the azure resource and gets deleted with it. If you search for microsoft entra identities within your azure subscription, you'll find one with the same name as your api or web resource. You can alternatively create a user defined managed identity, which exists seperately from the resource and can be managed entirely seperately.

Within the connection string, you'll also notice this

Authentication="Active Directory Managed Identity"

It explictly tells the authentication classes which method to attempt to connect to the DB with. I prefer to have it defined like this on your azure resource, it's more explicit so you can be sure that you're connecting with your managed identity. You can alternatively use this line

Authentication="Active Directory Default"

It'll attempt to use credentials in the order defined at this Azure docs page. I'd recommend using this locally, in some appsettings.development.json which is designed to override your config running on your server when running your application locally.

Within your database, you'll need a user for your system managed identity, please make sure you create your user within the database you intend to use and not the system wide master db. Something like this should sort you out...

CREATE USER [$apiName] FROM EXTERNAL PROVIDER;
ALTER ROLE db_datareader ADD MEMBER [$apiName];
ALTER ROLE db_datawriter ADD MEMBER [$apiName];

Next we'll have a look at the small code snippet you'll to get the managed identity running with entity framework,

services.AddDbContext<MyDbContextClass>(options =>
{
    var sqlConnection = new SqlConnection(builder.Configuration.GetConnectionString("MyConnection"));

    options.UseSqlServer(sqlConnection);
});

It should be pretty simple as above, assuming you have already created a context class when using entity framework. That should be you all need to setup and get running with a C# app with Entity framework which is connecting to your Sql server running on Azure using a system managed identity!

Deploying Azure Resources via powershell and bicep

2024-03-06T22:00:00+00:00

Deploying Azure Resources via Powershell

Deploying bicep templates

For the purpose of this article, I'm going to assume you already have your bicep templates made and have them stored in a convention which allows you to parameterise the paths to find them along with hosting profiles.

Start your script with any parameters that you might need

Param(
    [Parameter(Mandatory = $True)]
    [alias("n")][string]$applicationName,

    [Parameter(Mandatory)]
    [alias("e")][string]$environment
)

Given that most modern applications will look to deploy resources to more than one environment, maybe as many as four or five for larger companies, it's generally a good idea to define these environments within a hosting profile which your script can access to deploy them a generic manner. I'd strongly advise that a hosting profile only contains non-sensitive data or data which it is simply not possible to put in keyvault or an Azure app configuration instance. For my case, I've setup my bicep templates in a folder called Provisioning which then has sub folders for each applications bicep + hosting profile (which is a json file). We can then get the hosting profile and its environment specific data like this...

    $root = Get-Location
    $applicationName = $applicationName.ToLower()
    $environment = $environment.ToLower()

    # load our environment settings from our hosting profiles
    $hostingProfiles = Get-Content "$root\Provisioning\$applicationName.json" | ConvertFrom-Json;
    $hostingProfile = $hostingProfiles.$environment

My assumption here is that your hosting profile looks something like this...

    "dev": {},
    "qa": {},
    "live": {}

And you should never trust parameters, check the environment actually exists before you go any further with something like this

    if ($null -eq $hostingProfiles.$environment) {
        Write-Host "No profile is configured for environment $environment. Profile: $hostingProfiles"
        exit 1;
    }

Once we're a bit more sure that our hosting profile probably contains what we need, we should try to authenticate with azure.

    $tenantId = $hostingProfile.tenantId
    $subscriptionId = $hostingProfile.subscriptionId
    $resourceGroup = $hostingProfile.resourceGroup
    Connect-AzAccount -Tenant $tenantId -Subscription $subscriptionId

From there we simply get our template variable referenced and call the azure powershell module to deploy.

    # run templated deployment
    $templateFile = "$root\Provisioning\$applicationName.bicep"

    $deployment = New-AzResourceGroupDeployment `
        -ResourceGroupName $resourceGroup `
        -TemplateFile $templateFile `
        -applicationName $applicationName `
        -environment $environment

If our bicep templates gives us any output, we can iterate through it and display it.

    Foreach($output in $deployment.Outputs.Keys)
        {
            $keyValue = $deployment.Outputs.$output.Value
            if ($null -eq $keyValue)
            {
                Write-Host "Output from key : $output is null - this suggests strongly that the deployment failed."
                exit 1;
            }
            Write-Host $output " : " $keyValue
        }

We can also check if anything appears to have gone wrong with the bicep template and we want to exit the script with a code of 1, so that any script runners (i.e azure pipelines) will pick up as a failed script. Though in most cases I'd wrap the whole script in a try catch so we can get any other error messages.

Try 
{
    ....
}
Catch
{
    Write-Error "Terminating Script: $_"
    exit $LastExitCode;
}

Parallel tasks in powershell

2023-09-01T23:00:00+01:00

As the complexity of your application increases, it's a given that your build and release pipeline will get more complex. You'll notice the release time gradually creeps up which could lead to developer (or stakeholder!) frustration when waiting to validate a change when it hits your intended target environment.

The built in azure devops tasks can take you so far but again as complexity increases there is a good chance that you'll need to customise the build pipeline further and what better tool is there for that than powershell, it's extensible and powerful and runs on azure with great integration to azure with either the AZ modules or the full AZ CLI. I'm a fan, especially with my linux background, I almost prefer it bash at times.

Back to the point, increasing the complexity of your application will lead to increased build time so it would be great if we could have of those tasks run at the same time. There is a few options for this but really the best way, although the most complex, is to use powershells built in runspaces feature. Each runspace is a thread within your powershell process so this is an incredibly powerful feature which can cut your tasks runtime in half.

To get started you want to start a runspace pool like this

$RunspacePool = [runspacefactory]::CreateRunspacePool(1,2)  # Set up runspace pool
$RunspacePool.Open()

This defines a runspace pool where you can have a maximum of two runspaces (or threads) running concurrently.

$executionTime = Measure-Command -Expression {

}

Write-Host "Total execution time: $($executionTime.TotalSeconds) seconds." -ForegroundColor Cyan

For testing purposes, I like to wrap the actual work that'll be performed within the script in that Measure-Command, it helps to evaluate if the added complexity that runspaces brings is worth it for your task.

$projectDirectories = Get-ChildItem -Path $applicationPath -Filter "$applicationName.*"

    # Creating npm build runspace
    $npmBuildInstance = [powershell]::Create().AddScript({
        param($projectDirectories, $root, $applicationName)
        Set-Location $root
        . .\BuildService.ps1
        foreach($projectDirectory in $projectDirectories) {
            $buildService = [BuildService]::new($root, $applicationName)
            $buildService.RunNPMBuild($projectDirectory)
        }
    }).AddParameter('projectDirectories', $projectDirectories).AddParameter('root', $root).AddParameter('applicationName', $applicationName)

    $npmBuildInstance.RunspacePool = $RunspacePool
    $Runspaces += ,@{
        Instance = $npmBuildInstance
        Result = $npmBuildInstance.BeginInvoke()
    }

This is a relatively simple example as the actual work done is hidden within the BuildService class, however it gets the point across regarding how a thread is setup to run,

    $npmBuildInstance = [powershell]::Create().AddScript({

    })

    $npmBuildInstance.RunspacePool = $RunspacePool
    $Runspaces += ,@{
        Instance = $npmBuildInstance
        Result = $npmBuildInstance.BeginInvoke()
    }

Is how its done and the contents of the 'AddScript' function is what gets run! However, there's an important caveat, each runspace is its own powershell and has no context passed to it from the executing main script, so things like working directory or imported modules may have to be re imported or set.

The next step is to look at how we know the runspace is finished running, we'll loop round the the Runspaces object and check for all items to have their IsCompleted flag set to true.

while($Runspaces | ? { -not $_.Result.IsCompleted }) { 
    Write-Host "Awaiting completion"
    Start-Sleep -Seconds 1
}

The penultimate step is to kill the runspace and write any output from it to own our own session, otherwise we'll never know any of the output from the runspace.

# Process output or errors from each runspace
foreach($runspace in $Runspaces)
{
    $runspace.Instance.EndInvoke($runspace.Result)
    $runspace.Instance.Streams.Error | ForEach-Object {
        "[DEPLOY SERVICE ERROR] $($_.ToString())"
    }
    $runspace.Instance.Streams.Information | ForEach-Object {
        "[DEPLOY SERVICE INFO]  $($_.ToString())"
    }
}

And the final step is to close and dispose of the runspace pool and if you followed my example of wrapping the code in a Measure-Command block then you'll want to close it off and write the execution time out.

$RunspacePool.Close()
$RunspacePool.Dispose()


}
Write-Host "Total execution time: $($executionTime.TotalSeconds) seconds." -ForegroundColor Cyan

And thats it! Setting up a runspace pool on its own isn't overly complex however the real difficulty comes in any of the overlapping operations that might be occuring within each thread. It gives enormous benefits in speed, for my purposes I measured both build and publish times cut in half.

Limitations of bicep deployed function apps

2023-06-01T21:50:00+01:00

One of my clients was looking to deploy a function app which would be triggered by an event grid trigger, simple enough to set up and simple enough to write some bicep to deploy the resources for right? Well not quite.

The function that was being deployed was slightly more complex and made use of a few of the clients libraries and used DI (dependency injection) which is a pretty normal thing, so the function gets deployed using az publish commands through powershell. It was defined in bicep using something like

resource functionApp 'Microsoft.Web/sites@2020-06-01' = {
  name: appName
  location: location
  kind: 'functionapp'
  ........

Now normally that's all you need to do to get a function app resource ready, you just need that resource within your bicep. Bicep is ideally to be idempotent, so there should be no difference in the end result no matter how many times you run it, the outcome should be the same. You would have your azure resource to be deployed to then you can deploy your compiled code to this resource and let it run. In our case here, we want our function to be an eventgridtrigger function, which is simple enough to define within code, just something like this ....

namespace MyEventGridFunction
{
    public static class MyEventGridFunction
    {
        [FunctionName("MyEventGridFunction")]
        public static void Run([EventGridTrigger]EventGridEvent eventGridEvent, ILogger log)
        {
            log.LogInformation(eventGridEvent.Data.ToString());
        }
    }
}

If you get this in a proper project, flesh out the bicep above and deploy, it'll happily run. Where we encounter a problem is when we try to define the eventgridsubscription via bicep,

Flesh out this part - it's incomplete
resource eventGridSubscription 'Microsoft.EventGrid/eventSubscriptions@2020-06-01' = {
  name: eventGridSubscriptionName
  scope: eventGridTopic
  properties: {
    destination: {
      endpointType: 'WebHook'
      properties: {
        endpointUrl: eventGridSubscriptionUrl
      }
    }
  }
}

As you can see from the above bicep - the eventGridSubscription is linked to the function app, except when we run the bicep it responds with an error of the function must exist first! This is an issue as the if the code isn't deployed how can it exist? The eventgridsubscription is effectively looking for an endpoint which is not defined (Nor in my opinion SHOULD be defined!) from our bicep template. This clashes with our ideal of idempotency within our bicep templates, we want to be able to run our bicep and simply setup our infrastructure ready for our code. What if we can effectively define our function within our function app in our bicep template? This is a part of the job that I'm not fond of, the inaccuracy and heavily context dependency of language, in this terms a function is closer conceptually to a method, as the name function is already taken by type of the 'site' we have setup.

resource function 'Microsoft.Web/sites/functions@2020-06-01' = {
  name: '${functionApp.name}/${functionName}'
  properties: {
.......

This looks perfect, it sets up a function within the function app site we defined earlier. Except we have a problem, how do we deploy code to this? When we look into the folder structure on the azure resource we've setup, it will literally be

site\wwwroot\$functionAppName\FunctionName

It'll define it's own function.json etc but we can not deploy our code to it! This is a pretty fundamental flaw, we can do some pretty cool things with it like define simple functions within small c# files that get deployed with the bicep template or even have inline c# such as

resource function 'Microsoft.Web/sites/functions@2020-06-01' = {
  name: '${functionApp.name}/${functionName}'
  properties: {
    config: {
      disabled: false
      bindings: [
        {
          name: 'eventGridEvent'
          type: 'eventGridTrigger'
          direction: 'in'
        }
      ]
    }
    files: {
      'run.csx': '''
      #r "Microsoft.Azure.EventGrid"
      using Microsoft.Azure.EventGrid.Models;
      using Microsoft.Extensions.Logging;

      public static void Run(EventGridEvent eventGridEvent, ILogger log)
      {
          log.LogInformation(eventGridEvent.Data.ToString());
      }
      '''
    }
  }
}

But we can not setup a function like this that we deploy to from a seperate code pipeline. So we've hit a deadend with a single bicep execution to deploy all of our infrastructure.

What I ended up doing was to simplify the pipeline a little, deploy the site resource with a type of functionapp, then to deploy the function app source then have a different powershell task to deploy the eventgrid bicep.

Handling template inheritance within a monorepo.

2023-05-17T21:50:00+01:00

For my current client, they have all of their API's contained within a single repository as this allows for simpler code navigation from shared components such as DTO's or client classes which act as wrapper classes around an httpclient instance. This provides a lot of benefits especially when there is tighter coupling between some of the API's than say a nuget package would give.

It gives an interesting problem over reuse of yaml though, when defining a build pipeline or an integration pipeline, it would be silly to have the same pipeline copy/pasted into several different files and would give a bit of a maintenance headache when the shared powershell scripts that do 'the under the hood' work of the yaml pipelines change.

The solution we settled on was to create an application specific yaml file called say 'auth.yml' where the contents of it would simply be to extend and provide an application name parameter to a a beefier integration.yml pipeline.

# File: auth.yml
trigger:
- main
  paths:
    include:
    - Auth

extends:
  template: integration.yml
  parameters:
      applicationName: auth #

Then the integration.yml pipeline would look something like...

# File: integration.yml
parameters:
- name: applicationName

jobs:
  - job: Integration
    pool:
      vmImage: windows-2022
    steps:
    - checkout: self
      submodules: true
      persistCredentials: true

    - task: PowerShell@2
      inputs:
        targetType: 'filePath'
        filePath: $(System.DefaultWorkingDirectory)/Scripts/shared/build-app.ps1
        arguments: > # Use this to avoid newline characters in multiline string
          -n ${{ parameters.applicationName }}
      displayName: 'Build app'

    - task: PowerShell@2
      inputs:
        targetType: 'filePath'
        filePath: $(System.DefaultWorkingDirectory)/Scripts/prepare-artifacts.ps1
        arguments: > # Use this to avoid newline characters in multiline string
          -n ${{ parameters.applicationName }}
      displayName: 'Prepare artifacts'

    - task: PowerShell@2
    ########################### Other tasks...

The end result is one integration pipeline yaml to maintain which can have optionality built into it with application specific driver yaml files which act as parameter feeders.

Git submodules

2023-05-09T23:33:00+01:00

I was asked to look into how to simplify the management of a large number of scripts within several application repo's, each repo had a scripts folder which was roughly the same but with just enough variability to historically justify the straight forward option of isolating the scripts. This however lead to some problems when script changes in common were needed.

The solution I worked on, was to create a shared scripts folder within a common 'devops' repo. Each script within this repo, such as build/prepare/publish, would work out via execution branching and parameters picked up via hosting profiles (nice way of saying parameter files) exactly what it needed to build for each application.

The common scripts folder in the devops repo would be accessed via a git submodule - this is effectively a checked out commit of the devops repo which is detached from the HEAD. In other words, it's a snap shot in time of the repo. This works nicely as it allows each application repo to pick when to update it's scripts folder rather than be forced to take potentially breaking changes.

The gitsubmodule can be created via this command,

git submodule add https://github.com/admgreen/myrepo

This would add the submodule at the root of your repo in a folder with whatever your repo is called, in the above example it would be 'myrepo'.

When you come to commit your changes to your application repo, you'll find two newly added files,

.gitmodules 
& 
$externalRepoName

The .gitmodules file will contain the path to your repo and it's url, the $externalRepoName file will contain the commitId that you've checked out.

This is a great way to manage common scripts/code which you don't think you'll update often but the practice of packaging whatever it is upto into say a nuget package is the wrong option.

.Net Background Tasks

2022-01-31T22:07:00+00:00

Background Tasks

Running a background task

Running a task for your API, product or service used to involve creating a complex system with lots of custom code, even effectively a product in its own right just to trigger the code you want to run! This got a little easier with the addition of services like Hangfire, it gives you a nice admin page, it's well documented and commonly used. It's relatively easy to implement as well, though it is (I would argue) only worth adding to larger projects.

TaskRunner

In a previous job, they solved the problem of background tasks via a very complex system, but it was all that was available at the time. They had a SQL Server database called Tasks with a table also called Tasks which had fields such as

AssemblyName
Class name
procedure name
parameters
schedule (daily/hourly/adhoc)

Another table called TaskRequests could be populated with adhoc runs of processes. They had a pool of servers (called runners) with a primary server which would read continuously from these tables and check with the other servers in the pool if they were free to have a task assigned to them.

Visually the Architecture looked like this,

This system largely worked quite well, allowed for easy scaling of job processing with the only real pain point being load on the SQL database. Which could be considerable, however it was offset by the sheer power of those machines, they had 512GB of ram in 2014!

IHostedService

Both of the larger task runner service and hangfire are for larger more intensive requirements, what if you just want a simple background job for your API or web service? Recently at a client, they had a need to gather the status of multiple API's for monitoring purposes, originally I had designed the system so that each API was being polled by a service called the Registry, the registry would store a list of services which had registered with it, when it's triggered, it would call out to the status endpoint associated with each service that had registered itself.

This works quite nicely but it has its limitations, mainly that each service has an endpoint waiting to be called which returns data. The additional function app which runs on a timer trigger is also an unnecessary extra resource to maintain. A better solution is to use make use of IHostedService, it would allow each service to have a background task which runs on a timer to make a REST call directly rather than waiting for it's status endpoint to be called. Turns the status update conceptually from a get to a push.

public class MyBackGroundService : IHostedService, IDisposable
{
    public Task StartAsync(CancellationToken stoppingToken)
    {
        _timer = new Timer(DoWork, null, TimeSpan.Zero, 
            TimeSpan.FromMinutes(60));

        return Task.CompletedTask;
    }

    private void DoWork(object? state)
    {
        var response = await client.PostAsJsonAsync(...);
        response.EnsureSuccessStatusCode();
    }
}

Don't forget to add the service to your startup class!

services.AddHostedService<MyBackGroundService>();

As you can see, for simple background tasks, this is a much lighter and simpler way of getting the task done.

CaseNotes Django Upgrade

2021-12-13T21:41:00+00:00

CaseNotes is a clinical client note and administrative system which I've built for a client and maintain on an ongoing basis, it's built with python, django, bootstrap and django rest api (for the iOS app for carers to access when on the go). It handles client confidentiality, billing and a few other things as well.

With the latest django update, it's a good time to do some maintaince and get CaseNotes upgraded to ensure it's still easily maintainable.

First thing to do is switch to correct virtual envrionment (assuming you use mkvirtualenv!)

workon casenotes

Upgrade to latest django

pip install django

No breaking changes! Excellent!

I'll then run through the site to do some sanity testing, I usually do the following...

create a new note
create a new client
assign the new client to an employee
create a report for a webview and generate a pdf report
check a report on the billing side, do the sums to check that it adds up manually

Once no more package upgrades needed I'll then create new requirements.txt by doing

pip freeze > requirements.txt

And finally I'll run the unit tests by running this command

python manage.py test

At the point all thats left to do is commit the changes and work out when suits the client to deploy!

DotNet HttpClient Factory

2021-12-09T21:50:00+00:00

Recently at my current contract, I came accross an interesting problem, we were using typed http clients conceptually but not effectively or correctly. There are a few API's which at certain product launches can come under quite heavy load and unfortunately had both performance problems and could have exceptions thrown. The problem is in the use of the HttpClient class, it has an issue where even if you wrap its usage in a using statement, it won't release the sockets right away. The solution to this problem is to use IHttpClientFactory.

The design of my clients networking classes is hierarchical - an ApiClient class,then whatever the name of the consuming application is, lets make one up and call our application Consumer, so the class would be ConsumerApiClient which extends the partial ApiClient.

In code it looks something like this

public partial class ApiClient() : IApiClient
{
 public ApiClient(HttpClient httpClient)
 {
   _httpClient = httpClient;
 }

 public async Task Send()
 {}

}

Then the implementing class would look something like...

public class ConsumerApiClient() : ApiClient, IConsumerApiClient
{
    public ConsumerApiClient(HttpClient httpClient, AnotherClass className) 
    : base (httpClient)
    {}

    public async Task<Type> MyMethod(Params params) 
        => MethodProvidedByApiCient(settings.url, params);
}

The pattern used is designed to provide small client libraries with routes, DTO's and methods to obtain bearer tokens for authenticated access in a fashion that the developer who is interacting with the API is doing so in a controlled manner. Luckily this exact setup is pretty much ready to go for (in my opinion) the cleanest method of implementing the IHttpClientFactory pattern.

The ConsumerApiClient would be instantiated using DI, with a scoped lifetime I think it was but contained down at that ApiClient class, it would new up an instance of HttpClient to make whatever requests it needed.

Now when the ConsumerApiClient was underload, my client found that they often encountered the afore mentioned socket exhaustion exceptions. Not great when you've got a new product launch.

Now there's a really simple trick with this setup to really enhance the performance of the httpClient calls, instead of setting up the ConsumerApiClient in the DI with a scoped lifetime like ...

services.AddScoped<IConsumerApiClient, ConsumerApiClient>()

we simply do

services.AddHttpClient<IConsumerApiClient, ConsumerApiClient>()

Have the constructor require an instance of HttpClient (which the DI will provide) then pass it all the way down to the base client. What this does is move the responsibility of the lifetime of the httpclient instance to IHttpClientFactory, which is handled by the dotnet runtime instead.

The end result is performance improved and socket exhaustion errors stopped!