Returning structured data from an LLM via LangChain
Posted on Mon 22 September 2025 in ai, langchain, python, api, json
In my last article, we walked through how to turn our simple langchain -> ollama model into a simple chat bot over an API using with in memory history and seperate users, so each user gets their chat history with the bot. Another very useful case for llm's is categorisation and forming a structured json response from user input, say you have a triage system, where you want to extract key data and return it in a structured format for another service to consume. Let's stick with the same app structure as before, we'll add in a new service class since we want this llm instance to be configured seperated without history. We'll call our new file 'llm_service_structured_data' - very clear what it's going to do!
┣ app
┃ ┣ services
┃ ┃ ┣ llm_service_structured_data.py
┃ ┃ ┣ llm_service.py
┃ ┣ __init__.py
┃ ┣ llmroutes.py
┃ ┣ inMemoryHistory.py
┃ ┗ models.py
┣ config.py
┗ run.py
There are a few differences from before, we want to use a 'RunnablePassthrough' class instead of RunnableWithMessageHistory and we want to define our prompt template to strictly return json data in a parameterised way. We'll start with our imports
from langchain_core.output_parsers import JsonOutputParser
from langchain_ollama import OllamaLLM
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
Then lets sort our init for our new class, in the init we'll define our prompt template, tell it the format we expect back for our json and create a chain to call via a method.
class LLMServiceJson:
def __init__(self):
self.model = OllamaLLM(model="gpt-oss:20B")
self.prompt = ChatPromptTemplate.from_template("""
You are a service that extracts customer info.
Always respond with ONLY valid JSON in this format:
{{
"customerName": string,
"issueType": string,
"priority": "low" | "medium" | "high"
}}
User message: {message}
""")
self.parser = JsonOutputParser()
self.chain = (
{"message": RunnablePassthrough()} # passes user input
| self.prompt # formats prompt
| self.model # sends to LLM
| self.parser # enforces JSON parsing
)
And thats our class setup, now we just need a simple method, lets call it classify which will process user messages and return the json structure from the prompt. Very simple, takes in self, a message from the calling object and tells the caller to expect a string back.
def classify(self, message: str) -> str:
"""Process a chat message with persistent memory."""
response = self.chain.invoke(message)
return response
Let's add a new route in our llmroutes.py file, it'll just do some validation on the message and pass it through.
@llm_bp.route("/enquiries", methods=["POST"])
def enquiries():
data = request.get_json()
message = data.get("message", "")
if not message:
return jsonify({"error": "Message is required"}), 400
response = llm_structured_data.classify( message)
return jsonify({"response": response})
And now that's us ready to run and test our api! If I send in a message about my car being broken down roadside, I get a response like this...
curl -X POST http://localhost:8083/llm/enquiries \
-H "Content-Type: application/json" \
-d '{"message": "Hi, my name is Alasdair, my car has broken down and I need help at the roadside "}'
{
"response": {
"customerName": "Alasdair",
"issueType": "Roadside assistance for broken down car",
"priority": "high"
}
}
Which is great, thats a correctly formatted Json response that we can integrate into another service nicely. We can test it with other types of problem too,
curl -X POST http://localhost:8083/llm/enquiries \
-H "Content-Type: application/json" \
-d '{"message": "Hi, my name is Alasdair, my tap is leaking slowly, can I book a plumber for next week? "}'
{
"response": {
"customerName": "Alasdair",
"issueType": "tap leak",
"priority": "medium"
}
}
Great, it works across a few different types of enquiries that need to be classified!