PathLAB · Explore the improvement loop
Learning from your automations.
What could your next version do better? Start with an automation that does useful work, examine its results and try a change. A prompt. A retrieval strategy. A model specialised for your task.
PathLAB connects the run records, reviewed examples and comparisons. You set the question, the test cases and what counts as an improvement. Each experiment gives you evidence for the next decision.
Follow the code-search exampleAn example · Searching your code
From finding the wrong files to learning where to look.
Your automation searches the codebase, but useful files keep getting missed. What would help it find its way? Review the queries and results, identify the relevant files and correct the searches. Those examples give you a starting dataset for adapting an existing model with an external trainer.
Search
An automation searches your code. Its queries, tool calls and results become part of the run record.
Review and correct
Identify the useful results. Correct a missed file or an unhelpful search, keeping your assessment linked to the original run.
Build a dataset
Select the examples and fields you want to use. Keep their source and review history, with separate cases reserved for testing.
Train a candidate
Use an external trainer to adapt an existing model, for example with a LoRA adapter, to the search task.
Compare and choose
Test it against your existing setup on new searches. Compare quality, cost and speed. Choose what to adopt; keep the baseline if it wins.
The techniques behind the opportunity
Ways to specialise a model.
You can build on an existing model rather than train one from scratch. The task determines the approach and the examples it needs.
- Fine-tuning
- Teach through task examples.
Adapt an existing model using examples of the work you want it to do, such as choosing useful search queries and tool calls.
- LoRA adapters
- Train a compact set of changes.
Low-Rank Adaptation (LoRA) trains an adapter instead of updating all the model’s weights. It is one way to fine-tune a model for a particular task.
- Model distillation
- Teach a smaller model your kind of work.
Use reviewed examples from a larger model’s runs to train a smaller specialist. Classify incoming requests, extract information or propose the next action. Compare quality, speed and resource use through the same improvement loop. Your workflow’s controls stay in place.
An experiment you can build on.
A useful result can change your next run. An unsuccessful attempt can tell you what to try differently. Keep the examples, comparisons and decisions connected as you explore, from a prompt change to a trained candidate.
Explore controlled experiments in PathLAB