Llama self-hosted LLM engineering
Llama is what you pick when quality is not the deciding factor. Either the data is not allowed to leave your network, or you need a model nobody else can retire on you. Both are good reasons. Neither is free, and the bill arrives as people and hardware rather than as usage.
01Capabilities
What running a model yourself actually buys
Llama is a family of models from Meta whose weights you can download, so this page is judgement about when running one yourself is the right architecture and what you take on by choosing it. The build is AI and machine learning development sitting on top of a serious infrastructure job.
- Boundary
Answers that never cross your own edge
On your hardware or in your own cloud account, with nothing going out to a third party. This is the argument in healthcare and fintech, where the review question is not how good the model is but who else can see the data.
- Ownership
A tuned model that belongs to you
Trained a little further on your own material, so it knows your vocabulary and your document shapes. The result sits on your side of the boundary as something you hold, rather than as a capability you rent by the month.
- Steadiness
A version nobody can withdraw
A model you hold cannot be retired on somebody else's roadmap. Where changing it means re-approving an entire approved workflow, that stability is worth more than a few points of quality on a comparison nobody in the room believes anyway.
- Volume
Cost that stops tracking usage
Once traffic is high and steady, paying per word stops being the cheaper option. Batching, shrinking the model and honest arithmetic about how full the hardware runs are what turn that into a saving instead of an idle machine.
02Fit
The arithmetic that decides whether to own the hardware
Running your own model is a real commitment: machines, a serving stack, capacity planning, an upgrade path and somebody on call at three in the morning. We recommend it when a constraint forces it or when steady volume genuinely pays for it, and we talk people out of it the rest of the time. Four of the six rows below end somewhere other than a self-hosted project with us.
| Your situation | What we recommend |
|---|---|
| Data that may not leave your network, by contract or by law | Run Llama yourselfHere downloadable weights are not a preference. They are the only arrangement that satisfies the requirement at all. |
| An approved workflow where changing the model means approving it again | Hold your own copyOwning the model takes one external dependency out of your compliance story, which is often the whole point of the exercise. |
| Traffic that is low, spiky, or still a guess | Rent it insteadBelow steady use, owning hardware costs more and does less. Start at the OpenAI API or Claude and revisit with real numbers. |
| No graphics cards, and nobody whose job is keeping them running | Do not start hereThe model is the easy part. A serving platform with no owner degrades quietly until somebody notices that everything got slower last month. |
| You want to ship the model inside software you sell | Read the licence firstLlama ships under a Meta community licence rather than an approved open-source one, so Qwen may be the shorter legal conversation. |
| One narrow decision, made constantly, with your own examples of it | Train something smallA general model is an expensive way to output one of four labels, see PyTorch. It also runs on far less hardware. |
Scope
What we own on a self-hosted deployment is the capacity plan and the arithmetic that justified it, the decision to shrink the model and the accuracy that costs, the serving stack and how it behaves under a burst, the security boundary, and a live comparison against a rented model so nobody has to guess what independence is costing you in quality. unicrew has been building software since 2012, with 100+ senior in-house engineers across six countries, under an ISO 27001:2022 certified information-security management system. Where renting is the better answer, that is the answer you get.
03Delivery
Standing up inference your own team can operate
The first step is where the money is saved. More projects of this kind should stop at step one than actually do, and stopping there is a result rather than a failure.
- Confirm the constraint is real, in writingWhich regulation, which contract clause, which volume. Plenty of nothing-leaves-our-network rules turn out to be a preference nobody has tested against a regional hosted option with a signed processing agreement.
- Size it from the traffic, not from the model cardWords per second, how many people at once, how long the inputs are, and how spiky the peaks get. Then we test the model at the reduced size you would actually run, because quality claims are made at full size and you will not be serving at full size.
- Build the serving path properlyRequests batched as they arrive, a queue that pushes back when it is full, scaling that accounts for how long a model takes to load, and cold-start behaviour you have measured rather than hoped for, usually behind Docker and Kubernetes.
- Keep a rented baseline aliveThe same test set run against a frontier service on a schedule. It tells you what independence is costing in quality, and it turns next year's reconsideration into a data exercise instead of an argument. It goes through the same QA and test automation practice we use on our own builds.
04Stack
The platform underneath a self-hosted model
The pieces a deployment of this kind is actually made of, each with its own page if that is the decision you are actually making.
05Questions
Asked before anyone buys a graphics card
Six that decide whether this project should exist, answered the way we would answer them live.
Yes, and this pairing works well, because your infrastructure people know your network and we know the serving stack. What we will not do is supply engineers and leave the capacity and model-size decisions unowned, since those set both your bill and your output quality. An architect outside the delivery team reviews the design, and the platform work goes through the same review as any cloud infrastructure we build.
Only above a level of steady use, and that level is higher than most teams expect. A graphics card costs the same whether it is answering requests or idling overnight, so the comparison is your sustained throughput against usage pricing, not your busiest hour against a headline rate. Spiky traffic favours renting. Steady heavy traffic, or a rule about where data sits, favours owning it.
Not in the sense a lawyer means. The weights are downloadable and usable commercially, but the licence is a Meta community licence rather than an approved open-source one. It carries an acceptable-use policy, naming requirements, and a clause requiring a separate agreement above a very large user threshold. For most companies none of that bites. If you plan to resell the model inside your own product, have your legal team read it.
It follows from your traffic, your input lengths and how long an answer may take, not from the model's headline size. We size it from those, then test at the reduced precision you would really run, because shrinking a model is what makes the hardware affordable and it costs some accuracy. The honest answer is often less than expected for steady work and much more for spiky peaks.
A scoping call with an engineer, and the useful preparation is the constraint in writing plus your expected traffic. Those two facts decide most of the architecture before anything else is discussed. You get back a written recommendation covering own or rent, the sizing, the operational load you are taking on, and the quality gap to expect. Most engagements start within two to four weeks.
Three shapes, and which fits depends on how settled the scope is. Time and materials is billed hourly and quoted per project, which suits work still moving. Fixed price is outcome based, offered once the first read is done, because a fixed number on a system nobody has opened is a guess with a contract around it. Team extension is billed monthly per engineer. The rate depends on the seniority mix, so it is quoted rather than listed.
Told the data cannot leave your network?
Tell us the constraint, the traffic you expect and the hardware you have or can buy. You will get an engineer's read on whether running it yourself is justified, what it costs to keep running, and the quality gap to expect.