An on-premise AI system without an admin plan is a compliance problem wearing a nice interface.
You need a clean way to manage who reaches the models, what they can do once they're in, and what record survives the conversation. OpenWebUI's the standard open-source interface for running multiple language models locally, and it handles user permissions, conversation history, and usage logging without shipping a byte to a cloud platform.
This is the guide I walk an admin through on site, in the order I walk it: initial setup, user accounts, role-based model access, retention, and the network configuration that keeps the thing reachable and nothing more. It assumes you've already got the hardware standing with Ubuntu Server LTS running. None of it assumes where the box came from.
Initial Setup and Admin Login
OpenWebUI runs in Docker on your local network. Once the container's up, the first user created becomes the admin, which is worth knowing before somebody's nephew gets there first. You'll reach it at http://ai.yourfirm.local:8080 or http://192.168.x.x:8080 depending on your network configuration.
First login sets the admin password. That's a hard security boundary, so use a strong one: 20-plus characters, mixed case, numbers, symbols, stored in your password manager. Bitwarden, 1Password, or whatever vault your firm already runs all work. That password controls user management, model configuration, and conversation history exports, which is to say it controls everything.
From the admin panel in the top-right menu, go to Settings and handle three things before anything else. Disable public registration so you decide who gets an account. Confirm authentication's required for all access. Set your conversation retention window, which is covered below.
I don't leave a site until those three are done. They're the ones that quietly cost people later.
Creating User Accounts and Role Assignment
OpenWebUI separates users into three roles: Admin, User, and Moderator. Most organizations don't need more than Admin and User. Admins see the admin panel and can change settings. Users reach models and create conversations but can't touch global configuration.
To create an account, go to Admin Settings, then User Management, then Add User. Set the username, typically firstname.lastname or email, then a password and a role. You can make the password temporary and let people reset it on first login if you turn that option on.
A law firm thinks in Attorney, Paralegal, and Staff. OpenWebUI doesn't enforce role-based permissions at the model level natively, and there's a workaround for that below. Create the accounts first, then restrict the models.
In that order. I've watched an admin restrict models first and then spend an afternoon wondering why the new hire couldn't see anything at all.
Model Access Control by Role
Here's the step I won't prescribe from a blog post.
Which models are on your box depends on what Discovery turned up in your workflows, and who should reach which one depends on what those people do all day. I don't know that yet and neither does a checklist.
The shape's usually familiar, though. In a law firm you might structure it so attorneys and senior paralegals reach every model on the machine, junior paralegals reach the general-purpose ones but not the model doing fast draft analysis on live matters, and administrative staff reach a single model for document summarization only. One building, three exposure profiles.
OpenWebUI handles this through the Models section in Admin Settings. Each model carries an Allow list restricting which users can reach it. There's no built-in group management yet, so you're managing access per user. For a 50-person organization that means individual management, and yes, it's tedious.
A practical workaround: keep a simple spreadsheet tracking who should reach what, then batch-update OpenWebUI monthly. It's more manual than enterprise IAM. It also works reliably under 200 users and costs you nothing in additional software.
It's ugly. It's also auditable, and that's the trade I'd make every time.
Conversation History, Retention, and Compliance
OpenWebUI stores every conversation locally on your server. That's the compliance advantage in one sentence: the conversation data never leaves your network. It's also a responsibility, because you now own retention, backup, and access on purpose instead of by accident.
All conversation data lives in a SQLite database on your server, typically at /root/ollama/web/conversations.db depending on your deployment. Local to your server. Not cloud storage.
That's the sentence that justifies the whole build.
As admin you've got full visibility. The Conversations tab in Admin Settings shows every conversation across every user, searchable by date or content, exportable. If a discovery request or a regulatory audit lands on your desk, you've got complete access to the model interactions, and that's the entire point.
Set a retention window. Most organizations delete conversations older than 12 months automatically. Admin Settings, then System, then Conversation Retention, then set the number of days. Common settings: 365 days for law firms holding litigation-hold capability, 90 days for IT support teams, 180 days for medical practices.
You can also export conversations as JSON for archival. The export's got timestamps, model used, user, and full conversation text. Useful for migrations, full backups, or a move to a different system.
For HIPAA-regulated organizations, remember that conversations may carry protected health information. Your retention policy, backup procedure, and access controls all have to satisfy HIPAA. Encrypt the database file at rest and make sure only authorized personnel can reach the admin panel.
If you can't name who reaches that panel, you haven't set a retention policy. You've set a wish.
Usage Monitoring and Analytics
OpenWebUI logs basic usage: which user ran which model, and when. That data's stored locally and shows up in the admin panel under Analytics once you've enabled it. You'll see per-user token counts, model usage over time, and activity worth a second look, like somebody suddenly running hundreds of queries a day, which usually means programmatic API use rather than an unusually motivated paralegal.
It's enough to track adoption, spot who needs training, and catch anomalies, though it isn't enterprise-grade AI audit logging and I won't pretend otherwise. Some firms use it to bill AI usage back to practice areas by volume.
It's a smoke detector, not a security camera. Worth knowing which one you've got.
Network Configuration: Static IP and Hostname
Your inference server needs a stable address. Nobody's going to remember an IP, so don't make them.
Assign a static IP on your router or DHCP server. Something like 192.168.1.50 keeps the server from pulling a new address on reboot. Then give it a meaningful hostname, ai.yourfirm.local or gpt-local.yourfirm.local, and point that at the static IP in your internal DNS or hosts file. Now people reach OpenWebUI at http://ai.yourfirm.local:8080 instead of memorizing an address.
A Windows domain with Active Directory registers the hostname automatically. Without a domain, add it to the hosts file on each machine or configure it in your router's DNS settings.
Somebody's going to type it wrong the first week regardless. Pick a hostname that survives a typo.
Security: VPN and Network Isolation
Your inference server doesn't go on the internet. Not behind a rule, not "temporarily," not for the one consultant who asked nicely. Internal network only, and remote staff come in over a VPN.
Tailscale's the easiest path. Install it on the server and on user machines and it builds an encrypted mesh where only devices in your Tailscale account can reach each other. Remote staff hit OpenWebUI exactly like the people down the hall. It's free for personal use and runs about $10 per user per month for business accounts.
If you'd rather stay open-source and self-hosted, deploy WireGuard. It's lighter than traditional VPN protocols, handles remote access without a cloud intermediary, and hands you complete control. More configuration than Tailscale, nothing unreasonable.
Either way: no direct internet exposure, all remote access through the tunnel.
That one isn't negotiable, and it's the line I'd put in writing before the machine gets powered on.
Backup Strategy
Run a daily backup of the OpenWebUI database to a local NAS or external drive. A cron job covers it:
sqlite3 /root/ollama/web/conversations.db ".backup /mnt/nas/backup/conversations-$(date +\%Y\%m\%d).db"
Schedule that at 3 AM and you're covered.
Keep the backups on local network-attached storage, a Synology or Buffalo box works fine, and don't sync them to AWS S3, Google Drive, or Dropbox. You've got hardware in the building specifically to keep this data off cloud platforms. Syncing the backups hands it all back.
I've seen it happen. Once by an IT contractor who figured he was being helpful.
Keep 30 days of dailies, then delete older files. That's the balance between recovery and storage cost. If your NAS supports encrypted shares, turn it on. These database files may carry sensitive information and they belong encrypted at rest.
Common Admin Tasks
Adding a new model: after adding it to your Ollama installation, go to Admin Settings, then Models, and refresh the model list. The new model appears. Set permissions as needed.
Resetting a user password: User Management in the admin panel, find the user, reset. They'll be logged out and prompted to set a new one on next login.
Viewing a specific conversation: Admin Settings, then Conversations, search by user or date, click through. Full audit access, exactly what compliance asks for.
Exporting all conversations: some organizations run quarterly exports to a long-term archive. The export option in Admin Settings generates a JSON file with every conversation and its metadata.
Getting Started
If you're standing OpenWebUI up for your team, here's the checklist in order: create the admin password, disable registration, add users with appropriate roles, restrict models by user, set conversation retention, configure static IP and hostname, deploy VPN access, and set up database backups.
That's the procedure, and the procedure's the easy half.
What a checklist can't hand you is which of your people needs which model, and I don't learn that from a form. I learn it sitting next to your records clerk while she does the thing she's done for eighteen years, and then the access map more or less writes itself. Configuring this together on site, and onboarding the team afterward on the workflows and the agentic orchestration their vertical runs on, is where a deployment either takes or doesn't.
For specific compliance obligations, HIPAA, FERPA, or SOC 2, reach out and we'll walk your audit and retention requirements directly. OpenWebUI was built for exactly this: local, controlled, auditable AI with no cloud overhead. Set the admin layer up properly and it scales cleanly to dozens of users with a compliance posture to match.