WTIT surveyed over 100 customers in 2024 and asked them what they are doing with AI. The overwhelming answer was: shutting down any production use of ChatGPT.
Why? Respondents cited concerns around visibility, security, and intellectual property—and they’re not wrong! Without the proper guardrails to ensure governance, prompt security, cost controls, and visibility, public-facing GPTs create security, HR, and cost concerns. The good news is there has been an open-source revolution around large language models (LLMs), making AI more viable for companies that don’t have a deep bench of AI expertise.
Overview of the LLM Market
While ChatGPT was the first generative AI model of significance on the scene, the LLM market is now flush with great options that represent a varying scale of transparency and control that can enable accelerated AI adoption (if managed correctly). Categories that make up the LLM market include:
Commercial (public) closed-source models: Models such as ChatGPT provide a “black box” chat experience without access to code or parameters. It’s safe to assume with public models that any data you provide as part of your prompt can be leveraged back into the model lifecycle as training data. You also lack the insight to understand if the model supports your requirements around trustworthy AI utilization to avoid bias and other misaligned outcomes.
Private closed-source models: These private models deliver the same “black box” experience, but the model can be downloaded and run in a private DMZ so data and engagement stay protected and controlled with nothing running across the Internet.
Open-source models: Like the traditional definition of open source for software, open-source LLMs allow full access to source code and any information required to retrain the model from scratch. They offer the greatest flexibility, but adopters need a lot of technical know-how and effort to wield and customize these models. That said, this maturing category provides out-of-the-box answers, often for free, that are comparable to commercial models. They also provide total control over data privacy.
Open-weight models: Unlike open source which allows code-level access, open-weight models provide transparency around the pretrained parameters, or weights, of the neural network model. This enables fine-tuning (inference) while keeping the code, original training dataset, architecture, and other details unavailable. Customization is more limited, but there is reduced complexity, making this a good fit for less technical teams focused on speed.
Closed-weight models: These are effectively the same as closed-source models. Users cannot directly fine-tune model behavior by modifying its weights.
Figure 1. Market landscape of large language models
Regardless of the Model Type, You Need Security and Governance
AI applications introduce unique challenges compared with more traditional, deterministic applications. The wide variety of data sources, deployment models, and unpredictable nature of both incoming and outgoing information creates a moving target for both security and ensuring quality, trusted AI outcomes.
The question of whether a model is “good” or “bad” depends more on the target use case and the guardrails you put up around it. You need security and observability to ensure controlled user access, intelligent prompts, data privacy, and protection of your intellectual property.
Similar to how network proxies are used to control and restrict access and provide inspection and visibility into data loss prevention (DLP) and PII protection, you need those same corporate governance and controls over your AI workloads.
WTIT is proud to be a strategic partner of F5 to provide network and application security for your workloads, no matter where they run. We’ve witnessed F5 invest extensively to continue modernizing their portfolios, including their release of F5 AI Gateway—an integral part of WTIT’s AI strategy for its customers.
Why F5 AI Gateway for Your AI Workloads
AI Gateway inserts itself within the AI application workflow and inspects inbound prompts and outbound responses to prevent data leaks, mitigate AI-targeted attacks, and ensure reliability and performance. It makes it possible to safely entertain new use cases that involve sensitive or highly regulated data.
With the AI Gateway in place, you can:
- Institute security and compliance policy enforcement with automated detection and remediation against OWASP Top Ten for LLMs
- Protect speed and performance with semantic caching to improve the user experience and reduce operations costs
- Keep developers and DevSecOps focused on delivering AI-powered applications rather than worry about managing complex infrastructure
- Maintain real-time service availability and performance through load balancing, intelligent traffic routing, rate limiting, and more
Key Attributes of the AI Gateway
Figure 2. F5 AI Gateway capability overview
The gateway offers essential capabilities across five major categories:
Observability, monitoring, and analytics
You can’t protect what you can’t see. Adding observability, monitoring, and analytics gives teams more power to proactively act or intervene based on request and response content and AI workload behavior.
- Activity logging: <desc>
- Transaction logging: <desc>
- Performance metrics: <desc>
- Feedback: <desc>
- Watermarking and traceability: <desc>
- Language detection: <desc>
Performance optimization
Companies looking to scale their AI initiatives beyond a proof of concept quickly learn they can’t deliver the same performance at scale. AI Gateway can alleviate many of those issues.
- Caching: semantic caching drives faster response time and reduces operational costs by removing duplicate tasks from LLMs, avoiding the consumption of tokens. (source)
- Load balancing: <desc>
- Latency optimization: <desc>
Governance
New AI regulations are aggressively coming online, with some provisions of the comprehensive EU Artificial Intelligence Act coming into effect February 2026. Without automated, systematic, centralized policy creation and enforcement, it will be near impossible to maintain compliance over time.
- Encryption: <desc>
- Authentication: <desc>
- Rate limiting: <desc>
- Compliance Protocols: <desc>
Prompt security
Securing AI is essential to controlling who has access to what and keeping data and proprietary model information safe.
- Access controls: <desc>
- Prompt injection: occurs when user prompts alter the LLM’s behavior in unintended ways through direct or indirect inputs, potentially causing the model to violate guidelines, generate harmful content, enable unauthorized access, or influence critical decisions, even when the manipulated content is imperceptible to humans. (source)
- Data loss prevention: <desc>
- Response sanitization: <desc>
- Multi-view analysis: <desc>
- PII detection: <desc>
Cost control and prompt steering
Part of the success in scaling AI means efficient use of your resources and LLM services, which like cloud services, can rack up costs in unexpected and exponential ways.
- Backend service utilization: <desc>
- Intelligent prompt steering: <desc>
- Prompt decoration: <desc>
- Response minimization: <desc>
Benefits of the F5 AI Gateway Architecture
The AI Gateway was designed for flexibility, portability, and more streamlined integrations to support a wide range of AI architectures and use cases.
The core function of the gateway of course is policy enforcement. As noted in the diagram, this includes full inspection of prompt and response, sanitation of data, data leak prevention, fine-grained access control to data, prevention of unauthorized outputs, detection and prevention of attacks, and compliance with AI and data privacy regulations.
The AI Gateway is Kubernetes based, enabling consistent deployment options across clouds and data centers. Popular AI models including OpenAI, as well as generic HTTP LLMs and small language model (SLM) services are supported; an SDK is also available for further integration support.
A processor runs separately from the core and can come from F5, a trusted partner, or be developed by the customer (using the SDK). These functions can modify a request or response, for example to redact PII data, reject a request or response if it violates a policy, or annotate by adding tags or metadata to supply additional information for admins or other parts of the AI Gateway.
The benefit of leveraging these modular processers allows you to <detail> <sample text: simplify deployment of geo, industry, and corporate governance and security controls without compromising AI workload performance.>
Figure 3. AI Gateway architecture
AI Gateway Professional and Managed Services from WTIT
For companies that don’t have formally managed Kubernetes environments or need help developing and deploying AI Gateway services. In response, WTIT has designed a portfolio of professional and managed services to help our customers secure their AI workloads and ensure ongoing quality, compliance, and performance. WTIT services for AI Gateway include:
- Security assessment
- AI regulation readiness
- Custom processor development
- Cost optimization
- Integrations with the F5 portfolio including F5 NGINX, F5 BIG-IP Next, and F5 Distributed Cloud App Stack for edge deployments
- Fully hosted and managed delivery of AI Gateway as a service
This is part of a larger AI journey you can take with WTIT. We have honed the in-house expertise you need to help you adopt new AI models, create SaaS or cloud-based model delivery, and manage the ongoing health of your end-to-end AI solution.
Ready to Learn More?
Check out our latest video <URL> for another look at AI Gateway with WTIT. We’ll also be at AppWorld in Las Vegas, February 25-27—contact us at <@email> if you want to schedule a meeting or visit our booth. Hope to see you there!
Leave a Reply