Tag: API
-
DeepSeek-V3 (MoE)
DeepSeek-V3 is an open-source large language model that boast a 671-billion parameter Mixture-of-Experts architecture with only 37 billion parameters activated per token. This specific model uses Multi-Head Latent Attention (MLA) for inference this compresses the attention keys and values in a low dimensional latent representation. Additionally this has also the strategy of Auxiliary-Loss-Free load balancing…
-
Hacking Kubernetes via ServiceAccountTokens
Kubernetes has a large amount of advancements and inherent good security principles but these are dependent on configurations that are typically not well-known to end users. Predominantly the constructs of Service Accounts or (Non-human Identities) for the masses are populated in many services as they act as the go-between for service to authenticate and operate…
-
Image Policy Webhook
Image Policy Webhook is a native Kubernetes admission plugin that enforces security policies by validating container images before they are deployed. This ensures that only trusted and compliant images run in your environment. This will take the image that is attempted to be applied compare against predefined policies, and if those policies allow the image…
-
Garak Red Teaming LLMs
As Generative AI is playing a role in multiple organizations so is the popularity of tools for identifying risks and vulnerabilities. In this blog I’m exploring Garak a LLM vulnerability scanner developed by NVIDIA and is a OSS project to help strengthen LLM Security. When the term “Red Team” appears in the approach of simulation…
-
Bill of Materials CKS Refresher
A Software Bill of Materials (SBOM) is like the ingredients list on your food package—it reveals what components, libraries, and dependencies go into building the final software. Just as checking food labels helps you understand nutritional content and potential allergens, an SBOM provides transparency into third-party components, helping identify vulnerabilities early in the software supply…
-
Azure AI Foundry
Introduction This week at Microsoft Ignite, Azure AI Foundry was unveiled as the rebranded successor to “Azure AI Studio.” This marks a significant step toward unifying AI development tools under one cohesive platform. Azure AI Foundry provides a streamlined toolchain and an SDK designed for efficient consumption of AI models, supporting both OpenAI and Mistral…
-
AI Agents with LangGraph
Introduction Agents are the next iteration of taking traditional stateless interactions with LLM’s to a stateful interaction with the use of typically Assistants API or extending using a framework. Some popular ways to create a agent workflow are tools such as Promptflow, CrewAI, LangGraph, LangChain and others. For this blog post I’m going to demonstrate…
-
RouteLLM Unlocking Cost Effective LLM Routing
Introduction Costs associated with using closed-source large language models can add up in the use cases of complex tasks due to the nature of how tokens are priced for using APIs. RouteLLM is a open-sourced project that creates a method to determine based on the query a user sends which LLM to choose based on…
-
Batch Jobs in Azure OpenAI
Introduction In the existing landscape of Generative AI, optimizing API submissions is crucial for both cost and performance. Whether you’re fine-tuning token usage or streamlining context-aware requests using Retrieval-Augmented Generation (RAG), finding the right tools can make a significant difference. One of the most promising solutions is the Azure OpenAI Batch API, designed specifically for…
-
Adversarial Simulation in Azure AI Studio
Large Language Models present a powerful enabler for various use-cases for most enterprises but without some form of due diligence and testing can spew some unintended responses. Content safety is a preventative mechanism that is used for Azure AI Studio and can also be tested with the Prompt-flow SDK. In this blog post I’ve going…