Its that time again where the year ends and we begin the next year with the time frame being one more day left in 2024 it felt like it was time to release some predictions for 2025 and beyond. This list will be short but concise grounded in context and a healthy dose of foresight and intuition.
1. Agents in the Enterprise will rapidly be adopted with minimal thoughts on Security.
For what its worth I believe we are entering the Agent 2.0 Era as this year unfolded many Software-as-a-service solutions that are trying to address challenges beyond the ChatCompletionsClient. This is how most users interact with ChatGPT, Claude, Gemini and can be limiting while the API opens up the use of Assistants in OpenAI models this will likely set the foundation for having stateful interactions. LangChain produced a report on a sample of 1,300 professionals to learn the state of AI Agents in 2024 with insights showing 78% of respondents have active plans to implement agents into production soon. What remains to be seen at least as of today is platforms that are piecing together how to create a captivating story for Agents + having capabilities to perform tracing, observability and safe guards for the AI models all in one. This appears to be ripe for more startups that are targeting this space and the emergence of securing components associated with tools and ‘agents’ will come into focus.
2. Security for AI will take on a higher priority
The evolving nature of LLMs at least appeared to go mainstream end of 2023 and start of 2024 with many organizations leveraging some form of AI.
A research paper, The Rapid Adoption of Generative AI (September 18, 2024) highlights this trend with the following quote.
“We find that 39.4 percent of the U.S. population age 18-64 reported using generative AI during August of 2024, with 28.0 percent using at work and nearly one in nine workers reporting daily usage.”
The Rapid Adoption of Generative AI (September 18, 2024)
It is no longer just expected the boundary of AI will be confined to the user workstations this will likely touch other aspect of data management systems to enrich its responses with adaptive patterns based on use. Given that most of these LLMs are either confined such as SaaS equivalent more focus will be put on the vendors providing these services as many examples are showing signs of guard rails being either misused or outright abused to manipulate systems.
3. Deepfake attacks will evolve with Speech
Deepfakes have already made a significant impact on businesses, with a notable case in 2024 involving Arup, a British multinational design and engineering firm. In this instance, attackers used a sophisticated scheme to impersonate the company’s Chief Financial Officer, successfully deceiving the organization into transferring $25 million USD. Such attacks are likely to become even more costly as voice cloning technology from startups like ElevenLabs becomes increasingly accessible, making it easier to replicate voices convincingly
On the forefront of existential threats to the landscape this area will likely need solutions that can detect in real-time and also in air-gapped conditions depending on the level of communication. Future defense may center on something you know (pre-agreed safe word or detailed personal knowledge). These methods will likely have to evolve such as a safe word or very specific detailed knowledge that a person knows and can deceive the perpetrator.
4. Routing and use of multiple LLMs will go mainstream
This is not rooted yet in defined research I’ve come across but with the race from hyperscalers to capture AI in sales such as Amazon Bedrock, Azure AI Foundry and Google Vertex Studio the interchangeable nature of being model agnostic will likely be a sticky area for customers. This means less reliance on closed-source such as GPT-4o, o1 or Claude Sonnet but rather adoption of Llama from Meta and other open-source models. The first question you might be thinking from this prediction is the why model agnostic?
Costs matter and when performance becomes relatively close or superior to closed-source the costs of the open-source use and adoption will likely increase to a model agnostic view.
– Steven R
5. Cybersecurity infused with AI will get substantially more reliable
Notably many vendors depending on the vertical are shaping up to adopt some form in products the use of Generative AI whether its to help with SOC teams in summarization of events or enrichment of threat intel to streamline a workflow. This space is still relatively evolving and depending on the tech stack your organization is running the interoperability and support of third-party plugins or API workflows will be dependent on getting the most out of these tools. This is a increased area to watch for because at some point this will form a LLM that purely trained on Cybersecurity Domain Specific Knowledge and then optimized further by other components to augment many tedious areas. The human-in-the-loop will still be pivotal and here to stay I could see some semi-autonomous agents doing some tasks depending on the risk decided but not as much that is hyped up. I’ve created a video on YouTube showcasing the use of Copilot for Security as well with some examples on to use it. Check it out linked here. https://www.youtube.com/watch?v=qB-czlqJAM8&t=88s
Closing Thoughts
As 2024 comes to the tail end and finale much of what was anticipated on the threat of Generative AI Security attacks has relatively not come to fruition at least on a grand scale. This is not to diminish the attack surface but any cataclysmic downturn such as company held hostage for ransomware and massive bankruptcy filings for attack hasn’t occurred. While some legislation is starting to roll out as early as January in California to address Generative AI challenges it doesn’t appear on the radar to have sweeping overhaul at the national level in the US. The movement towards agents while at the surface may appear more nuanced then discussed here is a wide field for exploration and developing some best practices that are starting to come together from OWASP Top 10 recently linked here.
Resources
Connecticut State Colleges and Universities. The Rapid Adoption of Generative AI, 2024.
LangChain State of AI Agents Research. State of AI Agents, 2024
https://www.langchain.com/stateofaiagents
CNN. Finance worker pays out $25 million after video called with deepfake ‘chief financial officer, 2024.
https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html