Fastly has launched Fastly AI Accelerator, the company\u2019s first AI solution designed to improve performance and reduce costs across the use of similar prompts for LLM apps.<\/p>\n\n\n\n
AI Accelerator is designed to reduce API calls and costs with intelligent, semantic caching. Built on Fastly\u2019s Edge Cloud Platform and leveraging industry leading caching technology, it uses a specialised API gateway to improve performance for apps using popular LLMs.<\/p>\n\n\n\n
When using Fastly AI Accelerator, developers only need to update their app to use a new API endpoint, which typically only requires changing a single line of code. Fastly AI Accelerator will then transparently implement semantic caching for OpenAI compatible APIs. This approach goes beyond traditional caching as Fastly AI Accelerator can understand the context of the requests and queries and will send a similar response if two or more requests are alike.<\/p>\n\n\n\n
Anil Dash, Vice President of Developer Experience, Fastly, said: \u201cFastly AI Accelerator gives developers exactly what they want, by making the experience of their favourite LLMs a lot faster and more efficient, so they can focus on what makes their app or site unique – and what keeps their users happy.\u201d<\/p>\n\n\n\n
Stephen O\u2019Grady, Principal Analyst, RedMonk, said: “Developers and enterprises alike are turning in large numbers to medium and smaller models. Whether it\u2019s to lower costs, to shorten training cycles or to run on more limited hardware profiles, they\u2019re an increasingly important option.\u201d<\/p>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
Fastly has launched Fastly AI Accelerator, the company\u2019s first AI solution designed to improve performance and reduce costs across the use of similar prompts for LLM apps. AI Accelerator is designed to reduce API calls and costs with intelligent, semantic caching. Built on Fastly\u2019s Edge Cloud Platform and leveraging industry leading caching technology, it uses […]<\/p>\n","protected":false},"author":58,"featured_media":37970,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[30,43],"tags":[226,1837,1937,4342,4114],"class_list":["post-37969","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-intelligent-technology","category-top-stories","tag-ai","tag-api","tag-fastly","tag-llm","tag-openai"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/posts\/37969","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/users\/58"}],"replies":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/comments?post=37969"}],"version-history":[{"count":1,"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/posts\/37969\/revisions"}],"predecessor-version":[{"id":37971,"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/posts\/37969\/revisions\/37971"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/media\/37970"}],"wp:attachment":[{"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/media?parent=37969"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/categories?post=37969"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/north-america\/wp-json\/wp\/v2\/tags?post=37969"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}