{"id":87923,"date":"2023-07-05T14:00:52","date_gmt":"2023-07-05T13:00:52","guid":{"rendered":"https:\/\/www.intelligentcio.com\/me\/?p=87923"},"modified":"2023-07-13T16:04:06","modified_gmt":"2023-07-13T15:04:06","slug":"technology-innovation-institute-trained-open-source-falcon-40b-model-on-amazon-sagemaker","status":"publish","type":"post","link":"https:\/\/www.intelligentcio.com\/me\/2023\/07\/05\/technology-innovation-institute-trained-open-source-falcon-40b-model-on-amazon-sagemaker\/","title":{"rendered":"Technology Innovation Institute trained open-source Falcon 40B model on Amazon SageMaker"},"content":{"rendered":"\n

Amazon Web Services, announced that Technology Innovation Institute, TII, trained its open-source Falcon 40B model on AWS. Falcon 40B is a 40-billion-parameter large language model, LLM available under the Apache 2.0 license that ranked #1 in Hugging Face\u2019s Open LLM Leaderboard.<\/p>\n\n\n\n

It was trained on 1 trillion tokens on Amazon SageMaker, a fully managed service for developing, training, tuning, and hosting machine learning models, including LLMs. The launch represents a significant milestone in the UAE\u2019s commitment to fostering AI innovation and scientific contributions in line with the UAE National AI Strategy 2031.<\/p>\n\n\n\n

Customers can now deploy Falcon 40B from Amazon SageMaker JumpStart, a machine learning, ML hub that offers pre-trained models, giving customers access to Falcon 40B\u2019s state-of-the-art accuracy and industry- performance without having to build their own model from scratch.<\/p>\n\n\n

\n
\"\"
Dr Ebtesam Almazrouei, Executive Director, Acting Chief AI Researcher of AI Cross centre Unit and Project Lead for LLM Projects at TII<\/em><\/figcaption><\/figure><\/div>\n\n\n

Dr Ebtesam Almazrouei, Executive Director, Acting Chief AI Researcher of AI Cross centre Unit and Project Lead for LLM Projects at TII, highlighted Falcon 40B in a new blogpost, stating: \u201cWe proudly announce the official open-source release of Falcon-40B, the world’s top-ranking open-source language model.\u201d<\/p>\n\n\n\n

Wojciech Bajda, Managing Director, Public Sector Middle East, and Africa at AWS, said: “We are proud to have collaborated with the Technology Innovation Institute on the development of the Falcon LLM 40B model, which leverages Amazon SageMaker to train a model. The open-source release of Falcon-40B enables organisations to leverage its exceptional capabilities and advance AI-driven solutions to create new opportunities for progress and advancements.”<\/p>\n\n\n\n

Falcon 40B is an exceptional open-source model with 40B parameters, specifically designed as a causal decoder-only model. It was trained on a vast dataset of 1,000B tokens, including RefinedWeb enhanced with curated corpora. The model is available under the Apache 2.0 license, ensuring its accessibility and usability. The architecture of Falcon-40B is optimised for inference, incorporating FlashAttention and multi-query techniques.<\/p>\n","protected":false},"excerpt":{"rendered":"

Amazon Web Services, announced that Technology Innovation Institute, TII, trained its open-source Falcon 40B model on AWS. Falcon 40B is a 40-billion-parameter large language model, LLM available under the Apache 2.0 license that ranked #1 in Hugging Face\u2019s Open LLM Leaderboard. It was trained on 1 trillion tokens on Amazon SageMaker, a fully managed service […]<\/p>\n","protected":false},"author":59,"featured_media":88185,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[11661,6787,829,15989,517,15994,78,13,185,79],"tags":[3237,16022,13321,4478],"class_list":["post-87923","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-latest-threats-data","category-digital-transformation","category-editors-choice","category-facilities-and-servers","category-hardware","category-middle-east","category-regional-news-newsletter","category-top-stories","category-uae","category-used","tag-aws","tag-dr-ebtesam-almazrouei","tag-technology-innovation-institute","tag-wojciech-bajda"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/posts\/87923","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/users\/59"}],"replies":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/comments?post=87923"}],"version-history":[{"count":8,"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/posts\/87923\/revisions"}],"predecessor-version":[{"id":88430,"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/posts\/87923\/revisions\/88430"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/media\/88185"}],"wp:attachment":[{"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/media?parent=87923"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/categories?post=87923"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/me\/wp-json\/wp\/v2\/tags?post=87923"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}