Thirty years ago, stock markets around the globe suffered their greatest one-day losses ever. It\u2019s also the year (1987) the investment management firm AHL (now Man AHL) was founded to deliver a revolutionary approach to hedge fund management.<\/p>\n
Man AHL is a quantitative systematic investment manager, with more than US$19 billion in assets under management.<\/p>\n
‘Quantitative’ means all of the firm\u2019s investment decisions are based on mathematical models (algorithms); ‘systematic’ means those decisions are made solely by computer, without human involvement.<\/p>\n
Given this business model, Man AHL has attracted two extremely valuable sets of resources: the roughly 50 quantitative researchers (\u2018quants\u2019) who formulate the ideas driving the investment models and the 60-plus technologists who both develop the quantitative research platform and turn those ideas into trading strategies that can be executed by a computer.<\/p>\n
One of the firm\u2019s highest priorities is delivering a robust and ever-more-performant technology platform as a means of optimising the performance and productivity of its quants and technologists.
\n\u201cOur technology is key in giving the quants an environment in which they can explore and test their ideas, as well as a production environment that will faithfully implement the models they create,\u201d said Gary Collier, Co-CTO at Man AHL in London. Man AHL is part of Man Group, whose five investment management businesses manage over US$95 billion.<\/p>\n
Legacy storage fails to keep up with demands<\/strong><\/p>\n The demands on technology infrastructure at the firm are significant. \u201cIn the development of our trading models, we need to look at large amounts of data to run experiments and sift through all manner of possible ideas that might make money for our investors. Only those models in which we have sufficient conviction will then be put to work in the production trading environment,\u201d Collier said. \u201cWe had a HPC (high-performance computing) environment with shared storage that supported all our research and software development activity,\u201d noted James Blackburn, Head of Data Engineering at Man AHL. \u201cAt times, you would have people manipulating large numbers of files at the same time quants were trying to run large simulations. Our storage system just didn\u2019t have the IOPS to support all those demands simultaneously, which meant some people would get frustrated by the response times. We needed something that would provide much greater IOPS and bandwidth.\u201d<\/p>\n Collier noted: \u201cWe\u2019ve had poor experiences in the past with expensive \u2014 and supposedly highly parallel \u2014 storage solutions simply not being able to deliver the bandwidth and IOPS we require. This time, we wanted to make sure we conducted extreme due-diligence to explore every possibility.\u201d Blackburn was charged with managing the search for a new storage infrastructure.<\/p>\n In addition to high IOPS and bandwidth, \u201cwe also wanted the new storage to be highly-scalable,\u201d Collier added. \u201cWe\u2019re an extremely agile business and need to be able to add terabytes of storage almost on a whim. We just want to be able to start where we are, then add capacity and IOPS as necessary. In other words, we want something that scales linearly.\u201d<\/p>\n In addition, Collier noted: \u201cWe are not in the storage business and do not want to dedicate multiple engineers to the job of building and maintaining bespoke storage solutions, or to administering complex-to-manage vendor products. That\u2019s not what we\u2019re best at, that\u2019s not where the value-add from our technology lies.\u201d<\/p>\n The search for a new solution lead to pure storage<\/strong><\/p>\n Knowing that its need for high bandwidth and IOPS meant next-generation storage, the team investigated suppliers of all-flash products. \u201cPure Storage is very well-respected, both by organisations like Gartner and by users,\u201d Collier said. \u201cMore often than not, if you Google a storage vendor\u2019s name, you\u2019ll find horrific stories about data loss and other problems. But the customers who have purchased and deployed Pure take the time to write Internet posts about their positive experiences. That was a good sign for us from the start.\u201d<\/p>\n Based on its initial due diligence, the Man AHL team arranged for a proof of concept (PoC) trial of a Pure FlashBlade system from Pure Storage. FlashBlade is the first all-flash storage system purpose-built for modern analytics \u2014 architected from the ground-up to deliver a powerful cloud-era data platform that is fast, compact, infinitely scalable and easy to manage. FlashBlade is designed for customers who need parallel storage to support data-intensive applications involving parallel processing, such as the type of financial simulations conducted by Man AHL.<\/p>\n Man AHL\u2019s FlashBlade configuration has eight blades, each with 52TB raw capacity. Factoring in the overhead of the Purity OS and erasure coding, Man AHL is achieving 329.14 TiB of effective capacity thanks to an aggregate data reduction of 1.5:1 for all workloads running on FlashBlade.<\/p>\n \u201cIn the proof of concept phase, we pushed the Pure Storage system to the max,\u201d Blackburn reported. \u201cWe tested total throughput for large I\/O, maximum IOPS and other metrics. Overall, the system scaled as promised. We achieved 6GBs bandwidth for large I\/O reads and 3GBs for writes. And the 600,000 IOPS we achieved were far more than we ever had before.\u201d<\/p>\n Performance improvements create time-to-market advantage <\/strong><\/p>\n The impact of the new storage infrastructure was noticed immediately. \u201cMany of our researchers have found that the introduction of FlashBlade has made it easier to use Spark for performing multiple simulations. One of them experienced a 10x-to-20x improvement in throughput for his Spark workloads, compared to the previous storage system,\u201d Blackburn said. <\/p>\n Collier added: \u201cOur quants want to test a model, get the results and then test another one, and another one \u2014 all day long. So, a 10x-20x improvement in performance can be a game-changer when it comes to creating a time-to-market advantage for us.\u201d <\/p>\n The greatest benefit from installing Pure FlashBlade, Collier said, \u201cis significantly improved productivity for the team and accelerated time-to-market for new trading ideas.\u201d <\/p>\n The POC involved workloads from across the firm\u2019s research teams, \u201cand we picked jobs that were very demanding on I\/O, like our Jenkins builds and Spark users,\u201d Blackburn said. \u201cAs we progressed, we loaded more and more concurrent workloads and did benchmark testing to see how close we were to the limits of the system. What we found is that FlashBlade gives us significant headroom beyond what we need for the business today. This gives us confidence knowing we can simply add a blade to gain more capacity and preserve the investment we already have.\u201d <\/p>\n A month into the POC, the firm\u2019s entire research department had been migrated to FlashBlade. In addition, Man AHL runs a concurrent 50 TB Mongo database containing order-book data from stock exchanges and other sources. \u201cDuring the POC, we put a replica set of the Mongo data on FlashBlade using an NFS mount. It worked so well that we left it there,\u201d Collier said. <\/p>\n Beyond meeting the high-performance requirements of its core simulation applications, Blackburn said, \u201cthe real benefit of the FlashBlade is the consistent I\/O and metadata access. To have a shared substrate of I\/O that is consistent throughout the day makes users very happy. Storage used to be the bottleneck and now that has been removed.\u201d <\/p>\n Effortless management and a clear path for the future<\/strong> <\/p>\n Man AHL benefits from more than FlashBlade\u2019s raw performance. \u201cThere are significant savings from the simplicity of operation. It just works,\u201d Collier noted. \u201cThe fact that Pure Storage is plug-and-play adds to our operational efficiency. We\u2019ve essentially cut our headcount for storage management from one-and-a-half to the equivalent of a half-time role. And going forward, it\u2019ll approach zero. That means we can redeploy those assets to higher-priority activities.\u201d <\/p>\n Looking to the future, Collier said FlashBlade is a perfect fit for the firm\u2019s expanding storage and computing needs \u2014 but at times is unpredictable. \u201cFor a year we had been thinking about the challenges of shared NFS storage,\u201d he recalled, \u201cand FlashBlade came along at just the right time. FlashBlade is designed to scale out NFS, so it\u2019s ideal for the kind of use cases we have. This includes simulations using Spark, where you are processing lots and lots of unstructured data and working with millions of files.\u201d<\/p>\n \u201cIf we extrapolate the way we have been growing, we double our storage needs about every 18 months. And that\u2019s a scary prospect. A Pure Storage competitor that we looked at required 4x the rack space of FlashBlade, without the ability to add capacity. In contrast, FlashBlade is very compact and scales easily just by adding blades. Having something as flexible and expandable as FlashBlade is what our business demands.\u201d <\/p>\n Another attractive long-term benefit is Pure Storage support for the OpenStack initiative. \u201cWe are heavy users of Docker,\u201d Collier said, \u201cso we plan to take full advantage of OpenStack and Docker integration on the Pure data platform. That makes it very easy from a DevOps standpoint to create Docker volumes and attach them to containers.\u201d <\/p>\n Above and beyond product satisfaction, the Man AHL team has benefited from the extreme attentiveness of Pure Storage customer and technical support. As early as the POC, Blackburn said: \u201cWe saw how responsive the Pure team is. We had answers to questions generally within an hour of sending them, whereas we find other vendors can take weeks to respond. We\u2019ve had astonishing turnaround from Pure.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":" Thirty years ago, stock markets around the globe suffered their greatest one-day losses ever. It\u2019s also the year (1987) the investment management firm AHL (now Man AHL) was founded to deliver a revolutionary approach to hedge fund management. Man AHL is a quantitative systematic investment manager, with more than US$19 billion in assets under management. […]<\/p>\n","protected":false},"author":21,"featured_media":21376,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[42,205,1477,93,23],"tags":[271,6954,6953,6951,6952],"class_list":["post-21374","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-banking-finance","category-case-studies","category-hardware","category-top-stories","category-united-kingdom","tag-bandwidth","tag-high-performance-computing","tag-iops","tag-man-ahl","tag-pure-flashblade"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/posts\/21374","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/comments?post=21374"}],"version-history":[{"count":1,"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/posts\/21374\/revisions"}],"predecessor-version":[{"id":21377,"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/posts\/21374\/revisions\/21377"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/media\/21376"}],"wp:attachment":[{"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/media?parent=21374"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/categories?post=21374"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/eu\/wp-json\/wp\/v2\/tags?post=21374"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nIn 2016, the technologists at Man AHL had concerns about the ability of its legacy mass storage infrastructure to meet the demands of the quants and developers.<\/p>\n