Bringing All Types of System Devices to Life with Perfect Memory, Panmnesia
Visions
News & Demos
Special Seminar on Harnessing the Potential of CXL with Panmnesia
Panmnesia had the pleasure of being invited by Professor Byung-Gon Chun, the CEO of FriendliAI at Seoul National University, to present at his seminar. During our presentation, we shed light on two significant applications of the advanced Compute Express Link (CXL) technology. Firstly, we discussed a combined software-hardware system that enhances search capabilities for approximate nearest neighbors. This solution uses CXL to separate memory from host resources and optimize search performance despite CXL’s distant memory characteristics. It employs innovative strategies and utilizes all available hardware, outperforming current platforms in query latency. Secondly, we presented a robust system for managing large recommendation datasets. It harnesses CXL to integrate persistent memory and graphics processing units seamlessly, enabling direct memory access without software intervention. With a sophisticated checkpointing technique that updates model parameters and embeddings across training batches, we’ve seen a significant boost in training performance and energy savings. We’re looking forward to more such opportunities! Keep an eye out for our future endeavors! Link - Seminar Information
Panmnesia's CXL Prefetcher for Next Generation CXL-SSD
Panmnesia, Inc., the industry leader in data memory and storage solutions, is thrilled to announce that their cutting-edge SSD technology, Panmnesia’s ExPAND, has been accepted for presentation at the prestigious HotStorage 2023 conference, to be held in Boston this July. Panmnesia’s ExPAND is a groundbreaking development that integrates Compute Express Link (CXL) with SSDs, allowing for scalable access to large memory. Although this technology currently operates at slower speeds than DRAMs, ExPAND mitigates this by offloading last-level cache (LLC) prefetching from the host CPU to the CXL-SSDs, which dramatically improves performance. The cornerstone of Panmnesia’s ExPAND is its novel use of a heterogeneous prediction algorithm for prefetching, which ensures data consistency with CXL sub-protocols. The system also allows for detailed examination of prefetch timeliness, paving the way for more accurate latency estimations. Furthermore, ExPAND’s unique feature lies in its awareness of CXL multi-tiered switching, providing end-to-end latency for each CXL-SSD, as well as precise prefetch timeliness estimations. This breakthrough reduces the reliance on CXL-SSDs and enables most data to access the host cache directly. Early tests are already showing promise: Panmnesia’s ExPAND has been shown to enhance the performance of graph applications by a staggering 2.8 times, surpassing CXL-SSD pools with a variety of prefetching strategies. ExPAND’s acceptance into HotStorage 2023 underscores its potential to revolutionize the way we manage and access data. With ExPAND, Panmnesia, Inc. continues to solidify its reputation as a frontrunner in data storage solutions, pushing the boundaries of technology and driving the industry forward. The full details of ExPAND and its advantages will be presented at HotStorage 2023. Stay tuned for more exciting updates from the conference.
Hearst - Panmnesia's CXL Solution for AI Acceleration
Interested in discovering the potential applications of the CXL solution? We invite you to explore a recent white paper detailing Panmnesia’s CXL innovation, known as Hearst, which significantly enhances AI-driven applications (such as recommendation systems and vector search) through exceptional performance and cost efficiency. In summary, Hearst’s distinguishing features include 1. port-based routing (PBR) CXL switch for high scalability, 2. near-Data processing for high performance, and 3. DIMM pooling technology for a low cost. We hope you find this information helpful and enjoyable to read. Read The White Paper Download The PDF
Revolutionizing Billion-Scale AI-Powered Search Services with CXL Memory Pooling
We are thrilled to announce that Panmnesia’s groundbreaking research has been recognized by the esteemed USENIX ATC’23, boasting a competitive 18% acceptance rate. Our study explores the acceleration of nearest neighbor search services, handling billion-point datasets through innovative CXL memory pooling and coherence network topology. This approach enables substantial memory requirements for ANNS, such as Microsoft’s search engines (Bing and Outlook) which demand over 40TB for 100B+ vectors, each described by 100 dimensions. Likewise, Alibaba’s e-commerce platforms require TB-scale memory for their 2B+ vectors (128 dimensions). Join us in Boston this July to learn more about our exciting findings!
AI Acceleration Framework Empowering GPUs to Unleash a Massive Memory Space
Get ready for a revolution in AI acceleration with Panmnesia’s CXL-augmented system - making waves in StorageNewsletter! Our groundbreaking technology empowers GPUs to unleash a massive memory space of up to 4 Petabytes, supercharging the training of colossal machine learning models like recommendation systems and ChatGPT using just a few GPUs! Embrace substantial cost savings and push the boundaries of AI capabilities for your business! Don’t miss out - explore our trailblazing tech via the following link Link - Article (StorageNewsletter)
New Machine Learning Framework for Large-Scale Data Training with CXL 3.0
Panmnesia’s research team has developed an efficient machine learning framework designed for large-scale data training in datacenters, utilizing CXL 3.0 technology. Our recent achievements have been highlighted in Forbes, which can be accessed via the first link below. To gain a deeper understanding of our CXL 3.0-based training system, we recommend watching our informative video presentation via the second link below. The video demonstrates how CXL can enhance large-scale machine learning acceleration and provides insights for ML and AI system designers to consider when implementing this technology. We are excited to announce that our team will be showcasing this innovative system at the IPDPS ‘23 conference in Florida this May. We hope to see you there and look forward to discussing our advancements with you in person! Link - Article (Forbes) Link - Presentation Video
CXL-based Terabytes Memory Expansion Platform
Our team has successfully expanded memory by over a dozen terabytes, making it the largest CXL (Compute Express Link) hardware and software solution in the world. It’s designed to work seamlessly with multi-CPUs, CXL switches, and CXL memory expanders, offering a versatile option that can be easily integrated into any tech setup. We’re excited to share that we’ve successfully validated the system with the latest Linux full stack and have proven its capabilities in large-scale deep learning and machine learning-based recommendation systems. During our demonstration, we were able to assign all embedding features to the remote-side CXL memory expansion without the need for an assistant module. Our software and hardware IPs are seamlessly integrated with TensorFlow, which allows for effective management of training and servicing with ease. We’re committed to pushing the boundaries of innovation and are eager to showcase the range of state-of-the-art, memory-intensive applications running on our system. We’re also excited to introduce an array of new hardware-architected devices in the pipeline. Stay connected for further updates on our exciting new innovations! We’re open to exploring collaboration opportunities and would love to work with you. Please don’t hesitate to get in touch with us at [email protected] or [email protected] if you’re interested. Thanks for being a part of our journey!
Interview about CXL and Panmnesia
Panmnesia, a KAIST start-up company that focuses on developing cache coherent interconnect (CCI) technologies using Compute Express Link (CXL), has garnered significant attention from the technology industry, particularly in the field of computing and memory. Seoul Finance recently published an article that highlights Panmnesia as the first developer of a real CXL system with both working hardware and software modules. The article includes an interview with Panmnesia’s CEO, who explains the significance of CXL and how it could impact various computing and memory industries. CXL is a new cache coherent interconnect standard that supports multiple protocols and enables cache coherent communication between CPUs, memory devices, and accelerators. According to Panmnesia’s CEO, CXL is the most important technology in the next-generation memory semiconductor industry. The article explains that as the demand for high-performance and high-capacity memory semiconductor grows, CXL could be a viable solution for memory disaggregation, allowing memory resources to be efficiently and cost-effectively utilized. The article also discusses the potential impact of CXL on artificial intelligence (AI) acceleration and ChatGPT (Conversational AI based on the GPT model). Panmnesia’s CEO believes that CXL can play a crucial role in AI acceleration by offering large-scale memory subsystems to existing processing silicon and co-processing accelerators such as GPU, DPU, and TPU. This will enable faster data processing and more efficient use of resources. Panmnesia has already secured a significant number of intellectual property rights related to CXL, including the CXL standard IP, and is currently in talks with various domestic and international institutions for collaboration. Panmnesia aims to become the leading solution provider in the CXL industry and is focused on developing new and innovative CCI technologies to meet the demands of various industries. Link - Interview (Korean)
Emerging CXL Workshop with Leading Industry Experts and Renowned Academics
We had the privilege of attending a dynamic CXL workshop, where we were invited to share our knowledge and expertise as a leading vendor in CPU and memory technology. The workshop brought together industry experts from renowned companies such as Intel, AMD, Samsung, SK-Hynix, and our own company, Panmnesia. We were thrilled to participate in this high-level discussion, which explored the latest trends and innovations in CXL technology and its impact on the future of large-scale data-centric applications. As a participant in the workshop, we had the opportunity to give an invited talk and share our insights and perspectives on the direction that data-centric applications should take. We highlighted the importance of cutting-edge research directions such as near-data processing and AI acceleration, and how they can be leveraged to drive innovation and progress in this field. The discussions throughout the workshop were engaging and thought-provoking, and we were impressed by the wealth of knowledge and expertise shared by the attendees. We gained valuable insights into the latest trends and innovations in CXL technology, which will undoubtedly shape our thinking and approach to data-centric applications in the future. Overall, we were honored to have been a part of this workshop and grateful for the opportunity to share our knowledge and experience with other industry leaders. We look forward to continued engagement with CXL technology and its impact on the future of large-scale data-centric applications.
Invited Talk for Memory Pooling over CXL at Computer System Society 2023
We are thrilled to share our recent participation in the prestigious Computer System Society 2023 conference, where we had the honor of delivering an invited talk entitled “Memory Pooling over CXL”. Our talk was met with great enthusiasm from attendees, as we presented our history of development and research for CXL and memory expander technology from 2015 to 2023, including some specific use-cases. We also introduced a cutting-edge memory pooling stack and showcased a CXL 3.0 memory expander device reference board that we built from the ground up. As a leading vendor in CXL and memory technology, we are committed to driving innovation and progress in this field. We are particularly excited about our new memory pooling stack, which we believe has the potential to revolutionize the industry by providing a more efficient and cost-effective way to manage memory resources. Our CXL 3.0 memory expander device reference board is also a testament to our commitment to cutting-edge research and development, as we continue to push the boundaries of what is possible in this field. In addition to our talk at the Computer System Society 2023 conference, we are also honored to be participating in a closed CXL working symposium later this month, alongside industry leaders such as AMD, Intel, Samsung, and SK-Hynix. We look forward to sharing more technical details about our latest developments and discussing our vision for the future of CXL technology and its impact on data-centric applications. Overall, our participation in the Computer System Society 2023 conference was a tremendous success, and we are excited to continue to drive innovation and progress in this field. We remain committed to pushing the boundaries of what is possible in CPU and memory technology, and we look forward to sharing more updates and insights in the future. Check Full Program
TrainingCXL: Failure Tolerant Training with Persistent Memory Disaggregation over CXL
CXL 2.0 End-to-End System
CXL Forum at FMS'22: CXL-SSD
CXL Forum at FMS'22: CXL 2.0 End-to-End Solution
Technologies
The Panmnesia Hardware Development Toolkit (PanHDK) and Hardware Reference Board (PanHRB) are reliable and system-validated development vehicles to make your CXL development faster. PanHDK/HRB provides an environment that allows you to integrate Panmnesia’s state-of-the-art CXL logic into your hardware, such as CPU, co-processors, domain-specific/ML accelerators, and memory controllers, at the register-transfer level (RTL). The development results of PanHDK/PanHRB are compatible with any type of CXL components and devices. Fab-less and Silicon companies can have their own CXL development environment with the collaboration of Panmnesia.
The Panmnesia End-to-End (PanE2E) validator helps you verify, validate, and integrate their vendor-specific solutions into a CXL system in a fast and efficient manner. Even though you do not have the entire hardware components, you can do verify and validate all the functionalities and features that your device has to support with our PanE2E. For example, a hardware company has CXL memory expanders but cannot validate them without supporting CPU vendors and switches (and vice versa). Our PanE2E allows you to have the upper hand and make your CXL devices/prototypes verified and validated systems without vendor lock-in.
The Panmnesia System-on-Chip (PanSoC) is a holistic compute express link (CXL) infrastructure, including CPU-to-device, CPU-to-memory, and device-to-memory connections for hyperscalers. PanSoC offers extremely scalable, high-performance CXL switches and interconnect networks that even have intelligent near-data processing units (DPU) and processing-in-memory (PIM) units. PanSoC includes a set of hardware IP licenses for several CXL agents and engines, which can allow existing devices such as processors, diverse accelerators, and memory to be easily connected to CXL. PanSoC’s agents/engines can be integrated into the following devices by empowering CXL:
The Panmnesia Cluster (PanCluster) is a rack-scale resource disaggregation solution for high-performance datacenters and HPC. PanCluster is different from the conventional datacenter’s box architectures from the viewpoint of composability and interoperability. Panmnesia’s cache coherent interconnect and controller technologies (CXL 2.0/3.0) allow datacenters/HPC to completely disaggregate their resources from the computational complex and to see a scalable, high-capacity memory pool. PanCluster composes everything on the on-the-fly, giving you a free hand to have what you only need and to choose the best hardware/software without vendor or architecture-specific lock-in. Meet your capacity, efficiency, and update demands at a pretty low cost.
The Panmnesia Software Development Kit (PanSDK) offers an easy-to-use software development environment for hyperscalers, OS, kernel, and system developers. PanSDK includes various kernel drivers, runtime libraries, and interface sets, which can be used not only for Panmnesia’s hardware solutions (PanCluster, PanSoC, and PanHDK/HRB) but also for user-customized CXL-enabled solutions. PanSDK also includes CXL-aware data processing and acceleration software IPs, which can make your CXL system much faster. Application developers, such as databases, ML/AI frameworks, and graph analytics, can enjoy the cache coherent memory pool by satisfying a broad spectrum of demands.
The Panmnesia Hardware Development Toolkit (PanHDK) and Hardware Reference Board (PanHRB) are reliable and system-validated development vehicles to make your CXL development faster. PanHDK/HRB provides an environment that allows you to integrate Panmnesia’s state-of-the-art CXL logic into your hardware, such as CPU, co-processors, domain-specific/ML accelerators, and memory controllers, at the register-transfer level (RTL). The development results of PanHDK/PanHRB are compatible with any type of CXL components and devices. Fab-less and Silicon companies can have their own CXL development environment with the collaboration of Panmnesia.
The Panmnesia End-to-End (PanE2E) validator helps you verify, validate, and integrate their vendor-specific solutions into a CXL system in a fast and efficient manner. Even though you do not have the entire hardware components, you can do verify and validate all the functionalities and features that your device has to support with our PanE2E. For example, a hardware company has CXL memory expanders but cannot validate them without supporting CPU vendors and switches (and vice versa). Our PanE2E allows you to have the upper hand and make your CXL devices/prototypes verified and validated systems without vendor lock-in.
The Panmnesia System-on-Chip (PanSoC) is a holistic compute express link (CXL) infrastructure, including CPU-to-device, CPU-to-memory, and device-to-memory connections for hyperscalers. PanSoC offers extremely scalable, high-performance CXL switches and interconnect networks that even have intelligent near-data processing units (DPU) and processing-in-memory (PIM) units. PanSoC includes a set of hardware IP licenses for several CXL agents and engines, which can allow existing devices such as processors, diverse accelerators, and memory to be easily connected to CXL. PanSoC’s agents/engines can be integrated into the following devices by empowering CXL:
The Panmnesia Cluster (PanCluster) is a rack-scale resource disaggregation solution for high-performance datacenters and HPC. PanCluster is different from the conventional datacenter’s box architectures from the viewpoint of composability and interoperability. Panmnesia’s cache coherent interconnect and controller technologies (CXL 2.0/3.0) allow datacenters/HPC to completely disaggregate their resources from the computational complex and to see a scalable, high-capacity memory pool. PanCluster composes everything on the on-the-fly, giving you a free hand to have what you only need and to choose the best hardware/software without vendor or architecture-specific lock-in. Meet your capacity, efficiency, and update demands at a pretty low cost.
The Panmnesia Software Development Kit (PanSDK) offers an easy-to-use software development environment for hyperscalers, OS, kernel, and system developers. PanSDK includes various kernel drivers, runtime libraries, and interface sets, which can be used not only for Panmnesia’s hardware solutions (PanCluster, PanSoC, and PanHDK/HRB) but also for user-customized CXL-enabled solutions. PanSDK also includes CXL-aware data processing and acceleration software IPs, which can make your CXL system much faster. Application developers, such as databases, ML/AI frameworks, and graph analytics, can enjoy the cache coherent memory pool by satisfying a broad spectrum of demands.
Solutions
