Skip to Content
Find More Like This
Return to Search

Parallel file system with metadata distributed across partitioned key-value store

United States Patent

September 19, 2017
View the Complete Patent at the US Patent & Trademark Office
Los Alamos National Laboratory - Visit the Technology Transfer Division Website
Improved techniques are provided for storing metadata associated with a plurality of sub-files associated with a single shared file in a parallel file system. The shared file is generated by a plurality of applications executing on a plurality of compute nodes. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file generated by an application executing on the compute node and metadata for the at least one portion of the shared file on one or more object storage servers. The compute node is also configured to implement a partitioned data store for storing a partition of the metadata for the shared file, wherein the partitioned data store communicates with partitioned data stores on other compute nodes using a message passing interface. The partitioned data store can be implemented, for example, using Multidimensional Data Hashing Indexing Middleware (MDHIM).
Bent; John M. (Los Alamos, NM), Faibish; Sorin (Newton, MA), Grider; Gary (Los Alamos, NM), Torres; Aaron (Los Alamos, NM)
EMC IP Holding Company LLC (Hopkinton, MA), Los Alamos National Security, LLC (Los Alamos, NM)
13/ 931,832
June 29, 2013
STATEMENT OF GOVERNMENT RIGHTS This invention was made under a Cooperative Research and Development Agreement between EMC Corporation and Los Alamos National Security, LLC. The United States government has rights in this invention pursuant to Contract No. DE-AC52-06NA25396 between the United States Department of Energy and Los Alamos National Security, LLC for the operation of Los Alamos National Laboratory.