Planetary-scale indexing of sequencing data

Introduction

Nowadays a huge amount of planetary DNA and RNA sequencing data is available and continues to double every two years. However, efficiently analyzing this data is impossible due to its size measured in petabases. Many high-impact biological discoveries could be made but are prevented by the lack of fast search algorithms.

I recently demonstrated this potential by discovering an order of magnitude more RNA virus species within all public RNA samples. A global index, i.e. a planetary genomic search engine, would unlock instant and inexpensive search within petabase-scale data.

Hypothesis

I hypothesize that I can create a searchable index for all of the public DNA and RNA sequencing data. Leveraging my unique expertise across algorithms and data structures for biological sequences, my plan is to:

1. Design efficient methods to assemble and compress all available sequencing data.
2. Construct an external-memory index that will support versatile biological queries.

Potential Impact

A planetary sequencing data index will enable a myriad of bioinformatics analyses that are currently out of reach.

I will demonstrate the utility of the index by:

1. Constructing a database of human transcripts with novel disease associations.
2. Discovering novel microbial species.
3. Providing a search engine for environmental metagenomes.

The resulting unprecedented collection of assembled genomes and compressed reads will lift a major challenge in data accessibility, improving its efficiency by several orders of magnitude, and revolutionizing the scale of future bioinformatics analyses.