rbcor

An R package for reading BCOR (binary correlation) files generated by LDStore and other tools into R's matrix formats.

Overview

rbcor provides fast and memory-efficient access to correlation matrices stored in LDStore's BCOR format. The package supports:

Available matrix output formats include regular dense matrices, sparse matrices with thresholding, and memory-optimized packed symmetric matrices.

Installation

# remotes::install_github("mkanai/rbcor")
install.packages("rbcor", repos = c('https://mkanai.r-universe.dev', 'https://cloud.r-project.org'))

Features

Usage

Basic Usage

library(rbcor)

# Open example BCOR file included with the package
bcor_file <- system.file("extdata", "data.bcor", package = "rbcor")
bcor <- read_bcor(bcor_file)

# View basic information
print(bcor)
#> BCOR file: /path/to/rbcor/extdata/data.bcor
#> Number of SNPs: 55
#> Number of samples: 5363

# Get metadata
meta <- bcor$get_meta()
head(meta)
#>   rsid position chromosome allele1 allele2
#> 1  rs1        1         01       A       G
#> 2  rs2        2         01       A       G
#> 3  rs3        3         01       A       G
#> 4  rs4        4         01       A       G
#> 5  rs5        5         01       A       G
#> 6  rs6        6         01       A       G

Reading Correlation Matrices

# Read full correlation matrix
corr_full <- bcor$read_corr()
dim(corr_full)
#> [1] 55 55

# Read subset of SNPs
corr_subset <- bcor$read_corr(snps = c(1, 10, 20))
dim(corr_subset)
#> [1] 55  3

# Read rectangular submatrix
corr_rect <- bcor$read_corr(snps = 1:10, snps2 = 11:20)
dim(corr_rect)
#> [1] 10 10

Sparse Matrices

For large matrices with many small correlations, use sparse format to save memory:

# Only keep correlations above threshold
corr_sparse <- bcor$read_corr(sparse = TRUE, threshold = 0.1)
class(corr_sparse)
#> [1] "dgCMatrix"
#> attr(,"package")
#> [1] "Matrix"

# Check sparsity
nnzero(corr_sparse) / (nrow(corr_sparse) * ncol(corr_sparse))
#> [1] 0.8727273  # About 87% of values above threshold

# Sparse subset
corr_sparse_subset <- bcor$read_corr(
  snps = 1:20,
  sparse = TRUE,
  threshold = 0.05
)

Memory-Optimized Packed Matrices

For symmetric correlation matrices, packed storage uses approximately 50% less memory:

# Configure packed storage threshold (default: 1000)
# Lower threshold for demonstration with example data
bcor <- read_bcor(bcor_file, packed_threshold = 10)

# Use packed storage (example data: 55x55 >= 10)
corr_packed <- bcor$read_corr(packed = TRUE)
class(corr_packed)
#> [1] "dspMatrix"
#> attr(,"package")
#> [1] "Matrix"

# Packed matrices support all standard operations
result <- corr_packed %*% rep(1, ncol(corr_packed))
subset_value <- corr_packed[1, 5]

# Memory comparison
object.size(bcor$read_corr())          # Regular matrix
#> 24344 bytes
object.size(corr_packed)               # Packed matrix
#> 12584 bytes  # ~48% less memory

Performance Features

Automatic Optimization

The package automatically chooses the most efficient storage format:

# Get example data file
bcor_file <- system.file("extdata", "data.bcor", package = "rbcor")

# Small matrices: regular dense format (default threshold = 1000)
bcor_default <- read_bcor(bcor_file)
corr1 <- bcor_default$read_corr(packed = TRUE)
#> Warning: Matrix size (55x55) below packed threshold (1000). Returning regular matrix.
class(corr1)
#> [1] "matrix" "array"

# Lower threshold: packed symmetric format
bcor_packed <- read_bcor(bcor_file, packed_threshold = 10)
corr2 <- bcor_packed$read_corr(packed = TRUE)  # Returns dspMatrix
class(corr2)
#> [1] "dspMatrix"
#> attr(,"package")
#> [1] "Matrix"

Memory Usage Comparison

# Example with the included 55x55 correlation matrix
bcor_file <- system.file("extdata", "data.bcor", package = "rbcor")
bcor <- read_bcor(bcor_file, packed_threshold = 10)

# Regular matrix
corr_regular <- bcor$read_corr(packed = FALSE)
object.size(corr_regular)
#> 24344 bytes

# Packed matrix (48% memory savings)
corr_packed <- bcor$read_corr(packed = TRUE)
object.size(corr_packed)
#> 12584 bytes

# Values are identical
all.equal(as.matrix(corr_packed), corr_regular)
#> [1] TRUE

Extended Format Support

The package automatically detects and handles extended BCOR files that store diagonal values:

# Check if file uses extended format
bcor$is_extended
#> [1] FALSE  # Standard format in example

# Get diagonal values
diag_vals <- bcor$get_diagonal()
head(diag_vals)
#> [1] 1 1 1 1 1 1  # All 1.0 for standard format

# For extended format files:
# - Diagonal values are read from the file
# - All matrix operations automatically use these values
# - The API remains the same for both formats

File Format

BCOR files store correlation matrices in a compressed binary format:

Supported compression types:

Acknowledgements

This R implementation is based on the original Python implementation in the ldstore package by Christian Benner. The C++ code structure and binary file format parsing closely follow the Python reference implementation found in ldstore/bcor.py.

References

License

MIT