Flatiron Software

Recent advances in calcium imaging acquisition techniques are creating datasets of the order of Terabytes/week. Memory and computationally efficient algorithms are required to analyze in reasonable amount of time terabytes of data. This project implements a set of essential methods required in the calcium imaging movies analysis pipeline. Fast and scalable algorithms are implemented for motion correction, movie manipulation, and source and spike extraction. CaImAn also contains some routines for the analyisis of behavior from video cameras. In summary, CaImAn provides a general purpose tool to handle large movies, with special emphasis on tools for two-photon and one-photon calcium imaging and behavioral datasets.

`plenoptic` is a python library for model-based stimulus synthesis. It provides tools to help researchers understand their model by synthesizing novel informative stimuli, which help build intuition for what features the model ignores and what it is sensitive to. These synthetic images can then be used in future perceptual or neural experiments for further investigation.

Pynapple is a light-weight python library for neurophysiological data analysis. The goal is to offer a versatile set of tools to study typical data in the field, i.e. time series (spike times, behavioral events, etc.) and time intervals (trials, brain states, etc.). It also provides users with generic functions for neuroscience such as tuning curves and cross-correlograms.

Due to the string-like nature of neurons and blood vessels, they could be abstracted as curved tubes with center lines and radii. This representation could be used for morphological analysis, such as path length and branching angle. Given an accurate voxel segmentation, the computation of object centerlines and radii is called skeletonization. RealNeuralNetworks.jl is developed to do that. Unlike most related packages, it combines the synaptic connectivity graph with morphological features and could be used to explore the relationship between synaptic connectivity and morphology. Recently, a new arising programing language, called Julia, is getting popular in data science. RealNeuralNetworks.jl is a Julia package and the algorithms are written from scratch for less dependency and efficiency.

BridgeStan provides efficient in-memory access to a Stan model through Python, R, Julia, Rust, and C. Stan is a differentiable probabilistic programming language for coding Bayesian statistical models and their posterior predictive inferences. BridgeStan provides access to log densities and their gradients and Hessians, variable names and sizes, constraining and unconstraining parameter transforms, and predictive simulation.

Recent advances in calcium imaging acquisition techniques are creating datasets of the order of Terabytes/week. Memory and computationally efficient algorithms are required to analyze in reasonable amount of time terabytes of data. This project implements a set of essential methods required in the calcium imaging movies analysis pipeline. Fast and scalable algorithms are implemented for motion correction, movie manipulation, and source and spike extraction. CaImAn also contains some routines for the analyisis of behavior from video cameras. In summary, CaImAn provides a general purpose tool to handle large movies, with special emphasis on tools for two-photon and one-photon calcium imaging and behavioral datasets.

Fast sinc transform libraries which compute sums of the sinc and sinc2 kernels between N arbitrary points in 1, 2, or 3 dimensions. This has applications in MRI and band-limited function approximation. The naive cost is O(N2) whereas our algorithm is quasi-linear in N. Written by our 2017 summer intern Hannah Lawrence.

FFT-accelerated interpolation-based t-SNE (FIt-SNE) is an efficient implementation of t-SNE (stochastic neighborhood embedding) for dimensionality reduction and visualization of high dimensional datasets. This code is able to perform 1000 iterations of t-SNE on one million data points in under 2 minutes on a desktop, which is many times faster than any other existing code. Written by Manas Rachh with collaborators at Yale.

Figurl lets you use Python to generate shareable figURLs (permalinks) to interactive visualizations. With minimal configuration, these can be generated from any computer with access to the internet. Data objects required for the visualization are stored in kachery-cloud and are referenced by content-address strings. Domain-specific visualization plugins are also stored in the cloud and are developed using ReactJS. The central website, figurl.org, pairs the visualization plugin with the data object to create the shareable interactive views.

FINUFFT is a set of libraries to compute efficiently three types of nonuniform fast Fourier transform (NUFFT) to a specified precision, in one, two, or three dimensions, on a multi-core shared-memory machine. The library has a very simple interface, does not need any precomputation step, is written in C++ (using OpenMP and FFTW), and has wrappers to C, fortran, MATLAB, octave, and python. As an example, given M arbitrary real numbers xj and complex numbers cj, with j=1,…,M, and a requested integer number of modes N, the 1D type-1 (aka “adjoint”) transform evaluates the N numbers.

FMM3D is a set of libraries to compute N-body interactions governed by the Laplace and Helmholtz equations, to a specified precision, in three dimensions, on a multi-core shared-memory machine. The 3D fast multipole method evaluates potentials (and gradients, etc) at a large number of targets due to a large number of sources, in linear or quasi-linear time. Our implementation exploits efficient plane wave expansions, SIMD-accelerated kernel evaluations, and multi-threading.

IronClust is a fast and drift-resistant spike sorting pipeline. The accuracy of spike sorting is validated by multiple ground-truth datasets from a number of contributing labs. IronClust can take advantage of GPU or a compute cluster if available. IronClust requires Matlab with image, parallel, and signal processing toolboxes. IronClust supports Windows, Mac, and Linux.

ISO-SPLIT is an efficient clustering algorithm that handles an unknown number of unimodal clusters in low to moderate dimension, without any user-adjustable parameters. It is based on repeated tests for unimodality—using isotonic regression and a modified Hartigan dip test—applied to 1D projections of pairs of putative clusters. It handles well non-Gaussian clusters of widely varying densities and populations, and in such settings has been shown to outperform K-means variants, Gaussian mixture models, and density-based methods.
This repository contains an efficient single-threaded implementation in C++, with a MATLAB/MEX interface.
It was invented and coded by Jeremy Magland, with contributions to the algorithm and tests by Alex Barnett, at SCDA/Flatiron Institute.

Kachery-cloud is a network for sharing scientific data files, live feeds, mutable data and calculation results between lab computers and browser-based user interfaces. Resources are organized into projects which are accessed via registered Python clients. Using simple Python commands you can store files, data objects, mutables or live feeds, and then retrieve or access these on a remote machine (or in a browser via JavaScript) by referencing universal URI strings. In the case of static content, URIs are essentially content hashes, thus forming a content-addressable storage database. While the primary purpose of kachery-cloud at this time is to support figurl, it can also be used independently in collaborative scientific research workflows and for improving scientific reproducibility and dissemination.

MountainSort is spike sorting software developed by Jeremy Magland, Alex Barnett, and Leslie Greengard at the Center for Computational Biology, Flatiron Institute in close collaboration with Jason Chung and Loren Frank at UCSF department of Physiology. MountainSort is a plugin package to MountainLab, a general framework for scientific data analysis, sharing, and visualization.

MountainLab is data processing, sharing and visualization software for scientists. It is built around MountainSort, a spike sorting algorithm, but is designed to more generally applicable.

Riccati is an efficient numerical solver developed for a class of ordinary differential equations whose solution may exhibit extremely quick oscillations. Standard routines available from scientific computing libraries typically struggle with these types of equations: their runtime grows as the oscillation frequency. Riccati can achieve a frequency-independent (constant) runtime. The package is written in Python, complete with documentation, tests, and interactive examples. It implements the robust algorithm described in Agocs & Barnett (2022), which is able to switch between two different numerical methods on-the-fly, adapting to the behavior of the solution, as well as choose its own stepsize and other parameters to achieve a user-specified accuracy.

SpikeForest is a reproducible, continuously updating platform which benchmarks the performance of spike sorting codes across a large curated database of electrophysiological recordings with ground truth. It consists of this website for presenting our up-to-date findings, a Python package which contains the tools for running the SpikeForest analysis, and an expanding collection of electrophysiology recordings with ground-truth spiking information.

Stan is an open-source platform for statistical modeling and high-performance statistical computation. Users rely on Stan for statistical modeling, data analysis and prediction in the social, biological and physical sciences, engineering, business, medicine, finance, education and sports. Users specify log density functions in Stan’s probabilistic programming language and get full Bayesian statistical inference with MCMC sampling (NUTS, HMC), approximate Bayesian inference with variational inference (ADVI), and penalized maximum likelihood estimation with optimization and Laplace approximation (L-BFGS).Stan’s math library provides differentiable real and complex special functions, probability functions and linear algebra using reverse- and forward-mode automatic differentiation. Stan has interfaces supporting statistical workflow in R, Python and Julia.

Octopus is quantum mechanics package aimed at the ab initio virtual experimentation on an ever-increasing range of system types that enables a reliable and accurate simulation of light-induced changes in the physical and chemical properties of complex systems. The first principles real-space-based Octopus project provides an unique framework to describe non-equilibrium phenomena in molecular complexes, low dimensional materials, and extended systems by accounting for electronic, ionic, and photon quantum mechanical effects within a generalized time-dependent density functional theory framework (TDDFT) and beyond. The Octopus package enables the simulation and characterization spatial and time resolved spectroscopies, ultrafast phenomena in molecules and materials, and new emergent states of mater (QED-materials and QED-chemistry).

SkellySim is a simulation package for simulating cellular components such as flexible filaments, motor proteins, and arbitrary rigid bodies. It’s designed to be highly scalable, capable of both OpenMP and MPI style parallelism, while using the efficient STKFMM/PVFMM libraries for hydrodynamic resolution.

Galactic Dynamics is the study of the formation, history, and evolution of galaxies using the orbits of objects — numerically-integrated trajectories of stars, dark matter particles, star clusters, or galaxies themselves. Gala is an Astropy-affiliated Python package that aims to provide efficient tools for performing common tasks needed in Galactic Dynamics research. Much of this code uses Python for flexible, user-friendly interfaces that interact with wrappers around low-level code (primarily C) to enable fast computations. Common operations include gravitational potential and force evaluations, orbit integrations, dynamical coordinate transformations, and computing chaos indicators for nonlinear dynamics. Gala heavily uses the units and astronomical coordinate systems defined in the Astropy core package.