Search

What are you looking for?

  1. MIMIC: A Generative Multimodal Foundation Model for Biomolecules

    Publication
    We introduce MIMIC a generative multimodal foundation model trained across biomolecular sequences 3D structures and functional assays. MIMIC performs unified representation learning and conditional generation across nucleic acids …

  2. Three-dimensional Scene Reconstruction Using Roman Slitless Spectra

    Publication
    The Nancy Grace Roman Space Telescope will carry out a wide-field imaging and slitless spectroscopic survey of Type Ia Supernovae to improve our understanding of dark energy …

  3. A Fully Photometric Approach to Type Ia Supernova Cosmology in the LSST Era: Host Galaxy Redshifts and Supernova Classification

    Publication
    The upcoming Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) is expected to discover nearly a million Type Ia supernovae (SNeIa), offering an unprecedented …

  4. Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme

    Publication
    Conditional diffusion models provide a natural framework for probabilistic prediction of dynamical systems and have been successfully applied to fluid dynamics and weather prediction. However, in many settings …

  5. T-FIX: Text-Based Explanations with Features Interpretable to eXperts

    Publication
    As LLMs are deployed in knowledge-intensive settings (e.g., surgery, astronomy, therapy), users expect not just answers, but also meaningful explanations for those answers. In these settings …

  6. Multimodal Datasets with Controllable Mutual Information

    Publication
    We introduce a framework for generating highly multimodal datasets with explicitly calculable mutual information between modalities. This enables the construction of benchmark datasets that provide a novel testbed …

  7. Universal Spectral Tokenization via Self-Supervised Panchromatic Representation Learning

    Publication
    Sequential scientific data span many resolutions and domains, and unifying them into a common representation is a key step toward developing foundation models for the sciences. Astronomical spectra …

  8. AION-1: Omnimodal Foundation Model for Astronomical Sciences

    Publication
    While foundation models have shown promise across a variety of fields, astronomy still lacks a unified framework for joint modeling across its highly diverse data modalities. In this …

  9. AION-1: Omnimodal Foundation Model for Astronomical Sciences

    Publication
    While foundation models have shown promise across a variety of fields, astronomy lacks a unified framework for joint modeling across its highly diverse data modalities. In this paper …

  10. Impact of Pretraining Word Co-occurrence on Compositional Generalization in Multimodal Models

    Publication
    CLIP and large multimodal models (LMMs) have better accuracy on examples involving concepts that are highly represented in the training data. However, the role of concept combinations in …

  11. The Multimodal Universe: Enabling Large-Scale Machine Learning with 100TB of Astronomical Scientific Data

    Publication
    We present the MULTIMODAL UNIVERSE, a large-scale multimodal dataset of scientific astronomical data, compiled specifically to facilitate machine learning research. Overall, the MULTIMODAL UNIVERSE contains hundreds of …

  12. The Multimodal Universe: 100 TB of Machine Learning Ready Astronomical Data

    Publication
    We present the Multimodal Universe, a new framework collating over 100 TB of multimodal astronomical data for its first release, spanning images, spectra, time series, tabular and hyper …