AstroPath: Astronomy Meets Pathology to Study Cancer Immunotherapy

We have developed a detailed, automated, multi-step approach to multispectral staining and image analysis of multiplex immunofluorescent (mIF) slides of cancerous tissue. These support the study of cancer immunotherapy, the complex tumor microenvironment through quantification of key marker co-expression patterns on specific cells and spatial relationships between different interacting cell types.

Using the AstroPath pipelines, we have processed over 20 cohorts of 1,000 slides which have all been ingested into the AstroPath database to date. They contain a total of 60K High Powered Fields, 1.5B detected cells, of which our unique statistical sample contains 1B distinct cells, with full geometric boundaries for their nuclei and membranes. In order to support the TME studies, the databases contain precomputed neighbors within 25 microns to each cell, for a total of 3.5 billion cell pairs, with their respective phenotypes. We had to do complex image processing over 65 trillion pixels. For comparison, the Sloan Digital Sky Survey (SDSS) has collected about 6.4 trillion raw pixels from the sky over 16 years of imaging, with a budget of $250M.

Our group brings together astronomers, particle physicists, machine vision and deep learning experts with pathologists working on cancer. We have heavily relied on lessons from sky surveys. Over the last ten years we have worked very hard to adopt these to cancer immunotherapy in a systematic fashion. The resulting AstroPath platform demonstrates the feasibility of an automated system feeding into an open public database.  We are soon releasing open-source tumor-immune Atlases with billions of spatially mapped cells, enabling public analyses at unprecedented scales, similar to sky surveys.

Using AstroPath, in 2021 we have discovered the first predictive biomarker for melanoma, which can predict with a high significance whether a patient diagnosed with melanoma will respond to the current generation of immunotherapy drugs. Our spatial dataset is unique in its size and complexity. We are increasingly using AI, both for code development and in testing new hypotheses to discover new pathways in the cancer-Immune interactions. We are in the process of building a large multimodal foundation model, incorporating genomics data from The Cancer Genomic Atlas (TCGA), and integrating the melanoma collection with our spatial imaging from the same TCGA patients.

The talk will describe how lessons learned in astronomy have enabled us to short circuit the whole design and implementation process.