202409212324 Status: #lecture Tags: #ai #drug_discovery # Multi-modal and Generative Models for Drug Design Presented by Marinka Zitnik - Three core topics: - Integration of cellular contexts with 3D protein structure - Multimodal protein prediction - Protein optimization ### Integration of cellular contexts with 3D protein structure Paper link: https://www.biorxiv.org/content/10.1101/2023.07.18.549602v1 Some proteins' functions depend upon the cellular context that they exists within. How can we develop models that dynamically adjust their outputs to biological contexts in which they operate? This group developed a model called PINNACLE that is a multi-scale graph neural network for precise protein representation learning and cell type- and state-specific prediction. PINNACLE integrates single cell transcriptomics data with a protein interaction network, celltype interaction network, and tissue hierarchy to generate protein representations with celltype resolution. PINNACLE is a self-supervised model, optimizing to predict links between entities in the multi-scale graph (see below image). Results in MANY embeddings per protein, each tailored to a specific cell type. PINNACLE propagates message on proteins, celltypes, and tissues using attention mechanisms specific to each node and relationship type: - The protein-level objective function, which considers self-supervised link prediction on the protein interactions and celltype-identity classification on the protein nodes, enables PINNACLE to produce an embedding space that captures both the topology of the celltype-specific protein interaction networks and the celltype identity of proteins. - The celltype- and tissue-specific components in celltype- and tissue-specific objective functions are based solely on self-supervised link prediction to learn cellular and tissue organization. - Such information is propagated to the protein representations using an attention bridge, imposing tissue and cellular organization to the protein representations. PINNACLE enables contextualized, precise predictions of drug effects across cell types and cell states. ![[Pasted image 20240305122111.png]] ### Multimodal protein prediction Paper links: - SIPT: https://www.nature.com/articles/s42256-023-00647-z - Mulitmodal language model: Need to repurpose/change concepts from language modeling to customize them for biological sequences. There are opportunities for integrating biologically relevant information into large language models to make them more powerful. Some examples: - Structure-inducing pre-training (SIPT) allows users to specify the structure they want via a pre-training graph. Input to language model is both the pre-training graph and the protein sequences (see below image). - Multimodal language model that takes as input both a sequence and a natural language, plain-text description of the sequence. They use gene ontology, PDB, etc. descriptions of function to create an instruction-tuning dataset that allows researchers to use natural language to specify protein function for generation. ![[Pasted image 20240305124012.png]] ### Protein optimization Paper link: https://www.biorxiv.org/content/10.1101/2024.02.25.581968v1 Designing small-molecule-binding proteins, such as enzymes and biosensors, is crucial in protein biology and bioengineering. Generating high-fidelity protein pockets—areas where proteins interact with ligand molecules—is challenging due to complex interactions between ligand molecules and proteins, flexibility of ligand molecules and amino acid side chains, and complex sequence-structure dependencies. Here, the speaker introduces PocketGen, a deep generative method for generating the residue sequence and the full-atom structure within the protein pocket region that leverages sequence-structure consistency. PocketGen consists of a bilevel graph transformer for structural encoding and a sequence refinement module that uses a protein language model (pLM) for sequence prediction. The bilevel graph transformer captures interactions at multiple granularities (atom-level and residue/ligand-level) and aspects (intra-protein and protein-ligand) with bilevel attention mechanisms. For sequence refinement, a structural adapter using cross-attention is integrated into a pLM to ensure structure-sequence consistency. ![[Pasted image 20240305124052.png]] ![[Pasted image 20240305124119.png]] --- # References