Hierarchical Options-GAN: A Scaffold-Aware Generative Adversarial Network for De Novo Molecular Design

Abstract

We present the Hierarchical Options-GAN (HO-GAN), a novel generative adversarial network for de novo molecular design that addresses the long-standing challenge of sparse reward propagation in molecular graph construction. The core innovation lies in a two-tier generative architecture inspired by hierarchical reinforcement learning, which decouples global scaffold planning from local atom-level assembly. In our framework, a high-level controller sequentially samples reusable structural motifs, termed options, from a pre-learned embedding space of chemically valid subgraph templates; these options represent common ring systems, linkers, and functional groups with defined attachment points. A low-level policy then refines each selected motif by iteratively assigning atom types and bond orders under strict valence constraints, effectively building molecules as hierarchically composed fragments. To provide dense supervisory signals, we design a dual discriminator system: a global graph discriminator assesses overall molecular plausibility, while a local discriminator evaluates the chemical validity of each atomic expansion step. Furthermore, we integrate a property predictor that furnishes real-valued rewards for policy gradient updates of the high-level controller, thereby encouraging the generation of molecules with desired physicochemical or pharmacological profiles. The option set is constructed via unsupervised graph clustering of a large drug-like molecule database, and the entire architecture is pretrained on a motif decomposition task before end-to-end adversarial fine-tuning. Empirical results on benchmark molecular generation tasks demonstrate that HO-GAN consistently produces structurally diverse, chemically valid molecules with superior control over scaffold topology compared to flat-generation baselines.