4 MIN READ NASA Impact AI Team Blog UPDATED Sep 2, 2026 PUBLISHED Sep 2, 2026 Every year, millions of research papers, datasets, and technical reports flood scientific archives. As manual processing becomes nearly impossible, subject matter experts are left spending a significant amount of time tagging, classifying, and routing content, causing the backlog to grow exponentially.INDUS-SDE is a new specialized language model developed by researchers with NASA’s Marshall Space Flight Center Impact AI team and IBM Research and pre-trained on information from NASA’s Science Discovery Engine (SDE). The model is tailored for web-sourced scientific content and is effective at extracting domain-relevant terms while ignoring web clutter like navigation menus and boilerplate noise, allowing it to scale expert curation workflows.Weighted Dynamic Masking: Filtering Web ClutterThe core innovation driving INDUS-SDE is Weighted Dynamic Masking (WDM). Standard pretraining for Masked Language Modeling (MLM) masks tokens at random. While effective for clean content, this approach wastes substantial training effort on structural noise when applied to web-crawled scientific text. WDM bypasses this issue by using Yet Another Keyword Extractor (YAKE) algorithm to identify the most salient terms in each document. The system then masks those keywords at a 30% rate while masking the remaining text at approximately 4.5%. As a result, the model allocates more capacity for higher-value tasks like predicting informative tokens including "chloroplast," "spectral irradiance," and "perihelion," rather than repetitive noise such as "click here" or "copyright." Image The Weighted Dynamic Masking (WDM) pre-training strategy for INDUS-SDE prioritizes and masks critical scientific keywords to enhance model generalization.Credit: NASA Impact AI team Model Performance and Real-World Benchmark ResultsINDUS-SDE has achieved a top-1 masked language modeling accuracy of 78.1% on NASA's SDE test set, compared to 50.3% for the previous INDUS model and 65.0% for ModernBERT trained on the same corpus with standard masking. This performance advantage extends to real-world curation tasks. For example, on Global Change Master Directory (GCMD) keyword recommendation — assigning Earth science metadata keywords from a 3,240-label vocabulary — INDUS-SDE achieved a Mean Reciprocal Rank (MRR) of 0.791, meaning the correct keyword is consistently ranked at or near the top of suggestions.Additionally, on Time-Domain and Multi-Messenger Astrophysics (TDAMM) astrophysics classification, with only 408 hand-labeled training samples, it matched the performance of AstroBERT, a domain-specific model pretrained on 400,000 astrophysics articles. A content relevancy classifier was also derived from INDUS-SDE. The classifier achieved F1 scores above 0.98, serving as a reliable quality gate for the SDE ingestion pipeline.Semantic Discovery and Scalable Search with INDUS-SDE-STBeyond classification, this release introduces INDUS-SDE-ST, a sentence transformer designed for semantic scientific discovery. Instead of simply matching keywords, INDUS-SDE-ST retrieves content based on underlying meaning. For instance, a search for "sea surface warming" also retrieves datasets tagged as "SST anomalies." On the NASA SDE retrieval benchmark, INDUS-SDE-ST achieved a 45% higher MRR at 1 compared to the baseline INDUS sentence transformer and a 33% improvement over ModernBERT-ST.To deploy this architecture at scale across SDE's 600,000-document corpus, the team aims to utilize Embedding Quantization-Aware Training (EQAT). This method compresses embeddings to 1-bit binary format during training, reducing storage requirements by 10 to 16x while maintaining robust retrieval accuracy.Open Science Impact and Community RecognitionThe deployment of INDUS-SDE into NASA's SDE pipeline will mark a significant shift in data curation, successfully replacing hours of tedious expert labor with automated precision. With all models, datasets, and benchmarks now hosted openly on HuggingFace under the nasa-impact repository, the team offers an instructive case study for the broader AI community.Ultimately, INDUS-SDE challenges the “bigger is better” paradigm of modern AI. It demonstrates that the precise pretraining strategy can matter more than data volume alone. By prioritizing the right tokens, INDUS-SDE matches the output of specialized models trained on vastly larger pools of in-domain data, serving as a roadmap for any team filtering chaotic, web-sourced scientific content into clear scientific insights.The broader data science community recently learned more about advancements with INDUS-SDE at the 2026 Knowledge Discovery and Data Mining (KDD) Conference in Jeju, Korea. The researchers behind INDUS-SDE submitted a paper describing their work that was accepted and presented in the AI4Sciences track at this international forum for data science, AI and machine learning. The full INDUS-SDE paper can be directly accessed on the Association for Computing Machinery (ACM) Digital Library.INDUS-SDE development is funded by the Office of the Chief Science Data Officer within NASA's Science Mission Directorate at NASA Headquarters in Washington.To learn more about INDUS large language models and other AI tools for science, visit https://science.nasa.gov/artificial-intelligence-science.To learn more and explore the full capabilities of the newly released models, visit the INDUS-SDE and INDUS-SDE-ST repositories.
