JSC Joins New Trillion Parameter Consortium
JSC is a member of the new international Trillion Parameter Consortium (TPC) formed by federal laboratories, research institutes, academia, and industry to create trustworthy and reliable generative, large-scale AI models for science. TPC aims to address key challenges in advancing AI for science, which include developing scalable model architectures and training strategies; organizing and curating scientific data for training models; optimizing AI libraries for current and future exascale computing platforms; and developing deep evaluation platforms to assess progress with regard to scientific task learning and reliability and trust.
TPC addresses three aspects relating to the complexity of building large-scale AI models: Identifying and preparing high-quality training data of various scientific domains and data sources; designing and evaluating model architectures, performance, training, and downstream applications; and developing cross-cutting and foundational capabilities such as innovations in model evaluation strategies with respect to bias, trustworthiness, and goal alignment. The initiative aims to provide the community with a venue in which multiple large model-building initiatives can flexibly collaborate to leverage global efforts to accommodate the diverse goals of individual initiatives.
Trillion parameter models represent the frontier of large-scale AI, with only the largest commercial AI systems currently approaching this scale. Even using the exascale systems on the horizon, training a state-of-the-art one trillion parameter model will require months of dedicated time. Such efforts will therefore involve large, multi-disciplinary, multi-institutional teams. TPC is seen as a vehicle to support collaboration and cooperative efforts among and within such teams. By joining this initiative, JSC successfully continues to be involved in and show support for strong interdisciplinary communities like LAION, which are pushing the frontiers of research on large-scale open foundation models and datasets to enable reproducible and transparent large-scale machine learning research.
More information can be found in the TPC press release.
Contact: Dr. Jenia Jitsev
from JSC News No. 300, 12 December 2023