Abstract
We propose$S^2$ GPT-PINN, Sparse and Small Generative Pre-Trained Physics-Informed Neural Network, for solving parametric partial differential equations (PDEs).$S^2$ GPT-PINN is tailored to domain-specific (families of) PDEs and characterized by its compact architecture and minimal computational need. Leveraging a small amount of extremely high quality data via a mathematically rigorous greedy algorithm following the traditional Reduced Basis Method,$S^2$ GPT-PINN relies on orders of magnitude less parameters than PINNs to achieve extremely high efficiency via two levels of customizations and reductions. The first is knowledge distillation via task-specific activation functions that are transferred from Pre-Trained PINNs selected by the greedy algorithm. The second is a judicious hyper reduction when calculating the physics-informed loss of the network compressing the number of collocation points by orders of magnitude to the size of the small network.