Ultrastrongly coupled quantum hardware may increase the speed of quantum state processing in distributed architectures, allowing to approach fault-tolerant threshold. We show that circuit QED architectures in the ultrastrong coupling regime, which has been recently demonstrated with superconductors, may show substantial speedup for a class of adiabatic protocols resilient to the main source of errors, namely the interplay of dynamical Casimir effect and cavity losses.