Fusion energy could become a significant contributor to green energy in the 21st century. Several fusion energy concepts have proposed the use of high field (>20 tesla) superconducting magnets to create high performance plasmas in compact, lower cost devices. Achieving such strong magnetic fields requires the use of Rare Earth Barium Copper Oxide (REBCO) high temperature superconductor; however, the performance evolution of REBCO under fusion-relevant radiation damage and high-fidelity environmental conditions (e.g. temperature, magnetic field, transport current, axial strain, etc.) of an operating superconducting fusion magnet has not been well characterized to date. To directly address this shortcoming, we present the design of a cryogenic proton irradiation facility capable of providing in-situ, in-operando characterization of the critical current of REBCO tapes and initial data from the first experimental campaigns. REBCO samples have been irradiated with 1.2 MeV protons at temperatures ranging from 20 K to 300 K with critical current (Ic) and temperatures (Tc) measured as a function of radiation dose. Importantly, no thermal annealing took place between the cryogenic irradiation and Ic and Tc measurements, thus emulating the microstructural damage sustained at a given temperature and its effects on Ic and Tc as in an operating superconducting magnet. Thermal annealing to recover superconducting performance has been investigated, and we present results showing partial reversal of the radiation damage as a function of annealing temperature and time and confirming the importance of replicating the thermal history of an operating magnet in radiation damage studies.