There is growing evidence supporting the coeval growth of galaxies and their resident super-massive black hole (SMBH). Most studies also claim a correlation between the activity of the SMBH and the star formation of the host galaxy. It is unclear, however, whether this correlation extends to all redshifts and X-ray luminosities. Some studies find a weaker dependence at lower luminosities and/or a suppression of the star formation at high luminosities. We here use data from the X-ATLAS and XMM-XXL North fields and compile the largest X-ray sample up to date to investigate how X-ray selected AGN affect the star formation of their host galaxies in a wide redshift and luminosity baseline of 0.03 < z < 3 and log LX(2−10 keV) = (41−45.5) erg s−1. Our sample consists of 3336 AGN. 1872 of our sources have spectroscopic redshifts. For the remaining sources we calculate photometric redshifts using TPZ, a machine-learning algorithm. We estimate stellar masses (M⋆) and star formation rates (SFRs) by applying spectral energy distribution fitting through the CIGALE code, using optical, near-IR, and mid-IR photometry (SDSS, VISTA, and WISE). Of our sources, 608 also have far-IR photometry (Herschel). We use these sources to calibrate the SFR calculations of our remaining X-ray sample. Our results show a correlation between the X-ray luminosity (LX) and the SFR of the host galaxy at all redshifts and luminosities spanned by our sample. We also find a dependence of the specific SFR (sSFR) on redshift, while there are indications that the X-ray luminosity enhances the sSFR even at low redshifts. We then disentangle the effects of stellar mass and redshift on the SFR and again study its dependence on the X-ray luminosity. Towards this end, we estimate the SFR of main-sequence galaxies that have the same stellar mass and redshift as our X-ray AGN and compare them with the SFR of our X-ray AGN. Our analysis reveals that the AGN enhances the star formation of its host galaxy when the galaxy lies below the main sequence and quenches the star formation of the galaxy it lives in when the host lies above the main sequence. Therefore, the effect of AGN on the SFR of the host galaxy depends on the location of the galaxy relative to the main sequence.