We investigate the variations in the broadband luminosities of low and intermediate redshift galaxies due to the stochastic nature of the neutral hydrogen distribution present in the form of Ly$\alpha$ absorbers in the intergalactic medium. This effect is caused by variations in the distribution and properties (redshift, column density, Doppler parameter) of the absorbers along different lines-of-sight out to a given redshift. Using a set of observationally constrained redshift-, column density-, and Doppler parameter distributions we perform Monte Carlo simulations for a large number of lines-of-sight towards galaxies at a given redshift $z_{em}$ and calculate attenuated FUV/NUV magnitudes and corresponding 1-, 2-, and 3$\sigma$ variations. We predict significant variations in luminosity ranging from -0.22 to +0.48 magnitudes at the 1$\sigma$level for galaxies at$z=1.5$inGALEXdata.