The airglow continuum in the near infrared is a challenge to quantify due to its faintness, and the grating scattered light from atmospheric hydroxyl (OH) emission lines. Despite its faintness, the airglow continuum sets the fundamental limits for ground-based spectroscopy of faint targets, and makes the difference between ground and space-based observation in the interline regions between atmospheric emission lines. We aim to quantify the level of airglow continuum radiance in the VIS -- NIR wavelength range observable with silicon photodetectors for the site Observatorio del Roque de los Muchachos in a way that our measurement will not be biased by the grating scattered light. We aim to do this by measuring the airglow continuum radiance with a minimal and controlled contamination from the broad instrumental scattering wings caused by the bright atmospheric OH lines. We measure the airglow continuum radiance with longslit $\lambda/\Delta\lambda\sim4000$ spectrograph in $\sim$100\r{A} wide narrow band passes centered at 6720, 7700, 8700 and 10500\r{A} (in line with the R, I, and Z broadbands) with the 2.5-meter Nordic Optical Telescope under photometric dark sky conditions. The bandpasses are chosen to be as clean as possible from atmospheric absorption and the OH line emission keeping the radiation reaching the grating surface at minimum. We observe the zenith equivalent airglow continuum to be 22.5mag/arcsec2 at 6720\r{A} band, and 22mag/arcsec2 at 8700\r{A}. We derive upper limits of 22mag/arcsec2 at 7700\r{A} due to difficulty to find a clean part of spectrum for measurement, and 20.8mag/arcsec2 at 10500\r{A} due to low system sensitivity. Within measurement errors and the natural variability expected for the airglow emission our results for the Observatorio del Roque de los Muchachos are comparable to values reported for other major observatory sites. (abridged)