Obviously, we need to get clarity on the concept of "noise" before we can reason intelligently about "noise ceiling".
Some possible definitions:
"measurement fluctuations that are uncorrelated with the stimulus" → well, uncorrelated is a bit too strong here, I would say. But if you meant something like 'unrelated to the stimulus and tend to average out across trials' then I think that's pretty good.
"any signal you are not interested in" → This might be a bit too drastic... but maybe okay.
KK's preferred definition: "measurement fluctuations (in your data) that you as the scientist are fundamentally uninterested in and that you as the scientist don't really know how to remove (other than simple averaging)"
May be useful in some contexts to divide noise into different types: measurement noise, brain noise (or neural noise), motion-related noise, physiological noise, etc.
Noise ceiling
What is it?
It's a concept that refers to trying to understand the ceiling (i.e. the max) that one cannot exceed due to noise. In other words, noise may pose fundamental limits on some property of the data. For example, a computational model that starts with a stimulus (and knows only the stimulus) can only predict the exact same response for different trials of the same stimulus, and cannot predict the idiosyncratic neural response that one observes on a given trial (i.e. trial-to-trial variability).
The term "explainable variance" is often used to refer to the variance in your data that is actually explainable (based on the thing that is constant from trial from trial and which is accessible to the thing doing the explanation).
The noise ceiling is model-independent (in the sense of a model that is attempting to predict/explain the responses in the data based on some external measure (like a stimulus)). Hence, once you compute the noise ceiling, it should apply to any candidate model for the data. (But, it is true that you could argue that in computing a noise ceiling, we arerelying on a noise ceiling model.) Furthermore, the computation of the noise ceiling depends on some version of the data (i.e. preprocessed data), so one can reasonably question the fidelity/robustness/accuracy of those prepared data.
What is it good for?
Typically, it's used to discount how bad your model is. Like, "Of course, my model's performance is low because the noise ceiling is low."
For example, you can report the variance that your model explains in the data divided by the explainable variance (i.e. the noise ceiling that you calculated for your data).
General principles for calculating it:
Calculating a noise ceiling requires repeated trials, meaning you have to collect data for the same experimental manipulation for multiple trials (>1).
There are many approaches out there and they are potentially very different in nature/approach and also very different with respect to the numbers they provide. Proceed with caution!!
KK's favored theoretical approach:
Make basic assumptions about the distribution of signal and noise. For example:
Noise is zero-mean Gaussian (with some unknown standard deviation), independent of the stimulus, and additive
Signal is Gaussian-distributed
Estimate parameters of the noise and signal distributions from the data.
Calculate the amount of variance (in percentage units) in the experimental data that is expected to be driven by signal.
There are easy things one can do which are proxies for the noise ceiling. For example, split-half correlation/reliability is easy and quick-and-dirty, and under most reasonable scenarios monotonically related to the noise ceiling, but the value obtained is not a true noise ceiling.
Caveats/pitfalls
Be careful about blindly computing a specific noise celing estimate without fully understanding/vetting what it does.
Note that the noise ceiling is an estimate, and can itself be noisy!
When interpreting results/figures, sometimes people sort of hide or obscure the fact that they are actually reporting normalized performance (i.e. normalized by the noise ceiling)