In this paper we propose a method for learning the materials in a scene in an unsupervised manner making use of imaging spectroscopy data. Here, we view the input image spectra as a data point on a manifold which corresponds to a node in a graph whose vertices correspond to a set of parameters that should be inferred using the Expectation Maximisation (EM) algorithm. In this manner, we can pose the problem as a statistical unsupervised learning one where the aim of computation becomes the recovery of the set of parameters that allow for the image spectra to be projected onto a set of graph vertices defined a priori. Moreover, as a result of this treatment, the scene material prototypes can be recovered making use of a clustering algorithm applied to the parameter-set. This setting also allows, in a straightforward manner, for the visualisation of the spectra. We discuss the links between our method and self-organizing maps and illustrate the utility of the method as compared to other alternatives elsewhere in the literature.