Electroencephalography (EEG) signal has been used widely in health and medical fields. It is also used in brain-computer interface (BCI) systems for humans to continuously control mobile robots and wheelchairs. Recently, the research communities successfully explore the potential of using EEG as a new type of biometrics in user authentication. EEG-based user authentication systems have the combined advantages of both password-based and biometric-based authentication systems, yet without their drawbacks. In this paper, we propose to take the advantage of rich information, such as age and gender, carried by EEG signals for user authentication in multi-level security systems. Our experiments showed very promising results for the proposed multi-factor EEG-based authentication method.